-
Notifications
You must be signed in to change notification settings - Fork 134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
draft of data workflow diagram task #1528
Merged
Merged
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
6a5448f
draft of data workflow diagram task
vildead b766125
Merge branch 'master' of github.com:elixir-europe/rdmkit into pr/vild…
bedroesb f308355
add sidebar and news item
bedroesb 5062bc1
addressing comments of #1528
vildead 0b6a9ec
Merge branch 'master' into data-workflow-diagram
vildead ccb4c80
resolving comments from Nazeefa
vildead 5b9a0f5
fixing name of creating data workflow page in different places
vildead 9c2d9c9
fixing name of page in news
vildead File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,7 +4,7 @@ page_id: plan | |
description: Introduction to data management planning. | ||
contributors: [Siiri Fuchs, Korbinian Bösl, Minna Ahokas, Federico Bianchini, Flora D'Anna] | ||
related_pages: | ||
your_tasks: [compliance, costs, dmp, data_security, dm_coordination, machine_actionability] | ||
your_tasks: [compliance, costs, dmp, data_security, dm_coordination, machine_actionability, data_workflow_diagram] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @vildead can you also kindly update name here? |
||
training: | ||
- name: Training in TeSS | ||
registry: TeSS | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
--- | ||
title: Data workflow diagram | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
contributors: [Vilem Ded] | ||
description: Best practices to capture your planned data workflow in a diagram. | ||
page_id: data_workflow_diagram | ||
related_pages: | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
dsw: | ||
faircookbook: | ||
--- | ||
|
||
## Why you should draw a data workflow diagram? | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Description | ||
|
||
Creating a data workflow diagram for your research project greatly enhances your data management process. While Data Management Plans (DMPs) can span tens or even hundreds of pages, a diagram offers a more concise and accessible way to represent your workflow. This visual approach helps bridge the gap between partners who may have varying perceptions and levels of understanding, ensuring everyone is aligned. Diagrams make it easier to identify potential issues by highlighting undocumented data sources or destinations that might otherwise go unnoticed in a lengthy document. Additionally, these visual representations are invaluable during discussions and planning sessions, sparking deeper conversations and encouraging collaboration. A well-crafted workflow diagram also boosts the engagement of less-involved partners or stakeholders, giving them a clear and intuitive way to contribute to the planning process. | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Considerations | ||
|
||
* Even a simple diagram can significantly improve clarity and communication, but it's essential to first consider the complexity of the workflow. If the process is straightforward and can be easily described in a single paragraph, a diagram might not be necessary. However, for more complex workflows, a diagram is invaluable for identifying steps, dependencies, and potential issues. | ||
* If your workflow can be fully explained in a short, detailed paragraph, then a diagram may not add much value. However, for processes with multiple steps, data sources, or stakeholders, a diagram helps break down complexities and ensures everyone has a shared understanding. | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* The more complex the workflow, the more time and effort it will take to create and update the diagram. It’s important to plan carefully and ensure that the time invested in diagram creation and updates is manageable and that your contribution is clearly recognized. | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* A diagram can include sensitive or confidential information, so be cautious about sharing it publicly. Ensure that proper security measures are in place if the diagram contains details that should remain private. | ||
* Be aware that diagrams can reveal critical details that could be exploited by attackers, such as system vulnerabilities or data access points. Make sure any workflow diagram that includes sensitive information is properly protected to prevent security breaches. | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* While a diagram is a helpful tool, it is usually not legally binding without a formal, written description of the workflow. It’s good practice to complement your diagram with a full textual annex to ensure the process is well-documented and understood. | ||
* Content of the diagram will depend on many factors. Its good practice to start with definition of the targeted audience (project partners, data managers, funders, public) and main purpose (capturing life of project data, clarification of data protection framework, description of pre-processing steps). Based on the targeted audience and purpose, you can then more precisely define the scope of your diagram, e.i what (not) to include. These can be physical assets and entities (partners/people, storage locations, instruments, datasets, documents) or logical elements of your project (processes, data types, partner roles, ...). | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Same as DMP, the diagram is a living resource and should be updated as the research project progresses. | ||
|
||
### Solution | ||
|
||
* Start with simple top level diagram. Share it and get feedback on what can be refined. | ||
|
||
* You can follow these steps. | ||
1. List all assets. | ||
2. List all partners and actors. | ||
3. List all processes. | ||
4. Define data sources - instrument, patient, collaborator, lab, ... | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
5. Define final data locations (sinks) - archives, repositories, external processes, ingestion zones, ... | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
6. Start drawing visual elements representing the outer interface (sources and sinks) and move inwards. | ||
7. Iteratively refine the diagram based on items which were not yet included. | ||
|
||
* Too complex diagrams can be split. Define sub-processes and detect input/output flows /interface). | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Various attributes of visual elements can be mapped to aesthetics. These can be: | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* square for processes performed automatically vs. rectangle for processes performed manually (shape). | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* red line for sensitive data flow vs. blue for non-sensitive data (line color). | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* color, fill, border, ... | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Be consistent in mapping of aesthetics. For example, instead of using fill colors to depict encryption status (red vs blue) and collaborator's roles (green for researchers vs blue for contractors), you can use small key icon for the encryption status (shape). | ||
* Include legend if the aesthetics is complicated | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* If the mapping gets too complicated, include a legend. | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Don't forget to include date of last update, version and your name. | ||
|
||
* Source files for the diagrams should be vector based to facilitate portability and reuse. SVG | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* For dissemination, you can use PNG or other | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Use vector graphics tool of your preference. | ||
* open source: Draw.io, Inkscape, Libre Office Draw, ... | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* licensed: MS Visio*, Miro.com*,Corel Draw*, MS PowerPoint* | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Investigate your tool and all features it provides. E.g. Draw.io allows you to host the diagram in Github making it very convenient tool for collaborative editing. | ||
vildead marked this conversation as resolved.
Show resolved
Hide resolved
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bedroesb can you kindly update page name here after "News page"? The new name is "Creating Data Workflow"