-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
f75754a
commit 277059c
Showing
27 changed files
with
211 additions
and
82 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1 @@ | ||
<link href="./static/custom.css" rel="stylesheet" /> | ||
|
||
<link href="./static/custom.css" rel="stylesheet" /> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,55 +6,73 @@ order: 600 | |
--- | ||
# Data Preparation | ||
|
||
A dataset summarizes research data on a delimited situation in a structured manner. The research data should be prepared in such a way that **easy reuse** is possible. In principle, the data record should be **tabular and column-based**. In most cases, soil and agricultural data have a **spatial reference**. The spatial position of the measuring points or areas should be given in the table as detailed as possible. | ||
A dataset summarizes research data on a delimited situation in a structured manner. The research data should be prepared | ||
in such a way that **easy reuse** is possible. In principle, the data record should be **tabular and column-based**. In | ||
most cases, soil and agricultural data have a **spatial reference**. The spatial position of the measuring points or | ||
areas should be given in the table as detailed as possible. | ||
|
||
A typical dataset for data transfer to the BonaRes Data Centre has the following properties: | ||
|
||
Each column (attribute) of the table contains the attribute name in the first row and the attribute values in the following rows, which means the data within the **table are column-oriented**. Each table or dataset must be given a short, concise name. For widespread reuse, work should be done in English if possible. | ||
Each column (attribute) of the table contains the attribute name in the first row and the attribute values in the | ||
following rows, which means the data within the **table are column-oriented**. Each table or dataset must be given a | ||
short, concise name. For widespread reuse, work should be done in English if possible. | ||
|
||
Typically, each table contains the following standard attributes: | ||
|
||
_Table 1: Example table with typical standard attributes of a data table_ | ||
|
||
<div class="table-wrapper scrollbar overflow-hidden"> | ||
<table class="comfortable"> | ||
<thead style="font-size: 24px; background-color: #A8A8A8"> | ||
<tr> | ||
<th><strong>Identifier</strong></th> | ||
<th><strong>x-coordinate</strong></th> | ||
<th><strong>y-coordinate</strong></th> | ||
<th><strong>Date</strong></th> | ||
<th><strong>beetles</strong></th> | ||
<th><strong>Attribute 2</strong></th> | ||
<th><strong>Attribute n</strong></th> | ||
|
||
</tr> | ||
</thead> | ||
<tbody> | ||
<tr> | ||
<td>1</td> | ||
<td>52,460126<br> | ||
<td>13,296310<br> | ||
<td>17.04.2018<br> | ||
<td>5<br> | ||
<td>...<br> | ||
<td>...<br> | ||
</tr> | ||
</table> | ||
_Fig. 1 Example table with typical standard attributes of a data table_ | ||
<table class="comfortable"> | ||
<thead style="font-size: 24px; background-color: #A8A8A8"> | ||
<thead> | ||
<tr> | ||
<th><strong>Identifier</strong></th> | ||
<th><strong>x-coordinate</strong></th> | ||
<th><strong>y-coordinate</strong></th> | ||
<th><strong>Date</strong></th> | ||
<th><strong>beetles</strong></th> | ||
<th><strong>Attribute 2</strong></th> | ||
<th><strong>Attribute n</strong></th> | ||
</tr> | ||
</thead> | ||
<tbody> | ||
<tr> | ||
<td>1</td> | ||
<td>52,460126<br> | ||
<td>13,296310<br> | ||
<td>17.04.2018<br> | ||
<td>5<br> | ||
<td>...<br> | ||
<td>...<br> | ||
</tr> | ||
</tbody> | ||
</table> | ||
</div> | ||
|
||
### The following formal criteria must be met: | ||
|
||
The following formal criteria must be met: | ||
|
||
✓ tables are unformatted (do not contain grafical elements, colored lines, …) | ||
✓ there are no internal references or formulas within a table | ||
✓ tables contain only one worksheet (avoid Excel folders with multiple sheets) | ||
1. tables are unformatted (do not contain grafical elements, colored lines, …) | ||
2. there are no internal references or formulas within a table | ||
3. tables contain only one worksheet (avoid Excel folders with multiple sheets) | ||
4. in case of multiple sheets, upload each sheet as a separate table | ||
5. each column contains a unique attribute name (between upper and lower case not differentiated z. B. “beetles” and “Beetles”) | ||
6. column delimiter is consistent and clearly identifies throughout the table (e. g. ”;” or “,”) | ||
7. attribute names contain no spaces (an underscore „_“ is allowed to use), special characters, umlauts, “ß” or units | ||
8. ...are a maximum of 30 characters long | ||
9. ...do not start with a number | ||
10. ...are only assigned once | ||
11. missing values are explicitly marked and clearly differ from “none specification "and" 0 " | ||
12. values within a column are constant (no change of reference variable or unit) | ||
13. values in the cells contain no units, explanations or abbreviations e. g. "Under detection limit” or “< 0.00 ” | ||
14. only one value specified per cell | ||
15. cells are not connected | ||
16. ... do not contain separators such as “;” (decimal numbers are uniformly written with “.” or “,” and differ from column delimiter) | ||
17. IDs are only assigned once | ||
18. each line is available once(there are no double lines e. g. due to copy errors) | ||
19. cells contain no spaces | ||
|
||
✓ each column contains a unique attribute name | ||
(between upper and lower case not differentiated z. B. “beetles” and “Beetles”) | ||
✓ column delimiter is consistent and clearly identifies throughout the table (e. g. ”;” or “,”) | ||
The prepared dataset should be submitted in the **file format txt, csv, xls or xslx.** Avoid file formats that cannot be read with common programs. For example, formats for special company software for data loggers. In addition to tables, the BonaRes Repository also publishes all formats of research data that are common in science, such as pictures, videos, texts. The BonaRes Centre also is able to deal with complex file structures (Access, SQL, Shape, ...). In this case contact the [support of BonaRes Repository](mailto:[email protected]). | ||
|
||
✓ attribute names contain no spaces (an underscore „_“ is allowed to use), special characters, | ||
umlauts, “ß” or units | ||
✓ ... are a maximum of 30 characters long | ||
✓ ... do not start with a number | ||
✓ ... are only assigned once | ||
!!!success File Formats | ||
Datasets submitted in file formats, which can be integrated to the SQL database of the BonaRes Data Centre is made available with full service (options for file formats and coordinate system, description of the data model, ...). Other research data e.g., Pictures, videos or comparable formats can be downloaded by the re-user in the file format as submitted. If you have any questions, contact the [support of BonaRes Repository](mailto:[email protected]) | ||
!!! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,80 @@ | ||
--- | ||
label: Dataset and Metadata | ||
icon: code-square | ||
label: Dataset and Metadata Submission | ||
icon: project-roadmap | ||
expanded: false | ||
order: 30 | ||
--- | ||
# Dataset and Metada Submission | ||
|
||
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent tristique nibh in leo varius tincidunt nec id magna. Donec sit amet gravida purus. Aliquam molestie hendrerit aliquam. Nullam efficitur tortor nunc, ac porttitor nisi semper sit amet. In hac habitasse platea dictumst. Vivamus aliquet diam sem, facilisis fringilla ipsum commodo in. Mauris venenatis turpis sed metus tempus eleifend. Mauris pulvinar, mauris et faucibus cursus, orci eros ullamcorper metus, vel feugiat sapien metus porta quam. Fusce in sollicitudin quam. | ||
After completing the metadata template, click on the **“submit new dataset”** button and **fill** the upload form. | ||
|
||
Quisque vehicula, nisl ac pellentesque porta, lorem ligula vestibulum nisl, sit amet vulputate leo ligula eu nunc. Fusce eu pulvinar ex, a posuere arcu. Nullam rutrum volutpat gravida. Mauris eu ultrices tellus. Praesent neque erat, pharetra at lorem vitae, vestibulum eleifend metus. Nullam interdum, massa id semper vulputate, risus felis aliquet arcu, ut finibus turpis lorem at est. Nunc sit amet ultrices lectus, sit amet malesuada ex. Vestibulum viverra leo ut consequat fermentum. | ||
:::content-center | ||
![_Submit a new dataset_](/static/img/fig_submit_new_dataset.png) | ||
::: | ||
|
||
Ut dictum ligula a vehicula ornare. Suspendisse euismod semper tempor. Aliquam pellentesque ex in est aliquam, non iaculis velit cursus. Suspendisse dapibus diam tincidunt luctus porta. Curabitur volutpat sapien et ante rhoncus faucibus. Phasellus tempor iaculis porttitor. Duis est mauris, tristique at interdum eget, finibus eget ligula. Vestibulum suscipit, leo sit amet accumsan venenatis, velit justo tincidunt lorem, interdum aliquam diam mi nec sem. | ||
### Filling out the Upload form | ||
|
||
Donec vel fringilla nisi. Integer aliquet ligula vitae neque ultricies, ultricies tristique lorem fringilla. Phasellus iaculis eu risus auctor tempor. Praesent neque nunc, faucibus ut tempor non, lobortis ac arcu. Integer varius hendrerit massa. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi blandit quam sapien, vel porttitor lectus scelerisque sed. | ||
The first step in the upload form is to provide a title and a brief but understandable description of the dataset – analogue to the abstract of a paper but specified to this data. This information is the basis of the metadata. | ||
:::content-center | ||
![_Upload form: Title and description of the dataset_](/static/img/fig_title_description.png) | ||
::: | ||
The project in which the data were collected must be specified in the following field. You can choose from the BonaRes collaborative projects and some long-term field experiments. For all other data that cannot be assigned to any of these projects, “Other” must be specified. In the future, it should also be possible to manually enter the individual projects. | ||
:::content-center | ||
![_Upload form: Project_](/static/img/fig_project.png) | ||
::: | ||
When selecting the research domain, the most appropriate domain should be selected. Only one domain can be selected. The research domains are based on the categories of the [German Research Foundation (DFG)](https://www.dfg.de/en/dfg_profile/statutory_bodies/review_boards/subject_areas/index.jsp) (FK No. 207 Agriculture, Forestry and Veterinary Medicine). | ||
:::content-center | ||
![_Upload form: Selection of the relevant Research Domain_](/static/img/fig_domain.png) | ||
::: | ||
Now the research question for which the available data were determined can be specified. Of course, there are not "the" good and "the" bad data - but data are of high quality in relation to the research question they are supposed to answer. This enables the potential reuser to better assess the suitability of the data for her research. | ||
|
||
Aenean hendrerit rutrum dolor id maximus. Sed diam dui, volutpat nec scelerisque in, faucibus eget orci. Nulla vitae massa augue. Etiam et nunc nunc. Nulla quis orci vehicula, gravida sem ut, tempor nisl. In aliquam malesuada erat nec consectetur. Vestibulum pharetra venenatis mi, vel tempus nunc elementum a. Morbi ac elementum enim. Praesent vestibulum ornare nisl nec porttitor. Proin ut maximus sem. | ||
!!!success Note: | ||
Fields with two lines in the lower right corner can be enlarged if necessary. | ||
!!! | ||
|
||
[!ref](/submit_research_data/name.md) | ||
:::content-center | ||
![_Upload form: Specifying the research question_](/static/img/fig_research_question.png) | ||
::: | ||
If possible, one or more soil functions that match the dataset should be selected that address the underlying study. | ||
:::content-center | ||
![_Upload form: Selection the soil functions_](/static/img/fig_soil_functions.png) | ||
::: | ||
In the next step, the dataset, metadata, any additional material, such as pictures, plans or drawings, can be uploaded. Please upload just the actual dataset (only **one** table, **one** dataset, one worksheet). **Multiple worksheets should be split** and uploaded separately. | ||
To illustrate the research data, you can use a **data model** that shows the data, e.g. a diagram. | ||
:::content-center | ||
![_Upload dataset, metadata and supplemental material_](/static/img/fig_select_dataset.png) | ||
::: | ||
Afterwards, additional information can be given specifically for the BonaRes Centre. In addition, the BonaRes data guideline must be approved and assured that the authority exists to hand over the data to the repository and that it has not yet been published elsewhere. This means that the data owner does not necessarily have to submit the data himself and can also designate someone else as the data provider. | ||
|
||
If all fields have been filled in, the dataset can be transferred to the BonaRes Centre by clicking on **“Submit”**. The BonaRes Centre sends an automatic **confirmation email** afterwards with a ticket number. | ||
|
||
:::content-center | ||
![Additional Information](/static/img/fig_addtional_information.png) | ||
::: | ||
If the upload was successful, a summary of the submitted information appears. The processing status can also be viewed on this page. As long as the submitted data are formally checked | ||
|
||
:::content-center | ||
![_List of all records_](/static/img/fig_list_of_all_records.png) | ||
::: | ||
**(Status: Submitted)**, it is possible to make changes. As soon as the data have been added to the database and the metadata have been sent | ||
**(Status: in progress)**, contact must be made with the BonaRes Centre for subsequent changes. | ||
|
||
The history gives an overview of the completed processing steps of the respective dataset. | ||
|
||
An overview of all datasets submitted so far appears in the user account under [“Submission”](https://tools.bonares.de/submission/) under **“My dataset submissions”**. If you have any questions or problems, the ticket number should always be given. | ||
|
||
:::content-center | ||
![_Submission details_](/static/img/fig_submission_details.png) | ||
::: | ||
After a dataset has been transferred to the BonaRes Centre, it is checked formally and in terms of content (e. g. thematically fit into the BonaRes Repository) by data stewards. Formal checks are, e. g. empty columns, outliers or not plausible values. Conspicuous data are communicated back to the data provider. In the future, the test should carried out by an automated tool. After a dataset passed this formal check, further metadata will be prepared (see chapter Metadata), which can be downloaded as PDF. | ||
|
||
:::content | ||
### Prepared metadata with **information from the upload tool and the data table:** | ||
* Description: **Title, summary, research domain, research question, date of submission,** editor, reference system, geographical location (very rough) | ||
* Categorization: **Project category**, standard keywords for BonaRes with associated thesaurus, topic category, **soil functions** | ||
* Access: Restrictions on use, license for use | ||
* Distribution: Output format, data provider | ||
* Quality: Lineage statement | ||
* Metadata: Metadata standard and version | ||
* Data model: **First column descriptions of the data table** | ||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,59 @@ | ||
--- | ||
label: Download and Fill | ||
icon: code-square | ||
label: Download and fill the metadata Template | ||
icon: desktop-download | ||
expanded: false | ||
order: 40 | ||
--- | ||
# Download and Fill the metadata Template | ||
|
||
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent tristique nibh in leo varius tincidunt nec id magna. Donec sit amet gravida purus. Aliquam molestie hendrerit aliquam. Nullam efficitur tortor nunc, ac porttitor nisi semper sit amet. In hac habitasse platea dictumst. Vivamus aliquet diam sem, facilisis fringilla ipsum commodo in. Mauris venenatis turpis sed metus tempus eleifend. Mauris pulvinar, mauris et faucibus cursus, orci eros ullamcorper metus, vel feugiat sapien metus porta quam. Fusce in sollicitudin quam. | ||
!!!success Note | ||
If data are already published in another open repository, the BonaRes Centre offers to take over the metadata of the dataset and to disseminate it via its own infrastructure. There is a link to the location of the dataset. This means that the research data can also be found via the BonaRes Repository without the need for republication. **Contact the support of the BonaRes Repository here:**[[email protected]]([email protected]). | ||
!!! | ||
|
||
Quisque vehicula, nisl ac pellentesque porta, lorem ligula vestibulum nisl, sit amet vulputate leo ligula eu nunc. Fusce eu pulvinar ex, a posuere arcu. Nullam rutrum volutpat gravida. Mauris eu ultrices tellus. Praesent neque erat, pharetra at lorem vitae, vestibulum eleifend metus. Nullam interdum, massa id semper vulputate, risus felis aliquet arcu, ut finibus turpis lorem at est. Nunc sit amet ultrices lectus, sit amet malesuada ex. Vestibulum viverra leo ut consequat fermentum. | ||
### Download | ||
|
||
Ut dictum ligula a vehicula ornare. Suspendisse euismod semper tempor. Aliquam pellentesque ex in est aliquam, non iaculis velit cursus. Suspendisse dapibus diam tincidunt luctus porta. Curabitur volutpat sapien et ante rhoncus faucibus. Phasellus tempor iaculis porttitor. Duis est mauris, tristique at interdum eget, finibus eget ligula. Vestibulum suscipit, leo sit amet accumsan venenatis, velit justo tincidunt lorem, interdum aliquam diam mi nec sem. | ||
:::content-center | ||
![](/static/img/fig_download_metadata.png) | ||
_Download the metadata template [here](https://tools.bonares.de/submission/)_ | ||
::: | ||
|
||
Donec vel fringilla nisi. Integer aliquet ligula vitae neque ultricies, ultricies tristique lorem fringilla. Phasellus iaculis eu risus auctor tempor. Praesent neque nunc, faucibus ut tempor non, lobortis ac arcu. Integer varius hendrerit massa. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi blandit quam sapien, vel porttitor lectus scelerisque sed. | ||
### How to Fill out the metadata template | ||
|
||
Aenean hendrerit rutrum dolor id maximus. Sed diam dui, volutpat nec scelerisque in, faucibus eget orci. Nulla vitae massa augue. Etiam et nunc nunc. Nulla quis orci vehicula, gravida sem ut, tempor nisl. In aliquam malesuada erat nec consectetur. Vestibulum pharetra venenatis mi, vel tempus nunc elementum a. Morbi ac elementum enim. Praesent vestibulum ornare nisl nec porttitor. Proin ut maximus sem. | ||
- Title English: Max of xxxxx characters | ||
- Title German: Max of xxxx characters | ||
- Summary English: Maximum of xxxxxx characters | ||
- Summary German: Maximum of xxxxx characters | ||
- Embargo: refers to a period (max 24 months) during which the data cannot be downloaded. | ||
- Author(s): Include an ORCID | ||
- Contributor: click on “choose an element” and select an option. | ||
|
||
[!ref](/submit_research_data/name.md) | ||
:::content-center | ||
![](/static/img/fig_template_page1.png) | ||
~_Metadata template: page 1_~ | ||
::: | ||
|
||
- Funding information: provide cross ref links if available. | ||
- Geographic area: provide bounding box information from where the data was collected. | ||
- Keywords: click on links and select keywords then write them under each link. | ||
- Place keywords: precise location | ||
- Related data already published: click on “choose an element” and select an option of the Relation Type. | ||
|
||
:::content-center | ||
![](/static/img/fig_template_page2.png) | ||
~_Metadata template: page2_~ | ||
::: | ||
|
||
- Dataset details: describe what each column stands for and precise the units and methods if applicable | ||
- For large datasets use extra excel file. See below. Image/grapic: provide an image that represents your dataset | ||
:::content-center | ||
![](/static/img/fig_template_page3.png) | ||
~_Metadata template: page3_~ | ||
::: | ||
|
||
|
||
|
||
:::content-center | ||
![](/static/img/fig_metadata_excel_example.png) | ||
~_Metadata template: example of excel file for simple data sets_~ | ||
::: | ||
|
Oops, something went wrong.