Attached detached lite #390

ptsefton · 2025-01-10T04:48:03Z

This is another approach to clarifying Attached vs Detached that does not introduce much new terminology or conformsTo etc. Again, this is a first pass to see if this approach makes sense, it will need checking.

I moved a bit of stuff around in structure.md but there are no major changes there except to cover how to deal with a Detached RO-Crate Package. Also, did not try to deal with how you'd link one to a website.

In the section on Data Entites i further tidied up the logic around @ids and contentUrls -- I think this has made it clearer, and I don't think it will be hard to implement.

Thanks for your feedback @simleo about the complexity in my last try.

…hed to Local

…d/detached definitons

docs/_specification/1.2-DRAFT/data-entities.md

simleo · 2025-01-10T10:31:02Z

docs/_specification/1.2-DRAFT/data-entities.md

    *   an absolute URI
    *   a local reference beginning with `#`

+For an _Attached RO-Crate Package_:
+* The `@id` MUST be a relative path that resolves to a directory that is present in the _RO-Crate Root_. 


A Dataset in an attached crate can also be web-based, with an absolute URI as its @id: this is already allowed in RO-Crate 1.1, I don't think we should change that. The requirement that the directory be present is also absent in 1.1: adding it would basically force validators to perform a check on the file system for every Dataset in the crate, which could be quite expensive.

OK - I don't think I looked at this this on this edit -- what should it say?

I think the whole paragraph For an _Attached RO-Crate Package_ [...] in the _RO-Crate Root_. should be removed.

Change to:

If the @id is a relative path, then it MUST that resolve to a directory which must be present in the RO-Crate Root along with its parent directories.

The folder may be empty, e.g. because of contentUrl trick above.

The requirement that the directory be present is also absent in 1.1

@simleo I checked this on a call with @stain, and while the requirement isn't stated in this part of the spec, it is covered elsewhere.

In 1.1, data-entities.md has:

A Dataset (directory) Data Entity MUST have the following properties:

@type MUST be Dataset or an array where Dataset is one of the values.

@id MUST be either a URI Path relative to the RO Crate root, or an absolute URI. The id SHOULD end with /

But structure.md also has

Data Entities in the RO-Crate MUST either be payload files/directories present within the RO-Crate root directory or its subdirectories, or be Web-based Data Entities."

So in 1.2 this specific requirement is just clearer since the info is in one place.

simleo · 2025-01-10T11:33:04Z

docs/_specification/1.2-DRAFT/structure.md

+  - References to files and directories in the RO-Crate Metadata Document are present in the RO-Crate or available online as [Web-based Data Entities](data-entities.html#web-based-data-entities).
+2.  A _Detached RO-Crate Package_:
+    - Is defined by a stand alone RO-Crate metadata document which may be stored in a file or distributed via an API.
+    - If stored in a file, known as a _Detached RO-Crate Metadata File_, the filename SHOULD be `${slug}-ro-crate-metadata.json` where the variable `$slug` is a human readable version of the dataset's ID or name, to signal that the document should be interpreted as part of an _Attached RO-Crate Data Package_.


This looks like something that could break the algorithm for finding the metadata file and root data entity. I also have trouble understanding the use case, which doesn't seem to be mentioned elsewhere: can a detached crate be part of an attached crate? What would that imply?

It should not affect finding the RO Crate Metadata description as that is ALWAYS ro-crate-metadata.json in all cases -- it is essentially a magic string (this is already noted that if you get a crate over an API).

The intention of the proposed changes here is that when you are dealing with a Detached crate an algorithm should never have to find it, it will be passed in directly either as a file, a string or a URI to some endpoint. The point of recommending this slug is twofold - firstly to distinguish files people might have in their downloads -- ATM you end up with a lot of ro-crate-metadata.json files and secondly, to STOP detached crates from being accidentally treated as Attached RO-Crate Packages.

If this makes sense then I will look at making it clearer in the spec.

OK, I see. Then the paragraph should end with "... to signal that the document should be interpreted as part of a Detached RO-Crate Data Package"

I reworded it

docs/_specification/1.2-DRAFT/structure.md

simleo · 2025-01-10T13:25:40Z

docs/_specification/1.2-DRAFT/structure.md


-At the basic level, an Attached RO-Crate is a collection of files and resources represented as a Schema.org [Dataset], that together form a meaningful unit for the purposes of communication, citation, distribution, preservation, etc.  The _RO-Crate Metadata Document_ describes the RO-Crate, and MUST be stored in the _RO-Crate Root_. 
+In a _Detached RO-Crate Package_ the [root data entity](root-data-entity) SHOULD have an `@id` which is a URL that resolves to the _RO-Crate Metadata Document_.


Why would the @id of the root data entity resolve to the metadata document? They are two separate entities with different ids.

If you have a detached crate then in many cases that will have some kind of online home at an API or a website - this is saying the the @id should point at that so it should say something like.

In a Detached RO-Crate Package the root data entity SHOULD have an @id which is a URL that resolves to an online source for the RO-Crate Metadata Document which may be a on the web or available over an API.

Does that make sense to you @simleo ?

I guess I'm heavily influenced by the pre-1.2 mindset. So in a detached crate the metadata file descriptor will always have the "magic" string ro-crate-metadata.json as @id, while the root data entity's @id will (possibly) point to a resource where the RO-Crate metadata can actually be retrieved, right? I think changing "SHOULD have an @id which is a URL" to "SHOULD have an @id which is an absolute URL" would help make things clearer.

Co-authored-by: Simone Leo <[email protected]>

simleo · 2025-01-13T13:30:21Z

docs/_specification/1.2-DRAFT/structure.md

+  - References to files and directories in the RO-Crate Metadata Document are present in the RO-Crate or available online as [Web-based Data Entities](data-entities.html#web-based-data-entities).
+2.  A _Detached RO-Crate Package_:
+    - Is defined by a stand alone RO-Crate metadata document which may be stored in a file or distributed via an API.
+    - If stored in a file, known as a _Detached RO-Crate Metadata File_, the filename SHOULD be `${slug}-ro-crate-metadata.json` where the variable `$slug` is a human readable version of the dataset's ID or name, to signal that the document should be interpreted as part of an _Attached RO-Crate Data Package_.


Suggested change

- If stored in a file, known as a _Detached RO-Crate Metadata File_, the filename SHOULD be `${slug}-ro-crate-metadata.json` where the variable `$slug` is a human readable version of the dataset's ID or name, to signal that the document should be interpreted as part of an _Attached RO-Crate Data Package_.

- If stored in a file, known as a _Detached RO-Crate Metadata File_, the filename SHOULD be `${slug}-ro-crate-metadata.json` where the variable `$slug` is a human readable version of the dataset's ID or name, to signal that the document should be interpreted as part of a _Detached RO-Crate Data Package_.

stain

Went through with @elichad and we have indicated some changes. I didn't complete the review.

stain · 2025-01-13T14:39:26Z

docs/_specification/1.2-DRAFT/data-entities.md

+    b. An Absolute URI indicating that the entity is a [Web-based Data Entity](#web-based-data-entity).
+
+2. For a _Detached RO-Crate Package_ all [File] Data Entities are [Web-based Data Enties](#web-based-data-entity)
+   * If a `contentUrl`is present: `@id` MUST be a A valid relative URI reference and `contentURL` must be an absolute URI. The presence of the `contentUrl` property is an indication that the File content may be sourced from that URL and if the _Detached RO-Crate Package_ were to be converted to an _Attached RO-Crate Package_ the `@id` indicates the `filePath` to use for saving a local copy of the [File].


I can't allow relative paths of data entities in a detached RO-Crate.

For instance if http://example.com/crate?id=15 and http://example.com/crate?id=83 both declare a "@id": "file.txt" then these relative path both resolve to http://example.com/file.txt when interpreted as JSON-LD. And it makes it crucial to be able to detect if something is a detached crate to avoid fetching these deliberately broken URIs.

#337 suggested a new property localPath with a string that references desired path if so chosen to download. But in that case @id would need rewriting after.

stain · 2025-01-13T14:40:37Z

docs/_specification/1.2-DRAFT/data-entities.md

+
+Further constraints on the `@id` are dependent on whether the [File] entity is being considered as part of an _Attached RO-Crate Package_ or _Detached RO-Crate Package_.
+
+If an `@id` is a relative URI then it is treated as a `filePath`, which is calculated by appending the `@id` to the `RO-Crate Root`.


Avoid use of filePath code markdown syntax as it looks like a JSON key, I think file path may work.

stain · 2025-01-13T14:44:15Z

docs/_specification/1.2-DRAFT/data-entities.md

    *   an absolute URI
    *   a local reference beginning with `#`

+For an _Attached RO-Crate Package_:
+* The `@id` MUST be a relative path that resolves to a directory that is present in the _RO-Crate Root_. 


Change to:

If the @id is a relative path, then it MUST that resolve to a directory which must be present in the RO-Crate Root along with its parent directories.

stain · 2025-01-13T14:44:47Z

docs/_specification/1.2-DRAFT/data-entities.md

    *   an absolute URI
    *   a local reference beginning with `#`

+For an _Attached RO-Crate Package_:
+* The `@id` MUST be a relative path that resolves to a directory that is present in the _RO-Crate Root_. 


The folder may be empty, e.g. because of contentUrl trick above.

stain · 2025-01-13T14:46:30Z

docs/_specification/1.2-DRAFT/data-entities.md

+* The `@id` MUST be a relative path that resolves to a directory that is present in the _RO-Crate Root_. 
+
+For a _Detached RO-Crate Package_:
+* If the `@id` is a _URI Path it MAY be used to create a directory and MAY resolve to a service which returns a list of files


Similar issue as above, e.g. @id: "folder/subfolder/"from detached cratehttp://example.com/api/get_crate?1239` would wrongly resolve to http://example.com/api/folder/subfolder/

stain · 2025-01-13T14:54:53Z

docs/_specification/1.2-DRAFT/root-data-entity.md

@@ -67,20 +67,19 @@ property referencing the _Root Data Entity_'s `@id`.
 }
 ```

-{% include callout.html type="note" content="Even in [Detached RO-Crates](structure#detached-ro-crate) which do not have an _RO-Crate Metadata File_ present, the identifier `ro-crate-metadata.json` MUST be used." %}
+{: .note}


Change back to {% include callout... style

stain · 2025-01-13T14:56:50Z

docs/_specification/1.2-DRAFT/root-data-entity.md


-{% include callout.html type="tip" content="The `conformsTo` property MAY be an array, to additionally indicate 
-specializing [RO-Crate profiles](profiles)." %}
+{% include callout.html type="tip" content="In RO-Crate 1,2 The `conformsTo` property MAY not have more than one value, to additionally indicate 


I may or may not agree with this (breaks Workflow RO-Crate profile 1.0) and I see this is because the profiles should now be on the root Dataset, but I don't think this change should be snuck into this PR.

Should also be documented better as a PR or issue as technically it's a breaking change. I think we can still do it for 1.2 because it's the very same spot you see if it's 1.1 or 1.2

stain · 2025-01-13T15:00:59Z

docs/_specification/1.2-DRAFT/root-data-entity.md

@@ -128,14 +127,15 @@ be minimally valid.

 ## Direct properties of the Root Data Entity

-The _Root Data Entity_ MUST have the following properties:
+The _Root Data Entity_ of a _Valid RO-Crate Dataset_  MUST have the following properties:


terminology Valid RO-Crate Dataset is new, what does this mean?

stain · 2025-01-13T15:02:31Z

docs/_specification/1.2-DRAFT/root-data-entity.md


 *  `@type`: MUST be [Dataset] or an array that contains `Dataset`
 *  `@id`:  SHOULD be the string `./` or an absolute URI (see [below](#root-data-entity-identifier))
 *  `name`: SHOULD identify the dataset to humans well enough to disambiguate it from other RO-Crates
 *  `description`: SHOULD further elaborate on the name to provide a summary of the context in which the dataset is important.
 *  `datePublished`: MUST be a single string value in [ISO 8601 date format][DateTime], SHOULD be specified to at least the precision of a day, and MAY be a timestamp down to the millisecond. 
 *  `license`: SHOULD link to a _Contextual Entity_ or _Data Entity_ in the _RO-Crate Metadata Document_ with a name and description (see section on [licensing](contextual-entities#licensing-access-control-and-copyright)). MAY, if necessary, be a textual description of how the RO-Crate may be used. 
+* `conformsTo` with a value of {"@id": "https://w3id.org/ro/crate/1.2-DRAFT#ro-crate-dataset"} (may be an array with multiple values)


This must be a remnant from the 2.0 modularisation idea in #388 and should be removed from this PR

stain · 2025-01-13T15:03:18Z

docs/_specification/1.2-DRAFT/root-data-entity.md

-RO-Crates that have been assigned a _persistent identifier_ (e.g. a DOI) SHOULD indicate this using [identifier] on the Root Data Entity using the approach set out in the [Science On Schema.org guides], that is, through a `PropertyValue`. 
-
-{% include callout.html type="note" content="RO-Crate 1.1 and earlier recommended `identifier` to be plain string URIs. Clients SHOULD be permissive of an RO-Crate `identifier` being a string (which MAY be a URI), or a `@id` reference, which SHOULD be represented as an `PropertyValue` entity which MUST have a human readable `value`, and SHOULD have a `url` if the identifier is Web-resolvable. A citable representation of this persistent identifier MAY be given as a `description` of the `PropertyValue`, but as there are more than 10,000 known [citation styles], no attempt should be made to parse this string." %}
+{: note}


Use callout block instead

ptsefton added 20 commits January 1, 2025 09:51

Initial pass at clarifying Attached/Detached in which I changed Attac…

ad56acc

…hed to Local

Further tidying up after merge

9ecc6ae

Fix conflicts

2d04c59

Fixed wording

c3d4da1

Found some formatting issues and typos

d3e950d

Further tweaks to pseudo code

9aa0928

Removed conflict

90bcaeb

More tidying up of algorithm for File @id

dd86e94

ADded new Detached RO-Crate Metadata File

bb82e7e

Tightened up rules about detached File Data Entities

1ef66de

Merge branch 'ResearchObject:main' into main

963eba9

Reverted to attached/detatched a few tweaks

2119467

Added new general note

02a2426

Added new general note

d556f84

Futher edits to note on 1.2

2c08726

Update root-data-entity.md

bcc30d7

Removing some of the extra stuff I put in and simplifying the attache…

75da32c

…d/detached definitons

Cleaning up

700c1eb

Typos

b241c33

Draft simpler treatment of Attached / Detached

c56c8d8

ptsefton requested review from elichad, stain and simleo and removed request for elichad January 10, 2025 04:49

simleo reviewed Jan 10, 2025

View reviewed changes

ptsefton and others added 2 commits January 11, 2025 09:23

Update docs/_specification/1.2-DRAFT/data-entities.md

5f6f202

Co-authored-by: Simone Leo <[email protected]>

Update docs/_specification/1.2-DRAFT/data-entities.md

da2db55

Co-authored-by: Simone Leo <[email protected]>

ptsefton and others added 5 commits January 11, 2025 09:24

Update docs/_specification/1.2-DRAFT/data-entities.md

5090e85

Co-authored-by: Simone Leo <[email protected]>

Update docs/_specification/1.2-DRAFT/data-entities.md

78731d9

Co-authored-by: Simone Leo <[email protected]>

Update docs/_specification/1.2-DRAFT/structure.md

d4f5878

Co-authored-by: Simone Leo <[email protected]>

Update docs/_specification/1.2-DRAFT/structure.md

9958603

Co-authored-by: Simone Leo <[email protected]>

Update docs/_specification/1.2-DRAFT/structure.md

0812d95

Co-authored-by: Simone Leo <[email protected]>

simleo reviewed Jan 13, 2025

View reviewed changes

stain requested changes Jan 13, 2025

View reviewed changes

Clarfified use of Slug in detatched file name

2d02112

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Attached detached lite #390

Attached detached lite #390

ptsefton commented Jan 10, 2025 •

edited

Loading

simleo Jan 10, 2025

ptsefton Jan 10, 2025

simleo Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

elichad Jan 13, 2025

simleo Jan 10, 2025

ptsefton Jan 10, 2025

simleo Jan 13, 2025 •

edited

Loading

ptsefton Jan 13, 2025

simleo Jan 10, 2025

ptsefton Jan 10, 2025

simleo Jan 13, 2025

simleo Jan 13, 2025

stain left a comment

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025

stain Jan 13, 2025


		At the basic level, an Attached RO-Crate is a collection of files and resources represented as a Schema.org [Dataset], that together form a meaningful unit for the purposes of communication, citation, distribution, preservation, etc. The _RO-Crate Metadata Document_ describes the RO-Crate, and MUST be stored in the _RO-Crate Root_.
		In a _Detached RO-Crate Package_ the [root data entity](root-data-entity) SHOULD have an `@id` which is a URL that resolves to the _RO-Crate Metadata Document_.

	- If stored in a file, known as a _Detached RO-Crate Metadata File_, the filename SHOULD be `${slug}-ro-crate-metadata.json` where the variable `$slug` is a human readable version of the dataset's ID or name, to signal that the document should be interpreted as part of an _Attached RO-Crate Data Package_.
	- If stored in a file, known as a _Detached RO-Crate Metadata File_, the filename SHOULD be `${slug}-ro-crate-metadata.json` where the variable `$slug` is a human readable version of the dataset's ID or name, to signal that the document should be interpreted as part of a _Detached RO-Crate Data Package_.


		Further constraints on the `@id` are dependent on whether the [File] entity is being considered as part of an _Attached RO-Crate Package_ or _Detached RO-Crate Package_.

		If an `@id` is a relative URI then it is treated as a `filePath`, which is calculated by appending the `@id` to the `RO-Crate Root`.

Attached detached lite #390

Are you sure you want to change the base?

Attached detached lite #390

Conversation

ptsefton commented Jan 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simleo Jan 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stain left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ptsefton commented Jan 10, 2025 •

edited

Loading

simleo Jan 13, 2025 •

edited

Loading