diff --git a/optimade.rst b/optimade.rst index bebf10cb8..9ba6214ba 100644 --- a/optimade.rst +++ b/optimade.rst @@ -114,8 +114,9 @@ However, automating the retrieval of data is difficult if each database has a di This document specifies a standard API for retrieving data from materials databases. This API specification has been developed over a series of workshops entitled "Open Databases Integration for Materials Design", held at the Lorentz Center in Leiden, Netherlands and the CECAM headquarters in Lausanne, Switzerland. -The API specification described in this document builds on top of the `JSON API v1.0 specification `__. -In particular, the JSON API specification is assumed to apply wherever it is stricter than what is formulated in this document. +The API specification described in this document builds on top of the `JSON:API v1.1 specification `__. +More specifically, it defines specific `implementation semantics `__ allowed by the JSON:API standard, but which go beyond the restrictions imposed on `JSON:API profiles `__ and `extensions `__. +The JSON:API specification is assumed to apply wherever it is stricter than what is formulated in this document. Exceptions to this rule are stated explicitly (e.g. non-compliant responses are tolerated if a non-standard response format is explicitly requested). Definition of Terms @@ -358,11 +359,11 @@ A few suggestions and mandatory requirements of the OPTIMADE specification are s - When serving an index meta-database in the form of static files, it is RECOMMENDED that the response excludes the subfields in the top-level :field:`meta` field that would need to be dynamically generated (as described in the section `JSON Response Schema: Common Fields`_.) The motivation is that static files cannot keep dynamic fields such as :field:`time_stamp` updated. -- The `JSON API specification `__ requirements on content negotiation using the HTTP headers :http-header:`Content-type` and :http-header:`Accept` are NOT mandatory for index meta-databases. +- The `JSON:API specification `__ requirements on content negotiation using the HTTP headers :http-header:`Content-Type` and :http-header:`Accept` are NOT mandatory for index meta-databases. Hence, API Implementations MAY ignore the content of these headers and respond to all requests. The motivation is that static file hosting is typically not flexible enough to support these requirements on HTTP headers. -- API implementations SHOULD serve JSON content with either the JSON API mandated HTTP header :http-header:`Content-Type: application/vnd.api+json` or :http-header:`Content-Type: application/json`. However, if the hosting platform does not allow this, JSON content MAY be served with :http-header:`Content-Type: text/plain`. +- API implementations SHOULD serve JSON content with either the JSON:API mandated HTTP header :http-header:`Content-Type: application/vnd.api+json` or :http-header:`Content-Type: application/json`. However, if the hosting platform does not allow this, JSON content MAY be served with :http-header:`Content-Type: text/plain`. .. @@ -399,7 +400,7 @@ The API implementation MAY describe many-to-many relationships between entries a These relationships can be to the same, or to different, entry types. Response formats have to encode these relationships in ways appropriate for each format. -In the default response format, relationships are encoded as `JSON API Relationships `__, see section `Entry Listing JSON Response Schema`_. +In the default response format, relationships are encoded as `JSON:API Relationships `__, see section `Entry Listing JSON Response Schema`_. **For implementers**: For database-specific response formats without a dedicated mechanism to indicate relationships, it is suggested that they are encoded alongside the entry properties. For each entry type, the relationships with entries of that type can then be encoded in a field with the name of the entry type, which are to contain a list of the IDs of the referenced entries alongside the respective human-readable description of the relationships. @@ -572,7 +573,7 @@ Responses Response Format --------------- -This section defines a JSON response format that complies with the `JSON API v1.0 `__ specification. +This section defines a JSON response format that complies with the `JSON:API v1.1 `__ specification. All endpoints of an API implementation MUST be able to provide responses in the JSON format specified below and MUST respond in this format by default. Each endpoint MAY support additional formats, and SHOULD declare these formats under the endpoint :endpoint:`/info/` (see section `Entry Listing Info Endpoints`_). @@ -594,7 +595,7 @@ In the JSON response format, property types translate as follows: Every response SHOULD contain the following fields, and MUST contain at least :field:`meta`: -- **meta**: a `JSON API meta member `__ that contains JSON API meta objects of non-standard meta-information. +- **meta**: a `JSON:API meta member `__ that contains JSON:API meta objects of non-standard meta-information. It MUST be a dictionary with these fields: - **api\_version**: a string containing the full version of the API implementation. @@ -613,11 +614,6 @@ Every response SHOULD contain the following fields, and MUST contain at least :f :field:`meta` SHOULD also include these fields: - - **schema**: a `JSON API link `__ that points to a schema for the response. - If it is a string, or a dictionary containing no :field:`meta` field, the provided URL MUST point at an `OpenAPI `__ schema. - It is possible that future versions of this specification allows for alternative schema types. - Hence, if a :field:`meta` field is provided for the link and contains a field :field:`schema_type` that is not equal to the string :field-val:`OpenAPI` the client MUST not handle failures to parse the schema or to validate the response against the schema as errors. - - **time\_stamp**: a timestamp containing the date and time at which the query was executed. - **data\_returned**: an integer containing the total number of data resource objects returned for the current :query-param:`filter` query, independent of pagination. - **provider**: information on the database provider of the implementation. @@ -629,7 +625,7 @@ Every response SHOULD contain the following fields, and MUST contain at least :f :field:`provider` MAY include these fields: - - **homepage**: a `JSON API link `__, pointing to the homepage of the database provider, either directly as a string, or as an object which can contain the following fields: + - **homepage**: a `JSON API link `__, pointing to the homepage of the database provider, either directly as a string, or as an object which can contain the following fields: - **href**: a string containing the homepage URL. - **meta**: a meta object containing non-standard meta-information about the database provider's homepage. @@ -660,16 +656,17 @@ Every response SHOULD contain the following fields, and MUST contain at least :f - **name**: name of the implementation. - **version**: version string of the current implementation. - - **homepage**: a `JSON API link `__, pointing to the homepage of the implementation. - - **source\_url**: a `JSON API link `__ pointing to the implementation source, either downloadable archive or version control system. + - **homepage**: a `JSON API link `__, pointing to the homepage of the implementation. + - **source\_url**: a `JSON API link `__ pointing to the implementation source, either downloadable archive or version control system. + - **maintainer**: a dictionary providing details about the maintainer of the implementation, MUST contain the single field: - **email** with the maintainer's email address. - - **issue\_tracker**: a `JSON API link `__ pointing to the implementation's issue tracker. + - **issue\_tracker**: a `JSON API link `__ pointing to the implementation's issue tracker. - **warnings**: a list of warning resource objects representing non-critical errors or warnings. - A warning resource object is defined similarly to a `JSON API error object `__, but MUST also include the field :field:`type`, which MUST have the value :field-val:`"warning"`. + A warning resource object is defined similarly to a `JSON:API error object `__, but MUST also include the field :field:`type`, which MUST have the value :field-val:`"warning"`. The field :field:`detail` MUST be present and SHOULD contain a non-critical message, e.g., reporting unrecognized search attributes or deprecated features. The field :field:`status`, representing an HTTP response status code, MUST NOT be present for a warning resource object. This is an exclusive field for error resource objects. @@ -736,7 +733,13 @@ Every response SHOULD contain the following fields, and MUST contain at least :f // ... } -- **data**: The schema of this value varies by endpoint, it can be either a *single* `JSON API resource object `__ or a *list* of JSON API resource objects. + - **schema**: a `JSON:API links object `__ that points to a schema for the response. + If it is a string, or a dictionary containing no :field:`meta` field, the provided URL MUST point at an `OpenAPI `__ schema. + It is possible that future versions of this specification allow for alternative schema types. + Hence, if the :field:`meta` field of the JSON:API links object is provided and contains a field :field:`schema_type` that is not equal to the string :field-val:`OpenAPI` the client MUST NOT handle failures to parse the schema or to validate the response against the schema as errors. + **Note**: The :field:`schema` field was previously RECOMMENDED in all responses, but is now demoted to being OPTIONAL since there now is a standard way of specifying a response schema in JSON:API through the :field:`describedby` subfield of the top-level :field:`links` field. + +- **data**: The schema of this value varies by endpoint, it can be either a *single* `JSON:API resource object `__ or a *list* of JSON:API resource objects. Every resource object needs the :field:`type` and :field:`id` fields, and its attributes (described in section `API Endpoints`_) need to be in a dictionary corresponding to the :field:`attributes` field. Every resource object MAY also contain a :field:`meta` field which MAY contain the following keys: @@ -762,7 +765,7 @@ Every response SHOULD contain the following fields, and MUST contain at least :f The response MAY also return resources related to the primary data in the field: -- **links**: a `JSON API links object `__ is REQUIRED for implementing pagination. +- **links**: a `JSON API links object `__ is REQUIRED for implementing pagination. (see section `Entry Listing URL Query Parameters`_.) Each field of a links object, i.e., a "link", MUST be one of: @@ -792,6 +795,14 @@ The response MAY also return resources related to the primary data in the field: // ... } + The :field:`links` field SHOULD include the following links objects: + + - **describedby**: a links object giving the URL for a schema that describes the response. + The URL SHOULD resolve into a JSON formatted response returning a JSON object with top level :field:`$schema` and/or :field:`$id` fields that can be used by the client to identify the schema format. + + **Note**: This field is the standard facility in JSON:API to communicate a response schema. + It overlaps in function with the field :field:`schema` in the top level :field:`meta` field. + The following fields are REQUIRED for implementing pagination: - **next**: represents a link to fetch the next set of results. @@ -804,10 +815,14 @@ The response MAY also return resources related to the primary data in the field: - **last**: the last page of data. - **first**: the first page of data. -- **included**: a list of `JSON API resource objects `__ related to the primary data contained in :field:`data`. - Responses that contain related resources under :field:`included` are known as `compound documents `__ in the JSON API. + Finally, the :field:`links` field MAY also include the following links object: + + - **self**: a links object giving the URL from which the response was obtained. - The definition of this field is found in the `JSON API specification `__. +- **included**: a list of `JSON:API resource objects `__ related to the primary data contained in :field:`data`. + Responses that contain related resources under :field:`included` are known as `compound documents `__ in the JSON:API. + + The definition of this field is found in the `JSON:API specification `__. Specifically, if the query parameter :query-param:`include` is included in the request, :field:`included` MUST NOT include unrequested resource objects. For further information on the parameter :query-param:`include`, see section `Entry Listing URL Query Parameters`_. @@ -815,7 +830,7 @@ The response MAY also return resources related to the primary data in the field: If there were errors in producing the response all other fields MAY be present, but the top-level :field:`data` field MUST be skipped, and the following field MUST be present: -- **errors**: a list of `JSON API error objects `__, where the field :field:`detail` MUST be present. +- **errors**: a list of `JSON:API error objects `__, where the field :field:`detail` MUST be present. All other fields are OPTIONAL. An example of a full response: @@ -864,13 +879,28 @@ An example of a full response: ] } +- **@context**: A JSON-LD context that enables interpretation of data in the response as linked data. + If provided, it SHOULD be one of the following: + - An object conforming to a JSON-LD standard, which includes a :field:`@version` field specifying the version of the standard. + - A string containing a URL that resolves to such an object. + +- **jsonapi**: A `JSON:API object `__. + The :field:`version` subfield SHOULD be :field-val:`"1.1"`. + The :field:`meta` subfield SHOULD be included and contain the following subfields: + + - **api**: A string with the value "OPTIMADE". + - **api-version**: A string with the full version of the OPTIMADE standard that the processing and response adheres to. + This MAY be the version indicated at the top of this document, but MAY also be another version if the client, e.g., has used the query parameter :query-param:`api_hint` to request processing according to another version. + + If the server is able to handle serialization in such a way that it can dictate the order of the top level object members in the response, it is RECOMMENDED to put the :field:`jsonapi` as the first top level member to simplify identification of the response. + HTTP Response Status Codes -------------------------- All HTTP response status codes MUST conform to `RFC 7231: HTTP Semantics `__. The code registry is maintained by IANA and can be found `here `__. -See also the JSON API definitions of responses when `fetching `__ data, i.e., sending an HTTP GET request. +See also the JSON:API definitions of responses when `fetching `__ data, i.e., sending an HTTP GET request. **Important**: If a client receives an unexpected 404 error when making a query to a base URL, and is aware of the index meta-database that belongs to the database provider (as described in section `Index Meta-Database`_), the next course of action SHOULD be to fetch the resource objects under the :endpoint:`links` endpoint of the index meta-database and redirect the original query to the corresponding database ID that was originally queried, using the object's :field:`base_url` value. @@ -981,14 +1011,14 @@ Entry Listing URL Query Parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The client MAY provide a set of URL query parameters in order to alter the response and provide usage information. While these URL query parameters are OPTIONAL for clients, API implementations MUST accept and handle them. -To adhere to the requirement on implementation-specific URL query parameters of `JSON API v1.0 `__, query parameters that are not standardized by that specification have been given names that consist of at least two words separated by an underscore (a LOW LINE character '\_'). +To adhere to the requirement on implementation-specific URL query parameters of `JSON:API v1.1 `__, query parameters that are not standardized by that specification have been given names that consist of at least two words separated by an underscore (a LOW LINE character '\_'). -Standard OPTIONAL URL query parameters standardized by the JSON API specification: +Standard OPTIONAL URL query parameters standardized by the JSON:API specification: - **filter**: a filter string, in the format described below in section `API Filtering Format Specification`_. - **page\_limit**: sets a numerical limit on the number of entries returned. - See `JSON API 1.0 `__. + See `JSON:API 1.1 `__. The API implementation MUST return no more than the number specified. It MAY return fewer. The database MAY have a maximum limit and not accept larger numbers (in which case an error code -- 403 Forbidden -- MUST be returned). @@ -1011,22 +1041,22 @@ Standard OPTIONAL URL query parameters standardized by the JSON API specificatio - fetch page 2 of up to 50 structures per page: :query-url:`/structures?page_number=2&page_limit=50`. - fetch up to 100 structures above sort-field value 4000 (in this example, server chooses to fetch results sorted by increasing :field:`id`, so :field:`page_above` value refers to an :field:`id` value): :query-url:`/structures?page_above=4000&page_limit=100`. -- **sort**: If supporting sortable queries, an implementation MUST use the :query-param:`sort` query parameter with format as specified by `JSON API 1.0 `__. +- **sort**: If supporting sortable queries, an implementation MUST use the :query-param:`sort` query parameter with format as specified by `JSON:API 1.1 `__. An implementation MAY support multiple sort fields for a single query. - If it does, it again MUST conform to the JSON API 1.0 specification. + If it does, it again MUST conform to the JSON:API 1.1 specification. If an implementation supports sorting for an `entry listing endpoint `_, then the :endpoint:`/info/` endpoint MUST include, for each field name :field:`` in its :field:`data.properties.` response value that can be used for sorting, the key :field:`sortable` with value :field-val:`true`. If a field name under an entry listing endpoint supporting sorting cannot be used for sorting, the server MUST either leave out the :field:`sortable` key or set it equal to :field-val:`false` for the specific field name. - The set of field names, with :field:`sortable` equal to :field-val:`true` are allowed to be used in the "sort fields" list according to its definition in the JSON API 1.0 specification. + The set of field names, with :field:`sortable` equal to :field-val:`true` are allowed to be used in the "sort fields" list according to its definition in the JSON:API 1.1 specification. The field :field:`sortable` is in addition to each property description and other OPTIONAL fields. An example is shown in section `Entry Listing Info Endpoints`_. -- **include**: A server MAY implement the JSON API concept of returning `compound documents `__ by utilizing the :query-param:`include` query parameter as specified by `JSON API 1.0 `__. +- **include**: A server MAY implement the JSON:API concept of returning `compound documents `__ by utilizing the :query-param:`include` query parameter as specified by `JSON:API 1.0 `__. All related resource objects MUST be returned as part of an array value for the top-level :field:`included` field, see section `JSON Response Schema: Common Fields`_. - The value of :query-param:`include` MUST be a comma-separated list of "relationship paths", as defined in the `JSON API `__. + The value of :query-param:`include` MUST be a comma-separated list of "relationship paths", as defined in the `JSON:API `__. If relationship paths are not supported, or a server is unable to identify a relationship path a :http-error:`400 Bad Request` response MUST be made. The **default value** for :query-param:`include` is :query-val:`references`. @@ -1035,7 +1065,7 @@ Standard OPTIONAL URL query parameters standardized by the JSON API specificatio **Note**: A query with the parameter :query-param:`include` set to the empty string means no related resource objects are to be returned under the top-level field :field:`included`. -Standard OPTIONAL URL query parameters not in the JSON API specification: +Standard OPTIONAL URL query parameters not in the JSON:API specification: - **response\_format**: the output format requested (see section `Response Format`_). Defaults to the format string 'json', which specifies the standard output format described in this specification. @@ -1050,7 +1080,7 @@ Standard OPTIONAL URL query parameters not in the JSON API specification: Additional OPTIONAL URL query parameters not described above are not considered to be part of this standard, and are instead considered to be "custom URL query parameters". These custom URL query parameters MUST be of the format "". -These names adhere to the requirements on implementation-specific query parameters of `JSON API v1.0 `__ since the database-provider-specific prefixes contain at least two underscores (a LOW LINE character '\_'). +These names adhere to the requirements on implementation-specific query parameters of `JSON:API v1.1 `__ since the database-provider-specific prefixes contain at least two underscores (a LOW LINE character '\_'). Example uses of custom URL query parameters include providing an access token for the request, to tell the database to increase verbosity in error output, or providing a database-specific extended searching format. @@ -1068,7 +1098,7 @@ Entry Listing JSON Response Schema "Entry listing" endpoint response dictionaries MUST have a :field:`data` key. The value of this key MUST be a list containing dictionaries that represent individual entries. -In the default JSON response format every dictionary (`resource object `__) MUST have the following fields: +In the default JSON response format every dictionary (`resource object `__) MUST have the following fields: - **type**: field containing the Entry type as defined in section `Definition of Terms`_ - **id**: field containing the ID of entry as defined in section `Definition of Terms`_. This can be the local database ID. @@ -1078,16 +1108,16 @@ In the default JSON response format every dictionary (`resource object `__ can OPTIONALLY contain the field +- **links**: a `JSON:API links object `__ can OPTIONALLY contain the field - **self**: the entry's URL -- **meta**: a `JSON API meta object `__ that is used to communicate metadata. +- **meta**: a `JSON API meta object `__ that is used to communicate metadata. See `JSON Response Schema: Common Fields`_ for more information about this field. -- **relationships**: a dictionary containing references to other entries according to the description in section `Relationships`_ encoded as `JSON API Relationships `__. - The OPTIONAL human-readable description of the relationship MAY be provided in the :field:`description` field inside the :field:`meta` dictionary of the JSON API resource identifier object. - All relationships to entries of the same entry type MUST be grouped into the same JSON API relationship object and placed in the relationships dictionary with the entry type name as key (e.g., :entry:`structures`). +- **relationships**: a dictionary containing references to other entries according to the description in section `Relationships`_ encoded as `JSON:API Relationships `__. + The OPTIONAL human-readable description of the relationship MAY be provided in the :field:`description` field inside the :field:`meta` dictionary of the JSON:API resource identifier object. + All relationships to entries of the same entry type MUST be grouped into the same JSON:API relationship object and placed in the relationships dictionary with the entry type name as key (e.g., :entry:`structures`). Example: @@ -1213,7 +1243,8 @@ The single resource object's response dictionary MUST include the following fiel - **formats**: List of available output formats. - **entry\_types\_by\_format**: Available entry endpoints as a function of output formats. - **available\_endpoints**: List of available endpoints (i.e., the string to be appended to the versioned or unversioned base URL serving the API). - - **license**: A `JSON API link `__ giving a URL to a web page containing a human-readable text describing the license (or licensing options if there are multiple) covering all the data and metadata provided by this database. + - **license**: A `JSON API link `__ giving a URL to a web page containing a human-readable text describing the license (or licensing options if there are multiple) covering all the data and metadata provided by this database. + Clients are advised not to try automated parsing of this link or its content, but rather rely on the field :field:`available_licenses` instead. Example: :field-val:`https://example.com/licenses/example_license.html`. @@ -1229,15 +1260,16 @@ The single resource object's response dictionary MUST include the following fiel If the licensing information provided via the field :field:`license` omits licensing options specified in :field:`available_licenses`, or if it otherwise contradicts them, a client MUST still be allowed to interpret the inclusion of a license in :field:`available_licenses` as a full commitment from the database that the data and metadata is available, without exceptions, under the respective licenses. If the database cannot make that commitment, e.g., if only part of the data is available under a license, the corresponding license identifier MUST NOT appear in :field:`available_licenses` (but, rather, the field :field:`license` is to be used to clarify the licensing situation.) An empty list indicates that none of the SPDX licenses apply for the entirety of the database and that the licensing situation is clarified in human readable form in the field :field:`license`. + If this is an index meta-database base URL (see section `Index Meta-Database`_), then the response dictionary MUST also include the field: -- **relationships**: Dictionary that MAY contain a single `JSON API relationships object `__: +- **relationships**: Dictionary that MAY contain a single `JSON:API relationships object `__: - **default**: Reference to the links identifier object under the :endpoint:`links` endpoint that the provider has chosen as their "default" OPTIMADE API database. A client SHOULD present this database as the first choice when an end-user chooses this provider. This MUST include the field: - - **data**: `JSON API resource linkage `__. + - **data**: `JSON:API resource linkage `__. It MUST be either :field-val:`null` or contain a single links identifier object with the fields: - **type**: :field-val:`links` @@ -1497,12 +1529,13 @@ The resource objects' response dictionaries MUST include the following fields: - **name**: Human-readable name for the OPTIMADE API implementation, e.g., for use in clients to show the name to the end-user. - **description**: Human-readable description for the OPTIMADE API implementation, e.g., for use in clients to show a description to the end-user. - - **base\_url**: `JSON API link `__, pointing to the base URL for this implementation, either directly as a string, or as an object, which can contain the following fields: + - **base\_url**: `JSON API link `__, pointing to the base URL for this implementation, either directly as a string, or as an object, which can contain the following fields: + - **href**: a string containing the OPTIMADE base URL. - **meta**: a meta object containing non-standard meta-information about the implementation. - - **homepage**: a `JSON API link `__, pointing to a homepage URL for this implementation, either directly as a string, or as an object, which can contain the following fields: + - **homepage**: a `JSON API link `__, pointing to a homepage URL for this implementation, either directly as a string, or as an object, which can contain the following fields: - **href**: a string containing the implementation homepage URL. - **meta**: a meta object containing non-standard meta-information about the homepage. @@ -1667,7 +1700,7 @@ The API implementation is free to define roles of further URL path segments unde API Filtering Format Specification ================================== -An OPTIMADE filter expression is passed in the parameter :query-param:`filter` as a URL query parameter as `specified by JSON API `__. +An OPTIMADE filter expression is passed in the parameter :query-param:`filter` as a URL query parameter as `specified by JSON:API `__. The filter expression allows desired properties to be compared against search values; several such comparisons can be combined using the logical conjunctions AND, OR, NOT, and parentheses, with their usual semantics. All properties marked as REQUIRED in section `Entry list`_ MUST be queryable with all mandatory filter features. @@ -3272,7 +3305,7 @@ It relates an entry with any number of :entry:`references` entries. If the response format supports inclusion of entries of a different type in the response, then the response SHOULD include all references-type entries mentioned in the response. -For example, for the JSON response format, the top-level :field:`included` field SHOULD be used as per the `JSON API 1.0 specification `__: +For example, for the JSON response format, the top-level :field:`included` field SHOULD be used as per the `JSON:API 1.1 specification `__: .. code:: jsonc