Skip to content

2022 Proposal: a new metadata editor

François Prunayre edited this page Jun 22, 2022 · 7 revisions
Date 2022-03-24 Contacts [email protected], [email protected]
Status To be proposed Release TBD
Resources TBD Ticket # -
Source code -
Funding geo2France, DataGrandEst and other interested parties which are willing to join the effort

Overview

This is a proposal for starting the development of a new metadata editor in GeoNetwork. It contains the general vision that Camptocamp has for such a project as well as technical considerations.

Current situation

Authoring metadata in GeoNetwork has been a source of various complaints for a long time. This includes notably:

  • Too many fields, tabs, buttons etc. present in the form
  • Fields and inputs are unclear
  • Producing valid metadata requires going through trial-and-error iterations, involving unclear errors
  • The editing form is a direct representation of the underlying XML, one form cannot work across several schemas
  • Slow and unresponsive UI *...and more

Offering custom editor forms with a better UX is currently doable and is something that we often do for customers. It is time-consuming though, very error-prone and does not rely on any kind of stable API so things might break randomly on version bumps. It also does not let us do all the things we'd like and we are still subject to the general limitations of the editor workflow in GeoNetwork.

Our vision

We (Camptocamp) believe that a new metadata editor could be written as part of the geonetwork-ui project. This will allow us to solve most issues that users experience, while also opening up new possibilities for authoring metadata in other formats (inc. non XML ones).

The new metadata editor would be built on three principles:

  1. Accessible to everyone

    We want to offer an application that allows even the unexperienced users to describe their data without too much hassle, and without real training. The application will also accommodate for more advanced users by offering more customizations on the user's explicit request. By default, only a handful of relevant fields will be shown in each section to avoid burdening the user cognitive load too much. Producing valid metadata will be done by anticipation, meaning that invalid inputs will simply not be allowed.

  2. Forget about metadata schemas

    We want to build the new metadata editor in a way that enables it to work seamlessly on various metadata schemas; more on that in the technical section. Which metadata schema is used internally (if any) will absolutely not impact the user experience.

  3. Build your own editor

    We want to offer powerful tools and options for completely configuring the editor and the way it is laid out. Each organization will be able to choose what fields should be shown in what section, what are the default or allowed values, which parts should be hidden or conditionally shown, etc. This is similar in principle to what the config-editor.xml file currently allows in GeoNetwork, except that our approach will be used from the start and not as an optional way to "tweak" an existing form.

Technical Details

We currently cannot settle on a precise architecture for this new project. I will describe here an objective that is both achievable and involves significant additions to the GeoNetwork Microservices project.

This goal involves three components:

  • A frontend application using geonetwork-ui
  • A backend microservice for storing metadata
  • A backend microservice for transforming metadata from and to XML formats (iso19139, iso19115-3 etc.)

Frontend application

This will be a new application alongside the datahub and datafeeder. It will benefit from UX work from the start (mockups etc.), as well as existing work done in the datafeeder for user input.

Input fields for metadata will be organized into sections, with each section having at most 7 fields to limit cognitive load. An overview will be available to the user to give him/her a good understanding of what is expected of them.

The challenge for this application will be to design it so that every field in every section will be placed at runtime, i.e. not hardcoded in the HTML templates of the app.

Example mockup (french): image

Configuration

We want to make the editor form completely customizable from the ground up. Not in the sense that a default structure is provided and administrators can tweak around it, but in the sense that a new editor can be built from scratch simply using a configuration file.

Here is an example of what a configuration file could look like:

sections:

  - generalInfo:
    labelKey: editor.section.generalInfo
    fields:

        # optional, must be unique
      - id: record_title

        # name of the field in the data model
        # Note: this is a optional, a field could be included but directly
        # linked to a part of the data model (e.g. a toggle)
        model: title

        # translation keys for different parts of the field
        labelKey: editor.fields.title.label
        tipKey: editor.fields.title.tip
        helpKey: editor.fields.title.help

        # Types can include:
        # - text
        # - number
        # - rich: text with formatting, images etc.
        # - date
        # - list
        # - spatial_extent
        # - temporal_extent
        # - url
        # - file: file to upload or remote url
        # - toggle
        type: text

        # Special types an include:
        # - object: creates a embedded form just for this object (e.g. contacts)
        # - array: adds controls for adding and removing elements in the array

        # This field can include an expression, e.g.
        # required: ${{ isEmpty(record.fields['otherTitle']) }}
        required: true

        readOnly: false

        # Each field type comes with its sets of special rules, for instance:
        # - text.maxLength: 1234
        # - text.minLength: 12
        # - number.min: 100
        # - number.max: 500
        # - number.step: 50
        # - date.minDateTime: 167000234234
        # - date.maxDateTime: 167000854321
        # - list.fromApi: /api/keywords?lang=fre
        # - list.fromJson: https://my.site/files/list.json
        # - list.fromValues: [“Keyword 1”, “Keyword 2”, “Keyword 3”]
        # - rich.disallowFormatting: true
        # - file.allowFileUpload: true
        # - file.convertToInlineBase64: true
        # Each rule will be documented
        rules:
          text.maxRecommendedLength: 200

  - contactInfo:
    labelKey: editor.section.contactInfo
    fields:

        # An example of an array of objets
      - model: contactsForData
        type: array
        labelKey: editor.fields.contacts.label
        items:
          type: object
          labelKey: editor.fields.contact.label
          fields:
            - model: firstName
              labelKey: editor.fields.contact.firstName.label
              type: text
            - model: lastName
              labelKey: editor.fields.contact.firstName.label
              type: text
            - model: email
              labelKey: editor.fields.contact.firstName.label
              type: text
              required: true
              rules:
                - text.validateEmail: true
        required: true
        rules:
          - array.minCount: 1

  - distribution:
    labelKey: editor.section.distribution

  - accessAndUseConditions:
    labelKey: editor.section.accessAndUseConditions

  - datasetInfo:
    labelKey: editor.section.datasetInfo

To facilitate the process of creating custom editor forms, a sandbox-type app could be hosted e.g. on GH pages to let people experiment with a config file and see the result right away. This would be similar to the Swagger Editor.

Metadata editor microservice

In order to uncouple the process of authoring metadata with the XML schema used to store metadata in GeoNetwork, a new "metadata editor" backend microservice will be introduced. Its responsibility will be to offer an API for interacting with metadata records in a native format instead of XML documents. The data model used for this will be built to accommodate the most common usages, for example: title, abstract, contacts, links to online resources, usage and download constraints, etc.

Its API will allow reading records by UUID, editing records, deleting and creating new records. Internally this microservice will interact with the GeoNetwork database; it will also rely on the converter microservice (see next paragraph) to transform metadata records from and to iso19139. This means that all the metadata records present in the GeoNetwork database can be edited in the new editor, and all metadata management functions in GeoNetwork can still be used on the records that were authored in the new editor. GeoNetwork is still the source of truth in that scenario.

Metadata converter microservice

This service will be responsible for converting metadata from and to established schemas of any kinds. Its responsibility will be limited to applying predefined processes to either native metadata objects coming from the editor microservice, or XML metadata coming from the GeoNetwork database to be then transformed to native objects.

A schema is nothing more than:

  • A transformation from a native object to an interoperable document (out)
  • A transformation from an interoperable document to a native object (in)
  • A set of test cases validation the expected results for both transformations

How transformations will work exactly has yet to be determined.

Example architecture

This showcases what a new architecture relying on the above microservices would look like.

image

Voting History

Vote Proposed the 6th of April 2022

  • Jo Cook +1
  • Florent Gravin +1
  • Jeroen Ticheler +1
  • Jose Garcia +1
  • Simon Pigot +1
  • François-Xavier Prunayre
  • Emanuele Tajariol
  • Paul van Genuchten

Participants

  • All
Clone this wiki locally