Server error when running UDP

[ValentinaHutter]

I am trying to run a UDP, which I created with an EODC collection.
When I create and start a batch job, I get a 'ServerError: Collection not available'. Since this happens in the aggregator web-editor, but works in the EODC web-editor, it could be possible that only one backend is checked for UDPs? Is there something, we can do on the backend side, or would this be something to be updated on the aggregator?

[ValentinaHutter]

I now get this error message when I try to create a UDP with an EODC collection: Server error: OpenEoApiError('[401] TokenExchangeFailure: Authentication failed. Please login on our marketplace web undefined first, and then try authenticate again.')

-> do not think it is related to the eodc backend as the URL details say

Url: /jobs/vito-j-2403199b5d324c0a970d07a53f820ff3/results

[ValentinaHutter]

@soxofaan @jdries do you have any input on this, or could you forward this to someone else?

[jdries]

Hi @ValentinaHutter , it's indeed the case that we forward part of UDP handling mainly to vito backend, as there was no easy solution at aggregator level.
Hence I think it would require an aggregator PR to make this work.
It could help if you post the actual UDP and the requests that you are trying to do, to have a more accurate view on the use case.
An alternative option, which would require an extension of the openEO spec, is to allow referencing UDP's by URLs. This would potentially reduce some of the difficulties of doing UDP's in a federated context.

[soxofaan]

Indeed, the aggregator for openEO Platform, only interacts with the VITO backend for UDPs for now

more background and related tickets:

• UDP support (/process_graphs) #90
• https://github.com/openEOPlatform/architecture-docs/issues/368

[ValentinaHutter]

Thanks for the quick replies!

An alternative option, which would require an extension of the openEO spec, is to allow referencing UDP's by URLs. This would potentially reduce some of the difficulties of doing UDP's in a federated context.

Is this what you describe in here: Open-EO/openeo-api#515 ? So, as a first step, we would enable running UDPs which come from a public URL at EODC as well, right? :)

A very basic UDP would be for example for RGB image creation:

{
  "id": "RGB",
  "parameters": [
    {
      "description": "AOI",
      "name": "spatial_extent",
      "schema": {
        "description": "A bounding box with the required fields `west`, `south`, `east`, `north` and optionally `base`, `height`, `crs`. The `crs` is a EPSG code, a WKT2:2018 string or a PROJ definition (deprecated).",
        "properties": {
          "base": {
            "default": null,
            "description": "Base (optional, lower left corner, coordinate axis 3).",
            "type": [
              "number",
              "null"
            ]
          },
          "crs": {
            "anyOf": [
              {
                "description": "Specifies details about cartographic projections as [EPSG](http://www.epsg.org) code.",
                "examples": [
                  3857
                ],
                "minimum": 1000,
                "subtype": "epsg-code",
                "title": "EPSG Code",
                "type": "integer"
              },
              {
                "description": "Specifies details about cartographic projections as WKT2 string. Refers to the latest WKT2 version (currently [WKT2:2018](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html) / ISO 19162:2018) unless otherwise stated by the process.",
                "subtype": "wkt2-definition",
                "title": "WKT2 definition",
                "type": "string"
              },
              {
                "description": "**DEPRECATED.** Specifies details about cartographic projections as [PROJ](https://proj.org/usage/quickstart.html) definition.",
                "subtype": "proj-definition",
                "title": "PROJ definition",
                "type": "string"
              }
            ],
            "default": 4326,
            "description": "Coordinate reference system of the extent, specified as as [EPSG code](http://www.epsg-registry.org/), [WKT2 (ISO 19162) string](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html) or [PROJ definition (deprecated)](https://proj.org/usage/quickstart.html). Defaults to `4326` (EPSG code 4326) unless the client explicitly requests a different coordinate reference system."
          },
          "east": {
            "description": "East (upper right corner, coordinate axis 1).",
            "type": "number"
          },
          "height": {
            "default": null,
            "description": "Height (optional, upper right corner, coordinate axis 3).",
            "type": [
              "number",
              "null"
            ]
          },
          "north": {
            "description": "North (upper right corner, coordinate axis 2).",
            "type": "number"
          },
          "south": {
            "description": "South (lower left corner, coordinate axis 2).",
            "type": "number"
          },
          "west": {
            "description": "West (lower left corner, coordinate axis 1).",
            "type": "number"
          }
        },
        "required": [
          "west",
          "south",
          "east",
          "north"
        ],
        "subtype": "bounding-box",
        "title": "Bounding Box",
        "type": "object"
      }
    },
    {
      "description": "time",
      "name": "temporal_extent",
      "schema": {
        "description": "Left-closed temporal interval, represented as two-element array with the following elements:\n\n1. The first element is the start of the temporal interval. The specified instance in time is **included** in the interval.\n2. The second element is the end of the temporal interval. The specified instance in time is **excluded** from the interval.\n\nThe specified temporal strings follow [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html). Although [RFC 3339 prohibits the hour to be '24'](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.7), **this process allows the value '24' for the hour** of an end time in order to make it possible that left-closed time intervals can fully cover the day. `null` can be used to specify open intervals.",
        "examples": [
          [
            "2015-01-01T00:00:00Z",
            "2016-01-01T00:00:00Z"
          ],
          [
            "2015-01-01",
            "2016-01-01"
          ],
          [
            "00:00:00Z",
            "12:00:00Z"
          ],
          [
            "2015-01-01",
            null
          ]
        ],
        "items": {
          "anyOf": [
            {
              "description": "Date and time representation, as defined for `date-time` by [RFC 3339 in section 5.6](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.6).",
              "format": "date-time",
              "subtype": "date-time",
              "title": "Date with Time",
              "type": "string"
            },
            {
              "description": "Date only representation, as defined for `full-date` by [RFC 3339 in section 5.6](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.6). The time zone is UTC.",
              "format": "date",
              "subtype": "date",
              "title": "Date only",
              "type": "string"
            },
            {
              "description": "Time only representation, as defined for `full-time` by [RFC 3339 in section 5.6](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.6). Although [RFC 3339 prohibits the hour to be '24'](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.7), this definition allows the value '24' for the hour as end time in an interval in order to make it possible that left-closed time intervals can fully cover the day.",
              "format": "time",
              "subtype": "time",
              "title": "Time only",
              "type": "string"
            },
            {
              "description": "Year representation, as defined for `date-fullyear` by [RFC 3339 in section 5.6](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.6).",
              "maxLength": 4,
              "minLength": 4,
              "pattern": "^\\d{4}$",
              "subtype": "year",
              "title": "Year only",
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Processes and implementations may choose to only implement a subset of the subtypes specified here. Clients must check what back-ends / processes actually support."
        },
        "maxItems": 2,
        "minItems": 2,
        "subtype": "temporal-interval",
        "title": "Single temporal interval",
        "type": "array"
      }
    }
  ],
  "process_graph": {
    "load1": {
      "arguments": {
        "bands": [
          "B02",
          "B03",
          "B04"
        ],
        "id": "boa_sentinel_2",
        "spatial_extent": {
          "east": 19.38022689396854,
          "north": 52.43260595832486,
          "south": 41.388795496730424,
          "west": 1.4211922929976784
        },
        "temporal_extent": [
          "2015-07-04T12:00:00Z",
          "2021-06-22T12:00:00Z",
          {
            "from_parameter": "temporal_extent"
          }
        ]
      },
      "process_id": "load_collection"
    },
    "reduce2": {
      "arguments": {
        "data": {
          "from_node": "load1"
        },
        "dimension": "t",
        "reducer": {
          "process_graph": {
            "mean1": {
              "arguments": {
                "data": {
                  "from_parameter": "data"
                }
              },
              "process_id": "mean",
              "result": true
            }
          }
        }
      },
      "process_id": "reduce_dimension"
    },
    "save3": {
      "arguments": {
        "data": {
          "from_node": "reduce2"
        },
        "format": "NETCDF"
      },
      "process_id": "save_result",
      "result": true
    }
  }
}

Using the UDP in a job like:

  "process_graph": {
    "RGB": {
      "process_id": "RGB",
      "arguments": {
        "spatial_extent": {
          "west": 11.158127417005206,
          "east": 11.592552719418347,
          "south": 46.39131225178659,
          "north": 46.59804577340614
        },
        "temporal_extent": [
          "2018-03-01T00:00:00Z",
          "2018-03-31T00:00:00Z"
        ]
      },
      "result": true
    }
  }
} ```