Wrong state in transfer history?

[epanagou-ubitech]

This was originally meant to be a bug report but we are not sure if this behavior is a bug with the Connector UI or our implementation of the data source and data sink endpoints.

After a transfer occurs, the state displayed in the Transfer History page in the Connector UI is not always correct.

In particular:

• On the Consumer side, the state is always COMPLETED even if the transfer was unsuccessful

• On the Provider side, the state is always IN_PROGRESS even if the transfer was successful

In both cases (transfer successful or unsuccessful), our data source and data sink endpoints, developed with Express.js, respond with the correct HTTP status codes, and the Provider Connector seems aware of any errors that occur.

Example from the Provider Connector logs where the data source endpoint returns 404:

connector-core | 2023-10-12 14:17:48 Error processing data transfer request: 9cd2cc8a-42ae-4e01-8e21-5509bdb855a4
connector-core | org.eclipse.edc.spi.EdcException: Received code transferring HTTP data for request 9cd2cc8a-42ae-4e01-8e21-5509bdb855a4: 404 - Not Found. Source file does not exist

Example from the Provider Connector logs where the data sink endpoint is invalid (e.g. POST https://google.com):

connector-core | 2023-10-12 14:35:14 Error {405: } received writing HTTP data null to endpoint https://google.com/ for request: Request{method=POST, url=https://google.com/}

There are no logs in the Provider or the Consumer Connector when the transfer is successful.

We are aware that transfers are asynchronous. Is there a way to notify the Connector of the transfer state?

Versions used:

• edc: ghcr.io/sovity/edc-ce-mds:4.1.0
• edc-ui: v0.0.1-milestone-8-sovity12

[epanagou-ubitech]

A related issue sovity/edc-ce#576 was posted in the edc-extensions repo.

[tmberthold]

Have you tried this in the meantime with our newer sovity EDC CE versions? Does the problem still arise using the newer versions, since you are mentioning v4.1.0 (which has an older core-edc MS8-patch-1 under the hood)?

Starting with our sovity EDC CE v5.0.0 [1] we have migrated from core-edc Milestone 8 (patch 1) to core-EDC v0.2.1.
[1] https://github.com/sovity/edc-extensions/releases/tag/v5.0.0

As to the linked issue, this will most likely be fixed when we take on the next core-edc versions or create an individual solution.

[epanagou-ubitech]

Hello! We have tried ghcr.io/sovity/edc-dev:5.0.0 with ghcr.io/sovity/edc-ui:2.0.0.

Upon successful transfer (to webhook.site):

• On the consumer side, we get this line in the logs, and the transfer is stuck on STARTED in the Transfer History page:

  edc2_1          | 2023-11-22 13:40:40 No checker found for process 006f85fe-bc3d-45cb-ac71-3d3da9b7755b. The process will not advance to the COMPLETED state.
  

• On the provider side, the transfer is also stuck on STARTED in the Transfer History page.

Upon unsuccessful transfer (to google.com):

• On the consumer side, we get this line in the logs, and the transfer is stuck on STARTED in the Transfer History page:

  edc2_1          | 2023-11-22 13:47:09 No checker found for process 40a59365-a974-452a-9a92-48979bc95235. The process will not advance to the COMPLETED state.
  

• On the provider side, we get this line in the logs, and the transfer is stuck on STARTED in the Transfer History page:

  edc_1           | 2023-11-22 13:47:09 Error {413: } received writing HTTP data null to endpoint https://google.com/ for request: Request{method=POST, url=https://google.com/}
  

[tmberthold]

The "No checker found for process" message and being stuck on STARTED is a known problem of the core-edc version used by us, that was fixed with core-edc v0.3.0, which we have not yet migrated to.

Here is an already existing issue for that:
sovity/edc-ce#576

Once we migrated to => core-edc v0.3.0 this shouldn't be a problem any more.

[epanagou-ubitech]

Thank you!