chore: prepare 5.0.0 release (#564)

* chore: prepare release * fix: openapi.yaml, postman_collection.json Test Connector Remote for Eclipse EDC 0.2.1 * docs: fix API Wrapper documentation and client examples * docs: create deployment guides for 5.0.0 * chore: bump versions for local demo docker compose * chore: fix CHANGELOG.md structure * feat: add custom auth possibility to the java client * docs: fix asset creation and transfer initiation in docs * docs: migrate docs using the management api
sovity · Oct 13, 2023 · e026c22 · e026c22
1 parent 9754e5d
commit e026c22
Show file tree

Hide file tree

Showing 40 changed files with 4,001 additions and 1,327 deletions.
diff --git a/.env b/.env
@@ -1,6 +1,6 @@
 # Env variables for docker-compose.yaml
-EDC_IMAGE=ghcr.io/sovity/edc-dev:4.2.0
-TEST_BACKEND_IMAGE=ghcr.io/sovity/test-backend:4.2.0
-EDC_UI_IMAGE=ghcr.io/sovity/edc-ui:0.0.1-milestone-8-sovity13
+EDC_IMAGE=ghcr.io/sovity/edc-dev:5.0.0
+TEST_BACKEND_IMAGE=ghcr.io/sovity/test-backend:5.0.0
+EDC_UI_IMAGE=ghcr.io/sovity/edc-ui:2.0.0
 EDC_UI_ACTIVE_PROFILE=sovity-open-source
 
diff --git a/.github/ISSUE_TEMPLATE/release.md b/.github/ISSUE_TEMPLATE/release.md
@@ -12,13 +12,22 @@ assignees: ""
 
 Feel free to edit this release checklist in-progress depending on what tasks need to be done:
 
-- [ ] Release [edc-ui](https://github.com/sovity/edc-ui), this might require several steps.
+- [ ] Release [edc-ui](https://github.com/sovity/edc-ui), this might require several steps: _Link to EDC UI Release here_
 - [ ] Decide a release version depending on major/minor/patch changes in the CHANGELOG.md.
 - [ ] Update this issue's title to the new version
 - [ ] `release-prep` PR:
     - [ ] Write or review the current [Productive Deployment Guide](https://github.com/sovity/edc-extensions/blob/main/docs/deployment-guide/goals/production)
     - [ ] Write or review the current [Development Deployment Guide](https://github.com/sovity/edc-extensions/blob/main/docs/deployment-guide/goals/development)
     - [ ] Write or review the current [Local Demo Deployment Guide](https://github.com/sovity/edc-extensions/blob/main/docs/deployment-guide/goals/local-demo)
+    - [ ] For Major version updates: If we want to continue supporting the old major version:
+        - [ ] Keep the old Productive Development Guide in a separate location.
+            - [ ] Add a note to the old version about its deprecation status.
+            - [ ] Add a Link the old version in the new version for discoverability.
+            - [ ] Check all links in the old version.
+        - [ ] Keep the old Productive Development Guide in a separate location.
+            - [ ] Add a note to the old version about its deprecation status.
+            - [ ] Add a Link the old version in the new version for discoverability.
+            - [ ] Check all links in the old version.
     - [ ] Update the CHANGELOG.md.
         - [ ] Add a clean `Unreleased` version.
         - [ ] Add the version to the old section.
@@ -30,6 +39,7 @@ Feel free to edit this release checklist in-progress depending on what tasks nee
         - [ ] Write or review a release summary.
         - [ ] Write or review the compatible versions section.
         - [ ] Add a link to the EDC UI Release to the "EDC UI" section.
+        - [ ] Add a link to the EDC UI Release Deployment Migration Notes from the Deployment Migration section if the EDC UI has Deployment Migration Notes.
         - [ ] Remove empty sections from the patch notes.
     - [ ] Replace the existing `docker-compose.yaml` with `docker-compose-dev.yaml`.
     - [ ] Set the version for `EDC_IMAGE` of

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,43 +4,91 @@ All notable changes to this project will be documented in this file.
 
 ## [x.x.x] - UNRELEASED
 
-### Major Changes
+### Overview
+
+### EDC UI
+
+### EDC Extensions
 
-- Upgrade to core-EDC version `0.2.1`
-- Now using the `Dataspace Protocol`
-- Major changes to the management API
-  - Examples for the new requests are located in the postman collection in the `docs` folder
-  - The `OpenAPI` file has been updated to represent the EDC version `0.1.2`
-- Reworked Deployment Guide & Docker-Compose:
+#### Major Changes
+
+#### Minor Changes
+
+### Deployment Migration Notes
+
+#### Compatible Versions
+
+## [5.0.0] - 10.10.2023
+
+### Overview
+
+Migration from Eclipse EDC Milestone 8 to Eclispe EDC 0.2.1.
+
+The API Wrapper and API Client Libraries can now be used to fully control a sovity EDC Connector.
+
+### EDC UI
+
+https://github.com/sovity/edc-ui/releases/tag/v2.0.0
+
+### EDC Extensions
+
+#### Major Changes
+
+- Bump Eclipse EDC Version to `0.2.1`:
+  - Now using the Data Space Protocol (DSP) over the ~~IDS Protocol~~.
+  - Major changes to the Management API. See the postman collection / OpenAPI file.
+- The Getting Started Docker Compose file is no longer to be used as reference for deployments:
   - The Getting Started Docker Compose file now launches connectors for local demo purposes.
   - For productive deployments, a detailed deployment guide has been added.
   - The Dev-Images now also require a PostgreSQL Database.
 
-### Minor Changes
+#### Minor Changes
+
+- All Connector UI Endpoints were migrated to our UI API Wrapper. New UI API Wrapper Endpoints:
+    - Asset Page
+    - Create Asset
+    - Delete Asset
+    - Catalog / Data Offers
+    - Contract Definition Page
+    - Contract Negotiation Start / Detail
+    - Create Contract Definition
+    - Delete Contract Definition
+    - Policy Definition Page
+    - Create Policy Definition
+    - Delete Policy Definition
+    - Dashboard Page
+- New modules with common UI models and mappers for the Connector UI and Broker UI: `:extensions:wrapper:wrapper-common-api` and `:extensions:wrapper:wrapper-common-mappers`.
+- New module with centralized Vocab and utilities for dealing with EDC / DCAT JSON-LD: `:utils:json-and-jsonld-utils`
+- New module with utilities for parsing DCAT Catalog responses for use in the UI API Wrapper and the Broker Server: `:utils:catalog-parser`
+- New modules with utilities for E2E Testing Connectors: `:utils:test-connector-remote` and `:extensions:test-backend-controller`
+
+#### Patch Changes
+
+- New modules in `:launchers:common` and `:launchers:connectors` so building different variants no longer requires separate builds.
+- New module `:extensions:wrapper:wrapper-api` split from `:extensions:wrapper:wrapper` so integration tests in `wrapper` can use the Java Client Library.
+- New JUnit E2E Tests in `:launchers:connectors:sovity-dev` that start two connectors and test the data exchange.
 
-- UI API Wrapper Model:
-  - UiPolicy + PolicyMapper for EDC UI and Broker Server
-  - UiAsset + AssetMapper for EDC UI and Broker Server
-- UI API Wrapper Endpoints:
-  - Asset Page
-  - Create Asset
-  - Delete Asset
-  - Catalog / Data Offers
-  - Contract Definition Page
-  - Contract Negotiation Create / By ID
-  - Create Contract Definition
-  - Delete Contract Definition
-  - Policy Definition Page
-  - Create Policy Definition
-  - Delete Policy Definition
-  - Dashboard Page
-- DCAT Catalog Service: Parse DCAT Catalog responses for use in the UI API Wrapper and the Broker Server.
-- JSON and JSON-LD Utilities: Centralized Vocab collection for dealing with EDC / DCAT JSON-LD
-
-### Migration Notes
-
-1. The Connector Endpoint changed to `https://[FQDN]/api/dsp`, used to be `https://[FQDN]/api/v1/ids/data`.
-2. The `v1` Management API has been removed in favor of the `JSON-LD` `v2` Management API. All endpoints have a `v2` prefix now (example: `http://localhost:11002/api/v1/management/assets/request` is now available at `http://localhost:11002/api/management/v2/assets/request`)
+### Deployment Migration Notes
+
+1. Deployment Migration Notes for the EDC UI: https://github.com/sovity/edc-ui/releases/tag/v2.0.0
+2. The Connector Endpoint changed to `https://[FQDN]/api/dsp` from ~~`https://[FQDN]/api/v1/ids/data`~~. 
+3. The Management Endpoint changed to `https://[FQDN]/api/management` from ~~`https://[FQDN]/api/v1/management`~~.
+4. The `v1` Eclipse EDC Management API has been replaced by the Eclipse EDC `JSON-LD` `v2` Management API. Our Postman Collection shows some example requests. 
+   However, a switch to our [API Wrapper](extensions/wrapper/README.md) is recommended. Despite our Use Case API Wrapper API still being in development, 
+   the Connector UI API Wrapper is fully functional and can be used in concatenation with our type-safe generated API Client Libraries to both provide and 
+   consume data offers.
+5. The Connector now uses the Data Space Protocol (DSP) instead of the IDS Protocol. This requires different paths to be available from the internet. 
+   Please refer to our deployment guide for more information.
+6. If the old protocol endpoint required HTTP communication to pass as a workaround for a certain bug, this should be undone now, 
+   with all protocol endpoints being secured by HTTPS/TLS.
+
+#### Compatible Versions
+
+- Connector Backend Docker Images:
+  - Dev EDC: `ghcr.io/sovity/edc-dev:5.0.0`
+  - sovity EDC CE: `ghcr.io/sovity/edc-ce:5.0.0`
+  - MDS EDC CE: `ghcr.io/sovity/edc-ce-mds:5.0.0`
+- Connector UI Docker Image: `ghcr.io/sovity/edc-ui:2.0.0`
 
 ## [4.2.0] - 2023-09-01
 
@@ -52,7 +100,7 @@ MDS 1.2 release using MS8 EDC.
 
 - https://github.com/sovity/edc-ui/releases/tag/v0.0.1-milestone-8-sovity13
 
-### EDC-Extensions
+### Detailed Changes
 
 #### Patch Changes
 
@@ -64,7 +112,7 @@ MDS 1.2 release using MS8 EDC.
 
 - Connector Backend Docker Images:
     - Dev EDC: `ghcr.io/sovity/edc-dev:4.2.0`
-    - Sovity EDC CE: `ghcr.io/sovity/edc-ce:4.2.0`
+    - sovity EDC CE: `ghcr.io/sovity/edc-ce:4.2.0`
     - MDS EDC CE: `ghcr.io/sovity/edc-ce-mds:4.2.0`
 - Connector UI Docker Image: `ghcr.io/sovity/edc-ui:0.0.1-milestone-8-sovity13`
 
@@ -99,7 +147,7 @@ Security improvements of container image and enhancements for the `ReferringConn
 
 - Connector Backend Docker Images:
   - Dev EDC: `ghcr.io/sovity/edc-dev:4.1.0`
-  - Sovity EDC CE: `ghcr.io/sovity/edc-ce:4.1.0`
+  - sovity EDC CE: `ghcr.io/sovity/edc-ce:4.1.0`
   - MDS EDC CE: `ghcr.io/sovity/edc-ce-mds:4.1.0`
 - Connector UI Docker Image: `ghcr.io/sovity/edc-ui:0.0.1-milestone-8-sovity12`
 
@@ -127,7 +175,7 @@ No changes besides docker image versions.
 
 - Connector Backend Docker Images:
   - Dev EDC: `ghcr.io/sovity/edc-dev:4.0.1`
-  - Sovity EDC CE: `ghcr.io/sovity/edc-ce:4.0.1`
+  - sovity EDC CE: `ghcr.io/sovity/edc-ce:4.0.1`
   - MDS EDC CE: `ghcr.io/sovity/edc-ce-mds:4.0.1`
 - Connector UI Docker Image: `ghcr.io/sovity/edc-ui:0.0.1-milestone-8-sovity11`
 
@@ -163,7 +211,7 @@ No changes besides docker image versions.
 
 - Connector Backend Docker Images:
   - Dev EDC: `ghcr.io/sovity/edc-dev:4.0.0`
-  - Sovity EDC CE: `ghcr.io/sovity/edc-ce:4.0.0`
+  - sovity EDC CE: `ghcr.io/sovity/edc-ce:4.0.0`
   - MDS EDC CE: `ghcr.io/sovity/edc-ce-mds:4.0.0`
 - Connector UI Docker Image: `ghcr.io/sovity/edc-ui:0.0.1-milestone-8-sovity9`
 

diff --git a/docker-compose-dev.yaml b/docker-compose-dev.yaml
@@ -7,8 +7,8 @@ services:
     environment:
       - EDC_UI_ACTIVE_PROFILE=${EDC_UI_ACTIVE_PROFILE}
       - EDC_UI_CONFIG_URL=edc-ui-config
-      - EDC_UI_DATA_MANAGEMENT_API_URL=http://localhost:11002/api/management
-      - EDC_UI_DATA_MANAGEMENT_API_KEY=ApiKeyDefaultValue
+      - EDC_UI_MANAGEMENT_API_URL=http://localhost:11002/api/management
+      - EDC_UI_MANAGEMENT_API_KEY=ApiKeyDefaultValue
       - EDC_UI_CATALOG_URLS=http://edc2:11003/api/dsp
 
   edc:
@@ -51,8 +51,8 @@ services:
     environment:
       - EDC_UI_ACTIVE_PROFILE=${EDC_UI_ACTIVE_PROFILE}
       - EDC_UI_CONFIG_URL=edc-ui-config
-      - EDC_UI_DATA_MANAGEMENT_API_URL=http://localhost:22002/api/management
-      - EDC_UI_DATA_MANAGEMENT_API_KEY=ApiKeyDefaultValue
+      - EDC_UI_MANAGEMENT_API_URL=http://localhost:22002/api/management
+      - EDC_UI_MANAGEMENT_API_KEY=ApiKeyDefaultValue
       - EDC_UI_CATALOG_URLS=http://edc:11003/api/dsp
 
   edc2:

diff --git a/docker-compose.yaml b/docker-compose.yaml
@@ -3,20 +3,20 @@ services:
   edc-ui:
     image: ${EDC_UI_IMAGE}
     ports:
-      - '11000:80'
+      - '11000:8080'
     environment:
       - EDC_UI_ACTIVE_PROFILE=${EDC_UI_ACTIVE_PROFILE}
       - EDC_UI_CONFIG_URL=edc-ui-config
-      - EDC_UI_DATA_MANAGEMENT_API_URL=http://localhost:11002/api/v1/management
-      - EDC_UI_DATA_MANAGEMENT_API_KEY=ApiKeyDefaultValue
-      - EDC_UI_CATALOG_URLS=http://edc2:11003/api/v1/ids/data
+      - EDC_UI_MANAGEMENT_API_URL=http://localhost:11002/api/management
+      - EDC_UI_MANAGEMENT_API_KEY=ApiKeyDefaultValue
+      - EDC_UI_CATALOG_URLS=http://edc2:11003/api/dsp
 
   edc:
     image: ${EDC_IMAGE}
     depends_on:
       - postgresql
     environment:
-      MY_EDC_NAME_KEBAB_CASE: "example-connector"
+      MY_EDC_NAME_KEBAB_CASE: "my-edc"
       MY_EDC_TITLE: "EDC Connector"
       MY_EDC_DESCRIPTION: "sovity Community Edition EDC Connector"
       MY_EDC_CURATOR_URL: "https://example.com"
@@ -25,15 +25,15 @@ services:
       MY_EDC_MAINTAINER_NAME: "sovity GmbH"
 
       MY_EDC_FQDN: "edc"
-      EDC_API_AUTH_KEY: ApiKeyDefaultValue # Management API Key
+      EDC_API_AUTH_KEY: ApiKeyDefaultValue
 
       MY_EDC_JDBC_URL: jdbc:postgresql://postgresql:5432/edc
       MY_EDC_JDBC_USER: edc
       MY_EDC_JDBC_PASSWORD: edc
 
       # docker compose local dev environment overrides (don't use with non-dev images)
       MY_EDC_PROTOCOL: "http://"
-      MY_EDC_IDS_BASE_URL: "http://edc:11003"
+      EDC_DSP_CALLBACK_ADDRESS: http://edc:11003/api/dsp
       EDC_WEB_REST_CORS_ENABLED: 'true'
       EDC_WEB_REST_CORS_HEADERS: 'origin,content-type,accept,authorization,X-Api-Key'
       EDC_WEB_REST_CORS_ORIGINS: '*'
@@ -43,43 +43,41 @@ services:
       - '11003:11003'
       - '11004:11004'
       - '11005:5005'
-    volumes:
-      - ./docs/getting-started/secrets:/secrets
 
   edc-ui2:
     image: ${EDC_UI_IMAGE}
     ports:
-      - '22000:80'
+      - '22000:8080'
     environment:
       - EDC_UI_ACTIVE_PROFILE=${EDC_UI_ACTIVE_PROFILE}
       - EDC_UI_CONFIG_URL=edc-ui-config
-      - EDC_UI_DATA_MANAGEMENT_API_URL=http://localhost:22002/api/v1/management
-      - EDC_UI_DATA_MANAGEMENT_API_KEY=ApiKeyDefaultValue
-      - EDC_UI_CATALOG_URLS=http://edc:11003/api/v1/ids/data
+      - EDC_UI_MANAGEMENT_API_URL=http://localhost:22002/api/management
+      - EDC_UI_MANAGEMENT_API_KEY=ApiKeyDefaultValue
+      - EDC_UI_CATALOG_URLS=http://edc:11003/api/dsp
 
   edc2:
     image: ${EDC_IMAGE}
     depends_on:
       - postgresql2
     environment:
-      MY_EDC_NAME_KEBAB_CASE: "edc2"
-      MY_EDC_TITLE: "EDC Connector"
+      MY_EDC_NAME_KEBAB_CASE: "my-edc2"
+      MY_EDC_TITLE: "EDC Connector 2"
       MY_EDC_DESCRIPTION: "sovity Community Edition EDC Connector"
       MY_EDC_CURATOR_URL: "https://example.com"
       MY_EDC_CURATOR_NAME: "Example GmbH"
       MY_EDC_MAINTAINER_URL: "https://sovity.de"
       MY_EDC_MAINTAINER_NAME: "sovity GmbH"
 
       MY_EDC_FQDN: "edc2"
-      EDC_API_AUTH_KEY: ApiKeyDefaultValue # Management API Key
+      EDC_API_AUTH_KEY: ApiKeyDefaultValue
 
       MY_EDC_JDBC_URL: jdbc:postgresql://postgresql2:5432/edc
       MY_EDC_JDBC_USER: edc
       MY_EDC_JDBC_PASSWORD: edc
 
       # docker compose local dev environment overrides (don't use with non-dev images)
       MY_EDC_PROTOCOL: "http://"
-      MY_EDC_IDS_BASE_URL: "http://edc2:11003"
+      EDC_DSP_CALLBACK_ADDRESS: http://edc2:11003/api/dsp
       EDC_WEB_REST_CORS_ENABLED: 'true'
       EDC_WEB_REST_CORS_HEADERS: 'origin,content-type,accept,authorization,X-Api-Key'
       EDC_WEB_REST_CORS_ORIGINS: '*'
@@ -89,8 +87,6 @@ services:
       - '22003:11003'
       - '22004:11004'
       - '22005:5005'
-    volumes:
-      - ./docs/getting-started/secrets:/secrets
 
   postgresql:
     image: docker.io/bitnami/postgresql:11
@@ -100,7 +96,7 @@ services:
       POSTGRESQL_PASSWORD: edc
       POSTGRESQL_DATABASE: edc
     ports:
-      - '11543:5432'
+      - '54321:5432'
     volumes:
       - 'postgresql:/bitnami/postgresql'
 
@@ -112,10 +108,15 @@ services:
       POSTGRESQL_PASSWORD: edc
       POSTGRESQL_DATABASE: edc
     ports:
-      - '22543:5432'
+      - '54322:5432'
     volumes:
       - 'postgresql2:/bitnami/postgresql'
 
+  test-backend:
+    image: ${TEST_BACKEND_IMAGE}
+    ports:
+      - '33001:11001'
+
 volumes:
   postgresql:
     driver: local