fix: daps interaction issues, fix CI, deprecated MY_EDC_NAME_KEBAB_CA…

…SE (#642)

.github/ISSUE_TEMPLATE/release.md

@@ -53,6 +53,7 @@ Feel free to edit this release checklist in-progress depending on what tasks nee
     - [ ] Merge the `release-prep` PR.
 - [ ] Wait for the main branch to be green.
 - [ ] Test the release `docker-compose.yaml` with `RELEASE_EDC_IMAGE=ghcr.io/sovity/edc-dev:latest`.
+    - [ ] Ensure with a `docker ps -a` that all containers are healthy, and not `healthy: starting` or `healthy: unhealthy`.
 - [ ] Test the postman collection against that running docker-compose.
 - [ ] Create a release and re-use the changelog section as release description, and the version as title.
 - [ ] Check if the pipeline built the release versions in the Actions-Section (or you won't see it).

.github/workflows/ci.yml

@@ -76,11 +76,11 @@ jobs:
           GRADLE_ARGS="-PsovityEdcExtensionsVersion=${GITHUB_REF#refs/tags/v}"
           echo "GRADLE_ARGS=$GRADLE_ARGS" >> $GITHUB_ENV
       - name: "Gradle: Build"
-        uses: gradle/gradle-build-action@a4cf152f482c7ca97ef56ead29bf08bcd953284c
+        uses: gradle/gradle-build-action@v2.10.0
         with:
           arguments: build ${{ env.GRADLE_ARGS }}
       - name: "Gradle: Publish (Main & Release Only)"
-        uses: gradle/gradle-build-action@a4cf152f482c7ca97ef56ead29bf08bcd953284c
+        uses: gradle/gradle-build-action@v2.10.0
         if: ${{ github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') }}
         with:
           arguments: publish ${{ env.GRADLE_ARGS }}
@@ -152,7 +152,7 @@ jobs:
       - name: "Gradle: Validate Gradle Wrapper"
         uses: gradle/wrapper-validation-action@ccb4328a959376b642e027874838f60f8e596de3
       - name: "Gradle: Generate TS Code"
-        uses: gradle/gradle-build-action@a4cf152f482c7ca97ef56ead29bf08bcd953284c
+        uses: gradle/gradle-build-action@v2.10.0
         with:
           arguments: :extensions:wrapper:wrapper:build -x test
       - name: "NPM: Dist Tag & Version"

CHANGELOG.md

@@ -17,12 +17,18 @@ please see [changelog_updates.md](docs/dev/changelog_updates.md).
 
 #### Patch Changes
 
+- Marked `MY_EDC_NAME_KEBAB_CASE` as deprecated in favor of `MY_EDC_PARTICIPANT_ID`.
+- Fixed an issue preventing Contract Negotiations between MDS Connectors.
 - Improved `:extensions:wrapper:wrapper-common-mappers` for broker: `AssetJsonLdUtils`, made some methods public.
 - Added example for using the API Wrapper to offer and consume data.
 - Improved development documentation documenting our CHANGELOG.
 
 ### Deployment Migration Notes
 
+- Renamed ~~`MY_EDC_NAME_KEBAB_CASE`~~ to `MY_EDC_PARTICIPANT_ID`. ~~`MY_EDC_NAME_KEBAB_CASE`~~ continues working, but prints a warning on startup.
+- The value of `MY_EDC_PARTICIPANT_ID` must coincide with the claim value `referringConnector`
+  as configured for this Connector in the DAPS.
+
 #### Compatible Versions
 
 ## [6.0.0] - 2023-11-17

build.gradle.kts

@@ -54,7 +54,7 @@ allprojects {
     tasks.getByName<Test>("test") {
         useJUnitPlatform()
         testLogging {
-            events = setOf(TestLogEvent.FAILED)
+            events = setOf(TestLogEvent.SKIPPED, TestLogEvent.FAILED)
             exceptionFormat = TestExceptionFormat.FULL
             showExceptions = true
             showCauses = true

docker-compose-dev.yaml

@@ -16,7 +16,7 @@ services:
     depends_on:
       - postgresql
     environment:
-      MY_EDC_NAME_KEBAB_CASE: "my-edc"
+      MY_EDC_PARTICIPANT_ID: "my-edc"
       MY_EDC_TITLE: "EDC Connector"
       MY_EDC_DESCRIPTION: "sovity Community Edition EDC Connector"
       MY_EDC_CURATOR_URL: "https://example.com"
@@ -60,7 +60,7 @@ services:
     depends_on:
       - postgresql2
     environment:
-      MY_EDC_NAME_KEBAB_CASE: "my-edc2"
+      MY_EDC_PARTICIPANT_ID: "my-edc2"
       MY_EDC_TITLE: "EDC Connector 2"
       MY_EDC_DESCRIPTION: "sovity Community Edition EDC Connector"
       MY_EDC_CURATOR_URL: "https://example.com"

docs/deployment-guide/goals/production/README.md

@@ -7,25 +7,34 @@ Productive Deployment Guide
 
 This is a productive deployment guide for self-hosting a functional sovity CE EDC Connector or MDS CE EDC Connector.
 
-## Requirements
+## Prerequisites
 
-A productive EDC Connector deployment has strict requirements, with slight errors in configuration already causing
-contract negotiations / data transfer to fail.
+### Technical Skills
 
-In general a productive EDC Connector requires a DAPS Server, DAPS Credentials, a reverse proxy configured in detail due
-to technical reasons, reachability via the internet and well-defined URLs across all configurations.
+- Ability to deploy, run and expose containered applications to the internet.
+- Ability to configure ingress routes or a reverse proxy of your choice to merge multiple services under a single
+  domain.
+- Know-how on how to secure an otherwise unprotected application with an auth proxy or other solutions fitting
+  your situation.
+
+### Dataspace
+
+- Must have a running DAPS that follows the subset of OAuth2 as described in the DSP Specification.
+- You must have a valid Connector Certificate in the form of [a generated SKI/AKI pair and .jks file](#faq).
+- You must have a valid Participant ID / Connector ID, which is configured in the claim "referringConnector" in the
+  DAPS.
 
 ## Deployment Units
 
 To deploy an EDC multiple deployment units must be deployed and configured.
 
-| Deployment Unit                                                | Version / Details                                                                           |
-|----------------------------------------------------------------|---------------------------------------------------------------------------------------------|
-| An Auth Proxy / Auth solution of your choice.                  | (deployment specific, required to secure UI and management API)                             |
-| Reverse Proxy that merges the UI+Backend and removes the ports | (deployment specific)                                                                       |
-| Postgresql                                                     | 13 or compatible version                                                                    |
-| EDC Backend                                                    | edc-ce or edc-ce-mds, see [CHANGELOG.md](../../../../CHANGELOG.md) for compatible versions. |
-| EDC UI                                                         | edc-ui, see  [CHANGELOG.md](../../../../CHANGELOG.md) for compatible versions.              |
+| Deployment Unit                                                   | Version / Details                                                                           |
+|-------------------------------------------------------------------|---------------------------------------------------------------------------------------------|
+| An Auth Proxy / Auth solution of your choice.                     | (deployment specific, required to secure UI and management API)                             |
+| Reverse Proxy that merges multiple services and removes the ports | (deployment specific)                                                                       |
+| Postgresql                                                        | 13 or compatible version                                                                    |
+| EDC Backend                                                       | edc-ce or edc-ce-mds, see [CHANGELOG.md](../../../../CHANGELOG.md) for compatible versions. |
+| EDC UI                                                            | edc-ui, see  [CHANGELOG.md](../../../../CHANGELOG.md) for compatible versions.              |
 
 ## Configuration
 
@@ -79,13 +88,7 @@ EDC_UI_CONFIG_URL: "edc-ui-config"
 
 ## EDC Backend Configuration
 
-A sovity EDC CE or MDS EDC CE Backend deployment requires:
-
-- A running DAPS
-- (MDS Only) A running Clearing House
-- DAPS Access
-  and [a generated SKI/AKI pair and .jks file](#faq)
-- The following configuration properties
+A sovity EDC CE or MDS EDC CE Backend deployment requires the following environment variables:
 
 > [!WARNING]
 > Please be careful with overriding any of the ENV Vars set in our [launchers/.env](../../../../launchers/.env). Our
@@ -97,8 +100,9 @@ A sovity EDC CE or MDS EDC CE Backend deployment requires:
 # Connector Host Name
 MY_EDC_FQDN: "my-edc-deployment1.example.com"
 
-# Connector Technical Name
-MY_EDC_NAME_KEBAB_CASE: "example-connector"
+# Participant ID / Connector ID
+# Must be configured as the value of the "referringConnector" claim in the DAPS for this connector
+MY_EDC_PARTICIPANT_ID: "MDSL1234XX.C1234XX"
 
 # Connector Localized Name / Title
 MY_EDC_TITLE: "EDC Connector"
@@ -165,10 +169,5 @@ configuration and be accesible from the data provider via REST calls.
 
 ### Can I still use the deprecated Omejdn DAPS?
 
-For Omejdn one needs the following overrides in the backend:
-
-```yaml
-EDC_OAUTH_PROVIDER_AUDIENCE: idsc:IDS_CONNECTORS_ALL
-EDC_OAUTH_ENDPOINT_AUDIENCE: idsc:IDS_CONNECTORS_ALL
-EDC_AGENT_IDENTITY_KEY: client_id
-```
+In the current version of the sovity EDC CE Connector the Omejdn DAPS is not supported due to the Omejdn DAPS requiring
+a special OAuth2 extension and custom messages that exceed the default DSP Oauth2 Specification.

docs/getting-started/documentation/api_wrapper.md

@@ -42,4 +42,4 @@ Using The Client
 Feel free to use the endpoints of the aforementioned API groups.
 
 A full example providing and consuming a data offer using the API Wrapper Client Library can be found
-in [ApiWrapperDemoTest.java](../../../launchers/connectors/sovity-dev/src/test/java/de/sovity/edc/e2e/ApiWrapperDemoTest.java).
+in [ApiWrapperDemoTest.java](../../../tests/src/test/java/de/sovity/edc/e2e/ApiWrapperDemoTest.java).

docs/sovity-edc-api-wrapper.yaml

@@ -17,7 +17,7 @@ externalDocs:
   description: EDC API Wrapper Project in sovity/edc-extensions
   url: https://github.com/sovity/edc-extensions/tree/main/extensions/wrapper
 servers:
-- url: "https://[MY_EDC_FQDN]/api/management"
+- url: https://my-connector/api/management
 tags:
 - name: Enterprise Edition
   description: sovity Enterprise Edition EDC API Endpoints. Requires our sovity Enterprise
@@ -384,23 +384,6 @@ paths:
             application/json:
               schema:
                 $ref: '#/components/schemas/IdResponseDto'
-  /wrapper/use-case-api/data-offer:
-    post:
-      tags:
-      - Use Case
-      description: "Creates a new data offer, consisting of an asset, a policy definition\
-        \ and a contract definition."
-      operationId: createOffer
-      requestBody:
-        content:
-          application/json:
-            schema:
-              $ref: '#/components/schemas/CreateOfferingDto'
-      responses:
-        default:
-          description: default response
-          content:
-            '*/*': {}
   /wrapper/use-case-api/kpis:
     get:
       tags:
@@ -1215,6 +1198,7 @@ components:
       - assetName
       - contractAgreementId
       - counterPartyConnectorEndpoint
+      - counterPartyParticipantId
       - createdDate
       - direction
       - lastUpdatedDate
@@ -1243,6 +1227,9 @@ components:
         counterPartyConnectorEndpoint:
           type: string
           description: Other Connector's Endpoint
+        counterPartyParticipantId:
+          type: string
+          description: Other Connector's Participant ID
         assetName:
           type: string
           description: Asset Name
@@ -1328,150 +1315,6 @@ components:
           description: Additional transfer process properties. These are not passed
             to the consumer EDC
       description: "For type PARAMS_ONLY: Required data for starting a Transfer Process"
-    AssetEntryDto:
-      required:
-      - assetRequestId
-      - dataAddressProperties
-      type: object
-      properties:
-        assetRequestId:
-          type: string
-        assetRequestProperties:
-          type: object
-          additionalProperties:
-            type: object
-        dataAddressProperties:
-          type: object
-          additionalProperties:
-            type: string
-            description: At least a property 'type' must be set
-          description: At least a property 'type' must be set
-    AtomicConstraintDto:
-      required:
-      - leftExpression
-      - operator
-      - rightExpression
-      type: object
-      properties:
-        leftExpression:
-          type: string
-          description: Left part of the constraint.
-        operator:
-          $ref: '#/components/schemas/OperatorDto'
-        rightExpression:
-          type: string
-          description: Right part of the constraint.
-      description: Type-Safe OpenAPI generator friendly Constraint DTO that supports
-        an opinionated subset of the original EDC Constraint Entity.
-    ContractDefinitionRequestDto:
-      required:
-      - accessPolicyId
-      - assetsSelector
-      - contractPolicyId
-      type: object
-      properties:
-        id:
-          type: string
-        accessPolicyId:
-          type: string
-        contractPolicyId:
-          type: string
-        assetsSelector:
-          type: array
-          items:
-            $ref: '#/components/schemas/CriterionDto'
-    CreateOfferingDto:
-      type: object
-      properties:
-        assetEntry:
-          $ref: '#/components/schemas/AssetEntryDto'
-        policyDefinitionRequest:
-          $ref: '#/components/schemas/PolicyDefinitionRequestDto'
-        contractDefinitionRequest:
-          $ref: '#/components/schemas/ContractDefinitionRequestDto'
-      description: Necessary data for creating an offer
-    CriterionDto:
-      required:
-      - operandLeft
-      - operator
-      type: object
-      properties:
-        operandLeft:
-          type: object
-        operator:
-          type: string
-        operandRight:
-          type: object
-    ExpressionDto:
-      type: object
-      properties:
-        type:
-          $ref: '#/components/schemas/ExpressionType'
-        atomicConstraint:
-          $ref: '#/components/schemas/AtomicConstraintDto'
-        and:
-          type: array
-          items:
-            $ref: '#/components/schemas/ExpressionDto'
-        or:
-          type: array
-          items:
-            $ref: '#/components/schemas/ExpressionDto'
-        xor:
-          type: array
-          items:
-            $ref: '#/components/schemas/ExpressionDto'
-      description: Possible constraints for the permission
-    ExpressionType:
-      type: string
-      description: |
-        Expression types:
-        * `EMPTY` - No constraints for the policy
-        * `ATOMIC_CONSTRAINT` - A single constraint for the policy
-        * `AND` - Several constraints, all of which must be respected
-        * `OR` - Several constraints, of which at least one must be respected
-        * `XOR` - Several constraints, of which exactly one must be respected
-      enum:
-      - EMPTY
-      - ATOMIC_CONSTRAINT
-      - AND
-      - OR
-      - XOR
-    PermissionDto:
-      required:
-      - constraints
-      type: object
-      properties:
-        constraints:
-          $ref: '#/components/schemas/ExpressionDto'
-      description: Permission for this policy
-    PolicyDefinitionRequestDto:
-      required:
-      - id
-      - policy
-      type: object
-      properties:
-        id:
-          type: string
-          description: ID chosen by the user
-        policy:
-          $ref: '#/components/schemas/PolicyDto'
-      description: Data for creating a policy definition request
-    PolicyDto:
-      required:
-      - legacyPolicy
-      - permission
-      type: object
-      properties:
-        legacyPolicy:
-          type: string
-          description: Legacy JSON as built by the Management API. Will be replaced
-            in the future by a type-safe variant without polymorphisms that can be
-            used for our generated clients.
-        permission:
-          $ref: '#/components/schemas/PermissionDto'
-      description: Type-Safe OpenAPI generator friendly Policy DTO that supports an
-        opinionated subset of the original EDC Policy Entity.
     KpiResult:
       required:
       - assetsCount