Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: e2e framework #112

Merged
merged 19 commits into from
Jan 26, 2024
Merged

test: e2e framework #112

merged 19 commits into from
Jan 26, 2024

Conversation

amagyar-iohk
Copy link
Contributor

Overview

@goncalo-frade-iohk @antonbaliasnikov this is the draft of the e2e framework. few things are not ready yet and others could be improved

but the main points are the junit and html reports that create the output files that we are going to use for the releases. they are created using the targetDirectory folder -- which should point to the source code folder (E2E/e2etests/Target)

XML example:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites id="20240106_001826" name="swift-e2e-results - 06/01/2024, 0:18" time="137.0450 seconds" tests="6" failures="0">
    <testsuite id="750C5BC3-1E39-41E8-934F-5F202473D5A6" name="Receive anonymous credential" time="20.6792 seconds" tests="1" failures="0">
        <testcase id="19EA70E1-4C8E-4A2E-B076-DB8DA81BDDE6" name="Receive one anonymous credential" time="20.6764 seconds"></testcase>
    </testsuite>
    <testsuite id="385E4029-5B23-444F-A2D5-3618DE3E251E" name="Create connection" time="4.9239 seconds" tests="1" failures="0">
        <testcase id="2F6D8670-0089-497E-9817-DFB6C563315A" name="Create connection between Cloud and Edge agents" time="4.9232 seconds"></testcase>
    </testsuite>
    <testsuite id="BFF3566A-9376-4406-ABA2-0579F7F48525" name="Receive verifiable credential" time="83.5048 seconds" tests="3" failures="0">
        <testcase id="55EECD3A-9CCF-4BA9-9E68-B2C4B15CA929" name="Receive multiple verifiable credentials at once" time="17.3264 seconds"></testcase>
        <testcase id="7C5C8B10-121E-4335-870B-1F74B63359D4" name="Receive multiple verifiable credentials sequentially" time="45.4101 seconds"></testcase>
        <testcase id="94713D0C-4691-4F31-A224-97B2205D2EFF" name="Receive one verifiable credential" time="20.7514 seconds"></testcase>
    </testsuite>
    <testsuite id="B452F1A5-4D4F-4800-8B02-E79977C98C1E" name="Provide proof of request" time="27.7475 seconds" tests="1" failures="0">
        <testcase id="4C249232-AC24-4BAD-8F98-FC71785EC473" name="Respond to request proof" time="27.7455 seconds"></testcase>
    </testsuite>
</testsuites>

(checkout the output using online parser like this: https://lotterfriends.github.io/online-junit-parser)

"HTML" example (it's a txt file for now):

Executed 4 features
  Feature: Receive anonymous credential
    Scenario: Receive one anonymous credential
      (✔) Cloud Agent is connected to Edge Agent
            Cloud Agent create a connection using OEA API
            Cloud Agent remembers [invitation]
            Cloud Agent remembers [connectionId]
            Cloud Agent recalls [invitation]
            Edge Agent remembers [invitation]
            Edge Agent recalls [invitation]
            Edge Agent parses an OOB invitation using Swift SDK
            Edge Agent accepts an invitation using Swift SDK
            Cloud Agent recalls [connectionId]
            Cloud Agent waits until connection state to be ConnectionResponseSent using OEA API
      (✔) Cloud Agent offers an anonymous credential
            Cloud Agent recalls [connectionId]
            Cloud Agent offers an anonymous credential to 7f1fb47b-524e-4a6f-8cde-72d9ed6a13f7 using OEA API
            Cloud Agent remembers [recordId]
      (✔) Edge Agent should receive the credential
            Edge Agent waits until credential offer count to be 1 using Swift SDK
      (✔) Edge Agent accepts the credential
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
      (✔) Cloud Agent should see the credential was accepted
            Cloud Agent recalls [recordId]
            Cloud Agent waits until credential state is CredentialSent using OEA API
      (✔) Edge Agent wait to receive 1 issued credentials
            Edge Agent waits until wait for issued credentials to be 1 using Swift SDK
      (✔) Edge Agent process 1 issued credentials
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
    Status: SUCCESS

  Feature: Create connection
    Scenario: Create connection between Cloud and Edge agents
      (✔) Cloud Agent has a connection invitation
            Cloud Agent create a connection using OEA API
            Cloud Agent remembers [invitation]
            Cloud Agent remembers [connectionId]
      (✔) Cloud Agent shares invitation to Edge Agent
            Cloud Agent recalls [invitation]
            Edge Agent remembers [invitation]
      (✔) Edge Agent connects through the invite
            Edge Agent recalls [invitation]
            Edge Agent parses an OOB invitation using Swift SDK
            Edge Agent accepts an invitation using Swift SDK
      (✔) Cloud Agent should have the connection status updated to 'ConnectionResponseSent'
            Cloud Agent recalls [connectionId]
            Cloud Agent waits until connection state to be ConnectionResponseSent using OEA API
    Status: SUCCESS

  Feature: Receive verifiable credential
    Scenario: Receive multiple verifiable credentials at once
      (✔) Cloud Agent is connected to Edge Agent
            Cloud Agent create a connection using OEA API
            Cloud Agent remembers [invitation]
            Cloud Agent remembers [connectionId]
            Cloud Agent recalls [invitation]
            Edge Agent remembers [invitation]
            Edge Agent recalls [invitation]
            Edge Agent parses an OOB invitation using Swift SDK
            Edge Agent accepts an invitation using Swift SDK
            Cloud Agent recalls [connectionId]
            Cloud Agent waits until connection state to be ConnectionResponseSent using OEA API
      (✔) Edge Agent accepts 3 credentials offer at once from Cloud Agent
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to 5bbfdb03-6ddf-47cc-bb1f-68d97d40b28d using OEA API
            Cloud Agent remembers [recordId]
            Cloud Agent recalls [recordId]
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to 5bbfdb03-6ddf-47cc-bb1f-68d97d40b28d using OEA API
            Cloud Agent remembers [recordId]
            Cloud Agent recalls [recordId]
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to 5bbfdb03-6ddf-47cc-bb1f-68d97d40b28d using OEA API
            Cloud Agent remembers [recordId]
            Cloud Agent recalls [recordId]
            Cloud Agent remembers [recordIdList]
            Edge Agent waits until credential offer count to be 3 using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
      (✔) Cloud Agent should see all credentials were accepted
            Cloud Agent recalls [recordIdList]
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent waits until credential state is CredentialSent using OEA API
      (✔) Edge Agent wait to receive 3 issued credentials
            Edge Agent waits until wait for issued credentials to be 3 using Swift SDK
      (✔) Edge Agent process 3 issued credentials
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
    Status: SUCCESS

    Scenario: Receive multiple verifiable credentials sequentially
      (✔) Cloud Agent is connected to Edge Agent
            Cloud Agent create a connection using OEA API
            Cloud Agent remembers [invitation]
            Cloud Agent remembers [connectionId]
            Cloud Agent recalls [invitation]
            Edge Agent remembers [invitation]
            Edge Agent recalls [invitation]
            Edge Agent parses an OOB invitation using Swift SDK
            Edge Agent accepts an invitation using Swift SDK
            Cloud Agent recalls [connectionId]
            Cloud Agent waits until connection state to be ConnectionResponseSent using OEA API
      (✔) Edge Agent accepts 3 credential offer sequentially from Cloud Agent
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to e30c83dd-81e2-4e9a-ba8d-b2d22a13def3 using OEA API
            Cloud Agent remembers [recordId]
            Edge Agent waits until credential offer count to be 1 using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
            Cloud Agent recalls [recordId]
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to e30c83dd-81e2-4e9a-ba8d-b2d22a13def3 using OEA API
            Cloud Agent remembers [recordId]
            Edge Agent waits until credential offer count to be 1 using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
            Cloud Agent recalls [recordId]
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to e30c83dd-81e2-4e9a-ba8d-b2d22a13def3 using OEA API
            Cloud Agent remembers [recordId]
            Edge Agent waits until credential offer count to be 1 using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
            Cloud Agent recalls [recordId]
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent remembers [recordIdList]
      (✔) Cloud Agent should see all credentials were accepted
            Cloud Agent recalls [recordIdList]
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Cloud Agent waits until credential state is CredentialSent using OEA API
      (✔) Edge Agent wait to receive 3 issued credentials
            Edge Agent waits until wait for issued credentials to be 3 using Swift SDK
      (✔) Edge Agent process 3 issued credentials
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
    Status: SUCCESS

    Scenario: Receive one verifiable credential
      (✔) Cloud Agent is connected to Edge Agent
            Cloud Agent create a connection using OEA API
            Cloud Agent remembers [invitation]
            Cloud Agent remembers [connectionId]
            Cloud Agent recalls [invitation]
            Edge Agent remembers [invitation]
            Edge Agent recalls [invitation]
            Edge Agent parses an OOB invitation using Swift SDK
            Edge Agent accepts an invitation using Swift SDK
            Cloud Agent recalls [connectionId]
            Cloud Agent waits until connection state to be ConnectionResponseSent using OEA API
      (✔) Cloud Agent offers a credential
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to 77dd2fd9-29e3-43dc-86cc-da0101bcf232 using OEA API
            Cloud Agent remembers [recordId]
      (✔) Edge Agent should receive the credential
            Edge Agent waits until credential offer count to be 1 using Swift SDK
      (✔) Edge Agent accepts the credential
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
      (✔) Cloud Agent should see the credential was accepted
            Cloud Agent recalls [recordId]
            Cloud Agent waits until credential state is CredentialSent using OEA API
      (✔) Edge Agent wait to receive 1 issued credentials
            Edge Agent waits until wait for issued credentials to be 1 using Swift SDK
      (✔) Edge Agent process 1 issued credentials
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
    Status: SUCCESS

  Feature: Provide proof of request
    Scenario: Respond to request proof
      (✔) Cloud Agent is connected to Edge Agent
            Cloud Agent create a connection using OEA API
            Cloud Agent remembers [invitation]
            Cloud Agent remembers [connectionId]
            Cloud Agent recalls [invitation]
            Edge Agent remembers [invitation]
            Edge Agent recalls [invitation]
            Edge Agent parses an OOB invitation using Swift SDK
            Edge Agent accepts an invitation using Swift SDK
            Cloud Agent recalls [connectionId]
            Cloud Agent waits until connection state to be ConnectionResponseSent using OEA API
      (✔) Edge Agent has 1 credentials issued by Cloud Agent
            Cloud Agent recalls [connectionId]
            Cloud Agent offers a credential to d04f6cf3-f67a-4519-8f8a-48fbf779b3a3 using OEA API
            Cloud Agent remembers [recordId]
            Edge Agent waits until credential offer count to be 1 using Swift SDK
            Edge Agent gets the first credential offer using Swift SDK
            Edge Agent removes it from list using Swift SDK
            Edge Agent create a new prism DID using Swift SDK
            Edge Agent request a credential using Swift SDK
            Edge Agent send a message using Swift SDK
            Cloud Agent recalls [recordId]
            Cloud Agent waits until credential state is CredentialSent using OEA API
            Edge Agent waits until wait for issued credentials to be 1 using Swift SDK
            Edge Agent get an issued credential using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent process the credential using Swift SDK
      (✔) Cloud Agent asks for present-proof
            Cloud Agent recalls [connectionId]
            Cloud Agent ask a presentation proof to d04f6cf3-f67a-4519-8f8a-48fbf779b3a3 using OEA API
            Cloud Agent remembers [presentationId]
      (✔) Edge Agent sends the present-proof
            Edge Agent waits until wait for present proof request using Swift SDK
            Edge Agent get a verifiable credential using Swift SDK
            Edge Agent get proof request using Swift SDK
            Edge Agent remove it from list using Swift SDK
            Edge Agent make message using Swift SDK
            Edge Agent send message using Swift SDK
      (✔) Cloud Agent should see the present-proof is verified
            Cloud Agent recalls [presentationId]
            Cloud Agent waits until present proof state is PresentationVerified using OEA API
    Status: SUCCESS

Results running in xcode
image

Fixes #

Checklist

My PR contains...

  • No code changes (changes to documentation, CI, metadata, etc.)
  • Bug fixes (non-breaking change which fixes an issue)
  • Improvements (misc. changes to existing features)
  • Features (non-breaking change which adds functionality)

My changes...

  • are breaking changes
  • are not breaking changes
  • If yes to the above: I have updated the documentation accordingly

Documentation

  • My changes do not require a change to the project documentation
  • My changes require a change to the project documentation
  • If yes to the above: I have updated the documentation accordingly

Tests

  • My changes can not or do not need to be tested
  • My changes can and should be tested by unit and/or integration tests
  • If yes to the above: I have added tests to cover my changes
  • If yes to the above: I have taken care to cover edge cases in my tests

@@ -23,6 +23,7 @@ class KeychainMock: KeychainStore, KeychainProvider {
service: String,
account: String
) throws {
print("Add KEY", key, service, account)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just saw I have to remove this

E2E/e2eTests/Source/Abilities/Sdk.swift Outdated Show resolved Hide resolved
E2E/e2eTests/Source/Abilities/Sdk.swift Outdated Show resolved Hide resolved
E2E/e2eTests/Source/Abilities/Sdk.swift Show resolved Hide resolved
E2E/e2eTests/Source/Abilities/Sdk.swift Show resolved Hide resolved
Comment on lines +151 to +219
static private func createSecretsStream(
keyRestoration: KeyRestoration,
pluto: Pluto,
castor: Castor
) -> AnyPublisher<[Secret], Error> {
pluto.getAllPeerDIDs()
.first()
.flatMap { array in
Future {
try await array.asyncMap { did, privateKeys, _ in
let privateKeys = try await privateKeys.asyncMap {
try await keyRestoration.restorePrivateKey($0)
}
let secrets = try parsePrivateKeys(
did: did,
privateKeys: privateKeys,
castor: castor
)

return secrets
}
}
}
.map {
$0.compactMap {
$0
}.flatMap {
$0
} }
.eraseToAnyPublisher()
}

static private func parsePrivateKeys(
did: DID,
privateKeys: [PrivateKey],
castor: Castor
) throws -> [Domain.Secret] {
return try privateKeys
.map { $0 as? (PrivateKey & ExportableKey) }
.compactMap { $0 }
.map { privateKey in
let ecnumbasis = try castor.getEcnumbasis(did: did, publicKey: privateKey.publicKey())
return (did, privateKey, ecnumbasis)
}
.map { did, privateKey, ecnumbasis in
try parseToSecret(
did: did,
privateKey: privateKey,
ecnumbasis: ecnumbasis
)
}
}

static private func parseToSecret(did: DID, privateKey: PrivateKey & ExportableKey, ecnumbasis: String) throws -> Domain.Secret {
let id = did.string + "#" + ecnumbasis
let jwk = privateKey.jwk
guard
let dataJson = try? JSONEncoder().encode(jwk),
let stringJson = String(data: dataJson, encoding: .utf8)
else {
throw CommonError.invalidCoding(message: "Could not encode privateKey.jwk")
}
return .init(
id: id,
type: .jsonWebKey2020,
secretMaterial: .jwk(value: stringJson)
)
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to do this? Can't the SDK do this for you? What are your trying to achieve? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is part of what you did to help me. It's being used on the creation of the agent

E2E/e2eTests/Source/Interactions/UseSdk.swift Outdated Show resolved Hide resolved
Copy link

@antonbaliasnikov antonbaliasnikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't say much, to be honest, as I'm not Swift specialist, but from my perspective, it looks very good.

I would concentrate on a bit of documentation for configuring and running the tests + quick guide on how to add a new scenario.

For sure, JUnit report is lacking, but it can come later and included into the CI/CD checks.

But it's very nice job and we finally have something for Swift which is awesome and @goncalo-frade-iohk will be happy :)

@amagyar-iohk amagyar-iohk marked this pull request as ready for review January 23, 2024 19:16
Copy link
Member

@yshyn-iohk yshyn-iohk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks beautiful even if I'm not an expert in Swift 👍

let base64data: String = String(invitationUrl.split(separator: "?_oob=").last!)
let decodedData = Data(base64Encoded: base64data)!
let json = try (JSONSerialization.jsonObject(with: decodedData, options: []) as? [String: Any])!
let from = (json["from"] as? String)!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just FIY its preferable to use Codable and you could use our Message structure to parse this. But since this is a test framework its good enough :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean, can you provide an example?

@amagyar-iohk amagyar-iohk merged commit 78c53e0 into main Jan 26, 2024
3 checks passed
@amagyar-iohk amagyar-iohk deleted the test/e2e-rebased branch January 26, 2024 13:17
@amagyar-iohk amagyar-iohk restored the test/e2e-rebased branch January 26, 2024 13:24
@amagyar-iohk amagyar-iohk deleted the test/e2e-rebased branch January 26, 2024 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants