Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test session not marked as passed, despite all vector sets passing #939

Open
AlexThurston opened this issue Aug 4, 2020 · 1 comment
Open

Comments

@AlexThurston
Copy link

environment
Demo

testSessionId
88644

vsId
N/A

Algorithm registration
N/A

Endpoint in which the error is experienced
GET https://demo.acvts.nist.gov/acvp/v1/testSessions/88644/results

Expected behaviour
This test session has 3 vectors set which are all marked as passed but over-arching passed field is marked as false.

[
  {
    "acvVersion": "1.0"
  },
  {
    "passed": false,
    "results": [
      {
        "vectorSetUrl": "/acvp/v1/testSessions/88644/vectorSets/227932",
        "status": "passed"
      },
      {
        "vectorSetUrl": "/acvp/v1/testSessions/88644/vectorSets/227933",
        "status": "passed"
      },
      {
        "vectorSetUrl": "/acvp/v1/testSessions/88644/vectorSets/227934",
        "status": "passed"
      }
    ]
  }
]
@shaneshaffer
Copy link

The test session was last touched on 6/12, so it expired 30 days later. The "passed" value is based on our internal test session status, which is now Expired, so it winds up being false.

This is a case where the protocol specification doesn't match what we actually need to express here. We should be expressing a status, not just a simple pass/fail. While in this case the test session expired, we could wind up in similarly odd situations from other causes, like a test session being cancelled - even certifying a test session will cause this field to be inaccurate. This is something we should change to be more expressive in a future revision of the protocol spec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants