Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: e2e tests for sbomqs #292

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

viveksahu26
Copy link
Collaborator

@viveksahu26 viveksahu26 commented Jul 16, 2024

closes: #30

This PR brings following changes:

  • Added E2E test for score command covering all the NTIA cases.
  • Added a separate dir test/data which contains the sbom files. And these has been used for test. These files has been take from here.
  • Refractor score command functionalities.
    • I did this because earlier the name of the function and functionalities of that doesn't make a proper sense.
    • For wrting function wise test, it should return values. Currently the workflow or design of score command is forward way, that means it continue calling one after other function and at the end it prints out the result. Whereas now after making changes, we return the value and print the report at the starting it self. For more to understand this diagram will help:

image

FUTURE WORK:

  • To run these test as the Pull Request is made. This would help to identify that the changes made by contributor doesn't fails test. If test fails it ensure that the changes made by the contributor changed the core functionality, which shouldn't.
  • Add package wise test. To be more concise, add function wise test for each package.
  • Lastly, to add the codecov CI which will tell how much test it covers, and many more things.

Doing all these will fulfil the requirement of ScoreCard CI test.

More detail about changes:

  • Added test for Score command core functionalities:
    • Added test for ValidateFile function covering below cases: This function validate all files and return files.
      • File does not exist
      • File exists
      • File exists but cannot be opened
    • Added Tests for HandlePaths function: This function return all files retrieve from Paths
      • Directory containing sub-dir and files
      • Non-existent path
      • Non-existent path
    • Added Tests for GetDocsAndScore function: This function returns docs, score, etc.
      • Valid SBOM file
  • Tests for NTIA category features using SBOM files:
    • Missing author
    • Missing component name
    • Missing component version
    • Missing dependencies
    • Missing component supplier name
    • Missing timestamp
    • Missing component Unique IDs

Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
Signed-off-by: Vivek Kumar Sahu <[email protected]>
@viveksahu26
Copy link
Collaborator Author

Hey @riteshnoronha please have a look at it. And if you want, we can discuss more about it.

@viveksahu26 viveksahu26 marked this pull request as draft October 2, 2024 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate tests into CI before merge
1 participant