Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yardstick should be able to configure per-image max year for result sets #378

Closed
willmurphyscode opened this issue Aug 26, 2024 · 3 comments
Assignees

Comments

@willmurphyscode
Copy link
Contributor

What would you like to be added:

I would like to be able to tell yardstick that the default max year for a given image is later (or earlier) than the default_max_year specified in the top level of the yardstick config.

Why is this needed:

Some images don't have enough vulnerabilities to compare if we exclude CVEs after 2021. This difficulty is especially true when adding new distros or versions of distros that GA'ed long after 2021.

Additional context:

Discovered trying to add quality gates for anchore/vunnel#569.

@willmurphyscode willmurphyscode added the enhancement New feature or request label Aug 26, 2024
@willmurphyscode willmurphyscode self-assigned this Aug 26, 2024
@willmurphyscode
Copy link
Contributor Author

From my initial reading, the best approach is to add a max_year field to the result_set config object. Then, yardstick-based quality gates can have a 2021 result set and a 2022 result set, and we can just move images from one to another as enough 2022 labels are added to the image.

Other options considered

Make a per-image config

I could change the images field on the result_set config to be a list[ImageConfig] instead of a list[str] and could be the year there, and write some custom parsing so that a bare string in the config gets inflated to ImageConfig(img: <the string>, max_year: cfg.default_max_year).

Override in the individual quality gates

Yardstick quality gates are typically implemented as little python scripts themselves (see #126, grype's exampe, vunnel's example), and I could put code in these gate.py files to check which image is being scanned and update ask for a yardstick comparison with that max year instead of the default max year.

This option probably requires the least design and might be easiest, but it amounts to hard-coding around the lack of this feature everywhere we want it. It will be cleaner and easier long term to make this a first-class feature in yardstick.

@anchore anchore deleted a comment Aug 26, 2024
@willmurphyscode
Copy link
Contributor Author

The big disadvantage of making it a setting on the result sets is running multiple result sets everywhere, while most gates only run one result set today.

I'll put up an example PR making grype or vunnel use this method and see how it looks.

@willmurphyscode
Copy link
Contributor Author

#126 added this on a per-result-set level

@willmurphyscode willmurphyscode added changelog-ignore and removed enhancement New feature or request labels Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

1 participant