Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sonobouy should not use a skip list when running conformance tests #29

Open
spiffxp opened this issue Aug 21, 2018 · 4 comments
Open

Sonobouy should not use a skip list when running conformance tests #29

spiffxp opened this issue Aug 21, 2018 · 4 comments

Comments

@spiffxp
Copy link

spiffxp commented Aug 21, 2018

Describe the solution you'd like
Sonobuoy is used as the de facto solution for running conformance tests for kubernetes. It runs tests using this container, an artifact neither produced nor owned by the kubernetes project.

This has led to a situation where the list of tests sonobuouy runs is different than the list of tests the kubernetes project guards. As conformance is a concept defined by the kubernetes project, I humbly ask that the skip list used when running conformance tests be dropped entirely, and instead fed back upstream.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

  • Sonobuoy version: (use sonobuoy version):
  • Kubernetes version: (use kubectl version):
  • Kubernetes installer & version:
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
@timothysc
Copy link
Member

Happy to change if conformance working group agrees.

/cc @dankohn

Ideally, we could push this image upstream and all be leveraging from there.

@dankohn
Copy link

dankohn commented Aug 21, 2018

I'm very supportive of the change.

@WilliamDenniss
Copy link

+1, lists of which tests that should be run for Conformance should ideally come from upstream community repos.

PS. Thanks for all the hard work on Sonobuoy, it continues to provide an outstanding service for the conformance program, making it easy for participants to run the tests and get certified!

@johnSchnake
Copy link
Contributor

In kubernetes/kubernetes#74808 you mention testing dry-run with/without the skip and it didnt seem to impact the # of tests so I think this has been accomplished.

Need to just remove the value here now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants