Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/add certification process2 #781

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

garloff
Copy link
Member

@garloff garloff commented Oct 13, 2024

Move hints how the certification process works in practice into an implementation note attached to 0104.
Should be merged before SovereignCloudStack/docs#253.

Moved from docs repo in response to discussion on
PR docs/#253.

Signed-off-by: Kurt Garloff <[email protected]>
@garloff garloff added the standards Issues / ADR / pull requests relevant for standardization & certification label Oct 13, 2024
@garloff garloff self-assigned this Oct 13, 2024
Signed-off-by: Kurt Garloff <[email protected]>
Copy link
Contributor

@mbuechse mbuechse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good start that still needs some work, but more importantly, it highlights where our certification infrastructure still needs work...

Comment on lines +24 to +26
3. The operator must be a member ("shaper" or "advisor" level) of the Forum SCS-Standards in the
OSB Alliance (a non-profit) and pay the respective membership fees. Alternatively fees can
be paid without becoming a member.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be included in the main document?

Comment on lines +50 to +55
For the SCS-compatible IaaS v5 standard, the providers must — if they implement availability zones
at all (which is optional) — guarantee certain levels of independence for these. This can not
be fully tested by an automated test. The process thus envisions that providers must create some
documentation on the physical infrastructure and how it maps to availability zones and declare that
this documentation reflects the truth. SCS will review the docs and judge whether they meet the
criteria. In case of doubt, audits can be performed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be dropped, because it is redundant and it ages too quickly. Suggestion:

  • make the respective test output a note that documentation is required (mind you that it will show up as MISSING anyway, because the automated tests won't produce anything for the test cases in question)

On second thought, this is probably what is going to happen anyway. This paragraph is just a bit ahead of its time. Still, it should be replaced by a general remark that some test cases will turn up missing because documentation must be handed in.

Comment on lines +57 to +68
## Forum SCS-Standards @ OSBA

The SCS brand belongs to the Open Source Business Alliance e.V. (OSBA), an non-profit organization and
association for the Open Source Industry in Germany. After the completion of the funded SCS project
in the OSBA on 2024-12-31, the OSBA sets up the Forum SCS-Standards
which performs the work to evolve the SCS standards, develops the tests and perform the certification
process and thus becomes the SCS certification body.

Members of the OSBA can become also member of the Forum SCS-Standards for an additional membership
fee, providing the financial resources for the Forum SCS-Standards to do its work. Membership in the
OSBA is open to any organization that supports the goals of the OSBA.
Alternatively, a certification fee can be paid without any membership.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also ahead of its time. It sneakily uses present tense for something that is in the future:

After the completion of the funded SCS project in the OSBA on 2024-12-31, the OSBA sets up the Forum SCS-Standards

Maybe this should be a blog post before we add it to the official docs.

Comment on lines +78 to +81
Note that for public clouds, there will be a nightly job that tests the cloud for compliance, which will be
triggered by SCS infrastructure (zuul). For this, access to a tenant on the cloud needs
to be provided free of charge. (This only requires very low quota, one VM is created for a minute
in one of the tests.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This language is too strong. We offer to orchestrate the tests for the partners as a service, but (from my POV) they should be free to choose whether they want to run the tests themselves. It's a rather simple process that we are testing and refining right now with AOV.

Comment on lines +124 to +125
In the compliance manager (executing tests via zuul), we will add links to the log
files directly on the table, so it will be even easier to find the relevant log files.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a complication here. Special privileges are required to

  • see unfiltered test results (we currently hide fails for a period of 7 days if they aren't explicitly approved by us)
  • see log files

We have to hand out "API keys" to our partners so they can do these things.

garloff and others added 3 commits October 15, 2024 15:05
@garloff
Copy link
Member Author

garloff commented Oct 16, 2024

@mbuechse: I agree with your other comments as well. They require a bit more thinking than just merging a proposal unfortunately. Not sure if I can find the time b/w the Broadway Shows 😉
If you have improvements in mind, please don't feel shy at all to implement them and push them to the branch!

Signed-off-by: Matthias Büchse <[email protected]>
@mbuechse mbuechse linked an issue Oct 24, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
standards Issues / ADR / pull requests relevant for standardization & certification
Projects
Status: Backlog
Development

Successfully merging this pull request may close these issues.

scs-0004-v1: include more details about manual checks
2 participants