Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: run e2e tests for docker provider #1965

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

CI: run e2e tests for docker provider #1965

wants to merge 6 commits into from

Conversation

wainersm
Copy link
Member

This adds an initial version of CI for docker provider.

It was created a new workflow (e2e_docker.yaml) that run the e2e tests for docker on x86_64. In order to keep this implementation simple, it isn't installed KBS. Also the tests will be executed only alongside the libvirt nightly job, so not triggered on pull requests. The final status of libvirt nightly job shouldn't change if the docker tests fail because the jobs were marked continue-on-error, this way we will be able to proper test these changes on "production" (reason why I also didn't rename daily e2e tests for libvirt to something generic).

As a pre-req to run the e2e tests for docker, I had to ensure the podvm image for docker is built and pushed to the registry. So the very first commits changes the podvm_builder, podvm_binaries, podvm_mkosi, and podvm workflows to reuse existing code. Here I am trying really hard to not introduce new workflows. However, I will need to change further those workflows (on a respin PR) to make them more flexible to allow, for example, run the docker e2e tests on pull requests.

I managed to tests these changes on my forked tree, being the last execution https://github.com/wainersm/cc-cloud-api-adaptor/actions/runs/10171362457 . From that link you will notice that Build mkosi image passed but test-docker failed, the error being:

# github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e [github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e.test]
Error: test/e2e/docker_test.go:104:23: keyBrokerService.EnableKbsCustomizedPolicy undefined (type *provisioner.KeyBrokerService has no field or method EnableKbsCustomizedPolicy)
Error: test/e2e/docker_test.go:108:23: keyBrokerService.EnableKbsCustomizedPolicy undefined (type *provisioner.KeyBrokerService has no field or method EnableKbsCustomizedPolicy)

@bpradipt ^^^ I need help to understand what's going on.

Fixes: #1946

@liudalibj
Copy link
Member

I managed to tests these changes on my forked tree, being the last execution https://github.com/wainersm/cc-cloud-api-adaptor/actions/runs/10171362457 . From that link you will notice that Build mkosi image passed but test-docker failed, the error being:

# github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e [github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e.test]
Error: test/e2e/docker_test.go:104:23: keyBrokerService.EnableKbsCustomizedPolicy undefined (type *provisioner.KeyBrokerService has no field or method EnableKbsCustomizedPolicy)
Error: test/e2e/docker_test.go:108:23: keyBrokerService.EnableKbsCustomizedPolicy undefined (type *provisioner.KeyBrokerService has no field or method EnableKbsCustomizedPolicy)

@bpradipt ^^^ I need help to understand what's going on.

the EnableKbsCustomizedPolicy be replaced with EnableKbsCustomizedResourcePolicy or EnableKbsCustomizedAttestationPolicy
https://github.com/confidential-containers/cloud-api-adaptor/blob/main/src/cloud-api-adaptor/test/provisioner/trustee_kbs.go#L427

@wainersm
Copy link
Member Author

wainersm commented Aug 5, 2024

FYI @ldoktor

@bpradipt
Copy link
Member

@wainersm overall this looks good.
With the focus to switch to a single distro (fedora) and mkosi as part of 0.10.0, do you think this PR need to wait till the transition?

@stevenhorsman
Copy link
Member

the EnableKbsCustomizedPolicy be replaced with EnableKbsCustomizedResourcePolicy or EnableKbsCustomizedAttestationPolicy
https://github.com/confidential-containers/cloud-api-adaptor/blob/main/src/cloud-api-adaptor/test/provisioner/trustee_kbs.go#L427

FYI I hit this with the auth tests, so was working on a fix in a commit there, but have extracted it out to #2000 as it's useful here as well.

@wainersm
Copy link
Member Author

Hi @bpradipt !

@wainersm overall this looks good. With the focus to switch to a single distro (fedora) and mkosi as part of 0.10.0, do you think this PR need to wait till the transition?

I was thinking this could spark the transition actually, as this work introduces mkosi builds into the CI and docker e2e tests will leverage these builds which will give us (once passing) confidence on replacing ubuntu (packer). What do you think?

@bpradipt
Copy link
Member

Hi @bpradipt !

@wainersm overall this looks good. With the focus to switch to a single distro (fedora) and mkosi as part of 0.10.0, do you think this PR need to wait till the transition?

I was thinking this could spark the transition actually, as this work introduces mkosi builds into the CI and docker e2e tests will leverage these builds which will give us (once passing) confidence on replacing ubuntu (packer). What do you think?

I see.. Makes sense to me..

@wainersm
Copy link
Member Author

Hi @stevenhorsman !

the EnableKbsCustomizedPolicy be replaced with EnableKbsCustomizedResourcePolicy or EnableKbsCustomizedAttestationPolicy
https://github.com/confidential-containers/cloud-api-adaptor/blob/main/src/cloud-api-adaptor/test/provisioner/trustee_kbs.go#L427

FYI I hit this with the auth tests, so was working on a fix in a commit there, but have extracted it out to #2000 as it's useful here as well.

Thanks for the fix. I cherry-picked the commit to my tree and could now run docker job from this PR: https://github.com/wainersm/cc-cloud-api-adaptor/actions/runs/10354707202/job/28660607725

The tests, even the simplest one, are failing. It might be the build of mkosi or something else, I'm investigating.

@wainersm
workflows: build fedora builder and binaries
03b0e65 
The fedora-based (x86_64) podvm-builder and podvm-binaries images will be
built from the podvm_builder.yaml and podvm_binaries.yaml workflows. So the
binaries image will be used to build the mkosi podvm.

Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
@wainersm
workflows: make podvm_mkosi callable
c161725 
So that the podvm_mkosi.yaml workflow can be reused in podvm.yaml
to build mkosi artifacts.

Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
@wainersm
workflows: trigger mkosi image build
74cc785 
Changed podvm.yaml to call the podvm_mkosi.yaml so that mkosi images
will be built alongside the packer ones on pull-request, manually and
nightly. Because it might introduce instability on existing workflows,
it was set continue-on-error to disregard any error on mkosi builds.

Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
@wainersm
workflows: build podvm-docker-fedora-image in podvm_mkosi
ca3fbd9 
Re-use the resources/binaries-tree binaries already fetched in
the podvm_mkosi.yaml to build the podvm image for the docker provider.

Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
@wainersm
workflows: add test e2e workflow for docker
1b01072 
Add a callable workflow that run the e2e tests for the docker provider. This
workflow is similar to e2e_libvirt.yaml.

Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
@wainersm
workflows: enable docker e2e test in e2e_run_all
d217fff 
This will make the e2e tests for docker to run. Initially it's disabled
on pull requests, so only running with the nightly and manual triggers.

Notice that's set continue-on-error so that the e2e_run_all workflow
exit status won't change, i.e. any failure on e2e_docker is disregarded.

Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
@wainersm
Copy link
Member Author

Rebased to get the latest fixes. docker e2e tests running but still failing....

@bpradipt
Copy link
Member

bpradipt commented Sep 5, 2024

@wainersm is this ready for review ?

@stevenhorsman
Copy link
Member

@wainersm is this ready for review ?

Last I heard the tests were failing. I'm also trying to make it mutli-arch, but that can come as a follow up PR, so doesn't need to block this.

@wainersm
Copy link
Member Author

wainersm commented Sep 9, 2024

@wainersm is this ready for review ?

Last I heard the tests were failing. I'm also trying to make it mutli-arch, but that can come as a follow up PR, so doesn't need to block this.

It was ready to review but I wasn't able to run the tests on docker provider, even the simple Pod test used to fail. Now we have @stevenhorsman at al working on #2019 that's touching on same workflows as this. I talked with Steve and I will put this on hold, prioritizing 2019.

@wainersm wainersm added the hold label Sep 9, 2024
@stevenhorsman stevenhorsman mentioned this pull request Sep 9, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
hold
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Run e2e tests for docker provider on CI
4 participants
@wainersm @liudalibj @bpradipt @stevenhorsman