Adding e2e tests for cluster provisioning when OSM is used

[moadqassem]

User Story
In order to make sure that OSM is working as expected, we need to make sure that we test it thoroughly with every PR merge. That's why we need a proper infrastructure where we can build, run, integrate and test osm, to make sure everything is working as expected.

Acceptance Criteria

• Add the proper scripts to handle the deployment of OSM
• Make sure that OSM and OSM webhook are running and tested as well
• Make sure that all OSM dependencies are in place
• Run the e2e testing suite successfully with each master merge
• Introduce e2e tests for all cloud provider that we support by OSM
• Introduce e2e tests for all operating systems that we support by OSM
• Introduce e2e tests for all kubernetes versions that we support by OSM

[ahmedwaleedmalik]

With kubermatic/machine-controller#1369 in machine-controller, we switched the E2E tests to use OSM as default instead of the legacy user data. Hence, all the PRs in machine-controller are being tested against OSM as well.

Instead of reinventing the wheel, we can simply reuse those tests for OSM. The only significant change that we need for this is to build and use OSM image against the PR.

Prow job could like something like this:

  - name: pull-osm-e2e-aws
    decorate: true
    clone_uri: "ssh://[email protected]/kubermatic/operating-system-manager.git"
    extra_refs:
      - org: kubermatic
        repo: machine-controller
        base_ref: main
        clone_uri: 'ssh://[email protected]/kubermatic/machine-controller.git'
    labels:
      preset-aws: "true"
      preset-e2e-ssh: "true"
      preset-rhel: "true"
      preset-goproxy: "true"
      preset-kind-volume-mounts: "true"
      preset-docker-mirror: "true"
      preset-kubeconfig-ci: "true"
    spec:
      containers:
        - image: quay.io/kubermatic/build:go-1.20-node-18-kind-0.18-7
          command:
            # Build OSM container image and then call `./hack/ci/run-e2e-tests.sh` from machine-controller.
            - "./hack/ci/run-e2e-tests.sh"
          args:
            - "TestAWSProvisioningE2E"
          env:
            - name: CLOUD_PROVIDER
              value: aws
          securityContext:
            privileged: true
          resources:
            requests:
              memory: 7Gi
              cpu: 2
            limits:
              memory: 7Gi

NOTE: We should ideally take kubermatic/machine-controller#1626 in account as well while working on this ticket.

[ahmedwaleedmalik]

@moadqassem WDYT? I think this should significantly reduce the effort required to integrate and maintain E2E tests for OSM. We don't really need to implement anything per OS, cloud provider basis, etc. since MC already is taking care of all of those combinations.

[moadqassem]

@moadqassem WDYT? I think this should significantly reduce the effort required to integrate and maintain E2E tests for OSM. We don't really need to implement anything per OS, cloud provider basis, etc. since MC already is taking care of all of those combinations.

Yes that sounds good for me as well. I will close the epic.

[kubermatic-bot]

Issues go stale after 90d of inactivity.
After a furter 30 days, they will turn rotten.
Mark the issue as fresh with /remove-lifecycle stale.

If this issue is safe to close now please do so with /close.

/lifecycle stale