Skip to content

Latest commit

 

History

History
255 lines (224 loc) · 10.6 KB

integration_tests.md

File metadata and controls

255 lines (224 loc) · 10.6 KB

Running Integration Tests

Several tests are specified in .travis.yml. They usually follow this layout:

    stage: Some Integration Test (--platform=kubernetes --use-case=quality-gates)
    os: linux
    before_script:
      # download and install kubectl
      - curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.17.0/bin/linux/amd64/kubectl && chmod +x ./kubectl && sudo mv ./kubectl /usr/local/bin/kubectl
      - test/utils/download_and_install_keptn_cli.sh
      - test/utils/minikube_create_cluster.sh
    script:
      - kubectl get nodes || travis_terminate 1
      # finally install keptn quality gates
      - test/test_install_kubernetes_quality_gates.sh
      - keptn status
      - export PROJECT=musicshop
      - test/test_quality_gates_standalone.sh
    after_success:
      # delete google kubernetes cluster only on success (in case of an error we want to be able to dig into the cluster)
      - echo "Tests were successful, cleaning up the cluster now..."
    after_failure:
      # print some debug info
      - echo "Keptn Installation Log:"
      - cat ~/.keptn/keptn-installer.log
  • before_script: Set up kubectl, keptn cli and create a Kubernetes cluster
  • script: Verify connection to the kubernetes cluster, install Keptn (e.g., test/test_install_kubernetes_quality_gates.sh), verify connection to Keptn (keptn status) and run the actual test (e.g., test/test_quality_gates_standalone.sh)
  • after_success: Perform some cleanup (e.g., delete the cluster if necessary)
  • after_failure: Print some debug output

Run Integration Tests in Travis-CI

There are some caveats when creating a Kubernetes cluster on Travis-CI:

  • It is possible to create and connect to an external cluster (e.g., GKE, AWS, ...). However, this requires secrets for the cloud provider to be set on Travis-CI, and accessing those secrets only works for commits that are pushed within the current repository. It does not work for Pull Requests from forks.
  • It is possible to run docker on Travis-CI.
  • It is not possible to create a virtual-machine on Travis-CI. Therefore setting up Minishift or Minikube on Travis-CI only works with some specific settings (e.g., minishift start --vm-driver=generic or minikube start --vm-driver=none). With this, minishift and minikube use the local docker installation of Travis-CI. Please note, with this setup we are limited to 2 vCPUs and 4 GB of memory (see https://docs.travis-ci.com/user/reference/overview/#virtualisation-environment-vs-operating-system for details)

Above limitations lead to the following setup with Keptn's .travis.yml:

  • We test the full installation on GKE once per day on the master branch via a Travis-CI cron job.
  • We test the --use-case=quality-gates installation on Minikube, MicroK8s and Minishift for every push on the master branch (e.g., after merging a pull-request).
    • This type of installation is limited to --gateway=NodePort

Prepare your local environment to run integration tests

When running integration tests locally, we recommend using either Minikube or Minishift.

Set-up steps for Minikube

In case you plan to use Minikube, the following steps are necessary to get Keptn running on Minikube:

  1. Download and install Minikube (versions 1.4 to 1.10 should work) from https://github.com/kubernetes/minikube/releases for your operating system.

  2. Make sure you create a fresh Minikube cluster (using a VM):

    minikube stop # optional
minikube delete # optional
minikube start [--vm-driver=...] --cpus 2 --memory 4096 # or: minikube start --cpus 6 --memory 12200

    Note: In some cases you have to specify --vm-driver=... to select virtualbox, kvm, hyperv, etc... - please find out more that the official MiniKube docs

    Note: On Linux --vm-driver=docker should work in most cases

    Note: For the quality gates use case 2 CPUs and 4096 MB memory is enough, for the full installation we recommend at least 6 CPUs and 12200 MB memory

  3. Verify that everything has worked using kubectl get nodes

Set-up steps for Minishift

In case you plan to use Minishift, the following steps are necessary to get Keptn running on Minishift (1.34.2):

  1. Download and install Minishift 1.34.2 (from https://github.com/minishift/minishift/releases) for your operating system

  2. Setup Minishift profile, cpu and memory limits:

    # make sure you have a profile is set correctly
minishift profile set keptn-dev
# minimum memory required for the minishift VM
minishift config set memory 4GB # Note: use 12 GB for full installation
# the minimum required vCpus for the minishift vm
minishift config set cpus 2 # Note: use 6 for full installation
# Add new user called admin with password admin having role cluster-admin
minishift addons enable admin-user
# Allow the containers to be run with uid 0
minishift addons enable anyuid

    Note: For the quality-gates use-case 2 vCPUs and 4 GB memory is enough. For the full installation you should use more CPUs (e.g., 6) and more memory (e.g., 12 GB)

  3. Start Minishift:

    minishift start

  4. Enable admission WebHooks on your OpenShift master node:

    minishift openshift config set --target=kube --patch '{
    "admissionConfig": {
        "pluginConfig": {
            "ValidatingAdmissionWebhook": {
                "configuration": {
                    "apiVersion": "apiserver.config.k8s.io/v1alpha1",
                    "kind": "WebhookAdmission",
                    "kubeConfigFile": "/dev/null"
                }
            },
            "MutatingAdmissionWebhook": {
                "configuration": {
                    "apiVersion": "apiserver.config.k8s.io/v1alpha1",
                    "kind": "WebhookAdmission",
                    "kubeConfigFile": "/dev/null"
                }
            }
        }
    }
}'

  5. Login via oc cli (you might need to try a couple of times):

    oc login -u admin -p admin

    Note: It takes a couple of minutes before your Minishift cluster is ready. Expect error messages like

    Error from server (InternalError): Internal error occurred: unexpected response: 400

    and retry again.

  6. Set policies/permissions:

    oc adm policy --as system:admin add-cluster-role-to-user cluster-admin admin
oc adm policy  add-cluster-role-to-user cluster-admin system:serviceaccount:default:default
oc adm policy  add-cluster-role-to-user cluster-admin system:serviceaccount:kube-system:default

  7. Note down the Minishift server URL printed by oc status (e.g., https://192.168.99.101:8443)

Run the integration tests locally

Run the quality-gates integration test (with dynatrace-sli-service) locally

Pre-requesits:

  • Minikube (1.4 - 1.10) or Minishift (1.34.2) in a VM
  • kubectl
  • keptn cli
  • A local copy of this repository (keptn/keptn)
  • Linux, Mac OS or Linux Subsystem for Windows
  • Bash
  • Dynatrace Tenant with a service that has the tags FrontEnd and testdeploy
  • Dynatrace API Token that is allowed to read metrics from the Dynatrace Tenant
  • working internet connection
  • 4 GB of free memory (less is possible but you will notice a significant slowdown)
  • at least 4 CPU cores on your system (less is possible but you will notice a significant slowdown)
  1. Setup Minikube or Minishift (see above)
  2. Verify kubectl is properly configured: 
    kubectl get nodes
  3. Install Keptn
    • Minikube 
      keptn install --keptn-installer-image=keptn/installer:latest --platform=kubernetes --gateway=NodePort --verbose
      Note: You can skip the --gateway=NodePort if you run minikube tunnel
    • Minishift 
      keptn install --keptn-installer-image=keptn/installer:latest --platform=openshift --verbose
    Note: Change --keptn-installer-image=keptn/installer:latest to the desired installer version you want to use
  4. Verify the installation has worked 
    keptn status
  5. Verify which images have been deployed 
    kubectl -n keptn get deployments -owide
  6. Expose Keptn Bridge for better troubleshooting and debugging 
    keptn configure bridge --action=expose
  7. Verify accessing Keptn Bridge works
  8. Configuration for the test: 
    export PROJECT=easytravel
export SERVICE=frontend
export STAGE=hardening
export QG_INTEGRATION_TEST_DT_TENANT=<INSERT_YOUR_DT_TENANT_HERE>
export QG_INTEGRATION_TEST_DT_API_TOKEN=<INSERT_YOUR_DT_API_TOKEN_HERE>
  9. Run the test 
    bash ./test/test_quality_gates_standalone.sh

Run the full installation integration test locally

Pre-requesits:

  • Minikube (1.4 - 1.10) or Minishift (1.34.2) in a VM
  • kubectl
  • keptn cli
  • A local copy of this repository (keptn/keptn)
  • Linux, Mac OS or Linux Subsystem for Windows
  • Bash
  • working internet connection
  • 12 GB of free memory (less is possible but you will notice a significant slowdown)
  • at least 6 CPU cores on your system (less is possible but you will notice a significant slowdown)
  1. Setup Minikube or Minishift (see above)
  2. Install Keptn
    • Minikube 
      keptn install --keptn-installer-image=keptn/installer:latest --platform=kubernetes --gateway=NodePort --verbose
      Note: You can skip the --gateway=NodePort if you run minikube tunnel
    • Minishift 
      keptn install --keptn-installer-image=keptn/installer:latest --platform=openshift --verbose
    Note: Change --keptn-installer-image=keptn/installer:latest to the desired installer version you want to use
  3. Verify the installation has worked 
    keptn status
  4. Verify which images have been deployed 
    kubectl -n keptn get deployments -owide
  5. Expose Keptn Bridge for better troubleshooting and debugging 
    keptn configure bridge --action=expose
  6. Verify accessing Keptn Bridge works
  7. Configuration for the test: 
    export PROJECT=sockshop
  8. Run the test for onboarding a service 
    bash ./test/test_onboard_service.sh
  9. Run the test for sending a new artifact 
    bash ./test/test_new_artifact.sh