TESTING.adoc

Integration Testing for Add-ons

• Introduction
• Prerequisites
• Running Tests
• Writing Tests

Introduction

This repository uses the integration test suite from the minishift/minishift repository to test the functionality of Minishift add-ons. The test suite is written in Go and uses the Gherkin language to define test sets. This allows add-on contributors to write tests effectively using predefined, understandable sentences. For example:

  Scenario: Applying the CORS add-on
    Given Minishift has state "Running"
     When executing "minishift addons apply cors" succeeds
     Then stdout should contain "CORS is now allowed from any address"

Note	We recommend aligning white space to the ends of the Gherkin keywords Given, When, Then, and And.

See the Gherkin wiki for more information about Gherkin. More in-depth information about the Minishift integration test suite can be found in the Developing Minishift documentation.

Structure of Tests

Feature files which define the test cases for each individual add-on are stored in the test/integration/features directory. The feature files follow the naming convention of <addon-name>.feature and begin with the tag @<addon-name>.

Integration tests are implemented in the test/integration/integration_test.go file. This file imports the test suite from the Minishift repository and can be used for the import of additional step definitions.

Prerequisites

You must do the following to run the tests successfully:

• Install Minishift and its prerequisites.

• Install a recent Go distribution (>=1.8).

• Install the dep package with the following: go get -u github.com/golang/dep/cmd/dep

• git clone this repository into $GOPATH/src/github.com/minishift/minishift-addons.

• Get the latest minishift binary from https://github.com/minishift/minishift/releases/latest and place it in testing/bin.

• Vendor the needed Go packages by runining make vendor in the root directory of the minishift-addons repository.

Note	We recommend vendoring the packages after every git pull from the minishift-addons master branch.

Running Tests

To run integration tests for a specific add-on, go to the root directory of the minishift-addons repository and run the following command:

make integration ADDON=<name-of-addon>

This command will start tests on all Gherkin feature files present in tests/integration/features which contain the tag @<name-of-addon>. For most add-ons, this will mean using one feature file located at tests/integration/features/<name-of-addon>.feature.

To run the tests on all add-ons, you can use the following:

make integration

Additional Options

It is possible to run the tests with a Minishift binary located outside of the default location of testing/bin. To do so, use the MINISHIFT_BINARY parameter with the make integration target. For example:

make integration ADDON=xpaas MINISHIFT_BINARY=/home/user/minishift/minishift

Writing Tests

Create a new file named tests/integration/features/<name-of-add-on>.feature to create a new integration test for an add-on. The example.feature file demonstrates the basic structure of an integration test for an example add-on and may be used as a starting point.

Features should contain at least 6 key parts which secure that the tested add-on can be installed, applied, removed and uninstalled:

• Installation of the add-on: executing "minishift addons install ../../add-ons/<name-of-addon>" succeeds

• Starting of Minishift: executing "minishift start" succeeds

• Application of the add-on: executing "minishift addons apply <name-of-addon>" succeeds

• Removal of the add-on: executing "minishift addons remove <name-of-addon>" succeeds

• Add-on uninstallation: executing "minishift addons uninstall <name-of-addon>" succeeds

• Deletion of the Minishift instance: executing "minishift delete --force" succeeds

Advanced Steps

The Minishift testsuite package contains step definitions which can cover the functionality of an add-on in more detail. For example:

• Evaluation of Minishift commands: stdout of command "minishift ssh — docker ps" contains "my-container"

• Execution of oc commands: executing "oc login --username=developer --password=developer" succeeds

• Checking for OpenShift console accessibility: "status code" of HTTP request to "/console" of OpenShift instance is equal to "200"

• Checking if the HTTP endpoint of an application is accessible and contains the expected data: "body" of HTTP request to "/" of service "ruby-ex" in namespace "ruby" contains "Welcome to your Ruby application on OpenShift"

Check the implementation of the provided step definitions in Minishift’s testsuite package for more information.