CI CD Information

High level overview of nightlies

We have four sets of (cron) rules that launch e2e tests for all supported OSes. Those four sets are:

• stable (version = 3.1.0 at this point)
• stable enterprise (also version = 3.1.0)
• unstable (version = blank)
• unstable enterprise (also version = blank)

The st2tests repository has branches for each release of ST2.

For the stable test runs, the version input parameter specifies both the version of ST2 to use and the “version” (read: branch) of st2tests to use. This is because if we add a feature to the development version of ST2, we then need to add feature tests to the master branch of st2tests. However, if we ran an older version of ST2 against the newer tests in st2tests master branch, the older version wouldn’t have that feature, and the feature tests in st2tests would erroneously fail the build.

To avoid that, when we test the stable version of ST2, we test it using the same version branch of st2tests. If the feature doesn’t get backported to older versions of ST2, it doesn’t get backported to older versions/branches of st2tests, and the stable tests correctly continue to pass.

So we almost always want the st2tests version/branch to be the same as the version of ST2 that we’re testing.

The exception to that is when we want to test out changes to st2tests itself using a known-good version of ST2. For that usecase, we need to expose to the user the st2tests version/branch to pull/use/install.

This all means that we could actually implement CI for the st2tests repository, to ensure that any changes we make there won’t erroneously fail when they test known-good versions of ST2. This is something that we should implement before we do a lot of the work for v3.3, since that will involve a large amount of changes to st2tests.