Merge pull request #322 from kbalk/mock-ports

Add option to only use moto ports when testing

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.18.0
+current_version = 0.19.0
 commit = True
 message = Bumps version to {new_version}
 tag = False

CHANGELOG.md

@@ -4,6 +4,29 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).
 
+### 0.19.0
+
+**Released**: 2021.10.25
+
+**Commit Delta**: [Change from 0.18.0 release](https://github.com/plus3it/tardigrade-ci/compare/0.18.0..0.19.0)
+
+**Summary**:
+
+*   The new environment variable "ONLY_MOTO" can be used to specify that
+    moto should be used for mock AWS services, and not LocalStack or some
+    combination of moto and LocalStack.
+
+* Updates tool versions:
+    * bats 1.50
+    * cfn-lint 0.54.3
+    * terraform 1.0.9
+    * terraform-docs 1.16.0
+    * terragrunt 0.35.4
+    * terratest 0.38.2
+    * yq 4.13.5
+    * Docker now using golang:1.17.2-buster
+    * Docker now using python:3.10.0-buster
+
 ### 0.18.0
 
 **Released**: 2021.09.13
@@ -25,9 +48,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/) and this p
     * cfn-lint 0.54.1
     * pylint 2.11.1
     * pytest 6.2.5
-    * terragrunt 0.31.10
     * terraform 1.0.7
-    * terraform-docs 0.15.0
+    * terraform-docs 1.15.0
+    * terragrunt 0.31.10
     * terratest 0.37.8
     * yamllint 1.26.3
     * yq 4.13.2

INTEGRATION_TESTING.md

@@ -1,11 +1,12 @@
 # Tardigrade-ci Integration Testing
 
-The tardigrade-ci `Makefile` provides targets to facilitate automated 
-integration testing of terraform modules through the use of a mock AWS stack.
+The tardigrade-ci `Makefile` provides targets to facilitate
+integration testing of terraform modules.  The testing can be manual or
+automated; if automated, a mock AWS stack is provided by `LocalStack` and `moto`.
 
 This document describes the integration-specific `Makefile` targets and
 the environment variables to customize those targets.  In addition,
-the steps to try out a test are described as are the potential CI/CD
+the steps to execute a test are described, as are the potential CI/CD
 changes to incorporate automated integration testing.
 
 ## Integration test-specific targets and environment variables
@@ -20,8 +21,9 @@ changes to incorporate automated integration testing.
 
 Defaults:
 
-* `LocalStack` is used for the mock AWS stack, with moto serving ports for 
-services not yet suported by `LocalStack`.
+* `LocalStack` and `moto` are used for the mock AWS stack, with moto serving
+ports for services not yet suported by `LocalStack`.  However, there is an
+option ("only-moto") to specify that only `moto` ports be used.
 * The Terraform modules used for the integration tests are expected to
 be located in the directory `tests` off the repo\'s root directory.
 
@@ -30,6 +32,8 @@ be located in the directory `tests` off the repo\'s root directory.
 | Environment variable             | Default value |
 | -------------------------------- | --------------------------------------- |
 | INTEGRATION_TEST_BASE_IMAGE_NAME | $(basename $PWD)-integration-test |
+| ONLY_MOTO                        | False; adds "--only-moto" to PYTEST_ARGS |
+| PROVIDER_ALIAS                   | No default; adds "--alias \<alias\>" to PYTEST_ARGS |
 | PYTEST_ARGS                      | |
 | TERRAFORM_PYTEST_DIR             | $PWD/tests/terraform/pytest |
 
@@ -41,13 +45,15 @@ PYTEST_ARGS.
 | Command line option | Description |
 | ------------------- | ----------------------------------------------- |
 | --nomock            | Use AWS, not mocked AWS services |
+| --alias ALIAS       | Add a provider ALIAS to the Terraform test |
 | --alternate-profile | Configure an alternate profile in addition to default profile |
+| --only-moto         | Use moto ports only for mock AWS services |
 | --tf-dir=TF_DIR     | Directory of Terraform files under test; default: './tests' |
 
 ## Executing a Terraform test
 
-The tardigrade-ci `Makefile` expects the test Terraform modules to be under
-the repo\s `tests` directory.  There can be multiple sets of tests, each
+The tardigrade-ci `Makefile` expects Terraform test modules to be under
+the repo's `tests` directory.  There can be multiple sets of tests, each
 under their own `tests` subdirectory.
 
 If a test requires an initial test setup, then those Terraform "test setup"
@@ -92,15 +98,16 @@ make mockstack/clean
 
 Alternatively, in lieu of running `make mockstack/up`, LocalStack can be
 run from the command line in a separate window. This will provide you with
-a running log (although you could view the docker log (with no debugging)
-when using `make mockstack/up`).
+a running log, although you could view the docker log (with no debugging)
+when using `make mockstack/up`.
 
 ```
 pip install localstack
 
 # Most of the AWS services are started here, but the list can be tailored
 # to just the ones needed for the test.
-DEBUG=1 SERVICES=ec2,iam,sts,s3,kms,cloudformation,cloudwatch,events,lambda,route53,ssm,sns,sqs,glue,dynamodb localstack start
+DEBUG=1 SERVICES=ec2,iam,sts,s3,kms,cloudformation,cloudwatch,events,lambda,route53,ssm,sns,sqs,glue,dynamodb \
+localstack start
 
 # Wait for a LocalStack to issue the "Ready" message before starting a test.
 # To exit LocalStack, type Ctrl-C.
@@ -118,21 +125,15 @@ These are suggested changes and may not apply to all repos:
 tests/__pycache__/
 ```
 
-2.  For a Travis workflow, logic similar to the following would
-    need to be added to the `.travis.yml` file:
+2.  For a Travis workflow, logic similar to the following can be added
+    to the `.travis.yml` file:
 
 ```bash
-stage: test
-
-name: Terraform Integration Tests
-
-language: python
-
-python:
-  - "3.8"
-
-install: make mockstack/up
-
-script: make mockstack/pytest
-after_script: make mockstack/clean
+- stage: test
+  name: Terraform Integration Tests
+  install:
+    - make docker-compose/install
+    - make mockstack/up
+  script: make mockstack/pytest
+  after_script: make mockstack/clean
 ```

Makefile

@@ -290,10 +290,11 @@ python/format:
 # Run pytests, typically for unit tests.
 PYTEST_ARGS ?=
 PYTEST_ALIAS_ARG ?= $(if $(PROVIDER_ALIAS),--alias $(PROVIDER_ALIAS),)
+PYTEST_ONLY_MOTO_ARG ?= $(if $(ONLY_MOTO),--only-moto,)
 pytest/%: | guard/program/pytest
 pytest/%:
 	@ echo "[$@] Starting Python tests found under the directory \"$*\""
-	pytest $* $(PYTEST_ARGS) $(PYTEST_ALIAS_ARG)
+	pytest $* $(PYTEST_ARGS) $(PYTEST_ALIAS_ARG) $(PYTEST_ONLY_MOTO_ARG)
 	@ echo "[$@]: Tests executed!"
 
 ## Lints terraform files

tests/terraform_pytest/conftest.py

@@ -28,6 +28,11 @@ def pytest_addoption(parser):
         action="store",
         help="Name of Terraform provider alias to include in test",
     )
+    parser.addoption(
+        "--only-moto",
+        action="store_true",
+        help="Only use moto ports for mock AWS services",
+    )
 
 
 @pytest.fixture(scope="function")
@@ -80,6 +85,9 @@ def is_mock(request, aws_credentials):
 
     if request.config.option.alternate_profile:
         pytest.exit(msg="conflicting options: 'alternate_profile' and 'nomock'")
+    elif request.config.option.only_moto:
+        pytest.exit(msg="conflicting options: 'only_moto' and 'nomock'")
+
     return False
 
 
@@ -98,6 +106,12 @@ def provider_alias(request):
     return request.config.getoption("--alias")
 
 
+@pytest.fixture(scope="session")
+def only_moto(request):
+    """Return True if only moto ports are to be used for mock services."""
+    return request.config.option.only_moto
+
+
 def pytest_generate_tests(metafunc):
     """Generate list of subdirectories under test.
 

tests/terraform_pytest/test_terraform_install.py

@@ -59,7 +59,7 @@ def make_tf_test(tf_module):
 
 
 @pytest.fixture(scope="function")
-def tf_vars(is_mock):
+def tf_vars(is_mock, only_moto):
     """Return values for variables used for the Terraform apply.
 
     Set the Terraform variables for the hostname and port to differentiate
@@ -68,7 +68,7 @@ def tf_vars(is_mock):
     return (
         {
             "mockstack_host": MOCKSTACK_HOST,
-            "mockstack_port": MOCKSTACK_PORT,
+            "mockstack_port": MOTO_PORT if only_moto else MOCKSTACK_PORT,
             "moto_port": MOTO_PORT,
         }
         if is_mock