testing-best-practices.md

Testing Best Practices

Types of test

• Unit tests: In Terraform terminology they refer to tests that validate a resource schema. That is done automatically here for all resources and data sources using Terraform Framework Plugin. Additionally, we have general unit testing for testing a resource or unit without calling external systems like MongoDB Atlas.
• Acceptance (acc) tests: In Terraform terminology they refer to the use of real Terraform configurations to exercise the code in plan, apply, refresh, and destroy life cycles (real infrastructure resources are created as part of the test), more info here.
• Migration (mig) tests: These tests are designed to ensure that after an upgrade to a new Atlas provider version, user configs do not result in unexpected plan changes, more info here. Migration tests are a subset of Acceptance tests.

Test Organization

• A resource and associated data sources are implemented in a folder that is also a Go package, e.g. advancedcluster implementation is in internal/service/advancedcluster
• We enforce "black box" testing, tests must be in a separate "_test" package, e.g. advancedcluster tests are in advancedcluster_test package.
• Acceptance and general unit tests are in corresponding _test.go file as the resource or data source source file. If business logic is extracted into a separate file, unit testing for that logic will be including in its associated _test.go file, e.g. state_transition_search_deployment_test.go.
• Migration tests are in _migration_test.go files.
• When functions are in their own file because they are shared by resource and data sources, a test file can be created to test them, e.g. model_alert_configuration_test.go has tests for model_alert_configuration.
• All resource folders must have a main_test.go file to handle resource reuse lifecycle, e.g. here.
• internal/testutils/acc contains helper test functions for Acceptance tests.
• internal/testutils/mig contains helper test functions specifically for Migration tests.
• internal/testutils/replay contains helper test functions for Hoverfly. Hoverfly is used to capture and replay HTTP traffic with MongoDB Atlas.

Unit tests

• Unit tests must not create Terraform resources or use external systems, e.g unit tests using Atlas Go SDK must not call MongoDB Atlas.
• Mock of specific interfaces like admin.ProjectsApi is prefered to use the whole client, e.g. resource_project.go
• Testify Mock is used for test doubles.
• Altlas Go SDK mocked interfaces are generated in mockadmin package using Mockery, example of use in resource_project_test.go.

Acceptance tests

• There must be at least one basic acceptance test for each resource, e.g.: TestAccSearchIndex_basic. They test the happy path with minimum resource configuration.
• Basic import tests are done as the last step in the basic acceptance tests, not as a different test, e.g. basicTestCase. Exceptions apply for more specific import tests, e.g. testing with incorrect IDs. Import tests verify that the Terraform Import functionality is working fine.
• Data sources are tested in the same tests as the resources, e.g. commonChecks.
• Helper functions such as resource.TestCheckTypeSetElemNestedAttrs or resource.TestCheckTypeSetElemAttr can be used to check resource and data source attributes more easily, e.g. resource_serverless_instance_test.go.

Cloud Gov tests

1. Use PreCheck: PreCheckGovBasic
2. Use the acc.ConfigGovProvider together with your normal terraform config
3. Modify the checkExist and CheckDestroy to use acc.ConnV2UsingGov
4. Follow naming convention:
   1. TestAccGovProject_withProjectOwner, note prefix: TestAccGov
   2. TestMigGovProject_regionUsageRestrictionsDefault, note prefix: TestMigGov
   3. Although Gov tests can be run together with other acceptance tests, using the Test(Acc|Mig)Gov makes it easier to run only gov tests or find similar gov tests

Migration tests

• There must be at least one basic migration test for each resource that leverages on the basic acceptance tests using helper test functions such as CreateAndRunTest, e.g. TestMigServerlessInstance_basic.

Local testing

These enviroment variables can be used in local to speed up development process.

Enviroment Variable	Description
MONGODB_ATLAS_PROJECT_ID	Re-use an existing project reducing test run duration for resources supporting this variable
MONGODB_ATLAS_CLUSTER_NAME	Re-use an existing cluster reducing significantly test run duration for resources supporting this variable
REPLAY_MODE	Use Hoverfly, more info about possible variable values here

Shared resources

Acceptance and migration tests can reuse projects and clusters in order to be more efficient in resource utilization.

• A project can be reused using ProjectIDExecution. It returns the ID of a project that is created for the current execution of tests for a resource, e.g. TestAccConfigRSDatabaseUser_basic.
  • As the project is shared for all tests for a resource, sometimes tests can affect each other if they're using global resources to the project (e.g. network peering, maintenance window or LDAP config). In that case:
    • Run the tests in serial (resource.Test instead of resource.ParallelTest) if the tests are fast and saving resources is prefered, e.g. TestAccConfigRSProjectAPIKey_multiple.
    • Don’t use ProjectIDExecution and create a project for each test if a faster test execution is prefered even if more resources are needed, e.g. TestAccFederatedDatabaseInstance_basic.
• A cluster can be reused using ClusterNameExecution. This function returns the project id (created with ProjectIDExecution) and the name of a cluster that is created for the current execution of tests for a resource, e.g. TestAccSearchIndex_withSearchType. Similar precautions to project reuse must be taken here. If a global resource to cluster is being tested (e.g. cluster global config) then it's prefered to run tests in serial or create their own clusters.
• Plural data sources can be challenging to test when tests run in parallel or they share projects and/or clusters.
  • Avoid checking for a specific total count as other tests can also create resources, e.g. resource_network_container_test.go.
  • Don't assume results are in a certain order, e.g. resource_network_container_test.go.