Support developer tests

[jeroen]

A cleaner pattern to distinguish between public tests and developer tests.

Developer tests only run in CI or devtools, and can be used for tests that are not appropriate for public infrastructure such as CRAN or other downstream use, e.g. tests that require special tooling, are flaky, or consume much resources.

[DavisVaughan]

Possibly there is room for an explicit opt in/out env var that is very specific to this feature, like:

run_dev_tests <- function() {
  if (env_var_is_true("RUN_DEV_TESTS")) {
     return(TRUE)
  }

  if (env_var_is_false("RUN_DEV_TESTS")) {
     return(FALSE)
  }

  env_var_is_true("CI") || env_var_is_true("NOT_CRAN")
}

[jeroen]

Thinking a bit more about this after todays dicussion about how this works for reverse dependency checks. Perhaps the dev tests should be opt-in per package. So the variable should be named e.g. JSONLITE_RUN_DEV_TESTS to opt-in to dev tests from jsonlite only.

For example, if I want to run a reverse dependency check for curl, I also may want to run the developer tests for httr and httr2 but not for all the random packages with api wrappers that require custom auth tokens and so on.

[JosiahParry]

Just chiming in to say that I would really like to see this capability added to testthat. My particular use case is that my package arcpbf is used by arcgislayers and I want arcpbf to have integration tests for GHA CI but not to actually be tested on CRAN.