WIP: Populate tox with a script

[sentrivana]

WIP

• Fetch releases for packages to test from PYPI and pick a handful to test against
• Remove -latest tests; instead, we can rerun the script to regenerate tox.ini as it will now always also include the latest release
  • TODO: actually make an action for this to happen on PRs
• Add a CI check for making sure the script/template is in sync with the contents of tox.ini (so that folks get notified if they edit tox.ini directly instead of the template/script)
• Make Flags its own test group instead of dumping everything in Misc
• ❗Implement an automatic deprecation policy: only test releases that are at most X years old

[codecov]

❌ 454 Tests Failed:

Tests completed	Failed	Passed	Skipped
16111	454	15657	5260

View the top 3 failed tests by shortest run time

 tests.integrations.cohere.test_cohere

Stack Traces | 0s run time

ImportError while importing test module '.../integrations/cohere/test_cohere.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../site-packages/_pytest/python.py:599: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../site-packages/_pytest/pathlib.py:533: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.6.15.../x64/lib/python3.6/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:994: in _gcd_import
    ???
<frozen importlib._bootstrap>:971: in _find_and_load
    ???
<frozen importlib._bootstrap>:955: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:665: in _load_unlocked
    ???
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../_pytest/assertion/rewrite.py:171: in exec_module
    exec(co, module.__dict__)
.../integrations/cohere/test_cohere.py:3: in <module>
    import httpx
E   ModuleNotFoundError: No module named 'httpx'

 tests.integrations.cohere.test_cohere

Stack Traces | 0s run time

ImportError while importing test module '.../integrations/cohere/test_cohere.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.11-cohere-v4.4.2/lib/python3.11.../site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
.tox/py3.11-cohere-v4.4.2/lib/python3.11.../site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.11.11.../x64/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
.tox/py3.11-cohere-v4.4.2/lib/python3.11.../_pytest/assertion/rewrite.py:184: in exec_module
    exec(co, module.__dict__)
.../integrations/cohere/test_cohere.py:3: in <module>
    import httpx
E   ModuleNotFoundError: No module named 'httpx'

 tests.integrations.launchdarkly.test_launchdarkly

Stack Traces | 0s run time

ImportError while importing test module '.../integrations/launchdarkly/test_launchdarkly.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.7-launchdarkly-v6.12.2/lib/python3.7.../site-packages/_pytest/python.py:617: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
.tox/py3.7-launchdarkly-v6.12.2/lib/python3.7.../site-packages/_pytest/pathlib.py:567: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.7.17.../x64/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1006: in _gcd_import
    ???
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:967: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:677: in _load_unlocked
    ???
.tox/py3.7-launchdarkly-v6.12.2/lib/python3.7.../_pytest/assertion/rewrite.py:186: in exec_module
    exec(co, module.__dict__)
.../integrations/launchdarkly/test_launchdarkly.py:9: in <module>
    from ldclient.context import Context
E   ModuleNotFoundError: No module named 'ldclient.context'

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!