Switching Environments to Run Different Projects from One Script

[principalideal0]

TL;DR - How can I easily access Poetry environments programmatically via subprocess or the like in order to programmatically execute Python files that work great in their own, individual environments from one master program?

---

So let's say I have a Python project where I have one main folder with some pipeline files (let's call the main one pipeline.py) that run four other Python files (let's call them a.py, b.py, c.py, and d.py) in their own directories (with their own files and different dependencies). I've created Poetry pyproject.toml files for all four Python files that need to run, have installed a Poetry environment for each, and each runs well in its own environment. I also have a pyproject.toml file for the pipeline files, and have installed an environment there with all of the required dependencies.

How can I make sure that when pipeline.py calls one of the other files, like a.py, to be run, it runs a.py from its own environment rather than pipeline.py's?

I have tried several tricks here, and the only one that seems to work is running "poetry run [Python instance for a.py] [directory of a.py]/a.py". I can run this from any path, and it executes reliably. However I cannot find a way to get the paths to each Python instance for each project without manually going into the directory and running "poetry env info --executable".

---

To automate the latter I've even experimented with a Bash script (this doesn't work) to try to take me out of the Python runtime / Poetry context. I really would prefer to not have a monorepo. I saw this post: https://github.com/orgs/python-poetry/discussions/7962

It's a great solution for using a common library across multiple projects; if I did this in reverse though as I'd need with a pipeline file, I could have incompatible dependencies and a real nightmare on my hands.

Thank you in advance, and apologies if I failed to see something I should have in the docs.

[clintonroy]

I'm probably missing some nuance here, but: install it. Expose your script via the [tool.poetry.scripts] section in pyproject.toml, then pipx install the project (either pointing to a local checkout or directly from a git repository). Pipx creates a virtual environment for your tool, and exposes the script as ~/.local/bin/script, then your other tools can just call "tool", cause ~/.local/bin/ is in your PATH

[abn]

Alternatively, if you want locked dependencies, just call the script directly (if installed as above) or via the project's virtual environment (i'd use in-project venvs for this).

/path/to/project/.venv/bin/python script.py

/path/to/project/.venv/bin/script