Using Pex with Poetry

[jslaroch]

It would be nice if we could tell Pex to use the venv build using Poetry

[jsirois]

It sounds like you want to be able to run pex --import <venv path> .... where --import <venv path> would be a new feature that is an alternate way to specify requirements. You'd still have to pass -m or -c to set the entrypoint, etc - but all distributions installed in the virtual environment would be imported. If I've got that right then this feature would work for any virtual environment and Pex wouldn't need to learn anything special about Poetry.

[jslaroch]

Yes that’s exactly it. There is some duplication between Pex and Poetry but we need poetry for some features. However it would really be nice to use Pex to package our end product.
Since Poetry uses a standard venv I would suspect that it would not be very hard to add and a nice addition to Pex.

[jsirois]

However it would really be nice to use Pex to package our end product.

Well, that you can already do. Here I use Pex to package Poetry itself (which dogfoods itself; i.e.: it does not use setup.py, it uses the poetry-core PEP-517 builder).

/tmp/pex/issues/1174 $ git clone https://github.com/python-poetry/poetry
Cloning into 'poetry'...
remote: Enumerating objects: 15689, done.
remote: Total 15689 (delta 0), reused 0 (delta 0), pack-reused 15689
Receiving objects: 100% (15689/15689), 10.75 MiB | 1.78 MiB/s, done.
Resolving deltas: 100% (10693/10693), done.
/tmp/pex/issues/1174 $ cd poetry/
/tmp/pex/issues/1174/poetry (master) $ ls setup.py
ls: cannot access 'setup.py': No such file or directory
/tmp/pex/issues/1174/poetry (master) $ pex . -cpoetry -opoetry.pex
/tmp/pex/issues/1174/poetry (master) $ ./poetry.pex --version
Poetry version 1.2.0a0

The subtle thing there is that Pex accepts a directory as a requirement and I pass it ..

Is that all you needed?

[jslaroch]

Great ! I think so. I will try this later today and give you some feedback later
If it does the equivalent of your: pex --import .... where --import
Thats what I need. Thanks.

[jslaroch]

Huuummm. I don't think that this is it. I don't need to ship Poetry. Poetry is used for development. One thing it does for a project is build a standard venv that it manages from a .toml file (list of dependency).
So what I would like to do is use Pex to build my deliverable based on the venv build by Poetry.
More in the spirit of your first suggestion: pex --import

[jsirois]

@jslaroch I just used poetry as a (random) example project that uses poetry to build.
Pex can create a PEX file for any project that uses a PEP-517 builder. That's any project with a pyproject.toml with lines like (E.G.: https://github.com/python-poetry/poetry/blob/de0b32c245c72568cf546090600d4626917cd3a4/pyproject.toml#L61-L63):

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

I assume your project, which is managed by poetry, has those lines in its pyproject.toml. If so, you can create a PEX file of your project with just pex <path to your project directory> <rest of the Pex options you want>.

[jsirois]

To underline - Pex is fully compliant with PEP-517 and so supports building PEX files for Poetry projects "for free" as a result. It also can do this for flit projects, etc. - not because Pex knows flit, but because it supports PEP-517.

[jsirois]

For example - Pex itself uses flit to manage its build (basically for release to create an sdist and wheel and upload it to PyPI:
https://github.com/pantsbuild/pex/blob/43934ed57433de2b0569535daa364e9db24890c5/pyproject.toml#L1-L3
https://github.com/pantsbuild/pex/blob/43934ed57433de2b0569535daa364e9db24890c5/tox.ini#L215-L221

Pex also releases a PEX file of Pex using itself:
https://github.com/pantsbuild/pex/blob/43934ed57433de2b0569535daa364e9db24890c5/tox.ini#L197-L203
https://github.com/pantsbuild/pex/blob/43934ed57433de2b0569535daa364e9db24890c5/scripts/package.py#L25-L57

[jslaroch]

Thank for your nice explanation and patience. Sorry I did'nt get the previous explanation.
This is really nice and it should work. I will try to use Pex with one of our Python project tomorrow.

[jsirois]

Excellent! Definitely let me know if this works for you.

[jsirois]

Just checking in here @jslaroch - I hope you've found you can in fact create a PEX file for your poetry project with pex <path to poetry project> -o my-poetry-project.pexed.