Create poetry environment for use on another platform

[arneyjfs]

I'm trying to setup an environment inside a docker container that contains packages from a private google cloud repository.

I really don't want to have to authenticate from within the container as I would have to install gcloud cli then copy in a credential key file (that I would have to update when it expires) and then remove all of that before publishing the image. This also deosn't work well with making a key file from a github action secret when i move this to a CI/CD pipeline.

My idea was to poetry install outside the container where I am already authenticated (even in the github action I will be authenticated) and then copy the external site-packages into the container's global site packages. Using a local .venv environment outside the container this is actually quite easy. The problem is in the poetry install grabbing the .whl files for the external environment which then may be incompatible in the container.

This would be similar to doing a:

pip download -r requirements.txt -d path_to_the_folder
pip install -r requirements.txt --find-links=path_to_the_folder

So:

• can i force poetry to create my environment for a different platform?
• is this a terrible approach, is there an alternative?

[Sirivasv]

Perhaps something in the lines of using --platform:

# Adding one target platform
poetry add --platform "win32" requests
# or
poetry add --platform="win32" requests

# Adding multiple target platforms
poetry add --platform "win32 || linux" requests
# or
poetry add --platform="win32 || linux" requests

[arneyjfs]

My understanding of this command (though please correct me) was that it indicates the requested package is only needed on some systems.

I.e. if I run poetry add --platform "win32" requests within my environment, then poetry will add the requests package to my package's dependencies. Then if I publish my package and someone installs it on (for example) linux, poetry will not fetch the requests dependency.

Is this correct?

[Sirivasv]

Ah you are right I was under the assumption that it was a way to force certain packages with a specific platform.
Maybe a workaround would be with something from here:

[tool.poetry.dependencies]
python = "^3.10"
torch = [
    {platform = "darwin", url = "https://download.pytorch.org/whl/cpu/torch-2.0.1-cp310-none-macosx_11_0_arm64.whl"},
    {platform = "linux", url = "https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp310-cp310-linux_x86_64.whl"},
]

Effectively you could override the url for the whls so that for the platform outside the container you force to use a specific whl compatible with the container.

[arneyjfs]

interesting idea! This would probably negatively impact any of our users who genuinely want to install the package on their MacBook - but maybe I can do some trickery around an environment dependent build process.

Thanks for some good problem solving. Unless anyone comes with a better idea this might be the best solution it looks like

[dimbleby]

would have to ... and then remove all of that before publishing the image

sounds like this is a docker question rather than a poetry question and you want a multi-stage build