Wrong python version in new installation

[ricardo-reis-1970]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name:
• Poetry version:
• Link of a Gist with the contents of your pyproject.toml file:

Issue

I have two version of Python:

• 3.9.0
• 3.6.6

I'm currently developing a project that is somewhat locked to 3.6.6, but also running experiments on latest Python, so I need both. I installed 3.6.6 first, so when I run where python, I get

C:\Program Files\Python39\python.exe
C:\Program Files\Python36\python.exe

This means that simply running python will run 3.9.0, as it is ahead in the PATH.

Having said all this, 3.6.6 is still available and here's the very top of my pyproject.toml:

[tool.poetry.dependencies]
python = "^3.6.6"

With this, I was expecting to have poetry to create the virtual environment correctly, but it does not. It simply goes to the first python.exe it can find — 3.9.0 — and then nothing will run.

Workarounds:

• uninstall Python 3.9.0 whenever I need to recreate the .venv: highly inconvenient;
• create the virtual environment by hand: I am using poetry to do that.

How can I enforce the Python version? Am I doing something wrong?

This is not a current duplicate, but I came across complaints online from 2 years ago along the same lines, and these were supposed to have been fixed long ago.

[sinoroc]

As far as I can tell ^3.6.6 is equivalent to >=3.6.6 <4.0.0, so 3.9.0 is a valid Python version:

• https://python-poetry.org/docs/dependency-specification/#caret-requirements

So this could be normal expected behaviour.

[ricardo-reis-1970]

Dear @sinoroc,

Thank you for your reply. Indeed that is the case and you would be on the spot. Unfortunately — my bad, I forgot to mention — I obviously tried removing the caret and "freezing" that specific version, to no avail. That was indeed the very reason I decided to open this ticket.

In the end, instead of just uninstalling 3.9.0, I went to the environment variables and moved up the references to 3.6.6. I extensively tried this out with and without the caret (loads and loads of looking at scrolling logs) only to realise and pinpoint that the observable behaviour seems to unmistakably be "fetch the first Python you can muster and carry on".

As aforementioned, this seems to be the homecoming of an old glitch. This will not deter me, as I'm getting more and more used to poetry, but I really wish I could trust the tool a bit more. That would make it all the more easier to "sell" it to the raw pipsters in my team.

[sinoroc]

@ricardo-reis-1970 I recommend you to edit your initial bug report to reflect that. As it is now, your bug report does not make sense.

Additionally, I just tried to recreate your issue, but I could not, it seems to work perfectly fine for me. I am on Linux, though, so it might be a bug that only happens on Windows.

As you can see I have multiple Python interpreters installed:

$ whereis python
python: /usr/bin/python2.7-config /usr/bin/python3.9-config /usr/bin/python3.7 /usr/bin/python3.8-config /usr/bin/python3.7m /usr/bin/python3.7m-config /usr/bin/python2.7 /usr/bin/python3.6-config /usr/bin/python3.7-config /usr/bin/python /usr/bin/python3.5m /usr/bin/python3.8 /usr/bin/python3.10 /usr/bin/python3.6m /usr/bin/python3.6m-config /usr/bin/python3.5 /usr/bin/python3.9 /usr/bin/python3.10-config /usr/bin/python3.6 /usr/lib/python3.7 /usr/lib/python2.7 /usr/lib/python3.8 /usr/lib/python3.10 /usr/lib/python3.5 /usr/lib/python3.9 /usr/lib/python3.6 /etc/python3.7 /etc/python2.7 /etc/python /etc/python3.8 /etc/python3.10 /etc/python3.5 /etc/python3.9 /etc/python3.6 /usr/local/lib/python3.7 /usr/local/lib/python2.7 /usr/local/lib/python3.8 /usr/local/lib/python3.10 /usr/local/lib/python3.5 /usr/local/lib/python3.9 /usr/local/lib/python3.6 /usr/include/python3.7 /usr/include/python3.7m /usr/include/python2.7 /usr/include/python3.8 /usr/include/python3.10 /usr/include/python3.6m /usr/include/python3.9 /usr/include/python3.6 /usr/share/python /usr/share/man/man1/python.1.gz

I have a pyproject.toml that contains the following:

[tool.poetry.dependencies]
python = '3.5.*'

And poetry picks up the python3.5 interpreter as expected:

$ poetry install
The currently activated Python version 3.8.6 is not supported by the project (3.5.*).
Trying to find and use a compatible version. 
Using python3.5 (3.5.10)
Creating virtualenv poesie-aN_GiZ0h-py3.5 in /home/sinoroc/.cache/pypoetry/virtualenvs
Installing dependencies from lock file
[...]

[finswimmer]

How can I enforce the Python version? Am I doing something wrong?

That's what poetry env use is for.

To create a venv for C:\Program Files\Python39\python.exe (if it's not the default python) run:

``
$ poetry env use C:\Program Files\Python39\python.exe
$ poetry install

To create a venv for `C:\Program Files\Python36\python.exe` (if it's not the default `python`) run:

``
$ poetry env use C:\Program Files\Python36\python.exe
$ poetry install

If you don't have set virtualenvs.in-project to true both venv will be created. You can than always tell poetry which venv it should use when running a poetry command by running poetry env use /path/to/the/interpreter once.

fin swimmer

[ricardo-reis-1970]

@sinoroc I wish Linux were in the options for this project. Still fighting that good fight...

@finswimmer (cool handle!) I'm obviously a newbie to poetry. I was told that all it took was to run poetry update upon cloning the repo and voilà: env correctly created from toml, all good versions and all.

I'm starting to believe that a narrow street has been set up where my colleagues only have python 3.6.6, so a random (or any other kind of) choice out of one always goes right, right?

I'll be checking out the virtualenvs.in-project option, but in my case only one env is created: the first on PATH, regardless of the python version in the toml. I worked around it with PATH manipulation but I don't like that option.

In any case, thank you both very much. You certainly came to the rescue very promptly and provided some valuable input. And thank @jaharkes for taking notice.

[sinoroc]

@ricardo-reis-1970

1. I believe you should really edit your initial message, because as it is it shows perfectly normal behaviour. It does not show what is the issue you are confronted to. And according to your second message there might well be a bug (only on Windows?) that needs fixing. So again, I would recommend you to edit your first message to show what the actual bug is.

2. What @finswimmer is saying, is that you probably should not set virtualenvs.in-project to true if you want to use multiple Python versions on the same project (3.6 and 3.9).

3. In any case, I will repeat what finswimmer said (since the formatting of their message is a bit unclear). You can always call for a specific Python interpreter explicitly:

To use Python 3.9:

$ poetry env use "C:\Program Files\Python39\python.exe"
$ poetry install
$ poetry build
$ poetry shell
[...]

To use Python 3.6:

$ poetry env use "C:\Program Files\Python36\python.exe"
$ poetry install
$ poetry build
$ poetry shell
[...]

Those will create 2 different virtual environments, that can be used independently from one another.

4. If for now, your project is really only compatible with 3.6, while support 3.9 is still only in an experimental phase, then set:

[tool.poetry.dependencies]
python = '3.6.*'

and only edit that value locally to something else (maybe ^3.6.6), when experimenting with support for Python 3.9.

5. I looked at @jaharkes' PR #3475 and I am not entirely sure exactly how it relates to our issue here. I see some relation, but I am not sure it would be a fix for our issue here. It might be a fix for something slightly different. I might be missing something obvious, though.

[sinoroc]

Not sure if it is related, but I still link it here: #2117

[jaharkes]

I might have misread the bug reported here.

But if you try to specify a specific range like python>=3.6.6,python<3.7, or ~3.6.6 (or even a specific version) and the default python3 version is outside of that range, the existing code that searches for an alternate interpreter will not even bother checking the installed python3.6 binary because the test before running the binary to get the exact version is too strict.

[carantunes]

I have the same issue, poetry picks up the incorrect python version for a install without virtualenvs

Step 8/8 : RUN poetry --version && python -V && whereis python && poetry config virtualenvs.create false -vvv &&     poetry install --no-root --no-dev --no-interaction --no-ansi
 ---> Running in 23ac5b60a812
Poetry version 1.1.6
Python 3.8.3
python: /usr/bin/python /usr/bin/python3.6m /usr/bin/python3.6 /usr/bin/python3.8-x86_64-config /usr/bin/python3.8-config /usr/bin/python3.8 /usr/lib/python3.6 /usr/lib/python3.8 /usr/lib64/python3.6 /usr/lib64/python3.8 /usr/local/lib/python3.8 /usr/include/python3.6m /usr/include/python3.8 /usr/share/man/man1/python.1.gz
The currently activated Python version 3.6.8 is not supported by the project (^3.8).
Trying to find and use a compatible version. 
Using python3.8 (3.8.3)
Skipping virtualenv creation, as specified in config file.
Installing dependencies from lock file

  SolverProblemError

  The current project's Python requirement (3.6.8) is not compatible with some of the required packages Python requirement:
    - ipython requires Python >=3.7, so it will not be satisfied for Python 3.6.8

[sinoroc]

@carantunes Maybe show the pyproject.toml.

[carantunes]

I guess this is the part that matters..

[tool.poetry.dependencies]
python = "^3.8"

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.