Multi-job workflow with cache & poetry fails

[Vichoko]

Description:
I started migrating from actions/cache@v2 to setup-python's cache.

I followed the workflow schema for poetry shown in the README.md to create a workflow with 3 jobs:

steps:
- uses: actions/checkout@v3
- name: Install poetry
  run: pipx install poetry
- uses: actions/setup-python@v4
  with:
    python-version: '3.9'
    cache: 'poetry'
- run: poetry install
- run: poetry run pytest

When running a workflow with more than 1 job that uses poetry with cache enabled, only the first job finishes successfully but subsequent jobs fails with the following error:

Run poetry install
The currently activated Python version 3.6.9 is not supported by the project (^3.8).
Trying to find and use a compatible version. 
Using python3 (3.8.[12](https://github.com/thenotcompany/user-management-console-api/runs/6816676285?check_suite_focus=true#step:6:13))
Installing dependencies from lock file
Package operations: 57 installs, 0 updates, 0 removals
  • Installing protobuf (3.20.1)
  • Installing pyasn1 (0.4.8)
  • Installing pyparsing (3.0.9)
  • Installing six (1.16.0)
  FileNotFoundError
  [Errno 2] No such file or directory: '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip': '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip'
  at /usr/lib/python3.6/subprocess.py:[13](https://github.com/thenotcompany/user-management-console-api/runs/6816676285?check_suite_focus=true#step:6:14)64 in _execute_child
      1360│                     if errno_num != 0:
      1361│                         err_msg = os.strerror(errno_num)
      1362│                         if errno_num == errno.ENOENT:
      1363│                             err_msg += ': ' + repr(err_filename)
    → 1364│                     raise child_exception_type(errno_num, err_msg, err_filename)
      1365│                 raise child_exception_type(err_msg)
      1366│ 
      1367│ 
      1368│         def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
  FileNotFoundError
  [Errno 2] No such file or directory: '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip': '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip'
  at /usr/lib/python3.6/subprocess.py:1364 in _execute_child
      1360│                     if errno_num != 0:
      1361│                         err_msg = os.strerror(errno_num)
      1362│                         if errno_num == errno.ENOENT:
      1363│                             err_msg += ': ' + repr(err_filename)
    → 1364│                     raise child_exception_type(errno_num, err_msg, err_filename)
      1365│                 raise child_exception_type(err_msg)
      1366│ 
      1367│ 
      1368│         def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
  FileNotFoundError
  [Errno 2] No such file or directory: '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip': '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip'
  at /usr/lib/python3.6/subprocess.py:1[36](https://github.com/thenotcompany/user-management-console-api/runs/6816676285?check_suite_focus=true#step:6:37)4 in _execute_child
      1360│                     if errno_num != 0:
      1361│                         err_msg = os.strerror(errno_num)
      1362│                         if errno_num == errno.ENOENT:
      1363│                             err_msg += ': ' + repr(err_filename)
    → 1364│                     raise child_exception_type(errno_num, err_msg, err_filename)
      1365│                 raise child_exception_type(err_msg)
      1366│ 
      1367│ 
      1368│         def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
  FileNotFoundError
  [Errno 2] No such file or directory: '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip': '/home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8/bin/pip'
  at /usr/lib/python3.6/subprocess.py:1364 in _execute_child
      1360│                     if errno_num != 0:
      1361│                         err_msg = os.strerror(errno_num)
      1362│                         if errno_num == errno.ENOENT:
      1363│                             err_msg += ': ' + repr(err_filename)
    → 1364│                     raise child_exception_type(errno_num, err_msg, err_filename)
      1365│                 raise child_exception_type(err_msg)
      1366│ 
      1367│ 
      1368│         def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
Error: Process completed with exit code 1.

Action version:
actions/setup-python@v4

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Tools version:

• actions/setup-python@v4
• poetry 1.1.13

Repro steps:
This is happening in my linter workflow:

jobs:
  black:
    name: black
    runs-on: ubuntu-latest
    timeout-minutes: 15

    steps:
      - name: Checkout 🛎
        uses: actions/[email protected]

      - name: Install poetry 📚
        run: pipx install poetry

      - name: Set up Python 3.8 🐍
        uses: actions/setup-python@v4
        with:
          python-version: 3.8
          cache: "poetry"

      - name: Install dependencies 🤖
        run: poetry install

      - name: Lint with Black
        run: poetry run black src --check

  isort:
    name: isort
    runs-on: ubuntu-18.04
    timeout-minutes: 15

    steps:
      - name: Checkout 🛎
        uses: actions/[email protected]

      - name: Install poetry 📚
        run: pipx install poetry

      - name: Set up Python 3.8 🐍
        uses: actions/setup-python@v4
        with:
          python-version: 3.8
          cache: "poetry"

      - name: Install dependencies 🤖
        run: poetry install

      - name: Lint with isort
        run: 	poetry run isort src/**/* --check --settings-path ./pyproject.toml --diff

Expected behavior:

The second job should run smoothly.

Actual behavior:
The second job is failing on poetry install step with the error stated in the description of this issue. It seems to be trying to install the dependencies, even though they should be cached from the first job. So apparently, the cache isn't working. Should it skip the poetry install step if the cache was restored?

[dmitry-shibanov]

Hello @Vichoko. Thank you for your report. Could you please try to add this command: poetry env use "3.9" before installing poetry dependencies ?

[Vichoko]

Hello @Vichoko. Thank you for your report. Could you please try to add this command: poetry env use "3.9" before installing poetry dependencies ?

Thanks for your quick response! I was just trying that (but with py 3.8), and it continue failing but with a different error:

Run poetry env use "3.8"
  poetry env use "3.8"
  poetry install --no-interaction
  shell: /bin/bash -e {0}
  env:
    SSH_AUTH_SOCK: /tmp/ssh-qSrIUQ[2](https://github.com/thenotcompany/user-management-console-api/runs/6817336721?check_suite_focus=true#step:6:2)ulmML/agent.2094
    SSH_AGENT_PID: 2095
    pythonLocation: /opt/hostedtoolcache/Python/[3](https://github.com/thenotcompany/user-management-console-api/runs/6817336721?check_suite_focus=true#step:6:3).8.12/x6[4](https://github.com/thenotcompany/user-management-console-api/runs/6817336721?check_suite_focus=true#step:6:4)
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.12/x64/lib
  
Using virtualenv: /home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8
The currently activated Python version 3.6.9 is not supported by the project (^3.8).
Trying to find and use a compatible version. 
Using python3 (3.8.12)
The virtual environment found in /home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8 seems to be broken.
Recreating virtualenv user-management-console-api-UrIDKMyC-py3.8 in /home/runner/.cache/pypoetry/virtualenvs/user-management-console-api-UrIDKMyC-py3.8
Installing dependencies from lock file
  SolverProblemError
  The current project's Python requirement (3.6.9) is not compatible with some of the required packages Python requirement:
    - coverage requires Python >=3.7, so it will not be satisfied for Python 3.6.9
  Because coverage (6.4) requires Python >=3.7
   and no versions of coverage match >6.4,<7.0, coverage is forbidden.
  So, because user-management-console-api depends on coverage (^6.4), version solving failed.
  at /opt/pipx/venvs/poetry/lib/python3.6/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      24[5](https://github.com/thenotcompany/user-management-console-api/runs/6817336721?check_suite_focus=true#step:6:5)│                 PackageNode(self._package, packages), aggregate_package_nodes
  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
    For coverage, a possible solution would be to set the `python` property to "<empty>"
   [ https://python-poetry.org/docs/dependency-specification/#python-restricted-dependen](https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies)ci
   [ https://python-poetry.org/docs/dependency-specification/#using-environment-mar](https://python-poetry.org/docs/dependency-specification/#using-environment-markers)
Error: Process completed with exit code 1.

It seems to be trying to use python 3.6, but I'm not sure why.

[mrlubos]

I believe I’ve run into the same issue

[IvanZosimov]

Hi, @Vichoko 👋 ! Here you encountered a well-known Poetry's issue . Poetry tends to use the system's version of the python that is why in the first job (on Ubuntu 20.04) it uses: 3.8.12 and in the second (Ubuntu 18.04): 3.6.9 Actually, we have one open issue related to this topic with workarounds available in the comments.

By now I'm going to close this issue, If you have any questions - feel free to ping us!