Poetry Cheat Sheet

Using Poetry

The purpose of this doc is to serve as a quick reference for some of the common workflow commands. Most of the commands below can have -v|vv|vvv added to them for increased verbosity when running the commands, the number of v's added specifies the level,

1. normal output
2. verbose output
3. for debug You can find all the juicy details in the official documentation.

Managing Poetry's config

Poetry's configurations can be set either from the shell, or by environment variables.

# List the current config
poetry config --list

# Update a config by key
poetry config experimental.new-installer <true|false>

Using Environment Variables to override configs

When overriding a config with env vars, prefix the config key with POETRY_ and replace dots and dashes with underscores like so:

poetry config experimental.new-installer true
# Becomes
POETRY_EXPERIMENTAL_NEW_INSTALLER=true

Installing project dependencies

Install all Dependencies

poetry install 

Install dependency groups

There are a number of ways you can include, exclude, or otherwise specify dependency groups when installing. Dependency groups are specified in pyproject.toml with the following section specifier [tool.poetry.group.<desiredGroupName>.dependencies].

# Test dependencies only
poetry install --only test

# Exclude test dependencies
poetry install --without test

# Include test dependencies
poetry install --with test

Managing dependencies within pyproject.toml

Dependencies can be managed either via the shell or by editing pyproject.toml directly. When editing pyproject.toml directly, run poetry check to validate the pyproject.toml file before running poetry install.

pyproject.toml - Syntactical differences from requirements.txt

The simplest form

boto=="2.49.0"
# Becomes
boto = "2.49.0"

Specifying 'extras'

jinja2-cli[yaml]==0.8.2
# becomes
jinja2-cli = { extras = ["yaml"], version = "0.8.2" }

Pulling from a git repo

By commit revision

git+https://github.com/mitsuhiko/flask-sqlalchemy.git@500e732dd1b975a56ab06a46bd1a20a21e682262#egg=Flask-SQLAlchemy==2.3.2.dev20190108
# Becomes
Flask-SQLAlchemy = { git = "git+https://github.com/mitsuhiko/flask-sqlalchemy.git", rev = "500e732dd1b975a56ab06a46bd1a20a21e682262"}

By repo tag

git+https://github.com/cds-snc/[email protected]#egg=notifications-utils
# Becomes
notifications-utils = { git = "https://github.com/cds-snc/notifier-utils.git", tag = "50.1.0"}

Managing dependencies from the shell

Search for dependencies

poetry search <search term>

Adding dependencies

When adding a dependency and specifying a version, Version constraints can be used.

# Just the dependency name
poetry add pytest-mock-resources

# With a version
poetry add [email protected]

# Specify a dependency group target
poetry add [email protected] --group test

# Specify Extras (multiple can be specified separated by a space)
poetry add [email protected] --extras redis

# From a git url
poetry add git+https://github.com/mitsuhiko/flask-sqlalchemy.git

# From a git url with a revision
poetry add git+https://github.com/mitsuhiko/flask-sqlalchemy.git#500e732dd1b975a56ab06a46bd1a20a21e682262

Removing dependencies

# Just the dependency name
poetry remove pytest-mock-resources

# Specify a dependency group target
poetry remove [email protected] --group test

Working with Poetry.lock

When changes are made to the lock file it should be committed along with any other work done.

Locking project dependencies

# Validate the lock file
poetry lock --check
# Freeze the current requirements in the lock file
poetry lock

Re-Synchronize the lock file

When pyproject.toml is updated, either manually or via the CLI, the lock file needs to be refreshed.

poetry lock --no-update

When running poetry install and the lock file is out of sync, you'll be notified of this.

Working with source distribution and wheel packaging

When we need to package up a project, like notification-utils, we no longer need to use setup.py. Project metadata that was previously housed in setup.py for this purpose is now found in pyproject.toml:

[tool.poetry]
name = "notifications-utils"
version = "50.1.2"
description = "Shared python code for Notification - Provides logging utils etc."
authors = ["Canadian Digital Service"]
license = "MIT license"
readme = "README.md"
packages = []

Creating a source distribution

# Create a source distribution
poetry build --format sdist

# Create a wheel
poetry build --format wheel

Working with virtual environments

Poetry has built in virtual environment management. By default Poetry checks if it is being run from within a virtual environment and will utilize an existing environment. Otherwise Poetry will automatically create a virtual environment to isolate project dependencies from system dependencies.

Getting info about the current environment

# List all environments associated with he current project
poetry env list

# Get more detailed information about the currently activated env
poetry env info

Activating a virtual environment

# By path to python
poetry env use $(which python)

# By python version - if it exists in your path
poetry env use python3.10
poetry env use 3.10

# Explicitly use the system environment
poetry env use system

Removing environments

Environments can be removed in the same manner that they can be activated

# By path to python
poetry env use $(which python)

# By python version - if it exists in your path
poetry env use python3.10
poetry env use 3.10

# Explicitly use the system environment
poetry env use system

Controlling Poetry's venv behaviour

There are several useful configs for instructing poetry how to handle virtual environments:

# Enable or disable automatic venv creation
virtualenvs.create <true|false>

# Use the system environment, this is typically paired with virtualenvs.create false
virtualenvs.options.system-site-packages

# Specify the path for Poetry to use when creating virtual envs
virtualenvs.path <path>

# Instruct Poetry to create <project_root>/.venv and use it for creating virtual envs
virtualenvs.in-project <true|false>

# Use the currently installed python for creation of virtual environments
virtualenvs.prefer-active-python

Isolating Poetry's environment

In the case of CI pipelines and deployment environments it is useful to isolate Poetry from the system packages to avoid inadvertently removing or otherwise editing packages that either Poetry or the installed project depend on.

# Override Poetry's config to ensure it does not create venvs automatically
POETRY_VIRTUAL_ENVS_CREATE=false

# Set up your venv paths
POETRY_VENV=/poetry/belongs/here
PROJECT_VENV=/project/venv/belongs/here

# Add the virtual env paths to the PATH variable
PATH="#{PROJECT_VENV}/bin:${POETRY_VENV}/bin:$PATH"

# Create a venv to house poetry
python -m venv ${POETRY_VENV}

# Install poetry into it's venv
${POETRY_VENV}/bin/pip3 install poetry

# Create a venv for the project
python -m venv ${PROJECT_VENV}

# Activate the project venv and install dependencies
. ${PROJECT_VENV}/bin/activate
poetry install

For a more concrete example of how this plays out in practice see admin's Dockerfile.lambda.

Some cool extras

View the project's full dependency tree

poetry show --tree

Inspect a specific dependency

This command will list the package's dependencies, as well as which other packages depend on it.

poetry show SQLAlchemy
# Get the info as a tree instead
poetry show SQLAlchemy --tree

Show outdated dependencies

poetry show --outdated