Switch to poetry. (#12)

* Switch to poetry.

* Fix env and pytest call.

* Call flake8 and black with poetry.

* Run flake8 just on project files.

* Windows compatible.

* Remove venv setup.

* Update pre-commit hooks.

* Testing hooks.

* Test hooks.

* Test hooks.

* Update prehooks.

* Format.

* Update docs.

* Remove hatchling toml.

.flake8

@@ -1,5 +1,5 @@
 [flake8]
-exclude = .git,build
+exclude = .git, __pycache__, build, dist
 ignore = E203, E266, E501, W503, F403, F401, C901
 max-line-length = 100
 max-complexity = 18

.github/workflows/poetry.yml

@@ -0,0 +1,58 @@
+# Poetry GitHub Action: https://github.com/marketplace/actions/python-poetry-action
+name: pydevtips
+
+# on: [push, pull_request]
+on:
+  # trigger on pushes and PRs to main
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  build:
+
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      max-parallel: 12
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python-version: ["3.10", "3.11"]
+        poetry-version: ["1.8.4"]
+    steps:
+    - uses: actions/checkout@v4
+    - name: Checkout submodules
+      shell: bash
+      run: |
+        auth_header="$(git config --local --get http.https://github.com/.extraheader)"
+        git submodule sync --recursive
+        git -c "http.extraheader=$auth_header" -c protocol.version=2 submodule update --init --force --recursive --depth=1
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install poetry
+      uses: abatilo/[email protected]
+      with:
+        poetry-version: ${{ matrix.poetry-version }}
+    - name: Install dependencies
+      run: |
+        poetry install --with dev
+    - name: Lint with flake8
+      run: |
+        # stop the build if there are Python syntax errors or undefined names
+        poetry run flake8 pydevtips --count --select=B,C,E,F,W,T4,B9 --show-source --statistics --max-complexity=18 --max-line-length=100 --ignore=E203,E266,E501,W503,F403,F401,C901
+        poetry run flake8 examples --count --select=B,C,E,F,W,T4,B9 --show-source --statistics --max-complexity=18 --max-line-length=100 --ignore=E203,E266,E501,W503,F403,F401,C901
+        poetry run flake8 profile --count --select=B,C,E,F,W,T4,B9 --show-source --statistics --max-complexity=18 --max-line-length=100 --ignore=E203,E266,E501,W503,F403,F401,C901
+        poetry run flake8 tests --count --select=B,C,E,F,W,T4,B9 --show-source --statistics --max-complexity=18 --max-line-length=100 --ignore=E203,E266,E501,W503,F403,F401,C901
+    - name: Format with black
+      run: |
+        poetry run black pydevtips -l 100
+        poetry run black examples -l 100
+        poetry run black profile -l 100
+        poetry run black tests -l 100
+    - name: Test with pytest
+      run: poetry run pytest -v

.github/workflows/python.yml → .github/workflows/setuptools.yml

@@ -1,14 +1,11 @@
-name: pydevtips
+name: pydevtips setuptools
 
-# on: [push, pull_request]
+# HACK to not trigger GitHub action
 on:
-  # trigger on pushes and PRs to main
   push:
     branches:
-      - main
-  pull_request:
-    branches:
-      - main
+      - non-existant-branch
+
 
 jobs:
   build:

.gitignore

@@ -1,5 +1,6 @@
 project_env/
 .DS_Store
+dataset/
 
 # from hydra
 outputs/   
@@ -13,6 +14,7 @@ __pycache__/
 *.so
 
 # Distribution / packaging
+poetry.lock
 .Python
 build/
 develop-eggs/

.pre-commit-config.yaml

@@ -1,4 +1,11 @@
 repos:
+-   repo: https://github.com/python-poetry/poetry
+    rev: '1.8.4'  # add version here
+    hooks:
+    -   id: poetry-check
+    -   id: poetry-lock
+    -   id: poetry-export
+    -   id: poetry-install
 -   repo: https://github.com/psf/black
     rev: 22.12.0
     hooks:
@@ -7,4 +14,9 @@ repos:
 -   repo: https://github.com/PyCQA/flake8
     rev: 6.0.0
     hooks:
-    - id: flake8
+    - id: flake8
+-   repo: https://github.com/pycqa/isort
+    rev: 5.13.2
+    hooks:
+    - id: isort
+      name: isort (python)

README.rst

@@ -20,13 +20,13 @@ pydevtips: Python Development Tips
     :alt: Slides
 
 
-.. |ss| raw:: html
+.. .. |ss| raw:: html
 
-   <strike>
+..    <strike>
 
-.. |se| raw:: html
+.. .. |se| raw:: html
 
-   </strike>
+..    </strike>
 
 
 Reproducibility is important for software: *if it's not reproducible, 
@@ -54,13 +54,13 @@ But hopefully this gives you a starting point. Feel free to pick and
 choose the features that you like. This flexibility is one of the best
 (and worst parts) of open source. Some of the things we cover:
 
+* Packaging and distribution with `Poetry <https://python-poetry.org/>`_.
 * Virtual environments.
 * Version control.
 * Reproducible examples.
 * Documentation.
 * Code formatting.
 * Unit tests and continuous integration.
-* Packaging and distribution.
 * Remove development.
 
 The accompanying 
@@ -75,7 +75,7 @@ Feel free to modify and use it for your own purposes.
     Copilot <https://github.com/features/copilot>`_, which I highly recommend for development. If you don't like
     writing documentation, it is a great way to get started as it is able to 
     understand the functionality of your code and produce meaningful text to describe it. 
-    It should be used be used with caution, |ss| *but it can be a great tool for getting started* |se|
+    It should be used be used with caution, *but it can be a great tool for getting started*
     and you often you need to make a few tweaks (*like the previous repetition*).
     But it's a huge time-saver!
 
@@ -93,6 +93,8 @@ Or from source, e.g. with Anaconda / Miniconda:
 .. code:: bash
 
     # create new environment, press enter to accept
+    # -- important to set python version, otherwise `python` executable may not exist 
+    # -- (would be `python3` instead)
     conda create -n project_env python=3.11
 
     # view available environments
@@ -102,11 +104,10 @@ Or from source, e.g. with Anaconda / Miniconda:
     conda activate project_env
 
     # install package locally
-    (project_env) pip install -e .
+    (project_env) poetry install --with dev
 
     # run tests
-    # - one time: pip install pytest
-    (project_env) pytest
+    (project_env) poetry run pytest
 
     # deactivate environment
     (project_env) conda deactivate
@@ -123,19 +124,19 @@ repository, e.g.:
     python examples/real_convolve.py
 
 Parameter setting is done with `hydra <https://hydra.cc/>`_. More on that
-in the :ref:`Reproducible examples<Reproducible examples>` section of the 
-documentation.
+in the `Reproducible examples <https://pydevtips.readthedocs.io/en/latest/reproducible.html>`_
+section of the documentation.
 
 
 TODO
 ====
 
+- switch to ruff for code formatting: https://docs.astral.sh/ruff/
 - numba: https://numba.pydata.org/
 - picking a license
 - change documentation links to main branch
 - github page
 - point out features in scripts: object-oriented, asserts, tqdm, type hints
-- matplotlib, pytest, black in dev install
 - manifest file to not include file in package
 - GitHub actions for releasing to PyPi when changes to version
 - pytorch compatible

docs/source/clean_code.rst

@@ -16,10 +16,16 @@ tools that can help us with this task. The ones we use are:
 
 * `Black <https://github.com/psf/black>`_ which will reformat your code 
   in-place to conform to the PEP8 standard.
-* `Flake8 <https://flake8.pycqa.org/en/latest/>`__ which is a *linter* that 
+* `Flake8 <https://flake8.pycqa.org/en/latest/>`_ which is a *linter* that 
   will check your code for errors and style violations, but not reformat it. For
   example, for me it has identified code where I have unused variables or 
   scripts / functions that are too long.
+* `isort <https://pycqa.github.io/isort/>`_ which will sort your imports 
+  alphabetically and group them by type.
+
+There are many alternatives for there tools. An increasingly popular alternative
+is `ruff <https://docs.astral.sh/ruff//>`_, which is written in Rust and is meant
+to replace Flake8, Black, and isort.
 
 While you can use these tools manually, it is much more convenient to use them
 as pre-commit hooks. This means that before you commit your code, these tools
@@ -34,7 +40,7 @@ A few files are needed to setup pre-commit hooks:
   pre-commit hooks. It specifies which tools to use, and how to use them.
 * `.flake8 <https://github.com/ebezzam/python-dev-tips/blob/main/.flake8>`_: This file contains the configuration for Flake8. It specifies 
   e.g. which errors to ignore, and which line length to use. 
-* `pyproject.toml <https://github.com/ebezzam/python-dev-tips/blob/main/pyproject.toml>`_: This file contains the configuration for Black. It 
+* `pyproject.toml <https://github.com/ebezzam/python-dev-tips/blob/main/pyproject.toml>`_: This file contains the configuration for Black and isort. It 
   specifies e.g. which line length to use.
 
 You can then install the pre-commit hooks for your project by running the 
@@ -43,13 +49,18 @@ following commands:
 .. code:: bash
 
     # inside virtual environment
-    (project_env) pip install pre-commit
-    (project_env) pip install black
+    # -- black, flake8, isort are in the dev group
+    (project_env) poetry install --with dev
+
+    # -- if not using Poetry
+    # (project_env) pip install pre-commit black flake8 isort
 
     # Install git hooks
     (project_env) pre-commit install
     # pre-commit installed at .git/hooks/pre-commit
 
+More pre-commit hooks are available provided by `Poetry <https://python-poetry.org/docs/pre-commit-hooks/>`_.
+
 
 Avoiding long ``if-else`` statements with object instantiation
 --------------------------------------------------------------

docs/source/conf.py

@@ -15,9 +15,9 @@
 # sys.path.insert(0, os.path.abspath('.'))
 
 import datetime
+import os
 import sys
 from unittest import mock
-import os
 
 sys.path.insert(0, os.path.abspath(os.path.join("..", "..")))
 
@@ -55,7 +55,7 @@
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3/", None),
     "NumPy [latest]": ("https://docs.scipy.org/doc/numpy/", None),
-    "matplotlib": ("http://matplotlib.sourceforge.net/", None),
+    "matplotlib [stable]": ("https://matplotlib.org/stable/", None),
 }
 intersphinx_disabled_domains = ["std"]
 

docs/source/index.rst

@@ -15,13 +15,13 @@ Contents
 
 .. toctree::
 
+   packaging
    virtual_env
    version_control
    reproducible
    documentation
    clean_code
    testing
-   packaging
    remote_development
    badges