From 850ebae82163822f63fb78286df70af286b660c6 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 15 Aug 2024 16:23:33 +0100 Subject: [PATCH] Write down first draft of setuptools' policy for dependencies --- docs/development/developer-guide.rst | 69 +++++++++++++++++++++------- 1 file changed, 53 insertions(+), 16 deletions(-) diff --git a/docs/development/developer-guide.rst b/docs/development/developer-guide.rst index 4566fef8d0..26abda96e3 100644 --- a/docs/development/developer-guide.rst +++ b/docs/development/developer-guide.rst @@ -112,23 +112,60 @@ To build the docs locally, use tox:: .. _Sphinx: https://www.sphinx-doc.org/en/master/ .. _published documentation: https://setuptools.pypa.io/en/latest/ ---------------------- -Vendored Dependencies ---------------------- - -Setuptools has some dependencies, but due to `bootstrapping issues -`_, those dependencies -cannot be declared as they won't be resolved soon enough to build -setuptools from source. Eventually, this limitation may be lifted as -PEP 517/518 reach ubiquitous adoption, but for now, Setuptools -cannot declare dependencies other than through -``setuptools/_vendor/vendored.txt`` and -``pkg_resources/_vendor/vendored.txt``. - -All the dependencies specified in these files are "vendorized" using a -simple Python script ``tools/vendor.py``. -To refresh the dependencies, run the following command:: +.. _dependency-policy: + +---------------------------------- +Adopting and Updating Dependencies +---------------------------------- + +When developing ``setuptools``, contributors should strive to keep the codebase +efficient and maintainable. +Ideally, we would like to adhere to the DRY (Don’t Repeat Yourself) philosophy, +use the right tool for the job, and only make pragmatic compromises where +necessary (or if there's a strong case). + +This means that incorporating dependencies to streamline programming is in +principle possible, and that these would be readily adopted as needed. +However, ``setuptools`` is a :pep:`517`-compliant build-backend, and therefore +it is subject to a strong limitation on which dependencies it can or cannot +incorporate. + +To circumvent these limitations, ``setuptools`` bundles and distributes these +dependencies together into a nested directory in its source tree, in a process +called "vendorization", which is implemented using a simple Python script +``tools/vendor.py``. Unfortunately, this process still presents challenges, such as: + +- the increasing size of ``setuptools`` distribution archives and the associated + time to download, +- the existence of old and incompatible versions of dependencies available + elsewhere in ``sys.path``, +- the difficulty for downstream packagers (e.g. different Linux distributions) + to adopt and distribute ``setuptools``, +- the open bootstrapping issues discussed at `pypa/packaging-problems#342 + `_. + +Therefore, we recommend developers to observe the following policies before +deciding to adopt new dependencies: + +1. Developers **SHOULD NOT** introduce dependencies that bring along other + transient or indirect dependencies, unless there is a strong reason to. +2. Developers **MUST NOT** introduce dependencies (either direct or transient) + that require binary extension modules. +3. Developers **SHOULD** prefer similar alternatives available in the standard + library whenever possible. +4. Developers **SHOULD** avoid large dependencies but from which ``setuptools`` + uses a very small number of features/functions (high cost of adoption but + low utilisation). +5. Developers **SHOULD** ensure that a wide range of available versions of a + given dependency is compatible with ``setuptools`` codebase (ideally + ``setuptools`` should work with whichever version of the dependency the end + user already have installed in their environment). + +Once adopted, dependencies should be listed in the ``core`` optional-dependencies +group in ``setuptools``' ``pyproject.toml`` configuration file. + +To refresh the vendored dependencies, a contributor can run the following command:: $ tox -e vendor