Exclude statements in particular files by pattern?

[tgamblin]

We're considering using ruff for Spack, where we currently use black.

Spack is a package manager. It has core Python code, which we don't have many special formatting needs for. It also has 8,000 or so package.py recipe files, and the recipes have some special functions that implement a DSL.

So, for a package with lots of versions, we might write:

class SomePackage(CMakePackage):
    url = "https://example.com/some-package-1.21.0.tar.gz"

    version("1.22.2", sha256="374ea82b289ec738e968267cac59c7d5ff180f9492250254784b2044e90df5a9")
    version("1.22.1", sha256="79c9b91d7f109515a25fc3ecdaad125d67e6bdb54f6d4d98580f46799caea321")
    version("1.22.0", sha256="4d196c3d41a0d6c1dfc64d04e3cc1f608b0c436bd87b7060ce3e23234e1f4d5c")
    version("1.21.6", sha256="124926a62e45f78daabbaedb9c011d97633186a33c238ffc1e25320c02046248")
    version("1.21.5", sha256="285cbbdf4b6e6e62ed58f370f3f6d8c30825d6e56c5853c66d3c23bcdb09db19")

Idea

I would like to exclude version() and other directives containing a sha256 from formatting and line length checks. e.g., I'd like to be able to write something like this in ruff's config:

"packages/*/package.py"."^\s*version\(.*sha256=\"[a-f0-9]{64}\"\)" = ["E501"]

This seems like it might be complicated, as the regex would need to apply to a statement that might span multiple lines. For our purposes you could simplify it, since we really only need to match on the version( call, which is always first and never spread over multiple lines:

"packages/*/package.py"."^\s*version\(" = ["E501"]

I am curious whether something like this would be possible to implement in ruff, and whether ruff devs would be open to it.

Motivation

With long hashes, small argument changes can result in big formatting changes, e.g. if we deprecate those versions we might write:

    with default_args(deprecated=True):
        version("1.22.2", sha256="374ea82b289ec738e968267cac59c7d5ff180f9492250254784b2044e90df5a9")
        version("1.22.1", sha256="79c9b91d7f109515a25fc3ecdaad125d67e6bdb54f6d4d98580f46799caea321")
    ...

Or:

    version("1.22.2", sha256="374ea82b289ec738e968267cac59c7d5ff180f9492250254784b2044e90df5a9", deprecated=True)
    version("1.22.1", sha256="79c9b91d7f109515a25fc3ecdaad125d67e6bdb54f6d4d98580f46799caea321", deprecated=True)
    ...

Unfortunately, both of these now exceed the max line length for our black configuration, which we've set at 99 characters. So, these end up looking like:

    with default_args(deprecated=True):
        version(
            "1.22.4", sha256="fed720678e728a7ca30ba8d1ded1caafe27d16028fab0232b8ba8e22008fb784"
        )
        version(
            "1.22.2", sha256="374ea82b289ec738e968267cac59c7d5ff180f9492250254784b2044e90df5a9"
        )

or, worse:

    version(
        "1.22.4",
        sha256="fed720678e728a7ca30ba8d1ded1caafe27d16028fab0232b8ba8e22008fb784",
        deprecated=True,
    )
    version(
        "1.22.2",
        sha256="374ea82b289ec738e968267cac59c7d5ff180f9492250254784b2044e90df5a9",
        deprecated=True,
    )

This gets to be hard to read and causes headaches for devs when updating packages. It also adds more churn to the repo than we'd really like. We'd like the rest of the package.py file to be formatted with our regular code formatting rules, but directives like this (which really just provide metadata) should ideally be left untouched. We used to get this effect by pre-filtering our files and adding # noqa to lines that met certain criteria, but that got to be quite slow (copying thousands of files, modifying them, formatting them, copying them back). We simplified by choosing a slightly longer line length and standardizing on black, but a number of us still miss the exceptions.

[MichaReiser]

Thanks for starting this discussion

I'm a bit hesitant to add content-based filtering because it requires running through the entire file content to make a decision whether it should be included or not (the not can probably exit earlier). The filter configuration would have to be its own field to that projects pay this overhead when they aren't using the functionality. Either way, I'm worried about the performance impact

Spack is a package manager. It has core Python code, which we don't have many special formatting needs for. It also has 8,000 or so package.py recipe files, and the recipes have some special functions that implement a DSL.

What I understand is that you don't have specific formatting needs. Have you considered excluding those files from formatting altogether or using different formatting settings for them?