How to sort namespace packages like isort does?

[smartYSC]

At our company, we have all our packages in the same namespace (let's assume foo). The packages each contain one module and are developed in separate repos (let's call them bar and baz).

When using stuff from foo.baz inside foo.bar, all imports from foo.baz were sorted into the third-party category by isort. Which makes sense in our context, as those are libraries we consider third-party in the current package because they are developed elsewhere.

Now, we want to let ruff do the import sorting and it sorts everything in the foo namespace to first-party.

In our repo the code lives in src/foo/bar, with foo being a namespace package (no __init__.py).

I fiddled with all the different ruff config parameters but could not find the right combination to behave like isort does.

This came closest to the desired result:

[tool.ruff]
src = ["src"]
namespace-packages = ["foo"]

[tool.ruff.lint.isort]
known-third-party = ["foo.baz"]

But the above solution requires us that we list ALL our packages explicitly as third-party which is not feasible.

Instead, ruff should treat everything in src/foo/bar as first-party, and everything else in the foo namespace as third-party.

Could anyone help me finding the right configuration?

[charliermarsh]

Can you try disabling detect-same-package?

[smartYSC]

I just tried it again: no combination of the above settings and the following works:

[tool.ruff.lint.isort]
detect-same-package = false

Only known-third-party = ["foo.baz"] does the trick.

[charliermarsh]

I don't fully understand the setup but FWIW known-third-party supports globbing ("foo.*", IIRC).

[smartYSC]

Globbing also does not seem to help :-(

Let me try to explain in more detail:

We have one package which provides the module foo.baz. This is installed via pip into site-packages of a venv (if that makes any difference).

Then we have the current repository that we want to run ruff import sorting on. This repository contains the module foo.bar with the following structure:

my-repo/
├─ pyproject.toml (containing ruff config)
├─ src/
│  ├─ foo/                      (this folder has no __init__.py, so it is a namespace package)
│  │  ├─ bar/
│  │  │  ├─ __init__.py
│  │  │  ├─ mod1.py
│  │  │  ├─ mod2.py

Inside mod1.py we have the follow code:

# mod1.py

from pathlib import Path  # just some import from standard library

import foo.baz  # our other library we *consider* to be third-party code in the current repo
import numpy as np  # some actual third-party library

from foo.bar import mod2  # first-party import from the current repo

Let's run ruff check --extend-select=I --verbose src/foo/bar/mod1.py --fix to check what is going on.

ruff output without any special config

# mod1.py

from pathlib import Path 

from foo.bar import mod2
import foo.baz  
import numpy as np

Ruff considered both imports to be third-party imports and grouped it with numpy.

[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.baz' as Known(ThirdParty) (NoMatch)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'pathlib' as Known(StandardLibrary) (KnownStandardLibrary)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.bar' as Known(ThirdParty) (NoMatch)

configure src

Let's configure the src to make ruff detect first-party correctly:

[tool.ruff]
src = ["src"]

Resulting formatting:

# mod1.py

from pathlib import Path 

import numpy as np

from foo.bar import mod2 
import foo.baz 

Logs:

[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.baz' as Known(FirstParty) (SourceMatch("/home/user/my-repo/src"))
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'pathlib' as Known(StandardLibrary) (KnownStandardLibrary)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.bar' as Known(FirstParty) (SourceMatch("/home/user/my-repo/src"))

And here might actually be a bug in ruff, now that I see it. It considers foo.baz to match the source path. Likely, because it sees foo in src and decided this is enough confirmation, that foo.baz is inside there. But if it looked one level deeper, it would have figured out, that there is no baz in src/foo.

set namespace-package

Ok, let's try to help ruff by telling it foo is actually a namespace package (it does not seem to have figured it out before):

[tool.ruff]
src = ["src"]
namespace-packages = ["foo"]

This has no effect and produces the output as above.

set known-third-party

Let's make foo.baz explicitly a third party package:

[tool.ruff]
src = ["src"]
namespace-packages = ["foo"]

[tool.ruff.lint.isort]
known-third-party = ["foo.baz"]

Logs:

[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.baz' as Known(ThirdParty) (KnownThirdParty)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'pathlib' as Known(StandardLibrary) (KnownStandardLibrary)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.bar' as Known(FirstParty) (SourceMatch("/home/yannic/gits/file-python/src"))

This results in the correct output (similar to isort).

Using wildcard in known-third-party

As you suggested:

[tool.ruff]
src = ["src"]
namespace-packages = ["foo"]

[tool.ruff.lint.isort]
known-third-party = ["foo.*"]

All packages in foo, including the one in src are considered third-party now:

[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.baz' as Known(ThirdParty) (KnownThirdParty)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'pathlib' as Known(StandardLibrary) (KnownStandardLibrary)
[ruff_linter::rules::isort::categorize][DEBUG] Categorized 'foo.bar' as Known(ThirdParty) (KnownThirdParty)

Conclusion

ruff seems too quick to decide when checking the src folder if something is first-party or third-party. It only looks if the top level module name matches.

I tried my best to make it clearer what is going on here. I know these kinds of things are always hard to explain. Thank you for looking into this!

[smartYSC]

Just for completeness: This is the same issue reported in #12984 and the fix waits to be merged in #12987.