Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Third-party namespace packages can be incorrectly considered first-party packages by isort rules #12984

Open
tehkirill opened this issue Aug 19, 2024 · 4 comments · May be fixed by #12987
Open

Third-party namespace packages can be incorrectly considered first-party packages by isort rules #12984

tehkirill opened this issue Aug 19, 2024 · 4 comments · May be fixed by #12987
Assignees
@AlexWaygood
Labels
isort Related to import sorting

Comments

@tehkirill
Copy link

Hi, team. Thanks for your hard work.

I noticed a change in how known-first-party imports get classified recently.
If I'm working on my_namespace.a project, I expect my_namespace.b to be classified as a Known(ThirdParty) (so does isort). This was the case with ruff as well, up until version 0.5.7 .

Working on my_namespace.a project, with the following stucture:

ruff-namespace-src-imports-test/
  ├─ main.py
  ├─ ruff.toml
  └─ src/
     └─ my_namespace/
        └─ a/
           └─ ...

# ruff.toml
[lint.isort]
known-first-party = ["my_namespace.a"]
# main.py
import os

import my_namespace.b
import numpy

import my_namespace.a

^ I see it as a valid import order, so does v0.5.6:

$ python3.12 -m ruff check main.py --select I --diff -v
[2024-08-19][11:11:40][ruff::resolve][DEBUG] Using configuration file (via parent) at: C:\ruff-namespace-src-imports-test\ruff.toml
[2024-08-19][11:11:40][ruff_workspace::pyproject][DEBUG] `project.requires_python` in `pyproject.toml` will not be used to set `target_version` when using `ruff.toml`.
[2024-08-19][11:11:40][ruff::commands::check][DEBUG] Identified files to lint in: 4.5661ms
[2024-08-19][11:11:40][ruff::diagnostics][DEBUG] Checking: C:\ruff-namespace-src-imports-test\main.py
[2024-08-19][11:11:40][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'my_namespace.a' as Known(FirstParty) (KnownFirstParty)
[2024-08-19][11:11:40][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'os' as Known(StandardLibrary) (KnownStandardLibrary)
[2024-08-19][11:11:40][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'my_namespace.b' as Known(ThirdParty) (NoMatch)
[2024-08-19][11:11:40][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[2024-08-19][11:11:40][ruff::commands::check][DEBUG] Checked 1 files in: 2.7995ms

But 0.5.7+ (including the latest 0.6.1) sees otherwise:

$ python3.12 -m ruff check main.py --select I --diff -v
[2024-08-19][11:12:44][ruff::resolve][DEBUG] Using configuration file (via parent) at: C:\ruff-namespace-src-imports-test\ruff.toml
[2024-08-19][11:12:44][ruff_workspace::pyproject][DEBUG] `project.requires_python` in `pyproject.toml` will not be used to set `target_version` when using `ruff.toml`.
[2024-08-19][11:12:44][ruff::commands::check][DEBUG] Identified files to lint in: 4.8756ms
[2024-08-19][11:12:44][ruff::diagnostics][DEBUG] Checking: C:\ruff-namespace-src-imports-test\main.py
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'my_namespace.a' as Known(FirstParty) (KnownFirstParty)
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'os' as Known(StandardLibrary) (KnownStandardLibrary)
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'my_namespace.b' as Known(FirstParty) (SourceMatch("C:\\ruff-namespace-src-imports-test\\src"))
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'my_namespace.b' as Known(FirstParty) (SourceMatch("C:\\ruff-namespace-src-imports-test\\src"))
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'os' as Known(StandardLibrary) (KnownStandardLibrary)
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'numpy' as Known(ThirdParty) (NoMatch)
[2024-08-19][11:12:44][ruff_linter::rules::isort::categorize][DEBUG] Categorized 'my_namespace.a' as Known(FirstParty) (KnownFirstParty)
--- main.py
+++ main.py
@@ -1,6 +1,6 @@
 import os

-import my_namespace.b
 import numpy

 import my_namespace.a
+import my_namespace.b

[2024-08-19][11:12:44][ruff::commands::check][DEBUG] Checked 1 files in: 2.8168ms
Would fix 1 error.

Removing src/my_namespace correctly classifies my_namespace.b as known-third-party again.
@MichaReiser MichaReiser added the question Asking for support or clarification label Aug 19, 2024
@MichaReiser
Copy link
Member

Hi @tehkirill

This behavior changed with Ruff 0.6 where the directory src is now included in the src setting. You can get back the old behavior by setting src = ["."] which was the default until Ruff 0.6.

You can read more about it in our blog post.

@tehkirill
Copy link
Author

Thanks for the prompt response and linking to where it was documented, @MichaReiser
In my case - I am using src/ layout - don't think I should exclude it to make this work.

The way it's implemented at the moment makes sense for projects without namespaces, eg ./src/requests from requests repo.

It does, however, incorrectly identifies a third-party import under the same namespace as first-party, e.g. using an example from packaging.python.org:

mynamespace-subpackage-a/
    pyproject.toml
    src/
        mynamespace/
            subpackage_a/
                __init__.py

mynamespace-subpackage-b/
    pyproject.toml
    src/
        mynamespace/
            subpackage_b/
                __init__.py
            module_b.py

When working on mynamespace-subpackage-a, mynamespace-subpackage-b would be identified as first-party (in fact, anything starting with mynamespace. would), I don't think it's an expected behaviour.

@AlexWaygood AlexWaygood added the isort Related to import sorting label Aug 19, 2024
@AlexWaygood
Copy link
Member

When working on mynamespace-subpackage-a, mynamespace-subpackage-b would be identified as first-party (in fact, anything starting with mynamespace. would), I don't think it's an expected behaviour.

That's an interesting edge case! This seems to me to be a general issue with the way our isort rules detect first-party packages when it comes to namespace packages, that has been exposed in your project by the recent change to isort's configuration defaults, rather than an issue with the change in defaults itself.

@AlexWaygood AlexWaygood removed the question Asking for support or clarification label Aug 19, 2024
@AlexWaygood AlexWaygood changed the title Having src/my_namespace/a changes my_namespace.b from known-third-party to known-first-party Third-party namespace packages can be incorrectly considered first-party packages by isort rules Aug 19, 2024
@AlexWaygood AlexWaygood self-assigned this Aug 19, 2024
@AlexWaygood AlexWaygood linked a pull request Aug 19, 2024 that will close this issue
Draft
@notatallshaw notatallshaw mentioned this issue Aug 24, 2024
Draft
@smartYSC
Copy link

smartYSC commented Nov 6, 2024

I ran into the same problem recently but thought I was just too stupid to find the right settings combination. I opened a discussion about it here: #12038

We did not find a solution back then and I went on to other tasks. Now I revisted this and found this issue.

I built ruff from the linked PR #12987 and this indeed fixes the issue!

We now need to have this merged...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AlexWaygood AlexWaygood

Labels
isort Related to import sorting
Projects
None yet
Milestone
No milestone
4 participants
@MichaReiser @tehkirill @AlexWaygood @smartYSC