Replace flake8 with ruff in pre-commit and ci linting

[mraspaud]

This PR uses ruff instead of flake8 and mccabe.

It also add news rules for linting, so I'm expecting this to fail and start a discussion about what rules to use.

When that is done, I will fix satpy accordingly.

As first draft, here what I propose:

• port of previously used flake8 and plugins used in satpy

  • flake8-builtins
  • flake8-bugbear
  • pydocstyle
  • mccabe
  • flake8-debugger

• new checks

  • pycodestyle
  • pyflakes
  • isort
  • flake8-bandit
  • flake8-pyteststyle
  • flake8-quotes
  • flake8-tidyimports
  • pep8-naming

I'm also open to adding more checks, the full list is here:
https://docs.astral.sh/ruff/rules/

[pnuu]

What triggers this change? Is it necessary to switch over for some reason, or is ruff simply better in some way?

[mraspaud]

What triggers this change? Is it necessary to switch over for some reason, or is ruff simply better in some way?

Ruff has much better performance. Since we run linting on every commit (when pre-commit is installed), I thought it would be nice to save computing time for everyone, and for the linting ci.

Another good thing is that is supports pyproject.toml which flake8 does not do.

See a list of benefits on the top of this page https://docs.astral.sh/ruff/

[djhoese]

I am OK with adopting ruff, but I think we could drop the lint/style checks in the CI in favor of pre-commit.ci doing it all.

Also...adopt black?

[mraspaud]

I am OK with adopting ruff, but I think we could drop the lint/style checks in the CI in favor of pre-commit.ci doing it all.

Ok, I can do that

Also...adopt black?

Black is not a linter, this is about linting 😁

[djhoese]

Sure, a separate PR. But it would be run in pre-commit so it isn't way out of the realm of conversation here and is something that would automatically fix many syntax/style errors complained about here.

[mraspaud]

Indeed, I was just trying to avoid the question as I don't like black and I would rather not have it in satpy, which may not be a popular opinion.

I would like this PR to be about ruff, but we can make an issue about black to discuss the pros and cons of it if you like.

[djhoese]

I vote for removing the linting job in CI, but otherwise I'm good with whatever rules you want to apply. Can we tell ruff what version(s) of python to target so you don't have to have the B905 rule ignored?

[gerritholl]

Looks good, but let's merge this after and not before the new release.

[codecov]

Codecov Report

Merging #2586 (41759a8) into main (bc32c94) will increase coverage by 0.00%.
The diff coverage is 90.64%.

@@           Coverage Diff           @@
##             main    #2586   +/-   ##
=======================================
  Coverage   95.18%   95.18%           
=======================================
  Files         354      354           
  Lines       51253    51256    +3     
=======================================
+ Hits        48786    48789    +3     
  Misses       2467     2467           

Flag	Coverage Δ
behaviourtests	4.24% <1.40%> (-0.01%)	⬇️
unittests	95.81% <90.63%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
satpy/__init__.py	86.66% <100.00%> (ø)
satpy/_config.py	96.59% <100.00%> (ø)
satpy/_scene_converters.py	92.30% <ø> (ø)
satpy/composites/cloud_products.py	95.16% <100.00%> (ø)
satpy/composites/glm.py	100.00% <100.00%> (ø)
satpy/composites/spectral.py	100.00% <100.00%> (ø)
satpy/composites/viirs.py	94.76% <100.00%> (ø)
satpy/dataset/anc_vars.py	90.00% <100.00%> (ø)
satpy/dataset/data_dict.py	92.23% <100.00%> (ø)
satpy/dataset/metadata.py	100.00% <100.00%> (ø)
... and 279 more

... and 11 files with indirect coverage changes

[mraspaud]

Looks good, but let's merge this after and not before the new release.

Will do!

[mraspaud]

@djhoese @gerritholl ok, I think have to stop working on this, because it's way too time consuming for my health atm. I did not add all the rules I planned to, but the ones we have currently in main are implemented, and even a couple new ones.

If you want to review this, I highly recommend you check the changes by commit, since they should be consistent with one new rule per commit. Especially the one for pytest-style, which is a mouthful.

[djhoese]

I think have to stop working on this, because it's way too time consuming for my health atm.

I was honestly surprised when you said you were starting this. This is the same reason I didn't do it earlier.

[djhoese]

There are a lot of test failures.

[djhoese]

Thanks for putting all the work into this. There are test failures, but otherwise this looks good to me. Let's see if we can track down these failures.