Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(build/ci): switch to uv and ruff #5777

Conversation

the-wondersmith
Copy link

PR completes the "migration" to fully using uv and ruff for python environment management and linting/formatting (respectively).

There are numerous changes to the .py files, (and they should be eyeballed at the very least), but they're all just linting fixes to get the python side of things "up to snuff" with ruff's configured lints.

The important parts here that actually require significant scrutiny are the changes to all the non-.py files. If possible, I think it would be ideal for reviewers to pull down the branch locally into a "fresh" directory and attempt to run through "common" dev workflows to ensure everything still works as intended.

Also, it's worth noting that the CI workflow does install uv and ruff automatically, but I don't think anything in any of the makefiles does the same for non-ci environments. Presumably that should be rectified prior to merging the branch 🤔? Reviewers, please chime in on it.

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. python Pull requests that update Python code labels Aug 16, 2024
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Aug 20, 2024
Copy link
Member

@kflynn kflynn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty awesome and I love how fast uv & ruff are. Couple of small things, but also we need to have ruff wrap at 100 columns correctly, and mypy is making me nervous here because I can't get it to tell me about an error that you fixed.... 🙂

build-aux/lint.mk Outdated Show resolved Hide resolved
.github/actions/setup-deps/action.yml Outdated Show resolved Hide resolved
build-aux/tools.mk Show resolved Hide resolved
python/ambassador/src/ambassador/VERSION.py Outdated Show resolved Hide resolved
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Aug 20, 2024
Copy link
Member

@kflynn kflynn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Man, this is a huge PR. 😂 Approving with some comments that can be addressed in a future PR. The ones I feel most strongly about are that we shouldn't really have any #noqa: E402 comments (we should just be able to move the imports up above the executable code) and that I'd really prefer == when comparing specifically with True and False...

Thanks much!! This must've been a huge, huge, massive pain.

def __init__(self, local_config_path: str) -> None:
self.local_path = local_config_path
self.notices: List[Dict[str, str]] = []
self.local_path, self.notices = local_config_path, []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these are more readable on two lines. They're not related to each other, and vertical space is plentiful these days.


def reset(self):
local_notices: List[Dict[str, str]] = []
local_data = ""
local_data, local_notices = "", []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

self, cmd: str, arg: Optional[Union[str, Tuple[str, Optional[IR]], Tuple[str, str]]]
self,
cmd: str,
arg: Optional[Union[str, Tuple[str, Optional[IR]], Tuple[str, str]]],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me wonder why I didn't give this type a name back when. 😂

Comment on lines +1857 to +1864
% (
config_type,
snapshot,
service_count,
listener_count,
group_count,
cluster_count,
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Why does ruff like doing this? 🤦‍♂️)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because ruff preserves behavior from black, and therefore because of the trailing comma rule in black [ref], at least I assume?

@@ -143,7 +149,7 @@ def _test_errorresponse(yaml, expectations, expect_fail=False):
% (expected_body_format_override, expected_filter)
)
print("checking m: ", m)
actual_filter = m["filter"]
# actual_filter = m["filter"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another one to just delete, I think.

Comment on lines +1 to +2
# Import all the real tests from other files, to make it easier to pick and choose during development.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment should go if all the imports have gone.

IIRC the point of all this stuff had to do with the way pytest found tests -- I assume that's different now...

@@ -121,7 +124,7 @@ def check(self):
elif "X-Envoy-Overloaded" in result.headers:
pending_overloaded += 1

failed = False
# failed = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking this can just be deleted.

@@ -122,7 +128,7 @@ def check(self):
valid += 1
elif self.results[i].status == 500:
errors += 1
msg = "Errors: {}, Valid: {}".format(errors, valid)
# msg = "Errors: {}, Valid: {}".format(errors, valid)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should just delete this. Originally I think it was used as the assertion string (cf the ActiveHealthCheckTest function earlier in this file) but it's a terrible string for that anyway.

env:
DEBIAN_FRONTEND: noninteractive
run: |
python -m pip install -U awscli packaging
uv venv
uv --no-config pip install -U awscli
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think awscli is in the dev dependencies now, no?

(I also expect we'll likely get rid of it – I think it's only ALabs-specific stuff.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did move everything else that used to be installed in that step into the dev deps, but nothing in the Python packages explicitly used anything from awscli, so I figured maybe it was a CI-specific thing and just left it.

We can absolutely sunset it / remove it entirely though, if it's an ALabs-exclusive

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Aug 21, 2024
@the-wondersmith
Copy link
Author

...some comments that can be addressed in a future PR.

I'll get to work on them posthaste 😅

The ones I feel most strongly about are that we shouldn't really have any #noqa: E402 comments (we should just ...

Agreed. For historical purposes though - I left the imports "out of order" and added the E402 ignores because I didn't have the bandwidth to go through and ensure that the patching routine that's happening in between the import blocks isn't necessary any more w/ the new package structure and switch to uv.

... and that I'd really prefer == when comparing specifically with True and False...

Ummm.... that might be a sticking point? There actually is a meaningful difference between is and == in Python [ref]. Plus it's a linting violation in ruff specifically [ref], but also in several other linters / style guides.

Thanks much!! This must've been a huge, huge, massive pain.

My pleasure! It wasn't that big a pain - labors of love hyperfixation rarely are 😂

@the-wondersmith the-wondersmith merged commit d541135 into emissary-ingress:dev/v4 Aug 21, 2024
5 of 6 checks passed
@kflynn kflynn mentioned this pull request Sep 7, 2024
kflynn added a commit that referenced this pull request Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm This PR has been approved by a maintainer python Pull requests that update Python code size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants