-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[uv] port pyright script from pip -> uv (#20199)
## Summary & Motivation Companion PR: dagster-io/internal#8518 Make `scripts/run-pyright.py` use `uv` instead of `pip`. Experiment with size n=2 on BK shows a ~5-10m speedup in the OSS pyright step from 14m40s to (9m39s, 6m13s). `uv` also integrates some utilities from `pip-tools` that may be able to improve `make rebuild_pyright_pins` (though the process is already sped up a lot by this PR). I'll attempt to incorporate this functionality in a future PR. ## Notes on editable installs For some reason, even though no editable installs are altered in the pins, two imports in our examples started being flagged as type errors-- in both cases it was a sibling module import being flagged as unresolvable. Must be some subtle difference in how editable installs are being done between `pip` and `uv`, even with `--editable-mode=compat` set. In any case I was able to resolve the errors by making these two instances relative imports, which is arguably better in these cases anyway as it doesn't implicitly assume a working directory. ## Notes on `uv venv` python interpreter resolution The most tricky part of this was discovering and correcting for the python resolution logic `uv` uses to create virtual environments. Unlike the built-in `venv` package, `uv` (because it does not actually use python at all) _does not use the python it is installed under by default_. That is: ``` $ which pip # /some/pip (which goes with /some/python) $ pip install uv $ uv venv .venv # the created venv may not be using /some/python! ``` This had significant consequences on BK internal. The images used in internal contain a python at both: - `/bin/python` - `/usr/local/bin/python` The one at `/usr/local/bin/python` is the one used for all our tests and is the default target for `pip install`. But when creating a virtual env, `uv` actually chooses `/bin/python` by default. At time of this PR, the pyright env cannot be built against `/bin/python` (3.11.2) for obscure reasons having to do with PEP 517 build isolation and the failure of `pendulum==2.1.2` to accurately require `setuptools` as a buildtime dependency 💀. There's also some other nonsense with `xmlsec`. To fix these issues I had to point `uv` to `/usr/local/bin/python` when creating the venv on BK internal. ## How I Tested These Changes BK
- Loading branch information
Showing
6 changed files
with
149 additions
and
124 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 3 additions & 1 deletion
4
examples/docs_snippets/docs_snippets/integrations/duckdb/tutorial/resource/downstream.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.