Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for uv-installed pythons #842

Merged
merged 12 commits into from
Oct 2, 2024
Merged

Conversation

saucoide
Copy link
Contributor

A question more than a PR, now that uv supports bringing its own pythons, i wanted to try simplify a CI and not have to install python from other sources, but nox is currently now able to find uv-installed pythons (someone else's issue @UV about this same thing astral-sh/uv#6579 )

would it make sense to add a more fleshed out version of this? meaning passing the desired interpreter to uv venv -p {interpreter}, or should this be done by other mechanism that adds uv's python interpreters to $PATH ?

note: this is just a draft to show an example, kind of works but also breaks tests and probably breaks uv|virtualenv and other things

@henryiii
Copy link
Collaborator

henryiii commented Sep 6, 2024

I think it makes sense, especially if it's roughly this easy. :)

@saucoide
Copy link
Contributor Author

@henryiii i've updated this a bit, but there is a failing test locally test__resolved_interpreter_windows_path_and_version_fails which i dont quite understand though :/

@saucoide
Copy link
Contributor Author

im not happy about ignoring the uv-python cases for those windows tests, but i am not familiar with how nox works on windows to touch that part, so if anyone has any ideas it'll be helpful

@nugend
Copy link

nugend commented Sep 20, 2024

@saucoide Just want to express my appreciation for this PR. This has already made my life way easier!

caused problems with pypy
@saucoide
Copy link
Contributor Author

thanks but tbh i wouldn't use it just yet 😅 there is still something wrong with it, the tox-to-nox tests fails mixing up the interpreter depending on which order the tests are run, so there is something broken still

@saucoide
Copy link
Contributor Author

ok so this may be an issue with uv (or with how i thought it would behave) looks like if you run

uv python install pypy3.8 (there is a test that does it)

and then later on

uv venv -p python3.8 .venv

uv will go ahead and use the existing pypy to make the venv, will try to figure out if this is intended from uv's side or not

@henryiii
Copy link
Collaborator

I wonder if we could make pythonX -> cpythonX for uv? Just a thought.

@saucoide
Copy link
Contributor Author

I wonder if we could make pythonX -> cpythonX for uv? Just a thought.

yes I think we might have to, originally i thought passing it as-is to uv would be nice, because you can then specify the python with uv's own version syntax, and whatever versions they support e.g. 3.13t, without having to do any translation. But it looks like it it might not work out, depending on what they do here astral-sh/uv#7118

ill try to test tomorrow with replacing python/cpython

@saucoide
Copy link
Contributor Author

looks the uv people were faster so it works without changes now, this latest version seems ok

@saucoide saucoide marked this pull request as ready for review September 24, 2024 23:57
@saucoide
Copy link
Contributor Author

@henryiii could you review and let me know if you think this approach is ok?

@henryiii
Copy link
Collaborator

henryiii commented Oct 1, 2024

I think this is great, my only question is if we should have some control for this. We could in theory add downloading to the other backends (similar to how PDM and Hatch download Pythons - and it comes from the same place as uv's), so maybe we should have --download-python=auto,always,never? Not necessary for this PR unless the default wasn't auto.

@henryiii henryiii changed the title dummy support for uv-installed pythons feat: support for uv-installed pythons Oct 1, 2024
@saucoide
Copy link
Contributor Author

saucoide commented Oct 1, 2024

do you mean nox doing the downloading itself, or checking if hatch/pdm/uv are in path and use whatever is available to get the version requested?

@henryiii
Copy link
Collaborator

henryiii commented Oct 1, 2024

I believe PDM’s downloader is a standalone Python library we could use. But yes, nox downloading it itself.

@saucoide
Copy link
Contributor Author

saucoide commented Oct 1, 2024

yeah that sounds useful, what would be the behaviors of auto? I'd interpret it like

  • auto: prefer a present compatible version + download only if missing
  • always: always use the standalone python, even if a compatible version is already present from another source?
  • never: only if already preset

i can try to experiment a bit next week and see if i get something working (this week's a bit busy for me)

@henryiii
Copy link
Collaborator

henryiii commented Oct 2, 2024

Yes, that's exactly what I was thinking. The current behavior is "auto" for uv, and "never" for the other backends. (Actually, not sure about conda, that might be "always").

@henryiii henryiii merged commit a49c730 into wntrblm:main Oct 2, 2024
23 checks passed
@layday
Copy link
Contributor

layday commented Oct 16, 2024

This caught me completely by surprise in CI when I pre-installed Python with uv, only to have nox install and run my tests under a different version because I omitted the "t" suffix and even though I'd passed --error-on-missing-interpreters: https://github.com/layday/instawow/actions/runs/11359937505/job/31596873371#logs. I believe this should respect the --error-on-missing-interpreters flag.

@saucoide
Copy link
Contributor Author

saucoide commented Oct 16, 2024

@layday
:/ yeah it doesn't fail because there is an interpreter, the one it downloaded, i haven't had time to add the options henryii mentioned to have a download=never in nox, meanwhile i think it should be possible to disable it for uv directly setting UV_PYTHON_DOWNLOADS=never

@layday
Copy link
Contributor

layday commented Oct 18, 2024

meanwhile i think it should be possible to disable it for uv directly setting UV_PYTHON_DOWNLOADS=never

Thanks for the hint.

@henryiii
Copy link
Collaborator

henryiii commented Oct 26, 2024

Another issue I've noticed: this will install some uv pythons. Then subsequent uses of uv like uv run will suddenly start using an older Python instead of 3.13 because it installed some older missing version and it prefers its own Pythons. Not sure if there's a way to keep this from affecting uv usage.

uv run thing # Uses 3.13
nox ... # installs missing 3.12
uv run thing # Now uses 3.12

Simi-related, we should also make sure we fully support 3.13t as an identifier.

@saucoide
Copy link
Contributor Author

I tried to reproduce that but for me:

uv run python --version    # Python 3.12.3
uvx nox   # installs missing 3.10
uv run python --version    # Python 3.12.3

uv --version  # 0.4.26

That works for me and this project doesn't have a .python-version explicitly setting it, so i think it has to do with what python uv considers the default if not specified, to test it i made another project and for example:

uv run python --version    # Python 3.10.15
uvx nox   # installs missing 3.12
uv run python --version    # Python 3.12.7

So I think if no .python-version exists (e.g. uv python pin 3.13) and UV_PYTHON is unset, uv is probably defaulting to the highest version present, but 3.13 is for some reason not included yet?

what version of 3.13 was it? for me it currently installs an rc version so maybe it excludes it as non-stable

ps: i still want to try to add the options to control download + with other backends too but im currently travelling on vacations so i haven't found the time yet

@henryiii
Copy link
Collaborator

henryiii commented Nov 8, 2024

Any updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants