Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated pip and packaging versions to work with free-threading packages #2514

Merged
merged 1 commit into from
Jan 3, 2025
Merged

Updated pip and packaging versions to work with free-threading packages #2514

merged 1 commit into from
Jan 3, 2025

Conversation

vfdev-5
Copy link
Contributor

@vfdev-5 vfdev-5 commented Dec 19, 2024

We had an issue to install jaxlib with bazel when running the following command (using rules_python v0.39):

          bazel test \
              --repo_env=HERMETIC_PYTHON_VERSION=3.13-ft \
              --repo_env=JAX_NUM_GENERATED_CASES=$JAX_NUM_GENERATED_CASES \
              --repo_env=JAX_ENABLE_X64=$JAX_ENABLE_X64 \
              --repo_env=JAX_SKIP_SLOW_TESTS=$JAX_SKIP_SLOW_TESTS \
              --repo_env=PYTHON_GIL=$PYTHON_GIL \
              --repo_env=TSAN_OPTIONS="halt_on_error=1" \
              --//jax:build_jaxlib=false \
              --nocache_test_results \
              --test_output=all \
              //tests:cpu_tests

According to @vam-google, this was due to old pip/packaging versions. We updated them and this helped to make work the whole building/testing pipeline: jax-ml/jax#24898
So, we would like to upstream the patch: https://github.com/jax-ml/jax/pull/24898/files#diff-e3dc8d7d2bf5d057f95b86bcff7360b6c99fa1f458882fd112b58da4aceb53e4

@vfdev-5
Copy link
Contributor Author

vfdev-5 commented Dec 19, 2024

@aignas can you help with updating the tests to take new versions into account?

@aignas
Copy link
Collaborator

aignas commented Dec 20, 2024

Yeah, not sure about the fix, but it does seem that the suggested change is breaking the reproducibility of the compiled requirement files. I am wondering though if this is just a test setup issue.

Unfortunately, I have no time to work on this.

@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch from d76ba9a to c86039f Compare December 21, 2024 01:12
copybara-service bot pushed a commit to google/tsl that referenced this pull request Dec 21, 2024
@hawkinsp @copybara-github
Update packaging and pip versions in rules_python to enable freethrea…
60523e4 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 708471912
@copybara-service copybara-service bot mentioned this pull request Dec 21, 2024
Merged
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Dec 21, 2024
@hawkinsp @Google-ML-Automation
Update packaging and pip versions in rules_python to enable freethrea…
1b8d65d 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from #20723 for merging.

PiperOrigin-RevId: 708471912
@copybara-service copybara-service bot mentioned this pull request Dec 21, 2024
Merged
@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch from c86039f to 0dcfd8c Compare December 21, 2024 01:54
@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch from 0dcfd8c to c0d36d4 Compare December 21, 2024 02:12
@vfdev-5 vfdev-5 mentioned this pull request Dec 21, 2024
Closed
copybara-service bot pushed a commit to google/tsl that referenced this pull request Dec 21, 2024
@hawkinsp @copybara-github
Update packaging and pip versions in rules_python to enable freethrea…
bdc4d6c 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Dec 21, 2024
@hawkinsp @Google-ML-Automation
Update packaging and pip versions in rules_python to enable freethrea…
2721c86 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from #20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Dec 21, 2024
@hawkinsp @tensorflower-gardener
Update packaging and pip versions in rules_python to enable freethrea…
7f04517 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 708471912
@copybara-service copybara-service bot mentioned this pull request Dec 21, 2024
Merged
@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch 2 times, most recently from 6701c97 to 9b406af Compare December 23, 2024 09:29
@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch 4 times, most recently from a3249de to c9307ef Compare December 23, 2024 10:52
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Dec 23, 2024
@hawkinsp @Google-ML-Automation
Update packaging and pip versions in rules_python to enable freethrea…
44b1441 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from #20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Dec 23, 2024
@hawkinsp @tensorflower-gardener
Update packaging and pip versions in rules_python to enable freethrea…
1c2d9b6 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Dec 23, 2024
@hawkinsp @Google-ML-Automation
Update packaging and pip versions in rules_python to enable freethrea…
220c5aa 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from #20723 for merging.

PiperOrigin-RevId: 708471912
aignas added a commit to aignas/rules_python that referenced this pull request Dec 24, 2024
@aignas
fix(whl_library): track sources in whl_library
7b0dc0d 
This is reusing a bit of code used in `evaluate_markers` and makes use
of the RECORD files in the `whl` files that we use to extract whls in
`whl_library`. This should be merged before bazelbuild#2514 to avoid any cache
invalidation issues downstream.

Fixes bazelbuild#2468
@aignas aignas mentioned this pull request Dec 24, 2024
Merged
@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch 5 times, most recently from 0f921dc to 32ae8dc Compare December 24, 2024 14:17
@aignas
Copy link
Collaborator

aignas commented Dec 26, 2024

I think the current solution might be good enough.

aignas
aignas approved these changes Dec 31, 2024
View reviewed changes
Copy link
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix.

@aignas aignas added this pull request to the merge queue Dec 31, 2024
@aignas aignas removed this pull request from the merge queue due to a manual request Dec 31, 2024
@aignas
Copy link
Collaborator

aignas commented Dec 31, 2024

Could you please add a line in the Changelog file so that I could merge the PR?

github-merge-queue bot pushed a commit that referenced this pull request Jan 1, 2025
@aignas
fix(whl_library): track sources in whl_library (#2526)
6a04d38 
This is reusing a bit of code used in `evaluate_markers` and makes use
of the RECORD files in the `whl` files that we use to extract whls in
`whl_library`. This should be merged before #2514 to avoid any cache
invalidation issues downstream.

Fixes #2468
@vfdev-5 vfdev-5 force-pushed the updated-pip-packaging-versions branch from 32ae8dc to 49ab81c Compare January 2, 2025 14:43
vfdev-5
vfdev-5 commented Jan 2, 2025
View reviewed changes
CHANGELOG.md Show resolved Hide resolved
@aignas aignas added this pull request to the merge queue Jan 3, 2025
Merged via the queue into bazelbuild:main with commit fbf8bc1 Jan 3, 2025
4 checks passed
@vfdev-5 vfdev-5 deleted the updated-pip-packaging-versions branch January 3, 2025 13:46
copybara-service bot pushed a commit to google/tsl that referenced this pull request Jan 6, 2025
@hawkinsp @copybara-github
Update packaging and pip versions in rules_python to enable freethrea…
6073074 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jan 6, 2025
@hawkinsp @Google-ML-Automation
Update packaging and pip versions in rules_python to enable freethrea…
6b81489 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from #20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Jan 6, 2025
@hawkinsp @tensorflower-gardener
Update packaging and pip versions in rules_python to enable freethrea…
72c41b2 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 708471912
copybara-service bot pushed a commit to google/tsl that referenced this pull request Jan 6, 2025
@hawkinsp @copybara-github
Update packaging and pip versions in rules_python to enable freethrea…
f3cabbb 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 712538044
copybara-service bot pushed a commit to openxla/xla that referenced this pull request Jan 6, 2025
@hawkinsp @Google-ML-Automation
Update packaging and pip versions in rules_python to enable freethrea…
1a6361a 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from #20723 for merging.

PiperOrigin-RevId: 712538044
copybara-service bot pushed a commit to tensorflow/tensorflow that referenced this pull request Jan 6, 2025
@hawkinsp @tensorflower-gardener
Update packaging and pip versions in rules_python to enable freethrea…
5f9409e 
…ding support.

The same change has been sent upstream as bazelbuild/rules_python#2514

Forked from openxla/xla#20723 for merging.

PiperOrigin-RevId: 712538044
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aignas aignas aignas approved these changes

@rickeylev rickeylev Awaiting requested review from rickeylev rickeylev is a code owner

@groodt groodt Awaiting requested review from groodt groodt is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vfdev-5 @aignas