WIP: pytorch v2.6.0 #326

h-vetinari · 2025-01-18T13:41:55Z

Build the release candidates

Linux CI cancelled until builds for #322 are live

…nda-forge-pinning 2025.01.18.07.29.32

conda-forge-admin · 2025-01-18T13:43:30Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12863672331. Examine the logs at this URL for more detail.}

h-vetinari · 2025-01-18T22:17:28Z

This looks better than expected so far. Still have to double check dependency changes. Happy if someone could do that (even if it's just noting which bounds changed relative to the current recipe)

See pytorch/pytorch@2398e75 Partially reverts commit 46d7b46.

hmaarrfk · 2025-01-19T17:40:22Z

recipe/meta.yaml

@@ -305,27 +288,28 @@ outputs:
        - typing_extensions
        - {{ pin_subpackage('libtorch', exact=True) }}
      run:
+        - {{ pin_subpackage('libtorch', exact=True) }}


The old syntax was to workaround the fact that for the non megabuilds each libtorch may have different hashes making it incompatible.

Or maybe something like that. Maybe you have addressed the core problem and this is the better way.

Even for the megabuild, libtorch has a unique hash, because there's only one. It's pytorch itself that gets different hashes due to the different python versions

Even for the megabuild, libtorch has a unique hash

Not true. See https://anaconda.org/conda-forge/libtorch/files

Obviously cuda and non-CUDA create different libtorch hashes, but within a megabuild it's unique, which is what matters for pinning it in pytorch.

Unless the idea was to allow mixing CUDA-enabled pytorch with non-CUDA libtorch, but I don't see the sense in that.

It's more interesting for the blas_impl - pytorch could theoretically be independent of that (if all the blas calls go through libtorch), but also there we're already creating different pytorch hashes there due to the {{ pin_subpackage("libtorch", exact=True) }} in the host dependencies, so AFAICT we're not materially changing the various installations here, just making it impossible to install untested/unsupported combinations

For non-megabuilds, the idea was to allow libtorch from any of the builds (with the same features except python version) to work with any pytorch build. This way, you don't have to download a different libtorch for different python version. Note that this is for non-megabuilds only i.e. osx where we don't have CUDA builds.

Yeah, that makes sense! Let me try to reflect that in the run-deps.

h-vetinari · 2025-01-19T21:26:48Z

Aarch builds fail with

$SRC_DIR/third_party/XNNPACK/src/reference/unary-elementwise.cc:125:14: error: invalid 'static_cast' from type 'xnn_bfloat16' to type '_Float16'
  125 |       return static_cast<TOut>(x);
      |              ^~~~~~~~~~~~~~~~~~~~

don't rely on PKG_BUILDNUM resolving this correctly, which is either racy, or implicitly depends on a separate render pass after setting build.number

h-vetinari · 2025-01-20T07:29:41Z

Sigh, since when is conda-build applying patches through git rather than through patch? The former is stricter than the latter, and doesn't work in some situations (like applying a patch in one of the submodules):

Applying patch: /Users/runner/work/1/s/recipe/patches_submodules/0001-Fix-bazel-linux-aarch64-gcc13-workflow-and-resolve-a.patch
Applying: Fix `bazel-linux-aarch64-gcc13` workflow and resolve accompanying build errors.
error: sha1 information is lacking or useless (third_party/XNNPACK/src/reference/unary-elementwise.cc).
error: could not build fake ancestor

otherwise conda breaks ``` conda_build.exceptions.RecipeError: Mismatching hashes in recipe. Exact pins in dependencies that contribute to the hash often cause this. Can you change one or more exact pins to version bound constraints? Involved packages were: Mismatching package: libtorch (id cpu_generic_habf3c96_0); dep: libtorch 2.6.0.rc7 *0; consumer package: pytorch ```

danpetry · 2025-01-23T23:40:05Z

on osx-64

FAILED [0.0518s] test/test_nn.py::TestNN::test_batchnorm_nhwc_cpu - AssertionError: Tensor-likes are not close!

Mismatched elements: 1 / 8 (12.5%)
Greatest absolute difference: 1.430511474609375e-05 at index (0,) (up to 1e-05 allowed)
Greatest relative difference: 4.616721980710281e-06 at index (0,) (up to 1.3e-06 allowed)

skip?

h-vetinari · 2025-01-23T23:49:23Z

Yeah, this minor accuracy violation indeed sounds skippable, but I've deprioritised this PR until we get the windows builds for 2.5 fixed (and ideally your #318 merged as well).

danpetry · 2025-01-23T23:52:02Z

ok, good to know

danpetry · 2025-01-24T00:00:15Z

worth pointing out that as of 6 days time, pypi will have an up to date pytorch package whereas conda won't. Will have a look at that other PR

h-vetinari · 2025-01-24T00:17:27Z

worth pointing out that as of 6 days time, pypi will have an up to date pytorch package whereas conda won't.

Are you talking about rc's, or are we not looking at the same index? 2.6.0 GA hasn't been published AFAICT. Or are you saying that 2.6.0 will be released in 6 days?

In any case, this is no reason to rush. We didn't have windows packages for years, and I'm more concerned about fixing them, than lagging behind the PyPI release a bit (and we've often lagged for months in the past; this has gotten much better with the open-gpu server, but it still happens; 2.5.0 was released Oct 18th last year, we had first builds on Nov 3rd).

danpetry · 2025-01-24T21:38:05Z

are you saying that 2.6.0 will be released in 6 days

yes

I'm more concerned about fixing them, than lagging behind the PyPI release a bit

100%

h-vetinari added 3 commits January 18, 2025 14:33

pytorch v2.6.0.rc7

11db2ff

MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.2, and co…

5645301

…nda-forge-pinning 2025.01.18.07.29.32

rebase patches

1219e26

h-vetinari added 4 commits January 19, 2025 09:26

Remove bound on setuptools version

1e211b2

See pytorch/pytorch@2398e75 Partially reverts commit 46d7b46.

better sort pytorch run requirements

63c7c00

update dependencies for 2.6.0

bc3581d

pin libtorch in pytorch

113d413

hmaarrfk reviewed Jan 19, 2025

View reviewed changes

h-vetinari mentioned this pull request Jan 20, 2025

ARM build failed with recent XNNPACK update: third_party/XNNPACK/src/reference/unary-elementwise.cc:125:14: error: invalid ‘static_cast’ from type ‘xnn_bfloat16’ to type ‘_Float16’ pytorch/pytorch#141083

Open

h-vetinari added 5 commits January 20, 2025 14:01

backport fix for aarch build failure

c1b9cf6

reduce OMP_NUM_THREADS further due to OOMs in test

bf65aa4

use full paths to patched files in submodule

6757593

don't pin libtorch exactly for non-megabuilds

fa31c3f

make build number arithmetic explicit

49d134a

don't rely on PKG_BUILDNUM resolving this correctly, which is either racy, or implicitly depends on a separate render pass after setting build.number

disable patch in submodule

7cd981d

h-vetinari force-pushed the 2.6 branch 2 times, most recently from bd0bec7 to 022f063 Compare January 20, 2025 08:03

h-vetinari force-pushed the 2.6 branch from 022f063 to 1405263 Compare January 20, 2025 08:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: pytorch v2.6.0 #326

WIP: pytorch v2.6.0 #326

h-vetinari commented Jan 18, 2025

conda-forge-admin commented Jan 18, 2025 •

edited

Loading

h-vetinari commented Jan 18, 2025

hmaarrfk Jan 19, 2025

hmaarrfk Jan 19, 2025

h-vetinari Jan 19, 2025

isuruf Jan 19, 2025

h-vetinari Jan 19, 2025

h-vetinari Jan 19, 2025

isuruf Jan 20, 2025

h-vetinari Jan 20, 2025

h-vetinari commented Jan 19, 2025

h-vetinari commented Jan 20, 2025

danpetry commented Jan 23, 2025

h-vetinari commented Jan 23, 2025 •

edited

Loading

danpetry commented Jan 23, 2025

danpetry commented Jan 24, 2025

h-vetinari commented Jan 24, 2025

danpetry commented Jan 24, 2025

WIP: pytorch v2.6.0 #326

Are you sure you want to change the base?

WIP: pytorch v2.6.0 #326

Conversation

h-vetinari commented Jan 18, 2025

conda-forge-admin commented Jan 18, 2025 • edited Loading

h-vetinari commented Jan 18, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

h-vetinari commented Jan 19, 2025

h-vetinari commented Jan 20, 2025

danpetry commented Jan 23, 2025

h-vetinari commented Jan 23, 2025 • edited Loading

danpetry commented Jan 23, 2025

danpetry commented Jan 24, 2025

h-vetinari commented Jan 24, 2025

danpetry commented Jan 24, 2025

conda-forge-admin commented Jan 18, 2025 •

edited

Loading

h-vetinari commented Jan 23, 2025 •

edited

Loading