-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: make eltype of Traced/Concrete Arrays to be respective RNumbers #426
Conversation
e35a7e3
to
db906a4
Compare
@@ -1,20 +1,3 @@ | |||
struct XLAArray{T,N} <: RArray{T,N} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can get away with just doing this for the traced version and not the concrete variant (since a concrete number has higher overhead than otherwise, and it’ll obviously get optimized out for the traced case).
db906a4
to
32a9862
Compare
509901d
to
9d51948
Compare
9d51948
to
13b951c
Compare
Lux tests will fail for now. Needs LuxDL/Lux.jl#1140 |
what about #425 (comment)? isn't this way gonna be problematic too? the case that comes to my mind is Yao. if the following example works, then it's fine from my side (actually, maybe i should add a test for this? i think i forgot to add them) using YaoBlocks
f(x) = mat(Rx(x))
x = ConcreteRNumber(0.0)
@jit f(x) EDIT: added #431 to test that this PR doesn't break integration with YaoBlocks |
julia> @code_hlo f(x)
module {
func.func @main(%arg0: tensor<f64>) -> tensor<2x2xcomplex<f64>> {
%c = stablehlo.constant dense<1> : tensor<i64>
%c_0 = stablehlo.constant dense<0> : tensor<i64>
%cst = stablehlo.constant dense<(0.000000e+00,-1.000000e+00)> : tensor<complex<f64>>
%cst_1 = stablehlo.constant dense<2.000000e+00> : tensor<f64>
%cst_2 = stablehlo.constant dense<(0.000000e+00,0.000000e+00)> : tensor<2x2xcomplex<f64>>
%0 = stablehlo.divide %arg0, %cst_1 : tensor<f64>
%1 = stablehlo.cosine %0 : tensor<f64>
%2 = stablehlo.sine %0 : tensor<f64>
%3 = stablehlo.convert %2 : (tensor<f64>) -> tensor<complex<f64>>
%4 = stablehlo.multiply %cst, %3 : tensor<complex<f64>>
%5 = stablehlo.convert %1 : (tensor<f64>) -> tensor<complex<f64>>
%6 = stablehlo.reshape %5 : (tensor<complex<f64>>) -> tensor<1x1xcomplex<f64>>
%7 = stablehlo.dynamic_update_slice %cst_2, %6, %c_0, %c_0 : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
%8 = stablehlo.dynamic_update_slice %7, %6, %c, %c : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
%9 = stablehlo.reshape %4 : (tensor<complex<f64>>) -> tensor<1x1xcomplex<f64>>
%10 = stablehlo.dynamic_update_slice %8, %9, %c_0, %c : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
%11 = stablehlo.dynamic_update_slice %10, %9, %c, %c_0 : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
%12 = stablehlo.transpose %11, dims = [1, 0] : (tensor<2x2xcomplex<f64>>) -> tensor<2x2xcomplex<f64>>
return %12 : tensor<2x2xcomplex<f64>>
}
}
julia> @jit f(x)
2024-12-28 19:42:57.105196: I external/xla/xla/service/llvm_ir/llvm_command_line_options.cc:50] XLA (re)initializing LLVM with options fingerprint: 1154763464302348808
2×2 ConcreteRArray{ComplexF64, 2}:
1.0+0.0im 0.0+0.0im
0.0+0.0im 1.0+0.0im |
Great, then all in! |
commit 6556944 Author: Avik Pal <[email protected]> Date: Tue Dec 31 08:53:39 2024 -0500 feat: support Base.stack (EnzymeAD#433) * refactor: use scatter for generating diagm * refactor: directly generate the region for simple_scatter_op * feat: generalize diagm * feat: support Base.stack * fix: incorrect rebase * test: stack tests commit 9375f57 Author: Sergio Sánchez Ramírez <[email protected]> Date: Mon Dec 30 22:56:14 2024 +0100 Modularize Bazel build (EnzymeAD#421) * organize platforms and toolchains * hardcode libcxxwrap_julia path * format code * remove outdated hardcoded symbolic links * add third party bazel wrapper to libcxxwrap_julia * readd platforms * some small fixes * first step on moving externals to modular organization * refactor libcxxwrap_julia on top of `cc_import` * use modular workspaces * add `libcxxwrap_julia` as dependency * hardcode julia dep * export `reactant_*` functions * downgrade libcxxwrap_julia to v0.13.3 * fix major version when linking to libcxxwrap_julia * remove legacy export * move `API.cpp` to new `src/` folder to start modularizing code * export `register_julia_module` from libcxxwrap_julia * fix symbol visibility * clean code remove libcxxwrap and julia deps * format code * import hedron compile commands from Enzyme-JAX * move deps commits to `workspace.bzl` commit 3244204 Author: Avik Pal <[email protected]> Date: Mon Dec 30 16:33:52 2024 -0500 chore: bump jll (EnzymeAD#437) commit 25abfe4 Author: Avik Pal <[email protected]> Date: Mon Dec 30 12:03:20 2024 -0500 fix: try building with cudnn 9.4 (EnzymeAD#436) commit 241fd14 Author: Avik Pal <[email protected]> Date: Mon Dec 30 04:54:50 2024 -0500 feat: indexing using traced values (EnzymeAD#434) * feat: indexing using traced values * feat: implement repeat inner * feat: support scalar linear indexing + tests * fix: regression in cartesian index support * Update src/TracedRArray.jl commit 7d2b898 Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Date: Sun Dec 29 20:28:23 2024 -0500 Regenerate MLIR Bindings (EnzymeAD#435) Co-authored-by: mofeing <[email protected]> commit 8e4c095 Author: Avik Pal <[email protected]> Date: Sun Dec 29 13:48:27 2024 -0500 feat: add support for the remaining wrapper types (EnzymeAD#369) * feat: add materialize_traced_array for all other wrappers * refactor: use scatter for generating diagm * refactor: directly generate the region for simple_scatter_op * feat: generalize diagm * feat: efficient non-contiguous setindex * fix: non-contiguous indexing is now supported * feat: implement set_mlir_data for the remaining types * refactor: use `Ops.gather_getindex` to implement diag * fix: noinline ops * fix: incorrect rebase * fix: dispatches * fix: diagm for repeated indices and initial tests * fix: higher dimensional indexing + tests * fix: matrix multiplication of wrapper types * fix: de-specialize 3 arg mul! commit d4e7c76 Author: William Moses <[email protected]> Date: Sat Dec 28 23:05:51 2024 -0500 CUDA kernels take 3 (EnzymeAD#427) * CUDA take 3 * conditional run cuda * Update test/integration/cuda.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * bump enzymexla * fix * fix gpu reg * Update BUILD * Update BUILD * Update Project.toml * Update ReactantCUDAExt.jl * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix reactant method blocker * Update ReactantCUDAExt.jl * only do compile * use names in cache * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * cleanup further gc issues * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit b0a58bd Author: Avik Pal <[email protected]> Date: Sat Dec 28 21:53:29 2024 -0500 fix: make eltype of Traced/Concrete Arrays to be respective RNumbers (EnzymeAD#426) * feat: overlay eltype conversion * fix: overload the main methods * fix: make eltype of Traced/Concrete Arrays to be respective RNumbers * fix: handle more cases * fix: tracing of wrapped types * fix: arrayinterface overload * fix: python call commit f079a9d Author: mofeing <[email protected]> Date: Sun Dec 29 00:15:51 2024 +0000 Format code commit eeaf86c Author: glounes <[email protected]> Date: Sat Dec 28 04:24:38 2024 +0100 `stablehlo.sort` Ops (EnzymeAD#374) * `stablehlo.sort` Ops * review * use `return_dialect` * feedback * fix test GPU commit 925544f Author: William Moses <[email protected]> Date: Tue Dec 24 15:34:52 2024 -0500 Cuv2 (EnzymeAD#423) * Kernel-supporting jll * fix rulescc * adapt to hedron dep * init target * fixup * additional fixups * fixup * fix * registry utils * callname * reg * fix * fix bld * cleanup * no pip * fix * force rules python to older version before bug * fixup jll * with proto * fix * fix * Update WORKSPACE * more deps for apple * bump * fix * workspace bump * workspace * Update Compiler.jl * Update ReactantCUDAExt.jl * Update ReactantCUDAExt.jl * Update ReactantCUDAExt.jl * Update ReactantCUDAExt.jl * Update ReactantCUDAExt.jl * Update Project.toml * Update ReactantCUDAExt.jl * Update Project.toml * Update Project.toml * fix * Update ReactantCUDAExt.jl * Update ReactantCUDAExt.jl * Update ReactantCUDAExt.jl * Update cuda.jl * Update cuda.jl * Update cuda.jl * Cuda kernel v2 * Update Project.toml * Update API.cpp * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: William Moses <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 057e6b8 Author: Avik Pal <[email protected]> Date: Tue Dec 24 21:46:23 2024 +0530 fix: handle traced array returns inside objects (EnzymeAD#417) * fix: handle traced array returns inside objects * test: add EnzymeAD#416 as a test * fix: propagate track_numbers correctly * fix: aliasing and add a test * test: use updated API for the tests * feat: cache new arrays * fix: traced_getfield commit 0b6dafc Author: Sergio Sánchez Ramírez <[email protected]> Date: Tue Dec 24 10:43:18 2024 +0100 Bump Reactant_jll to v0.0.32 commit a02fd5b Author: William Moses <[email protected]> Date: Mon Dec 23 21:58:40 2024 -0500 Update WORKSPACE commit 6e1710d Author: William Moses <[email protected]> Date: Mon Dec 23 19:48:12 2024 -0500 disable absint of absint (EnzymeAD#424) * disable absint of absint * no typeinf ext * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 228732f Author: William Moses <[email protected]> Date: Mon Dec 23 18:50:34 2024 -0500 Fix error on global (EnzymeAD#422) commit 695cc80 Author: William Moses <[email protected]> Date: Mon Dec 23 13:59:31 2024 -0500 Update Project.toml commit 38916f5 Author: Avik Pal <[email protected]> Date: Mon Dec 23 21:46:22 2024 +0530 feat: add zero and fill! for ConcreteRArray (EnzymeAD#420) * feat: add zero and fill! for ConcreteRArray * test: add tests commit 6571d54 Author: William S. Moses <[email protected]> Date: Sun Dec 22 23:49:03 2024 -0500 bump enzymexla commit commit 5b89b56 Author: William Moses <[email protected]> Date: Sun Dec 22 21:23:33 2024 -0500 Fix ReactantPythonCallExt.jl (EnzymeAD#419) * Fix ReactantPythonCallExt.jl * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 4cc000c Author: William Moses <[email protected]> Date: Sun Dec 22 19:55:58 2024 -0500 Improve reactant error messages (EnzymeAD#418) * Improve reactant error messages * More exported symbols * Update XLA.jl * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 2759c3c Author: jumerckx <[email protected]> Date: Mon Dec 23 01:35:17 2024 +0100 Inference cache (EnzymeAD#405) * add inference cache * start from `typeinf_ircode` * julia 1.10 * Apply formatting suggestions Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * remove debug logging * vendor in type inference code for v1.10 To avoid having to build a MethodInstance twice (performance hazard) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jules Merckx <[email protected]> commit f9c43ad Author: William S. Moses <[email protected]> Date: Sun Dec 22 19:33:25 2024 -0500 Bump enzymexla
No description provided.