Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: make eltype of Traced/Concrete Arrays to be respective RNumbers #426

Merged
merged 7 commits into from
Dec 29, 2024

Conversation

avik-pal
Copy link
Collaborator

No description provided.

@avik-pal avik-pal force-pushed the ap/prototype_eltype2 branch from e35a7e3 to db906a4 Compare December 24, 2024 18:34
@@ -1,20 +1,3 @@
struct XLAArray{T,N} <: RArray{T,N}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can get away with just doing this for the traced version and not the concrete variant (since a concrete number has higher overhead than otherwise, and it’ll obviously get optimized out for the traced case).

@avik-pal avik-pal force-pushed the ap/prototype_eltype2 branch from db906a4 to 32a9862 Compare December 28, 2024 17:10
@avik-pal avik-pal force-pushed the ap/prototype_eltype2 branch 2 times, most recently from 509901d to 9d51948 Compare December 28, 2024 18:25
@avik-pal avik-pal force-pushed the ap/prototype_eltype2 branch from 9d51948 to 13b951c Compare December 28, 2024 18:28
@avik-pal avik-pal marked this pull request as ready for review December 28, 2024 18:56
@avik-pal
Copy link
Collaborator Author

Lux tests will fail for now. Needs LuxDL/Lux.jl#1140

@mofeing
Copy link
Collaborator

mofeing commented Dec 28, 2024

what about #425 (comment)? isn't this way gonna be problematic too?

the case that comes to my mind is Yao. if the following example works, then it's fine from my side (actually, maybe i should add a test for this? i think i forgot to add them)

using YaoBlocks

f(x) = mat(Rx(x))
x = ConcreteRNumber(0.0)
@jit f(x)

EDIT: added #431 to test that this PR doesn't break integration with YaoBlocks

@mofeing mofeing mentioned this pull request Dec 28, 2024
@avik-pal
Copy link
Collaborator Author

what about #425 (comment)? isn't this way gonna be problematic too?

the case that comes to my mind is Yao. if the following example works, then it's fine from my side (actually, maybe i should add a test for this? i think i forgot to add them)

using YaoBlocks

f(x) = mat(Rx(x))
x = ConcreteRNumber(0.0)
@jit f(x)

EDIT: added #431 to test that this PR doesn't break integration with YaoBlocks

julia> @code_hlo f(x)
module {
  func.func @main(%arg0: tensor<f64>) -> tensor<2x2xcomplex<f64>> {
    %c = stablehlo.constant dense<1> : tensor<i64>
    %c_0 = stablehlo.constant dense<0> : tensor<i64>
    %cst = stablehlo.constant dense<(0.000000e+00,-1.000000e+00)> : tensor<complex<f64>>
    %cst_1 = stablehlo.constant dense<2.000000e+00> : tensor<f64>
    %cst_2 = stablehlo.constant dense<(0.000000e+00,0.000000e+00)> : tensor<2x2xcomplex<f64>>
    %0 = stablehlo.divide %arg0, %cst_1 : tensor<f64>
    %1 = stablehlo.cosine %0 : tensor<f64>
    %2 = stablehlo.sine %0 : tensor<f64>
    %3 = stablehlo.convert %2 : (tensor<f64>) -> tensor<complex<f64>>
    %4 = stablehlo.multiply %cst, %3 : tensor<complex<f64>>
    %5 = stablehlo.convert %1 : (tensor<f64>) -> tensor<complex<f64>>
    %6 = stablehlo.reshape %5 : (tensor<complex<f64>>) -> tensor<1x1xcomplex<f64>>
    %7 = stablehlo.dynamic_update_slice %cst_2, %6, %c_0, %c_0 : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
    %8 = stablehlo.dynamic_update_slice %7, %6, %c, %c : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
    %9 = stablehlo.reshape %4 : (tensor<complex<f64>>) -> tensor<1x1xcomplex<f64>>
    %10 = stablehlo.dynamic_update_slice %8, %9, %c_0, %c : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
    %11 = stablehlo.dynamic_update_slice %10, %9, %c, %c_0 : (tensor<2x2xcomplex<f64>>, tensor<1x1xcomplex<f64>>, tensor<i64>, tensor<i64>) -> tensor<2x2xcomplex<f64>>
    %12 = stablehlo.transpose %11, dims = [1, 0] : (tensor<2x2xcomplex<f64>>) -> tensor<2x2xcomplex<f64>>
    return %12 : tensor<2x2xcomplex<f64>>
  }
}

julia> @jit f(x)
2024-12-28 19:42:57.105196: I external/xla/xla/service/llvm_ir/llvm_command_line_options.cc:50] XLA (re)initializing LLVM with options fingerprint: 1154763464302348808
2×2 ConcreteRArray{ComplexF64, 2}:
 1.0+0.0im  0.0+0.0im
 0.0+0.0im  1.0+0.0im

@mofeing
Copy link
Collaborator

mofeing commented Dec 29, 2024

Great, then all in!

@wsmoses
Copy link
Member

wsmoses commented Dec 29, 2024

@avik-pal, since @mofeing's test case seems to pass (and also strictly this is an improvement to cases where the assumption is made in julia programs in that typeof(getindex(A, idx)) == eltype(A) is now actually satisfied by tracedrarrays), this is fine to merge when ready imo

@avik-pal avik-pal merged commit b0a58bd into main Dec 29, 2024
26 of 38 checks passed
@avik-pal avik-pal deleted the ap/prototype_eltype2 branch December 29, 2024 02:53
jumerckx added a commit to jumerckx/Reactant.jl that referenced this pull request Jan 1, 2025
commit 6556944
Author: Avik Pal <[email protected]>
Date:   Tue Dec 31 08:53:39 2024 -0500

    feat: support Base.stack (EnzymeAD#433)

    * refactor: use scatter for generating diagm

    * refactor: directly generate the region for simple_scatter_op

    * feat: generalize diagm

    * feat: support Base.stack

    * fix: incorrect rebase

    * test: stack tests

commit 9375f57
Author: Sergio Sánchez Ramírez <[email protected]>
Date:   Mon Dec 30 22:56:14 2024 +0100

    Modularize Bazel build (EnzymeAD#421)

    * organize platforms and toolchains

    * hardcode libcxxwrap_julia path

    * format code

    * remove outdated hardcoded symbolic links

    * add third party bazel wrapper to libcxxwrap_julia

    * readd platforms

    * some small fixes

    * first step on moving externals to modular organization

    * refactor libcxxwrap_julia on top of `cc_import`

    * use modular workspaces

    * add `libcxxwrap_julia` as dependency

    * hardcode julia dep

    * export `reactant_*` functions

    * downgrade libcxxwrap_julia to v0.13.3

    * fix major version when linking to libcxxwrap_julia

    * remove legacy export

    * move `API.cpp` to new `src/` folder to start modularizing code

    * export `register_julia_module` from libcxxwrap_julia

    * fix symbol visibility

    * clean code

    remove libcxxwrap and julia deps

    * format code

    * import hedron compile commands from Enzyme-JAX

    * move deps commits to `workspace.bzl`

commit 3244204
Author: Avik Pal <[email protected]>
Date:   Mon Dec 30 16:33:52 2024 -0500

    chore: bump jll (EnzymeAD#437)

commit 25abfe4
Author: Avik Pal <[email protected]>
Date:   Mon Dec 30 12:03:20 2024 -0500

    fix: try building with cudnn 9.4 (EnzymeAD#436)

commit 241fd14
Author: Avik Pal <[email protected]>
Date:   Mon Dec 30 04:54:50 2024 -0500

    feat: indexing using traced values (EnzymeAD#434)

    * feat: indexing using traced values

    * feat: implement repeat inner

    * feat: support scalar linear indexing + tests

    * fix: regression in cartesian index support

    * Update src/TracedRArray.jl

commit 7d2b898
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date:   Sun Dec 29 20:28:23 2024 -0500

    Regenerate MLIR Bindings (EnzymeAD#435)

    Co-authored-by: mofeing <[email protected]>

commit 8e4c095
Author: Avik Pal <[email protected]>
Date:   Sun Dec 29 13:48:27 2024 -0500

    feat: add support for the remaining wrapper types (EnzymeAD#369)

    * feat: add materialize_traced_array for all other wrappers

    * refactor: use scatter for generating diagm

    * refactor: directly generate the region for simple_scatter_op

    * feat: generalize diagm

    * feat: efficient non-contiguous setindex

    * fix: non-contiguous indexing is now supported

    * feat: implement set_mlir_data for the remaining types

    * refactor: use `Ops.gather_getindex` to implement diag

    * fix: noinline ops

    * fix: incorrect rebase

    * fix: dispatches

    * fix: diagm for repeated indices and initial tests

    * fix: higher dimensional indexing + tests

    * fix: matrix multiplication of wrapper types

    * fix: de-specialize 3 arg mul!

commit d4e7c76
Author: William Moses <[email protected]>
Date:   Sat Dec 28 23:05:51 2024 -0500

    CUDA kernels take 3 (EnzymeAD#427)

    * CUDA take 3

    * conditional run cuda

    * Update test/integration/cuda.jl

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * bump enzymexla

    * fix

    * fix gpu reg

    * Update BUILD

    * Update BUILD

    * Update Project.toml

    * Update ReactantCUDAExt.jl

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fix reactant method blocker

    * Update ReactantCUDAExt.jl

    * only do compile

    * use names in cache

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * cleanup further gc issues

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fix

    ---------

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit b0a58bd
Author: Avik Pal <[email protected]>
Date:   Sat Dec 28 21:53:29 2024 -0500

    fix: make eltype of Traced/Concrete Arrays to be respective RNumbers (EnzymeAD#426)

    * feat: overlay eltype conversion

    * fix: overload the main methods

    * fix: make eltype of Traced/Concrete Arrays to be respective RNumbers

    * fix: handle more cases

    * fix: tracing of wrapped types

    * fix: arrayinterface overload

    * fix: python call

commit f079a9d
Author: mofeing <[email protected]>
Date:   Sun Dec 29 00:15:51 2024 +0000

    Format code

commit eeaf86c
Author: glounes <[email protected]>
Date:   Sat Dec 28 04:24:38 2024 +0100

    `stablehlo.sort` Ops (EnzymeAD#374)

    * `stablehlo.sort` Ops

    * review

    * use `return_dialect`

    * feedback

    * fix test GPU

commit 925544f
Author: William Moses <[email protected]>
Date:   Tue Dec 24 15:34:52 2024 -0500

    Cuv2 (EnzymeAD#423)

    * Kernel-supporting jll

    * fix rulescc

    * adapt to hedron dep

    * init target

    * fixup

    * additional fixups

    * fixup

    * fix

    * registry utils

    * callname

    * reg

    * fix

    * fix bld

    * cleanup

    * no pip

    * fix

    * force rules python to older version before bug

    * fixup jll

    * with proto

    * fix

    * fix

    * Update WORKSPACE

    * more deps for apple

    * bump

    * fix

    * workspace bump

    * workspace

    * Update Compiler.jl

    * Update ReactantCUDAExt.jl

    * Update ReactantCUDAExt.jl

    * Update ReactantCUDAExt.jl

    * Update ReactantCUDAExt.jl

    * Update ReactantCUDAExt.jl

    * Update Project.toml

    * Update ReactantCUDAExt.jl

    * Update Project.toml

    * Update Project.toml

    * fix

    * Update ReactantCUDAExt.jl

    * Update ReactantCUDAExt.jl

    * Update ReactantCUDAExt.jl

    * Update cuda.jl

    * Update cuda.jl

    * Update cuda.jl

    * Cuda kernel v2

    * Update Project.toml

    * Update API.cpp

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    ---------

    Co-authored-by: William Moses <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 057e6b8
Author: Avik Pal <[email protected]>
Date:   Tue Dec 24 21:46:23 2024 +0530

    fix: handle traced array returns inside objects (EnzymeAD#417)

    * fix: handle traced array returns inside objects

    * test: add EnzymeAD#416 as a test

    * fix: propagate track_numbers correctly

    * fix: aliasing and add a test

    * test: use updated API for the tests

    * feat: cache new arrays

    * fix: traced_getfield

commit 0b6dafc
Author: Sergio Sánchez Ramírez <[email protected]>
Date:   Tue Dec 24 10:43:18 2024 +0100

    Bump Reactant_jll to v0.0.32

commit a02fd5b
Author: William Moses <[email protected]>
Date:   Mon Dec 23 21:58:40 2024 -0500

    Update WORKSPACE

commit 6e1710d
Author: William Moses <[email protected]>
Date:   Mon Dec 23 19:48:12 2024 -0500

    disable absint of absint (EnzymeAD#424)

    * disable absint of absint

    * no typeinf ext

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    ---------

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 228732f
Author: William Moses <[email protected]>
Date:   Mon Dec 23 18:50:34 2024 -0500

    Fix error on global (EnzymeAD#422)

commit 695cc80
Author: William Moses <[email protected]>
Date:   Mon Dec 23 13:59:31 2024 -0500

    Update Project.toml

commit 38916f5
Author: Avik Pal <[email protected]>
Date:   Mon Dec 23 21:46:22 2024 +0530

    feat: add zero and fill! for ConcreteRArray (EnzymeAD#420)

    * feat: add zero and fill! for ConcreteRArray

    * test: add tests

commit 6571d54
Author: William S. Moses <[email protected]>
Date:   Sun Dec 22 23:49:03 2024 -0500

    bump enzymexla commit

commit 5b89b56
Author: William Moses <[email protected]>
Date:   Sun Dec 22 21:23:33 2024 -0500

    Fix ReactantPythonCallExt.jl (EnzymeAD#419)

    * Fix ReactantPythonCallExt.jl

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    ---------

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 4cc000c
Author: William Moses <[email protected]>
Date:   Sun Dec 22 19:55:58 2024 -0500

    Improve reactant error messages (EnzymeAD#418)

    * Improve reactant error messages

    * More exported symbols

    * Update XLA.jl

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * Apply suggestions from code review

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    ---------

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 2759c3c
Author: jumerckx <[email protected]>
Date:   Mon Dec 23 01:35:17 2024 +0100

    Inference cache (EnzymeAD#405)

    * add inference cache

    * start from `typeinf_ircode`

    * julia 1.10

    * Apply formatting suggestions

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * remove debug logging

    * vendor in type inference code for v1.10

    To avoid having to build a MethodInstance twice (performance hazard)

    ---------

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    Co-authored-by: Jules Merckx <[email protected]>

commit f9c43ad
Author: William S. Moses <[email protected]>
Date:   Sun Dec 22 19:33:25 2024 -0500

    Bump enzymexla
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants