Add Lux tests

[gdalle]

DIT extensions

• Set up a Lux extension to define scenarios, modeled after the Flux one
• Define the scenarios (out-of-place gradient with respect to parameters), their ground truth values (FiniteDifferences) and comparison functions (LuxTestUtils.check_approx)

DIT tests

• Test Lux scenarios with one backend (Zygote)

DI tests

• Set up a folder test/Down/Lux for Lux tests and add it to CI
• Test Lux scenarios with one backend (Zygote)

---

See also:

• DifferentiationInterface testing LuxDL/Lux.jl#769
• Testing NNLib / Lux / Flux #105
• Tests on generic structs #343

[avik-pal]

Gradient with respect to parameter only? Or parameter + state in a ComponentVector?

The latter won't work because state can contain non-array-like objects like Val.

Is Zygote the ground truth or Finite Differences?

Its Finite Differences (except in edge cases like Float16 testing where the error is quite high for finite differencing). I left more detailed comment on the issue on Lux

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.63%. Comparing base (2ee0b56) to head (a07ff9b).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #372      +/-   ##
==========================================
+ Coverage   96.62%   96.63%   +0.01%     
==========================================
  Files         102      103       +1     
  Lines        4974     4996      +22     
==========================================
+ Hits         4806     4828      +22     
  Misses        168      168              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.