Allow N tracers to be advected

[FlorianDeconinck]

GEOS-FP advects 28 tracers. DSL port does 9 hard coded tracers.

Parent: GEOS-ESM/SMT-Nebulae#55

---

• ndsl dynamical core can advect N tracers

[FlorianDeconinck]

GEOS breakdown:

• Bundle of tracers, some are flagged friendly to dynamics, the flagged are all advected
  • Within the dynamics tracers, FV_State mode singles out the water/cloud species
  • AdvCore is a standalone advection, which borrows some (how much?) code from FV and is used to
    • Do offline advection for Chemistry
    • Do advection of the non-water species on a separate DT
• nwat is used. ncnst is not

[FlorianDeconinck]

pyFV3 breakdown:

• Code is split between TracerAdvection and Remapping:mapn_tracers.
• The "advectable" tracers are denoted by NQ in constants.py, which means order of tracers in the array matters (sic)
• TracerAdvection actually names the tracers in its code, NQ is passed but unused
• Mapn_tracers uses NQ to loop over the tracers and remap them

[FlorianDeconinck]

Potential solutions:

• Replicate the "bundle" of tracers via DataClass/Register combo, with a way to show water species and non-water species on top of names
• Allow the initialization of stencils by name of species so that we can do a simple for q_name in advectable_tracers: stencil[q_name](). That way on orchestration we will be able to use the nounroll and have a clean and faster SDFG

[FlorianDeconinck]

PR: NOAA-GFDL#37

[FlorianDeconinck]

Orchestration remain an issue.

Two main problems:

• The dict solution sported in the PR is very anti-DaCe, which doesn't allow any complex structure like that. It will work but the code will be unrolled. Fine with 7 tracers, painful with 31, unthinkable with 200+
• The newly implemented reduction over all nodes (MPI.Allreduce) are not DaCe friendly. Though there's a DaCe MPI it doesn't work out of the box. Callbacking might be a way out, but we need much stronger callbacks.