Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: MultiFabRegister #5230

Merged
merged 385 commits into from
Sep 24, 2024
Merged

Conversation

ax3l
Copy link
Member

@ax3l ax3l commented Sep 9, 2024

Add a central location to store all MultiFabs in. The ablastr::fields::MultiFabRegister provides us with a central location to allocate, initialize and exchange fields now.

This is useful to add new fields at runtime (even Python!), access them from anywhere for read & write (incl. Python), alias them from anywhere, redistribute them on load balancing by default (unless you say: no, don't do this, during allocation), and to simplify function calls that access a lot of fields (e.g., solvers just need to pass the ablastr::fields::MultiFabRegister & WarpX::m_fields around instead of including WarpX.H).

New Headers

New ABLASTR header:

#include <ablastr/fields/MultiFabRegister.H>

New Type Aliases

using ScalarField = amrex::MultiFab *;
using VectorField = std::array<amrex::MultiFab *, 3>;
using MultiLevelScalarField = amrex::Vector<ScalarField>;

and const variants.

New Functions

MultiFabRegister provides calls to allocate & initialize (alloc_init, alias_init), test (has), access (get, get_mr_levels, get_alldirs, get_mr_levels_alldirs), list (list), remove (erase, clear_level), and remake on AMR/load balance (remake_level).
Please see the Doxygen page for all details.

These functions can be used both with warpx::fields::FieldType enums (preferred) as well as std::string (e.g., for purely runtime logic or Python hackery).

TODO

  • make sure all test pass
  • write more extended PR description
  • Fields.H: warpx::fields::FieldType add per-enum-entry inline doc string
  • use throw logic on get with missing keys
  • cleanup

@ax3l ax3l added the hackathon Let's address this topic during the GPU hackathon label Sep 9, 2024
@ax3l ax3l requested review from dpgrote, RemiLehe, WeiqunZhang and JustinRayAngus and removed request for WeiqunZhang September 9, 2024 19:04
@ax3l ax3l force-pushed the hackathon-field-register branch 2 times, most recently from 15c6119 to b9c1d78 Compare September 9, 2024 19:53
@ax3l ax3l mentioned this pull request Sep 9, 2024
Source/WarpX.cpp Outdated Show resolved Hide resolved
Source/WarpX.cpp Outdated Show resolved Hide resolved
Source/ablastr/fields/MultiFabRegister.H Outdated Show resolved Hide resolved
Source/ablastr/fields/MultiFabRegister.H Outdated Show resolved Hide resolved
Source/ablastr/fields/MultiFabRegister.H Show resolved Hide resolved
Source/ablastr/fields/MultiFabRegister.H Outdated Show resolved Hide resolved
Comment on lines +56 to +74
/** Components (base vector directions) of vector/tensor fields.
*
* Because of different staggering, the components of vector/tensor fields are stored
* in separate (i)MultiFab.
*
* @todo: synchronize with AMReX "enum class Direction"
*/
struct Direction
{
int dir = 0;

bool operator<(const Direction& other) const
{
return other.dir < this->dir;
}

/* TODO: just temporary int compatibility */
operator int() const { return dir; }
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there is a Direction available in AMReX, which should serve the same purpose, maybe we can try to use that one and see if it works for us. AMReX's direction is defined here:

enum class Direction : int { AMREX_D_DECL(x = 0, y = 1, z = 2) };

This could be another separate follow-up issue/PR.

Copy link
Member Author

@ax3l ax3l Sep 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, see the @todo in the Doxygen comment block above.

We currently need to add some implicit casts and comparison operators, thus we cannot use it.

Examples/CMakeLists.txt Outdated Show resolved Hide resolved
@ax3l ax3l enabled auto-merge (squash) September 24, 2024 00:01
@ax3l ax3l merged commit 086b0a3 into ECP-WarpX:development Sep 24, 2024
33 of 37 checks passed
@ax3l ax3l deleted the hackathon-field-register branch September 24, 2024 03:33
@ax3l
Copy link
Member Author

ax3l commented Sep 24, 2024

Oh ha, clang-tidy is not a required test it seems.

@ax3l
Copy link
Member Author

ax3l commented Sep 24, 2024

Settings updated and fix in #5309

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hackathon Let's address this topic during the GPU hackathon
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants