Initial MIR component #1438

BradWhitlock · 2024-10-03T20:12:48Z

This PR is adds an initial MIR component to Axom. EquiZ MIR was implemented as the first algorithm because it was more familiar and my first goal was to shake out problems with infrastructure for writing algorithms against Blueprint data. The MIR algorithm takes Blueprint input and generates Blueprint output.

This PR does the following:

Supercedes previous MIR work (previous work was either adapted or moved to "reference" directory)
Implements EquiZ (without iteration for now) that runs on CPU/GPU and works on 2D/3D meshes with various topology and shape types.
Provides views to simplify writing algorithms that use Blueprint data.
Provides several utility classes that perform mesh operations (extract zones, merge meshes, etc.)
Adds documentation
Adds example programs
Adds a little core infrastructure

NOTE:

The Elvira algorithm will be added in a future PR.
The previous serial MIR implementation was moved to a "reference" directory that we can remove once EquiZAlgorithm has iteration support (future work).
The component requires Conduit, RAJA, Umpire - I'll probably have to iterate with the CI some to make that work everywhere.
There are some files that changed because of "make style"

… other bugs.

Arlie-Capps · 2024-11-05T23:19:44Z

src/axom/mir/EquiZAlgorithm.hpp

+using MaterialID = int;
+using MaterialIDArray = axom::Array<MaterialID>;
+using MaterialIDView = axom::ArrayView<MaterialID>;
+using MaterialVF = float;


Is float a design choice over double?

Yes, but I think I could change it to use the material view's FloatType for MaterialVF so it stays true to the types specified in the material view. I think I had some compiler problems in this code but I could try again.

kennyweiss

Continuing to make my way through this PR... this review covers the MIR examples.

When you get a chance, could please share some performance results and visualizations?

src/axom/mir/tests/mir_smoke.cpp

src/axom/mir/tests/mir_testing_helpers.hpp

kennyweiss · 2024-10-31T00:37:48Z

src/axom/mir/tests/mir_testing_helpers.hpp

+    std::cout << "a={";
+    for(axom::IndexType i = 0; i < a.size(); i++)
+    {
+      if(i > 0) std::cout << ", ";


Note:
~~Our clang-format can't enforce this until we go past clang-10, but we require braces around single line conditionals/loops~~
Since we're on clang-14 for our clang-format, we should update our rules to automatically add braces around single line conditionals

kennyweiss · 2024-10-31T23:50:22Z

src/axom/CMakeLists.txt

+# Add MIR if the prerequisites are found.
+if(RAJA_FOUND AND UMPIRE_FOUND AND CONDUIT_FOUND)
+    axom_add_component(COMPONENT_NAME mir
+                       DEFAULT_STATE  ${AXOM_ENABLE_ALL_COMPONENTS})
+else()
+    message(STATUS "Axom Component MIR turned off due to missing RAJA, UMPIRE, or Conduit.")
+    set(AXOM_ENABLE_MIR OFF CACHE BOOL "")
+endif()


I wonder if we should add lines to our spack package to disable MIR when RAJA and Conduit are not present.
And similarly, to disable Sidre, Inlet and Klee when axom is configured without conduit.

For your specific question, I think we'd need to add -DAXOM_ENABLE_MIR=OFF to the *_noraja* CI configs, e.g. here

axom/azure-pipelines.yml

Line 48 in e5b5707

CMAKE_EXTRA_FLAGS: '-DBUILD_SHARED_LIBS=ON -DAXOM_QUEST_ENABLE_EXTRA_REGRESSION_TESTS:BOOL=ON -U RAJA_DIR'

src/axom/mir/examples/concentric_circles/CMakeLists.txt

src/axom/mir/examples/tutorial_simple/mir_tutorial_simple.cpp

src/axom/mir/examples/CMakeLists.txt

src/axom/mir/examples/tutorial_simple/mir_tutorial_simple.cpp

src/axom/mir/examples/tutorial_simple/runMIR.hpp

…x OpenMP macro in example program.

… from writing output files during CI tests. I/O might be causing the failures.

BradWhitlock · 2024-11-09T01:00:30Z

I added a script that will make BASH scripts to run the mir_concentric_circles example program on various platforms and gather the caliper data results. I plotted the MIR runtime and some things look good, some very bad at the moment. I think SEQ mode could be optimized more to bring it down, likely bringing down time in the other modes too. OMP has inexplicably bad performance. I've seen some hints of this before during development but focused more on GPU so far. I'll need to profile OMP and see what's the matter.

rzansel42{whitlocb}144: env OMP_NUM_THREADS=44 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 17.601138     17.601138     17.601138 88.542790 
rzansel42{whitlocb}145: env OMP_NUM_THREADS=88 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 20.347344     20.347344     20.347344 89.191530 
rzansel42{whitlocb}146: env OMP_NUM_THREADS=32 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 16.865811     16.865811     16.865811 88.723097 
rzansel42{whitlocb}147: env OMP_NUM_THREADS=22 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 12.256273     12.256273     12.256273 85.147964 
rzansel42{whitlocb}148: env OMP_NUM_THREADS=2 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                  6.044385      6.044385      6.044385 73.873691 
rzansel42{whitlocb}149: env OMP_NUM_THREADS=1 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                  3.880441      3.880441      3.880441 64.487702 
rzansel42{whitlocb}150: ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 30.048744     30.048744     30.048744 90.253846

Arlie-Capps · 2024-11-15T20:28:02Z

src/axom/mir/tests/mir_equiz2d.cpp

+void conduit_debug_err_handler(const std::string &s1, const std::string &s2, int i1)
+{
+  std::cout << "s1=" << s1 << ", s2=" << s2 << ", i1=" << i1 << std::endl;
+  // This is on purpose.


Thank you: this is a necessary comment.

gunney1 · 2024-11-20T15:18:13Z

src/axom/core/memory_management.hpp

+inline int getAllocatorIDForAddress(void* ptr)
+{
+  umpire::ResourceManager& rm = umpire::ResourceManager::getInstance();
+  return rm.getAllocator(ptr).getId();


What if ptr was not Umpire-allocated? A check that the allocator could be found would be helpful to avoid crashing. Would it make sense to return INVALID_ALLOCATOR_ID in that case?

Good suggestion. I added a test that calls getAllocatorIDForAddress() with a pointer not allocated by Umpire, which made Umpire throw an exception. I added exception handling and return INVALID_ALLOCATOR_ID in that case.

src/axom/core/utilities/Sorting.hpp

…h a pointer not allocated by umpire. Added test.

…mall changes too.

Arlie-Capps · 2024-12-13T23:38:09Z

src/axom/mir/utilities/CoordsetBlender.hpp

+   *
+   * \note The coordset view must agree with the coordset in n_input. We pass both
+   *       a view and the coordset node since the view may not be able to contain
+   *       come coordset metadata and remain trivially copyable.


"come coordset metadata" Should this be "some coordset metadata"?

BradWhitlock added 30 commits July 15, 2024 17:59

Compilation fixes

9b143d1

Change accessing edges

a2fa467

Fixed some coordset views. Added distance field

0930c1c

SLIC errors

fc240cb

Fix bit functions

eefee2b

Added mechanism for Conduit to allocate through Axom.

6af4fa2

Added mechanism for making Conduit allocate through Axom. Fixed a few…

7f0d095

… other bugs.

Debugging ClipField

719329b

Populate slice data so we can map element fields to new mesh

72d3240

Added ClipOptions

7b293b0

More testing

a5ff0ac

templated

07bdecf

Changes that let ClipField run on GPU with HIP

395c37c

Initial mixed topo support

24a78d5

Try out the mixed shape view

41e6a46

Some cleanup/refactoring

4ba2c84

code cleanup

6ff5327

Adjust 3D shapes

721e970

debugging

6db7e4b

Some refactoring to see if that helps for debugging.

39b3e33

wedge is bad too

8cd51b2

Added a test for explicit coordset view

acf1ac2

Fix typo

5331c0a

Fixed a clip bug from incorrect shape traversal

b46d992

Fixed wedge edges

40152ce

Fixes for wedge clip

43d9c47

start trying to work around OMP issue

e8c8474

Broke ClipField::execute into methods.

2f39c3e

code cleanup

a7d9e1e

Attempts to work around OMP problems.

dd75375

BradWhitlock added 2 commits November 4, 2024 10:26

Updated data directory to main.

cf008dc

Merged develop into branch.

9a0f9c2

Arlie-Capps reviewed Nov 5, 2024

View reviewed changes

kennyweiss reviewed Nov 6, 2024

View reviewed changes

BradWhitlock added 15 commits November 6, 2024 16:17

Removed unused test. Updated copyrights

cc74af3

Use join

438de42

Some CLI11 improvements.

0d55d81

Run example programs as tests.

3a478ef

make style

95961ab

Only allow certain test policies if umpire and raja are available. Fi…

639a154

…x OpenMP macro in example program.

Disable MIR if RAJA and Umpire are not in the Spack spec.

28673c5

make style

8d967a6

Added --disable-write flag to example programs so we can prevent them…

c86654d

… from writing output files during CI tests. I/O might be causing the failures.

Enable caliper on some tests. Increase Azure time limit.

1e4e913

Merged develop to branch.

d277541

Minor fixes, build changes

128da10

Added mir_concentric_circles.py script

8229455

Fix mistake from previous merge in macos on Azure.

2a4987f

Speedup for MeshTester case that generates circle materials.

558b844

Arlie-Capps reviewed Nov 15, 2024

View reviewed changes

gunney1 reviewed Nov 20, 2024

View reviewed changes

src/axom/core/utilities/Sorting.hpp Show resolved Hide resolved

BradWhitlock added 6 commits November 20, 2024 14:46

Added exception handling for when getAllocatorIDForAddress called wit…

275851d

…h a pointer not allocated by umpire. Added test.

Added comments about template specialization in sorting. Made a few s…

a034b56

…mall changes too.

Fixed issue introduced when speeding up MeshTester.

42cb6fc

make style

e4c8ca1

Merged develop to branch.

4537b07

Merge branch 'develop' into feature/whitlock/mir_update

ab3e869

Arlie-Capps reviewed Dec 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial MIR component #1438

Initial MIR component #1438

BradWhitlock commented Oct 3, 2024 •

edited

Loading

Arlie-Capps Nov 5, 2024

BradWhitlock Nov 5, 2024

kennyweiss left a comment

kennyweiss Oct 31, 2024 •

edited

Loading

kennyweiss Oct 31, 2024

BradWhitlock commented Nov 9, 2024 •

edited

Loading

Arlie-Capps Nov 15, 2024

gunney1 Nov 20, 2024

BradWhitlock Nov 20, 2024

Arlie-Capps Dec 13, 2024

Initial MIR component #1438

Are you sure you want to change the base?

Initial MIR component #1438

Conversation

BradWhitlock commented Oct 3, 2024 • edited Loading

Arlie-Capps Nov 5, 2024

Choose a reason for hiding this comment

BradWhitlock Nov 5, 2024

Choose a reason for hiding this comment

kennyweiss left a comment

Choose a reason for hiding this comment

kennyweiss Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

kennyweiss Oct 31, 2024

Choose a reason for hiding this comment

BradWhitlock commented Nov 9, 2024 • edited Loading

Arlie-Capps Nov 15, 2024

Choose a reason for hiding this comment

gunney1 Nov 20, 2024

Choose a reason for hiding this comment

BradWhitlock Nov 20, 2024

Choose a reason for hiding this comment

Arlie-Capps Dec 13, 2024

Choose a reason for hiding this comment

BradWhitlock commented Oct 3, 2024 •

edited

Loading

kennyweiss Oct 31, 2024 •

edited

Loading

BradWhitlock commented Nov 9, 2024 •

edited

Loading