Mesh preprocessing operations #260

sebastiangrimberg · 2024-06-05T23:31:23Z

Convert all elements to simplices (enable with "MakeSimplex")
Convert all elements to hexahedra (enable with "MakeHexahedral", only for tet meshes currently, compatible with "MakeSimplex")
Serial refinement levels ("SerialUniformLevels")

All mesh conversions work with high-order curved meshes by interpolating the nodal grid function between meshes using GSLIB. Also exposes via configuration file a number of mesh preprocessing options which used to be hard-coded.

hughcars

A few minor questions and comments, but LGTM.

hughcars · 2024-06-21T19:02:18Z

palace/fem/interpolator.cpp

+    int npts = 0, offset = 0;
+    for (int i = 0; i < dest_mesh.GetNE(); i++)
+    {
+      npts += V.FESpace()->GetFE(i)->GetNodes().GetNPoints();


All points are interior to elements or if there is duplication at intersections we don't mind? I would guess it makes indexing etc way more complicated so probably not worth it.

Yeah the latter, we just find the duplicate shared points nodes between elements and they are disregarded because we SetSubVector later on. Certainly a future optimization could be to remove these duplicates from the GSLIB search and just maintain the indexing separately.

hughcars · 2024-06-21T19:14:56Z

palace/utils/geodata.cpp

@@ -212,7 +285,7 @@ void RefineMesh(const IoData &iodata, std::vector<std::unique_ptr<mfem::ParMesh>
  // reorient only the coarse mesh so that the refinements are still true refinements of
  // the original mesh (required for geometric multigrid). Otherwise, it happens after
  // refinement.
-  if (iodata.model.reorient_tet && mesh.capacity() > 1)
+  if (iodata.model.reorient_tet_mesh && mesh.capacity() > 1)


If mesh.empty() == true this could still be triggered as capacity is the largest value seen. I think not an actual problem as we're always in the case where size() == capacity(), but worth fixing.

The check at https://github.com/awslabs/palace/pull/260/files#diff-0642b48eb210cdd7e98a30345d4857eb87c192985610bc00696294583cd09a43R259 should prevent that from ever being the case. The idea here is that if we are storing a multigrid hierarchy of meshes, we want to reorient the coarse mesh, otherwise we will wait until the end to do the reorientation.

hughcars · 2024-06-21T19:15:17Z

palace/utils/geodata.cpp

@@ -360,7 +433,7 @@ void RefineMesh(const IoData &iodata, std::vector<std::unique_ptr<mfem::ParMesh>
  // not required after MFEM's PR #1046, but in case you want to be absolutely sure, we
  // reorient only the mesh after refinement if there is a single mesh (doesn't work with
  // h-refinement geometric multigrid).
-  if (iodata.model.reorient_tet && mesh.size() == 1)
+  if (iodata.model.reorient_tet_mesh && mesh.capacity() == 1)


Changing from size to capacity, implies a vector where pop() etc. had been called shouldn't on this branch, is that intentional?

See my comment above, but in particular the change here is just to be consistent in use of capacity for both checks.

hughcars · 2024-06-26T14:07:16Z

palace/utils/geodata.cpp

+  if (!use_mesh_partitioner)
+  {
+    MPI_Comm_split_type(comm, MPI_COMM_TYPE_SHARED, Mpi::Rank(comm), MPI_INFO_NULL,
+                        &node_comm);
+  }


To check I am getting this correctly, the idea is to split the communicator into N communicators, one for each node. Then within each communicator, only the root rank loads the serial, thereby avoiding the memory overloading issue of every rank on a node loading the mesh for preprocessing purposes.

Correct. Just one process on each node does the serial mesh preprocessing operations to save memory overhead, and then broadcasts the final serial mesh to all other processes on the node.

palace/utils/geodata.cpp

hughcars · 2024-06-26T14:33:36Z

palace/utils/geodata.cpp

+      if (end == start + 1)
+      {
+        new_fespace.GetElementVDofs(start, vdofs);
+        new_nodes.SetSubVector(vdofs, vals);
+        start = end;
+        continue;
+      }


In the MakeSimplex and MakeHex methods, neither allow mixed presently, so is this an edge case that can ever be hit? I.e. all tets get converted, and all hexes get converted. Is this just anticipating mixed meshes?

Actually MakeSimplex allows for a mixed mesh, so this code gets hit for a mixed tet-prism mesh, for example.

hughcars · 2024-06-26T14:37:31Z

palace/utils/geodata.cpp

+        for (int j = 0; j < pointmat.Width(); j++)
+        {
+          for (int d = 0; d < sdim; d++)
+          {
+            // Use default ordering byNODES.
+            xyz(d * npts + offset + j) = pointmat(d, j);
+          }
+        }


For this loop, rather than jumping back and forwards by npts in xyz for each write, it might be more cache coherent to have the d loop on the outer. This loops the slow way through pointmat (assuming column major), but pointmat is smaller than xyz, so the npt jumps about in xyz might be causing more reads etc.

Yeah good call about pointmat being small so the jumps not being a big issue. Fixed.

…ry element addition)

…d interface boundary element additions (rather than hard-coding)

… of curvature when splitting mesh into simplices

…esh partitioner class

sebastiangrimberg added enhancement New feature or request mesh Related to meshes and mesh generation labels Jun 5, 2024

sebastiangrimberg force-pushed the sjg/mesh-simplex-dev branch from 789a272 to b2f178a Compare June 10, 2024 18:20

sebastiangrimberg force-pushed the sjg/mesh-simplex-dev branch 2 times, most recently from 24fc0d8 to 77cbc97 Compare June 21, 2024 18:44

sebastiangrimberg requested a review from hughcars June 21, 2024 18:44

hughcars reviewed Jun 26, 2024

View reviewed changes

sebastiangrimberg added 14 commits June 26, 2024 15:02

Preliminary cleanup for mesh cracking (remove unused subdomain bounda…

71cc19e

…ry element addition)

Add configuration file options for mesh reordering, mesh cleaning, an…

651af79

…d interface boundary element additions (rather than hard-coding)

Minor docs cleanup

f3e307f

Add function interpolation between meshes, which enables preservation…

cab5f61

… of curvature when splitting mesh into simplices

Fix interpolation for non-H1 spaces

f4bdc61

Add checks for pyramid elements when converting to simplices

22af1ff

Add tet-to-hex mesh conversion option, also supporting high-order meshes

0bbd579

Add serial refinement capability for debugging

f0c9bcf

Improve mesh partitioning for nonconforming meshes/meshes not using m…

4c18884

…esh partitioner class

Improve high-order node transfer for mesh splitting

67fd635

Fix bug for certain MPI distributions

f319200

MakeHex -> MakeHexahedral

54d6631

Update changelog

5053453

Address PR feedback

3e03a64

sebastiangrimberg force-pushed the sjg/mesh-simplex-dev branch from 5ae3332 to 3e03a64 Compare June 26, 2024 22:02

hughcars approved these changes Jul 1, 2024

View reviewed changes

sebastiangrimberg merged commit 385804b into main Jul 1, 2024
17 checks passed

sebastiangrimberg deleted the sjg/mesh-simplex-dev branch July 1, 2024 16:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mesh preprocessing operations #260

Mesh preprocessing operations #260

sebastiangrimberg commented Jun 5, 2024 •

edited

Loading

hughcars left a comment

hughcars Jun 21, 2024

sebastiangrimberg Jun 26, 2024

hughcars Jun 21, 2024

sebastiangrimberg Jun 26, 2024

hughcars Jun 21, 2024

sebastiangrimberg Jun 26, 2024

hughcars Jun 26, 2024

sebastiangrimberg Jun 26, 2024 •

edited

Loading

hughcars Jun 26, 2024

sebastiangrimberg Jun 26, 2024

hughcars Jun 26, 2024

sebastiangrimberg Jun 26, 2024

Mesh preprocessing operations #260

Mesh preprocessing operations #260

Conversation

sebastiangrimberg commented Jun 5, 2024 • edited Loading

hughcars left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastiangrimberg Jun 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastiangrimberg commented Jun 5, 2024 •

edited

Loading

sebastiangrimberg Jun 26, 2024 •

edited

Loading