Organization of steps for time dependent data and collection of time independent data

[ampdes]

For the FEM library DOLFINx, we would like to create checkpointing using ADIOS2.

The data consists of

1. Time dependent data: mesh
2. A collection of time independent data : meshtags (example, say three sets of meshtags)
3. Time dependent data: functions

The Step concept in ADIOS2 is an abstract one, and it could represent time points, or iterations or even collection of data (meshtags for example).

1. What is the recommended way to organize this?
2. When using only one IO, the reader has to be aware of the steps the writer took in order to read back. This is not practical.
3. Should one save to one folder say simulation.bp and use different IO and Engine to write?
   As a simple test case, writing using multiple IO, the bpls couldn't query such a dataset.

[pnorbert]

Hi, can you explain what is not practical for the readers? The recommended way to organize your data is to make the reader's life easy but not ruin the performance of the writer.

Suggestion 3 is not feasible with adios. Just because you call a folder something.bp, adios will not able to process it as a hierarchy of multiple outputs.

Other codes usually write the time-independent data into an output, and then write the time-dependent data into another output (consisting of many steps), or in case of a checkpoint, maybe a separate output per step, or overwrite the one output.

And the reader needs to know that there are two separate datasets to read from.

[ampdes]

Hi,
The BeginStep and EndStep collective functions are designed per write function, and we might write in the order

write_mesh(io, engine, mesh, t=0)
write_meshtags(io, engine, tag1)
write_meshtags(io, engine, tag2)
write_meshtags(io, engine, tag3)
write_function(io, engine, function1, t=0)
write_mesh(io, engine, mesh, t=0.5)
write_function(io, engine, function1, t=0.5)

And when reading, we might only be interested in reading mesh at all time steps. So keeping track of which step corresponds to the intended data seems not practical. Or is there a way?

Do you then recommend to create one ADIOS2 instance and attach as many IO and use as many Engine instances with separate folders for the mesh, meshtags, and each function?

Thanks.

[pnorbert]

Sorry, I misread first, now I see that mesh is time-dependent. Is it changing/saved every output step along with the data, or is it only written (occasionally) when it changes?

In the former case, I would restructure the code to have write_mesh, write_meshtags and write_function in a single step. i.e. call BeginStep before and EndStep after these calls in separate calls. When one opens the dataset, mesh tags will have appear to have a single step, so it is easy to read, while one has to pick the step (same step) for mesh and function to read a single step.

In the latter case, I would write the mesh into a separate output with its own steps. Since you have the t indicating the time, that can be used to find what mesh is "in play" when reading the data at a given t. I don't think there is an easy, straight-forward way of doing this without managing and searching among steps of mesh vs steps of functions, using the t in each dataset.

[pnorbert]

Well, I don't understand how you want the reader to work and go through the steps without being "aware of the steps the writer took". Can you explain please?