romio: implement binding generation for ROMIO #7228
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hzhou
force-pushed
the
2308_romio_binding
branch
14 times, most recently
from
966f859 to
d00e9f7
Compare
|
test:mpich/ch3/tcp ✔️ test:mpich/custom |
hzhou
force-pushed
the
2308_romio_binding
branch
2 times, most recently
from
35655c4 to
3c43546
Compare
|
test:mpich/custom |
hzhou
force-pushed
the
2308_romio_binding
branch
4 times, most recently
from
06fc378 to
a4c17e9
Compare
hzhou
force-pushed
the
2308_romio_binding
branch
from
a4c17e9 to
eaebe60
Compare
|
test:mpich/ch3/tcp test:mpich/custom |
1 similar comment
|
test:mpich/ch3/tcp test:mpich/custom |
raffenet
approved these changes
Jan 14, 2025
There was a problem hiding this comment.
There's a minor conflict that looks simple enough to resolve. @roblatham00 want to give a review? I didn't scrutinize everything since the generated code makes the actual change set pretty huge.
roblatham00
force-pushed
the
2308_romio_binding
branch
from
ca0ea65 to
ca6f98b
Compare
roblatham00
approved these changes
Jan 22, 2025
roblatham00
force-pushed
the
2308_romio_binding
branch
from
ca6f98b to
59b031e
Compare
Need double cast to suppress warnings when converting between pointers and integers. Also remove an unused macro, HAVE_INT_LT_POINTER.
Define the complete MPI-IO interface in mpi.h. In all_romio_symbols.c, we can use MPI_Request instead of MPIO_Reuqest since MPIO_Request is not longer defined in mpi.h (since it no longer includes mpio.h).
Add io_api.txt and generate io bindings
This function is now defined in src/binding/c/io_api.txt.
We don't have error checking for async_thing, amode, whence.
Wrap the code that generates c_bindings.c in a function. This prepares for later adding io binding generations. ROMIO bindings will be thinner than non-IO functions. So we will a separate function to generate IO bindings.
Refactor code to function dump_body_of_routine. This is a preparation for generating io bindings. Since ROMIO is designed on top of the rest of MPI, we will handle differently for parameter filtering and error checking, but the part that handles large count and calling the internal impl functions can be shared.
MPICH will generate the binding layer. Now that libromio just supply internal code, we no longer need worry about libromio vs libpromio. We don't separate man pages and include mpio.h in mpi.h either.
We no longer have any convenience libraries that need go into libmpi.so when libpmpi.so is also built. Thus, the legacy mpi_convenience_libs variable is no longer in use. However, we need differentiate libromio from the other convenience libraries for the ABI build. ROMIO depends on MPI, thus will need separate version for ABI and non-ABI build. Thus, we create two variables -- mpi_convenience_libs and abi_convenience_libs. Note: the former reuses the name but now with different meanings.
Generate io bindings including prototypes in mpi_proto.h and mpir_impl.h. src/binding/c/io.c caintains all the IO binding functions that calls the corresponding _impl functions. But it is not included in the Makefile yet. Dump the prototypes for io impl functions separately to allow inserting necessary missing declarations such as MPIR_Ext_cs_enter/exit. IO functions need use MPIR_Ext_cs_enter/exit for global cs. Now it gets more complicated, refactor into dump_global_cs_enter/exit for better readability.
Because ROMIO is built on top of the rest of MPI, it is difficult to use the abi wrappers because we need either - a. convert from ABI into MPICH abi and then convert back to ABI before calling into ROMIO. This seems silly. Or - b. build ROMIO on MPICH abi. But in libmpi_abi.so, MPICH abi is not available. Thus we need - c. Switch ROMIO to call MPICH impl functions instead of `MPI/PMPI` functions. We'll need to build a layer in ROMIO switch between using `MPI/PMPI` or using `MPIR_Xxx_impl` depending on build options. This will result in ROMIO having full access to MPICH internals! But let's delay work until we have a convincing need. Thus, this commit implements io abi bindings without the abi wrappers. We implemented a separate path rather than making the existing code more complex.
hzhou
force-pushed
the
2308_romio_binding
branch
from
59b031e to
014d053
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
[note: split preparation PR in #7232]
Integrate ROMIO into MPICH's binding generation framework. This will unify and simplify the IO binding tasks such as large count support, fortran support, man page support.
We'll retain the current stand-alone nature of ROMIO as much as we can in this PR. There shouldn't be any noticeable difference for users who build ROMIO separate from MPICH.
MPI_File_open) into a binding layer and an implementation layer --MPI_File_openinopen.candMPI_File_open_implinio_impl.cFROM_MPICHdefined stays exactly the samesrc/binding/c/io.c, which calls into theimpllayer that is defined in ROMIO.mpl.#include "mpio.h"inmpi.hAuthor Checklist
Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
Commits are self-contained and do not do two things at once.
Commit message is of the form:
module: short descriptionCommit message explains what's in the commit.
Whitespace checker. Warnings test. Additional tests via comments.
For non-Argonne authors, check contribution agreement.
If necessary, request an explicit comment from your companies PR approval manager.