Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile time scales with MPI ranks (and it shouldn't) #386

Closed
mofeing opened this issue Dec 16, 2024 · 1 comment
Closed

Compile time scales with MPI ranks (and it shouldn't) #386

mofeing opened this issue Dec 16, 2024 · 1 comment

Comments

@mofeing
Copy link
Collaborator

mofeing commented Dec 16, 2024

cc @Todorbsc

we are seeing this strange behavior when running Reactant.compile in several MPI ranks at the same time. this should be flat, as there is no communication happening.

i have no idea of what's going around, except that MPI and Julia compiler don't like each other

Warning

note that this is not precompilation or load time, but just time taken by first call to Reactant.compile.

Image

@mofeing mofeing changed the title Compile time scales with MPI ranks Compile time scales with MPI ranks (and it shouldn't) Dec 16, 2024
@mofeing
Copy link
Collaborator Author

mofeing commented Dec 28, 2024

The issue was unrelated to Reactant: MPI was initiating Julia in the same core. Thread pinning solved the problem.

@mofeing mofeing closed this as completed Dec 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant