Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move distributed pixel data to node shared memory #478

Open
tskisner opened this issue Sep 19, 2021 · 0 comments
Open

Move distributed pixel data to node shared memory #478

tskisner opened this issue Sep 19, 2021 · 0 comments
Assignees
Milestone

Comments

@tskisner
Copy link
Member

Currently, the PixelDistribution and PixelData classes store a sparse set of pixel values on each process. These pixel values correspond to the pixels covered by the local detector data on each process. For some scenarios (high resolution pixelization, high cadence scanning), the sky fraction covered by a single process can be quite large. The impact of this high per-process memory use is that fewer processes per node can be used.

One possible solution is currently in development which uses MPI shared memory for storing a node-local copy of pixels seen by all processes on the node. This incurs a cost when updating the pixel values, since locking or similar turn-taking must be done by processes on the node. However, the hope is that this cost will be more than offset by enabling the use of more processes per node.

@tskisner tskisner added this to the Release 3.0 milestone Sep 19, 2021
@tskisner tskisner self-assigned this Sep 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant