Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PaRSEC TTG: Offload stream reductions #256

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

devreal
Copy link
Contributor

@devreal devreal commented May 26, 2023

When we encounter a streaming terminal we schedule a dedicated
reducer task. Subsequent inputs are put into a LIFO. Once the
reducer task runs it will process all available inputs for that
terminal but will not block waiting for all inputs. In the
worst case, we create N tasks (one for each input), likely less.

Currently we use the priority of the task we stream into but this should be revisited, maybe a higher priority could be useful.

@devreal
Copy link
Contributor Author

devreal commented May 26, 2023

@therault the tests are failing because the madness tag is unavailable. Do you have a tag to use instead?

@therault
Copy link
Contributor

therault commented May 26, 2023 via email

@devreal devreal force-pushed the offload-stream-reductions branch from 7bd063b to ab86822 Compare May 26, 2023 19:19
@devreal devreal force-pushed the offload-stream-reductions branch 2 times, most recently from e34d232 to 35a1002 Compare June 5, 2023 19:09
devreal added 10 commits July 7, 2023 11:50
This is needed to work-around missing support for offsetof in virtual
classes. Instead we store the pointer to the derived class inside the
non-virtual base class. This allows us to store data copies in parsec
lifo or lists and get the virtual derived object back.

Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
When we encounter a streaming terminal we schedule a dedicated
reducer task. Subsequent inputs are put into a LIFO. Once the
reducer task runs it will process all available inputs for that
terminal but will not block waiting for all inputs. In the
worst case, we create N tasks (one for each input), likely less.

Signed-off-by: Joseph Schuchart <[email protected]>
The thread-local variable was a hack that is not needed anymore, and
apparently didn't work properly.

Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
…puts

In the PaRSEC backend, reducer tasks will be treated as any other
task and be deferred if there are readers on their first input.
All other inputs are read-only and will be handled that way.
This avoids extra copies if there are no other writers on these copies.

Signed-off-by: Joseph Schuchart <[email protected]>
@devreal devreal force-pushed the offload-stream-reductions branch from dd5c0e7 to 68a9852 Compare July 7, 2023 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants