Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Kokkos / CUDA / ROCM / oneAPI terminology table for hierarchical parallelism #33

Open
rbourgeois33 opened this issue Jan 9, 2025 · 3 comments

Comments

@rbourgeois33
Copy link

rbourgeois33 commented Jan 9, 2025

Kokkos CUDA (Nvidia) ROCM (AMD) oneAPI/DPC++ (Intel)
Team (of threads) Block Workgroup/Block Workgroup
Scratch pad memory (of a team) Shared Memory Shared Memory Local Memory
League size Number of Blocks Number of Workgroups/Blocks Number of Workgroups
Team size Number of Threads per Block Number of Threads per Workgroup/Block Number of Threads per Workgroup
League rank Index of the Block Index of the Workgroup/Block Index of the Workgroup
Team rank Index of the Thread within a Block Index of the Thread within a Workgroup Index of the Thread within a Workgroup
team member thread thread ?
Team Barrier __syncthreads() __syncthreads() item.barrier()
Core clusters Streaming Multiprocessors (SMs) Compute Units (CUs) I am unsure
@pzehner
Copy link
Member

pzehner commented Jan 9, 2025

That's a cool guide to add somewhere in the cheat sheet. Maybe that could make a third sheet "Terminologies cheat sheet."

I think we would need to differentiate/complete:

  • Memories (global memory, shared memory, local memory, register, UVM);
  • Execution chips (streaming multiprocessor, streaming processor, warp, thread);
  • Hierarchical parallelism itself.

@rbourgeois33
Copy link
Author

Agreed, let's start simple with only CUDA/Kokkos then (as this will help out most users)

@rbourgeois33
Copy link
Author

https://docs.google.com/spreadsheets/d/11aH1Dk1JOKUyF7ro__qtqBpenLCZ5I-Gv5_cDEU8FbU/edit?usp=sharing

I've started a cleaner set of table there (markdown is not ideal)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants