Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrap UCX processes outside of MPI #594

Open
2 of 3 tasks
Tracked by #575
nirandaperera opened this issue Jul 15, 2022 · 1 comment
Open
2 of 3 tasks
Tracked by #575

Bootstrap UCX processes outside of MPI #594

nirandaperera opened this issue Jul 15, 2022 · 1 comment
Assignees

Comments

@nirandaperera
Copy link
Collaborator

nirandaperera commented Jul 15, 2022

Parent issue: #575

  • Figure out how to create a context out of locally spawned processes (determine what metadata is required to spawn processes)
  • Get a UCC example running without MPI
  • Spawn UCX processes outside of MPI (use Dask/ ssh+bash to do this)

Possible options for a distributed key value store

  • Redis
  • ZeroMQ
  • Dask (cluster)
@kaiyingshan
Copy link
Collaborator

kaiyingshan commented Jul 18, 2022

Process of initializing UCX context:

  • get rank and world size (currently by MPI)
    init UCP context (with cylon::ucx::initContext)
    init send/recv workers (with cylon::ucx::initWorker)
  • allgather ucp recv worker address (currently by MPI), and create endpoint map
  • build with MPI if UCC is not present

Initializing UCC context:
get rank and world size from UCX communicator assigned by constructor

  • create ucc_context with oob allgather (currently using MPI)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants