Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ucc reduce ops #660

Closed
wants to merge 54 commits into from
Closed

Ucc reduce ops #660

wants to merge 54 commits into from

Conversation

mstaylor
Copy link
Collaborator

@mstaylor mstaylor commented Jun 1, 2023

This PR exposes AllReduce for usage in python via the UCXUCCCommunicator

kaiyingshan and others added 30 commits February 23, 2023 17:21
@kaiyingshan @mstaylor
add redis
19fc2fc
@kaiyingshan @mstaylor
able to run without mpi
399c6ee
@kaiyingshan @mstaylor
remove useless file
ffdfcca
@kaiyingshan @mstaylor
separate oob logic from ucx/ucc communicators
5b2da21
@kaiyingshan @mstaylor
re-enable tests
6001c24
@kaiyingshan @mstaylor
code placements
720d71e
@kaiyingshan @mstaylor
minor fixes
3a221bf
@kaiyingshan @mstaylor
added python script to run cylon ucx/ucc without mpirun
82f4bda
@kaiyingshan @mstaylor
mimic gather with allgather
ccb323a
@mstaylor
Fixes missing MPI_Comm in UCXConfig
9aa2ce2
@mstaylor
Adds CYLON_USE_REDIS flag to allow UCC/UCX builds that don't require …
aed0a44 
…redis
@mstaylor
Changes ucc_operations to reflect code in cylondata/cylon main
7670557
@mstaylor
Changes adds defaults for CYLON_UCX, CYLON_UCC and CYLON_USE_REDIS
cdadc14
@mstaylor
Changes adds defaults for CYLON_UCX, CYLON_UCC and CYLON_USE_REDIS
6eb56f9
@mstaylor
Changes adds defaults for CYLON_UCX, CYLON_UCC and CYLON_USE_REDIS
da99851
@mstaylor
Changes adds defaults for CYLON_UCX, CYLON_UCC and CYLON_USE_REDIS
2e15781
@mstaylor
Adds missing constructor for UCXUCCCommunicator from main
6de16fd
@mstaylor
Further resolution of differences between main and branch
103eccd
@mstaylor
Fix merge issue where CommType Type() was private and results in a co…
d337a68 
…nflict in python tests
@mstaylor
Fix issue with UCX build where CreateChannel in UCX Communicator is n…
b7b0173 
…ot implemented
@mstaylor
Fixes issue with UCX (non-UCC) tests
9f7e38b
@mstaylor
Adds support for redis build git workflow
2ff91a0
@mstaylor
fix hiredis workflow
f7b6fee
@mstaylor
Adds support for redis build git workflow
3694a2c
@mstaylor
Adds support for redis build git workflow
2ad10fa
@mstaylor
Adds support for redis build git workflow - root install via sudo
4bcbae3
@mstaylor
moves OOBType to separate hpp + cython support
0ce6df6
@mstaylor
adds oob_context cython + updates to build.py and setup.py in support…
a65a7fe 
… of includes and linking libraries in python
@mstaylor
adds oob_context cython + updates to build.py and setup.py in support…
7211adf 
… of includes and linking libraries in python
@mstaylor
adds oob_context cython + updates to build.py and setup.py in support…
1d45b39 
… of includes and linking libraries in python
@mstaylor
adds oob_context cython + updates to build.py and setup.py in support…
fdbe7c9 
… of includes and linking libraries in python
@mstaylor
adds oob_context cython + updates to build.py and setup.py in support…
2c47ef7 
… of includes and linking libraries in python
@mstaylor
separates redis oob contexts in separate source files to facilitate c…
3f9ba96 
…ython processing
@mstaylor
refactoring related to non-redis environments + introducing UCXRedisO…
5022567 
…OBContext
@mstaylor
refactoring related to non-redis environments + introducing UCXRedisO…
32005ce 
…OBContext - part 2 (bug fixes)
@mstaylor
introduces UCCRedisOOBContext and adds calls to wrapper class
11d3ed1
@mstaylor
adds UCC Config
fbbba2e
@mstaylor
adds UCC Config (removes redis hard dependency)
f836668
@mstaylor
adds necessary hooks in lib.pxd, lib.pyx and context for initDistributed
e218ac0
@mstaylor
includes aws cf scripts for redis and minor change for operator examp…
3d92e3c 
…le to execute for a redis environment
@mstaylor
minor changes to oob contexts to support Cython initialization + upda…
ccb6166 
…ted cylon-redis to reflect new cloudbank account
@mstaylor
adds redis example and minor changes related to redis oob context
3e7e123
@mstaylor
fixes for running redis example
7974db9
@mstaylor
updates to redis_example to take argument for world size, redis host …
ec3b2f9 
…and port and demo_join example
@mstaylor
adds support for ReduceOp
a9f9573
@mstaylor
fixes circular dependency when using CScalar
33d45f4
@mstaylor
UCC/UCX AllReduce partial
83bc34e
@mstaylor
UCC/UCX AllReduce partial
d9f3abf
@mstaylor
UCC/UCX AllReduce partial
3a5fe29
@cylon-manager
Copy link
Collaborator

@mstaylor please cherry-pick commits from a9f9573 and open a new PR. Previous merge has resulted in some conflicts, but these should be trivially addressed by a cherry-pick.

cylon-manager
cylon-manager requested changes Aug 3, 2023
View reviewed changes
Copy link
Collaborator

@cylon-manager cylon-manager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mstaylor please cherry-pick commits from a9f9573 and open a new PR. Previous merge has resulted in some conflicts, but these should be trivially addressed by a cherry-pick.

@nirandaperera
Copy link
Collaborator

closing this PR as these changes have already merged in PR #672

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cylon-manager cylon-manager cylon-manager requested changes

@nirandaperera nirandaperera Awaiting requested review from nirandaperera

@arupcsedu arupcsedu Awaiting requested review from arupcsedu

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mstaylor @cylon-manager @nirandaperera @kaiyingshan