About the compatibility of Colossal and torch.distributed.

[ZhuXMMM]

In my code, the original distributed sequences were using functions such as torch.distributed.all_reduce(), torch.distributed.get_backend(), torch.distributed.all_gather(), torch.distributed.is_available(), etc. I want to ask whether these functions are compatible with Colossal. If I want to replace them, which functions in Colossal should I use, or how can I find out?

[kurisusnowdeng]

We maintain the similar information of all distributed groups in a global context. You shall use gpc in your codes like from colossalai.core import global_context as gpc. Note that a parallel mode is usually needed to specify a certain group