Better Document Distributed Jobs

[Quentin-Anthony]

Currently there's no documentation on how to run gpt-neox on multiple nodes. For example, the deepspeed_mpi, deepspeed_slurm, and launcher config args aren't well documented.

• Add README section explaining the different ways to launch gpt-neox

[gauravaror]

I am keen to take up this issue. Can you please assign this issue to me unless this has already been done?

I am new to the project, so doing this will help me understand how to use the project and different configurations. What's the best place to ask questions if I have some questions GitHub issues or discord?

[StellaAthena]

I am keen to take up this issue. Can you please assign this issue to me unless this has already been done?

I am new to the project, so doing this will help me understand how to use the project and different configurations. What's the best place to ask questions if I have some questions GitHub issues or discord?

Welcome! This is still open.

GitHub is best for documenting work, Discord (esp. the #gpt-neox channel) is best for synchronous conversations.