DeadLock with multiple gpus

[bbjy]

When I pretrained the model from scratch, the traning process hangs during the first epoch. The GPU usage is stuck at 100%. The environment I used is as follows:

torch=1.8.0
pytorch_lightning=1.2.3
deepspeed=0.3.12

Is there anyone else encountered the same problem, and can give me some advice to tackle it？ Thank you！