WideNet: An Implementation based on Colossal-AI

This is the re-implementation of model WideNet from paper Go Wider Instead of Deeper:

@misc{xue2021wider,
      title={Go Wider Instead of Deeper}, 
      author={Fuzhao Xue and Ziji Shi and Futao Wei and Yuxuan Lou and Yong Liu and Yang You},
      year={2021},
      eprint={2107.11817},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

• The original implementation was in TensorFlow+TPU. This re-implementation is in Colossal-AI+GPU.
• This repo applies DALI datalodaer for ImageNet-1k dataset in TFRecord form

Experiment Settings

CV

NLP

Results

CV

NLP

Colossal-AI

Colossal-AI is an integrated large-scale model training system with efficient parallelization techniques.

• Site
• Github
• Examples