Skip to content

Commit

Permalink
fix markdown lint issue
Browse files Browse the repository at this point in the history
  • Loading branch information
tocean committed Oct 18, 2023
1 parent 71d0a94 commit ac1cc19
Show file tree
Hide file tree
Showing 6 changed files with 26 additions and 14 deletions.
2 changes: 1 addition & 1 deletion .mdlrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
rules "~MD013", "~MD033", "~MD046"
rules "~MD013", "~MD033", "~MD046", "~MD034"
12 changes: 6 additions & 6 deletions docs/developer-guides/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ MS-AMP is an open-source project. Your participation and contribution are highly
#### What content can be added to MS-AMP

1. Bug fixes for existing features.
2. Performance improvement.
3. New features such as support for new distributed training framework.
1. Performance improvement.
1. New features such as support for new distributed training framework.

If you would like to contribute a new feature on MS-AMP, please submit your proposal first. In [GitHub Issues](https://github.com/azure/MS-AMP/issues) module, choose `Enhancement Request` to finish the submission. If the proposal is accepted, you can submit pull requests to origin `main` branch.

Expand All @@ -37,7 +37,7 @@ MS-AMP is an open-source project. Your participation and contribution are highly
If you would like to contribute to the project, please follow below steps of joint development on GitHub.

1. `Fork` the repo first to your personal GitHub account.
2. Checkout from main branch for feature development.
3. When you finish the feature, please fetch the latest code from origin repo, merge to your branch and resolve conflict.
4. Submit pull requests to origin main branch.
5. Please note that there might be comments or questions from reviewers. It will need your help to update the pull request.
1. Checkout from main branch for feature development.
1. When you finish the feature, please fetch the latest code from origin repo, merge to your branch and resolve conflict.
1. Submit pull requests to origin main branch.
1. Please note that there might be comments or questions from reviewers. It will need your help to update the pull request.
12 changes: 9 additions & 3 deletions docs/developer-guides/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,22 @@ Follow [System Requirements](../getting-started/installation.mdx).
## Set up

Clone code.

```bash
git clone --recurse-submodules https://github.com/azure/MS-AMP
cd MS-AMP
```

Install MS-AMP.

```bash
python3 -m pip install --upgrade pip
python3 -m pip install -e .[test]
python3 -m pip install -e .[test]
make postinstall
```

Install MSCCL and preload msamp_dist library
Install MSCCL and preload msamp_dist library.

```bash
cd third_party/msccl
# H100
Expand All @@ -47,16 +50,19 @@ export LD_PRELOAD="/usr/local/lib/libmsamp_dist.so:${NCCL_LIBRARY}:${LD_PRELOAD}
## Lint and Test

Format code using yapf.

```bash
python3 setup.py format
```

Check code style with mypy and flake8
Check code style with mypy and flake8.

```bash
python3 setup.py lint
```

Run unit tests.

```bash
python3 setup.py test
```
Expand Down
4 changes: 3 additions & 1 deletion docs/getting-started/run-msamp.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ id: run-msamp
---

# Run examples

After installing MS-AMP, you can run several simple examples using MS-AMP. Please note that before running these commands, you need to change work directory to [examples](https://github.com/Azure/MS-AMP/tree/main/examples).

## MNIST

### 1. Run mnist using single GPU

```bash
Expand Down Expand Up @@ -38,4 +40,4 @@ deepspeed cifar10_deepspeed.py --deepspeed --deepspeed_config ds_config_msamp.js
deepspeed cifar10_deepspeed.py --deepspeed --deepspeed_config ds_config_zero_msamp.json
```

For more comprehensive examples, please go to [MS-AMP-Examples](https://github.com/Azure/MS-AMP-Examples).
For more comprehensive examples, please go to [MS-AMP-Examples](https://github.com/Azure/MS-AMP-Examples).
2 changes: 1 addition & 1 deletion docs/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,4 @@ Here are the results for Swin-1.0B and ViT-1.2B.

![Image](./assets/gpu-memory.png)

For detailed setting and results, please go to [MS-AMP-Example](https://github.com/Azure/MS-AMP-Examples).
For detailed setting and results, please go to [MS-AMP-Example](https://github.com/Azure/MS-AMP-Examples).
8 changes: 6 additions & 2 deletions website/blog/2023-07-20-release-0-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,23 @@ We are very happy to announce that **MS-AMP 0.2.0 version** is officially releas

You can install and try MS-AMP by following [Getting Started Tutorial](https://azure.github.io/MS-AMP/docs/getting-started/installation).

## MS-AMP 0.2.0 Release Notes
# MS-AMP 0.2.0 Release Notes

## MS-AMP Improvements

- Add O3 optimization for supporting FP8 in distributed training frameworks
- Support ScalingTensor in functional.linear
- Support customized attributes in FP8Linear
- Improve performance
- Improve performance
- Add docker file for pytorch1.14+cuda11.8 and pytorch2.1+cuda12.1
- Support pytorch 2.1
- Add performance result and TE result in homepage
- Cache TE build in pipeline

## MS-AMP-Examples Improvements

Add 3 examples using MS-AMP:

- [GPT-3 model](https://github.com/Azure/MS-AMP-Examples/tree/main/gpt3)
- [RoBERTa model](https://github.com/Azure/MS-AMP-Examples/tree/main/RoBERTa)
- [CIFAR10 model](https://github.com/Azure/MS-AMP/tree/main/examples#cifar10)

0 comments on commit ac1cc19

Please sign in to comment.