-
Notifications
You must be signed in to change notification settings - Fork 466
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into flex_scheduler
- Loading branch information
Showing
185 changed files
with
7,368 additions
and
4,279 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,15 +1,15 @@ | ||
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help. | ||
<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> | ||
|
||
## Motivation | ||
|
||
Please explain the motivation behind this PR and the goal you aim to achieve with it. | ||
<!-- Explain the purpose of this PR and the goals it aims to achieve. --> | ||
|
||
## Modification | ||
## Modifications | ||
|
||
Briefly describe the changes made in this PR. | ||
<!-- Describe the changes made in this PR. --> | ||
|
||
## Checklist | ||
|
||
1. Ensure pre-commit `pre-commit run --all-files` or other linting tools are used to fix potential lint issues. | ||
2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness. | ||
3. Modify documentation as needed, such as docstrings or example tutorials. | ||
- [ ] Format your code according to the [Contributor Guide](https://github.com/sgl-project/sglang/blob/main/docs/en/contributor_guide.md). | ||
- [ ] Add unit tests as outlined in the [Contributor Guide](https://github.com/sgl-project/sglang/blob/main/docs/en/contributor_guide.md). | ||
- [ ] Update documentation as needed, including docstrings or example tutorials. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
name: Accuracy Test | ||
|
||
on: | ||
push: | ||
branches: [ main ] | ||
paths: | ||
- "python/sglang/**" | ||
- "test/**" | ||
pull_request: | ||
branches: [ main ] | ||
paths: | ||
- "python/sglang/**" | ||
- "test/**" | ||
workflow_dispatch: | ||
|
||
concurrency: | ||
group: accuracy-test-${{ github.ref }} | ||
cancel-in-progress: true | ||
|
||
jobs: | ||
accuracy-test: | ||
if: github.repository == 'sgl-project/sglang' || github.event_name == 'pull_request' | ||
runs-on: 1-gpu-runner | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Install dependencies | ||
run: | | ||
pip install --upgrade pip | ||
pip install -e "python[all]" | ||
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall | ||
git clone https://github.com/merrymercy/human-eval.git | ||
cd human-eval | ||
pip install -e . | ||
- name: Evaluate Accuracy | ||
timeout-minutes: 20 | ||
run: | | ||
cd test/srt | ||
python3 test_eval_accuracy_large.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
name: Cancel PR Workflows on Merge | ||
|
||
on: | ||
pull_request_target: | ||
types: | ||
- closed | ||
|
||
permissions: | ||
actions: write | ||
|
||
jobs: | ||
cancel: | ||
if: github.event.pull_request.merged == true | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Cancel Previous Runs | ||
uses: styfle/[email protected] | ||
with: | ||
workflow_id: all | ||
access_token: ${{ secrets.GITHUB_TOKEN }} | ||
ignore_sha: true | ||
pr_number: ${{ github.event.pull_request.number }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
name: MoE Test | ||
|
||
on: | ||
push: | ||
branches: [ main ] | ||
paths: | ||
- "python/sglang/**" | ||
- "test/**" | ||
pull_request: | ||
branches: [ main ] | ||
paths: | ||
- "python/sglang/**" | ||
- "test/**" | ||
workflow_dispatch: | ||
|
||
concurrency: | ||
group: moe-test-${{ github.ref }} | ||
cancel-in-progress: true | ||
|
||
jobs: | ||
moe-test: | ||
if: github.repository == 'sgl-project/sglang' || github.event_name == 'pull_request' | ||
runs-on: 2-gpu-runner | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Install dependencies | ||
run: | | ||
pip install --upgrade pip | ||
pip install -e "python[all]" | ||
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall | ||
- name: Benchmark MoE Serving Throughput | ||
timeout-minutes: 10 | ||
run: | | ||
cd test/srt | ||
python3 -m unittest test_moe_serving_throughput.TestServingThroughput.test_default | ||
- name: Benchmark MoE Serving Throughput (w/o RadixAttention) | ||
timeout-minutes: 10 | ||
run: | | ||
cd test/srt | ||
python3 -m unittest test_moe_serving_throughput.TestServingThroughput.test_default_without_radix_cache |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.