Update 2024-10-26 04:42:14

sgl-project · Oct 26, 2024 · 727d88e · 727d88e
1 parent ff2d44a
commit 727d88e
Show file tree

Hide file tree

Showing 19 changed files with 39 additions and 39 deletions.
diff --git a/README.html b/README.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/README.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/README.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/README.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/README.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/_sources/troubleshooting.md b/_sources/troubleshooting.md
@@ -5,9 +5,9 @@ This page lists some common errors and tips for fixing them.
 ## CUDA error: an illegal memory access was encountered
 This error may be due to kernel errors or out-of-memory issues.
 - If it is a kernel error, it is not easy to fix.
-- If it is out-of-memory, sometimes it will report this error instead of "Out-of-memory." In this case, try setting a smaller value for `--mem-fraction-static`. The default value of `--mem-fraction-static` is around 0.8 - 0.9. https://github.com/sgl-project/sglang/blob/1edd4e07d6ad52f4f63e7f6beaa5987c1e1cf621/python/sglang/srt/server_args.py#L92-L102
+- If it is out-of-memory, sometimes it will report this error instead of "Out-of-memory." In this case, try setting a smaller value for `--mem-fraction-static`. The default value of `--mem-fraction-static` is around 0.8 - 0.9.
 
 ## The server hangs
 If the server hangs, try disabling some optimizations when launching the server.
 - Add `--disable-cuda-graph`.
-- Add `--disable-flashinfer-sampling`.
+- Add `--sampling-backend pytorch`.
diff --git a/backend.html b/backend.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/backend.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/backend.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/backend.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/backend.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/benchmark_and_profiling.html b/benchmark_and_profiling.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/benchmark_and_profiling.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/benchmark_and_profiling.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/benchmark_and_profiling.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/benchmark_and_profiling.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/choices_methods.html b/choices_methods.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/choices_methods.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/choices_methods.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/choices_methods.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/choices_methods.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/contributor_guide.html b/contributor_guide.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/contributor_guide.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/contributor_guide.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/contributor_guide.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/contributor_guide.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/custom_chat_template.html b/custom_chat_template.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/custom_chat_template.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/custom_chat_template.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/custom_chat_template.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/custom_chat_template.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/embedding_model.html b/embedding_model.html
@@ -256,7 +256,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/embedding_model.ipynb?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/embedding_model.ipynb?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -273,7 +273,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/embedding_model.ipynb" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/embedding_model.ipynb" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/frontend.html b/frontend.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/frontend.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/frontend.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/frontend.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/frontend.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/hyperparameter_tuning.html b/hyperparameter_tuning.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/hyperparameter_tuning.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/hyperparameter_tuning.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/hyperparameter_tuning.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/hyperparameter_tuning.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/index.html b/index.html
@@ -254,7 +254,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/index.rst?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/index.rst?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -271,7 +271,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/index.rst" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/index.rst" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/install.html b/install.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/install.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/install.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/install.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/install.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/model_support.html b/model_support.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/model_support.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/model_support.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/model_support.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/model_support.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/release_process.html b/release_process.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/release_process.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/release_process.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/release_process.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/release_process.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/sampling_params.html b/sampling_params.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/sampling_params.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/sampling_params.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/sampling_params.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/sampling_params.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/searchindex.js b/searchindex.js
diff --git a/send_request.html b/send_request.html
@@ -256,7 +256,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/send_request.ipynb?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/send_request.ipynb?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -273,7 +273,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/send_request.ipynb" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/send_request.ipynb" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/setup_github_runner.html b/setup_github_runner.html
@@ -253,7 +253,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/setup_github_runner.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/setup_github_runner.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -270,7 +270,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/setup_github_runner.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/setup_github_runner.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"

diff --git a/troubleshooting.html b/troubleshooting.html
@@ -252,7 +252,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/blob/main/docs/en/troubleshooting.md?plain=1" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/blob/main/troubleshooting.md?plain=1" target="_blank"
    class="btn btn-sm btn-source-file-button dropdown-item"
    title="Show source"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -269,7 +269,7 @@
 
 
 
-      <li><a href="https://github.com/sgl-project/sglang/edit/main/docs/en/troubleshooting.md" target="_blank"
+      <li><a href="https://github.com/sgl-project/sglang/edit/main/troubleshooting.md" target="_blank"
    class="btn btn-sm btn-source-edit-button dropdown-item"
    title="Suggest edit"
    data-bs-placement="left" data-bs-toggle="tooltip"
@@ -431,15 +431,15 @@ <h2>CUDA error: an illegal memory access was encountered<a class="headerlink" hr
 <p>This error may be due to kernel errors or out-of-memory issues.</p>
 <ul class="simple">
 <li><p>If it is a kernel error, it is not easy to fix.</p></li>
-<li><p>If it is out-of-memory, sometimes it will report this error instead of “Out-of-memory.” In this case, try setting a smaller value for <code class="docutils literal notranslate"><span class="pre">--mem-fraction-static</span></code>. The default value of <code class="docutils literal notranslate"><span class="pre">--mem-fraction-static</span></code> is around 0.8 - 0.9. https://github.com/sgl-project/sglang/blob/1edd4e07d6ad52f4f63e7f6beaa5987c1e1cf621/python/sglang/srt/server_args.py#L92-L102</p></li>
+<li><p>If it is out-of-memory, sometimes it will report this error instead of “Out-of-memory.” In this case, try setting a smaller value for <code class="docutils literal notranslate"><span class="pre">--mem-fraction-static</span></code>. The default value of <code class="docutils literal notranslate"><span class="pre">--mem-fraction-static</span></code> is around 0.8 - 0.9.</p></li>
 </ul>
 </section>
 <section id="the-server-hangs">
 <h2>The server hangs<a class="headerlink" href="#the-server-hangs" title="Link to this heading">#</a></h2>
 <p>If the server hangs, try disabling some optimizations when launching the server.</p>
 <ul class="simple">
 <li><p>Add <code class="docutils literal notranslate"><span class="pre">--disable-cuda-graph</span></code>.</p></li>
-<li><p>Add <code class="docutils literal notranslate"><span class="pre">--disable-flashinfer-sampling</span></code>.</p></li>
+<li><p>Add <code class="docutils literal notranslate"><span class="pre">--sampling-backend</span> <span class="pre">pytorch</span></code>.</p></li>
 </ul>
 </section>
 </section>