Skip to content
Test SHARK

[WIP][Llama2] Add KVCache for prefill stage + interactive chat mode in llm_runner + StreamingLLM. #14

Sign in to view logs

[WIP][Llama2] Add KVCache for prefill stage + interactive chat mode in llm_runner + StreamingLLM.

[WIP][Llama2] Add KVCache for prefill stage + interactive chat mode in llm_runner + StreamingLLM. #14

Run time

Learn about OS pricing on GitHub Actions
Job Run time
test-shark (3.11, nodai-ubuntu-builder-large)
9m 36s
9m 36s