Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lora doesn't work when kv_cache is disabled #2543

Open
4 tasks
ShuaiShao93 opened this issue Dec 5, 2024 · 0 comments
Open
4 tasks

lora doesn't work when kv_cache is disabled #2543

ShuaiShao93 opened this issue Dec 5, 2024 · 0 comments
Assignees
Labels
Investigating Lora/P-tuning triaged Issue has been triaged by maintainers

Comments

@ShuaiShao93
Copy link

System Info

x86_64, debian 11, L4 gpu

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

  1. Install trtllm 0.15.0
  2. Build engine with awq int4 and lora, disable kv cache
  3. Run TensorRT-LLM/examples/run.py by passing lora dir

Expected behavior

Get results

actual behavior

Segmentation fault

additional notes

N/A

@ShuaiShao93 ShuaiShao93 added the bug Something isn't working label Dec 5, 2024
@github-actions github-actions bot added triaged Issue has been triaged by maintainers Investigating labels Dec 6, 2024
@nv-guomingz nv-guomingz added Lora/P-tuning and removed bug Something isn't working triaged Issue has been triaged by maintainers Investigating Lora/P-tuning labels Dec 10, 2024
@github-actions github-actions bot added triaged Issue has been triaged by maintainers Investigating labels Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Investigating Lora/P-tuning triaged Issue has been triaged by maintainers
Projects
None yet
Development

No branches or pull requests

3 participants