Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

not able to reproduce the passkey retrieval accuracy #195

Open
zhuconv opened this issue Sep 16, 2024 · 4 comments
Open

not able to reproduce the passkey retrieval accuracy #195

zhuconv opened this issue Sep 16, 2024 · 4 comments

Comments

@zhuconv
Copy link

zhuconv commented Sep 16, 2024

I have directly run instructions for python3 passkey_retrival for Llama 7B but get accuracy all zeros.
image

@zhuconv
Copy link
Author

zhuconv commented Sep 16, 2024

btw, i also cannot get high passkey retrieval accuracy when finishing the finetuning of llama 7B with length 8192. generally the accuracies are 0.5.

@zhuconv
Copy link
Author

zhuconv commented Sep 17, 2024

my mistake for python3 passkey_retrival for Llama 7B. the results should be right. but the problem of low accuracy for longlora-llama-7b with length 8192 remains.

@micelvrice
Copy link

my mistake for python3 passkey_retrival for Llama 7B. the results should be right.

hey, what's your mistake? i get the same result as your show

@micelvrice
Copy link

ok i find the mistake too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants