Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add missing logit index increment in sampling metadata prep #1059

Merged
merged 1 commit into from
Dec 30, 2024

Conversation

AlpinDale
Copy link
Member

This little oversight was causing output quality to tank.

aphrodite run NousResearch/Meta-Llama-3.1-8B-Instruct --max-model-len 2048 -tp 2 --enable-chunked-prefill --enable-prefix-caching --max-seq-len-to-capture 2048
lm_eval --model local-completions --tasks gsm8k --model_args model="NousResearch/Meta-Llama-3.1-8B-Instruct",base_url=http://localhost:2242/v1/completions,num_concurrent=256,max_retries=3,tokenized_requests=False

Main:

Tasks Version Filter n-shot Metric Value Stderr
gsm8k 3 flexible-extract 5 exact_match 0.0099 ± 0.0027
strict-match 5 exact_match 0.0000 ± 0.0000

PR:

Tasks Version Filter n-shot Metric Value Stderr
gsm8k 3 flexible-extract 5 exact_match 0.7748 ± 0.0115
strict-match 5 exact_match 0.7483 ± 0.0120

@AlpinDale AlpinDale merged commit 0b5588d into main Dec 30, 2024
5 checks passed
@AlpinDale AlpinDale deleted the sampling_metadata_fix branch December 30, 2024 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant