Skip to content

Commit

Permalink
LanguageModel: Disable top-p sampling by default
Browse files Browse the repository at this point in the history
This matches upstream llama2.c, and prevents a confusing message with the basic example, which specifies a temperature (thus disabling the default top-p sampling).
  • Loading branch information
gohai committed Aug 12, 2023
1 parent 1920026 commit a645b2c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/LanguageModel/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ class LanguageModel extends EventEmitter {
tokenizerUrl: '', // if set, tokenizer.bin will be preloaded from provided URL (assumed to be embedded in llama2.data if not)
maxTokens: 0, // how many tokens to generate (defaults to model's maximum)
temperature: 1.0, // 0.0 = (deterministic) argmax sampling, 1.0 = baseline, don't set higher
topp: 0.9, // p value in top-p (nucleus) sampling, 0 = off
topp: 0, // p value in top-p (nucleus) sampling, 0 = off
stopOnBosOrEos: true, // stop when encountering beginning-of-sequence or end-of-sequence token
};

Expand Down

0 comments on commit a645b2c

Please sign in to comment.