You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At present Agent-Zero only has the basic 'temperature' sampling parameter and as it cannot be disabled - may mess up the more advanced sampling options configured in the model.
For context - temperature is a very basic sampling option which essentially injects noise (randomness) into part of the prediction algorithm and is very rudimentary and can be problematic for code generation (should pretty much always be set to 0 unless you're using min_p). Initially top_p replaced temperature as the most effective simple way of controlling "creativity" aka randomness, then min_p came out which is a good upgrade from top_p. In other words - we're two evolutions past setting 'temperature' alone.
It would be great if Agent Zero additionally let you set:
min_p
top_p
repetition penalty
and let you unset temperature (as in - don't send it with API requests to Ollama).
Example of a sensible default configuration for a coding model (Qwen 2.5 Coder 32b in this case) that you don't want to be too ridged but want to retain high quality tokens:
### min_p sampling ##
# 1.0 disables top_p, so we can use min_p
PARAMETER top_p 1.0
# min_p works best with a small amount of temperature
PARAMETER temperature 0.2
PARAMETER min_p 0.9
PARAMETER repeat_penalty 1.05
The text was updated successfully, but these errors were encountered:
At present Agent-Zero only has the basic '
temperature
' sampling parameter and as it cannot be disabled - may mess up the more advanced sampling options configured in the model.For context -
temperature
is a very basic sampling option which essentially injects noise (randomness) into part of the prediction algorithm and is very rudimentary and can be problematic for code generation (should pretty much always be set to 0 unless you're using min_p). Initiallytop_p
replacedtemperature
as the most effective simple way of controlling "creativity" aka randomness, thenmin_p
came out which is a good upgrade fromtop_p
. In other words - we're two evolutions past setting 'temperature
' alone.It would be great if Agent Zero additionally let you set:
min_p
top_p
repetition penalty
and let you unset
temperature
(as in - don't send it with API requests to Ollama).Further reading on why
min_p
is important huggingface/transformers#27670Example of a sensible default configuration for a coding model (Qwen 2.5 Coder 32b in this case) that you don't want to be too ridged but want to retain high quality tokens:
The text was updated successfully, but these errors were encountered: