Adjust the temperature to get more stable speech

[normatovjj]

From FAQ:

Bark's generations sometimes differ from my prompts. What's happening?

Bark is a GPT-style model. As such, it may take some creative liberties in its generations, resulting in higher-variance model outputs than traditional text-to-speech approaches.

In fact, in order NOT to get random style each time, we can change the temp in the generation.py file. A few simple changes are necessary so that the end product looks like below: (copy the code below and paste in the beginning of the generaion.py file, these dictionaries serve as params that can be changed quickly whenever you want).

For this to work (!), the functions also need to be modified, see the second part of the code. Changing the temp to lower value allows for less "creative" output and a more stable speech. Try it for yourself. Once the generation.py is modified, you can run the model as normal.

There is also a way to output even more stable output/style using set_seed(). Let me know if someone is interested, I will reply below.

default_temp.mp4

lower_temp.mp4

# Parameter values for generate_coarse
PARAMS_COARSE = {
    'temp': 0.1,  #0.7
    'top_k': None,
    'top_p': None,
    'silent': False,
    'max_coarse_history': 630,
    'sliding_window_len': 60,
    'use_kv_caching': False
}


# Parameter values for generate_text_semantic
PARAMS_TEXT_SEMANTIC = {
    'temp': 0.1,  #0.7
    'top_k': None,
    'top_p': None,
    'silent': False,
    'min_eos_p': 0.2,
    'max_gen_duration_s': None,
    'allow_early_stop': True,
    'use_kv_caching': False
}

# Parameter values for generate_fine
PARAMS_FINE = {
    'temp': 0.1,  #0.5
    'silent': False #True
}

Change functions to read the params:

def generate_text_semantic(
    text,
    history_prompt=None,
    temp=PARAMS_TEXT_SEMANTIC['temp'],
    top_k=PARAMS_TEXT_SEMANTIC['top_k'],
    top_p=PARAMS_TEXT_SEMANTIC['top_p'],
    silent=PARAMS_TEXT_SEMANTIC['silent'],
    min_eos_p=PARAMS_TEXT_SEMANTIC['min_eos_p'],
    max_gen_duration_s=PARAMS_TEXT_SEMANTIC['max_gen_duration_s'],
    allow_early_stop=PARAMS_TEXT_SEMANTIC['allow_early_stop'],
    use_kv_caching=PARAMS_TEXT_SEMANTIC['use_kv_caching']
):

def generate_coarse(
    x_semantic,
    history_prompt=None,
    temp=PARAMS_COARSE['temp'],
    top_k=PARAMS_COARSE['top_k'],
    top_p=PARAMS_COARSE['top_p'],
    silent=PARAMS_COARSE['silent'],
    max_coarse_history=PARAMS_COARSE['max_coarse_history'],
    sliding_window_len=PARAMS_COARSE['sliding_window_len'],
    use_kv_caching=PARAMS_COARSE['use_kv_caching']
):

def generate_fine(
    x_coarse_gen,
    history_prompt=None,
    temp=PARAMS_FINE['temp'],
    silent=PARAMS_FINE['silent']
):

[JonathanFly]

Sometimes lowering the temp like that increases stability, but it's not a general rule. It tends to be voice and prompt-style dependent.

[normatovjj]

Agreed, prompt plays a big role. I wonder why Suno does not let users set temp or seed by default, but very grateful that it's open source and can be changed in the code