Make Whisper to Recognise Names

[felipehertzer]

Hello, I would like to inquire whether it is possible to make whisper to transcribe names correctly, some sort of vocabulary input, or other technique. Thanks.

I'm using Hugging Face model.

[jianfch]

Lets say the model transcribes "Emily" but you want it to be 'Emilie', you can prompt it with 'Emilie':

model = stable_whisper.load_hf_whisper('base')
prompt_ids = model._pipe.tokenizer.get_prompt_ids('Emilie', return_tensors="pt").to(model._pipe.model.device)
result = model.transcribe('audio.wav', prompt_ids=prompt_ids)

If the prompt shows up at the beginning of the transcription (huggingface/transformers#27836), updating transformers should fix it.

[felipehertzer]

Thank you, this is exactly what I was looking for. However, I was unable to implement it successfully with distil-whisper/distil-large-v3.