Can refinement be used with any ASR?

[entrider]

Can .refine() be used in this example?

result = stable_whisper.transcribe_any(inference, './demo.wav', vad=True)
print(result.to_srt_vtt(word_level=False))

P.S. I use Whisper API.

[jianfch]

Refinement is only supported on models loaded by stable_whisper.load_model().
But you can use it to refine results from stable_whisper.transcribe_any().

[entrider]

What's wrong?

result = stable_whisper.transcribe_any(inference, './demo.wav', vad=True)
model = stable_whisper.load_model('base')
model.refine('./demo.wav', result)

NotImplementedError: The are missing tokens in the result. Refinement currently only supports results produced by ``transcribe()`` on vanilla Whisper models.

[jianfch]

If the Whisper API does not return any tokens, you can manually add them to the result.

from stable_whisper.whisper_compatibility import get_tokenizer

result = stable_whisper.transcribe_any(inference, './demo.wav', vad=True)
model = stable_whisper.load_model('base')
tokenizer = get_tokenizer(model=model)
for word in result.all_words():
    word.tokens = tokenizer.encode(word.word)
model.refine('./demo.wav', result)

Note: the result must have word timestamps to begin with.