Consider switching to whisper.cpp

[djmaze]

Whisper.cpp is a high-performance alternative to the original implementation. There is also an Android example included. Might make sense to switch to it?

[MichaelMcCulloch]

Amazed I didnt see this before. Thank you.

[djmaze]

Mhh.. Does whisper.cpp really deprecate this? AFAICS there is no voice keyboard application using whisper.cpp yet, and this would be really nice to have.

[MichaelMcCulloch]

Hey that's a good point. Sorry, that was a knee-jerk reaction. I've only just cloned whisper.cpp and I'm looking into integrating it.

[MichaelMcCulloch]

The tiny and base models cant understand my friend say 'Burt Reynolds'. The medium comes close, but it takes a minute and a half, instead of seconds. I would want to use the medium or, ideally large models on the GPU. But the repo intentionally doesn't support GPU, however there's an entry point mentioned in the FAQ.

[MichaelMcCulloch]

Given that whisper.cpp intentionally does not support GPU and was built for CPU with a 'bespoke' 'tensorflow' implementation, it would require more effort on my part to get it working with the GPU, than to modify the existing model to work with the GPU or NNAPI.

Unless someone with expertise in android GPU or NNAPI hardware would be willing to pair on this solution path, I'm going to mark this as wontfix