QNN backend

[Francis235]

Hello, everyone. How is llama.cpp's support for qualcomm QNN backend now? Is there any usage tutorials?

[ExtReMLapin]

#9662

Closed PR : #6869

[Francis235]

#9662 is still not merged, have you done any verification? How should I use QNN-backend in llama.cpp?

[FranzKafkaYu]

there is a little hope for QNN backend in llama.cpp,I found that there is a project which you may find useful:https://github.com/UbiquitousLearning/mllm

[bingo787]

mlllm run would codedump on the Qualcomm Soc

[bingo787]

Hello, everyone. How is llama.cpp's support for qualcomm QNN backend now? Is there any usage tutorials?

The call of a single operator is very inefficient and must be optimized from the model to build a static graph on QNN-backend
Maybe you can use qualcomm Genie SDK.

[Francis235]

Thanks for you kindly reply, I am trying both mllm and Genie SDK now.