Help! Want a toy example to run matmul with q40 weight by cuda kernel

[Eutenacity]

Sorry, i am not familiar with the library, I want to run a matmul between a tensor created by pytorch and the q40 weight read from gguf.
I can read the weight from gguf and convert it to pytorch tensor.
But I have no idea to run the matmul between a tensor created by pytorch and the q40 weight by cuda kernel.

[slaren]

Some resources:
https://huggingface.co/blog/introduction-to-ggml
https://github.com/ggerganov/ggml/blob/master/examples/simple/simple-backend.cpp

[Eutenacity]

Is it possible to turn ggml_tensor to pytorch tensor on gpu without copy?