Any minimum GPU requirments to run with cublas on?

[klosax]

I successfully compiled main (llama.cpp-master-54bb60e) with make LLAMA_CUBLAS=1, but running it exits with the error:

...
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size =  59.11 KB
llama_model_load_internal: mem required  = 5809.32 MB (+ 1026.00 MB per state)
cuBLAS error 1 at ggml-cuda.cu:219

Using cuda tools 12.1 and gcc 9.4.0.

Are there any minimum GPU requirement to get this to work?
Tried using GTX 1660 with 6 GB VRAM.

[slaren]

It seems that cublasCreate is failing with CUBLAS_STATUS_NOT_INITIALIZED. From what I found, this may happen if the GPU is out of memory or some hardware error. I would check that there aren't any other programs using GPU memory and that CUDA is working properly in your system.

[klosax]

Turned out that there was an error in the gpu driver installation.

Test result: The numbers for "prompt eval time" are cut to a third with cublas enabled, but the "eval time" stays the same.