Shared memory usage

[dancixx]

I tried to use Phi3v on a Mac, and it worked slowly but without CUDA. However, if I switched to CUDA RTX 4070 12GB, I always got an "out of memory" error. I tried the same with the Huggingface transformers python lib using a larger model and it seems that it can handle loading the model partially into the GPU and Memory. I am still not as experienced in LLMs as I would, but maybe I should add some extra settings to the example code, or mistral.rs cannot support this right now?