Multi-GPU support for inferencing

[philpax]

With #325, we now have GPU acceleration. However, this is limited to one GPU at present. We'll need to mimic the functionality from llama.cpp in order to distribute tensors between models as appropriate.

[jacohend]

Possibly addressed in #419 ?

Still getting some strange assert failures on llama.cpp's end when I try to set cuda device 1 instead of 0.