CUDA out of memory error on an 11 GB GPU: Any easy ways to use multiple GPUs?

[yashjakhotiya]

Hi,

When running experiments.evaluate on ROME on gpt2-xl, I get an OOM after 4 cases on an 11 GB RTX 2080 Ti GPU. Given that each case runs sequentially, is this expected? If it is, do you think there are any trivial ways to extend evaluation on multiple GPUs?

Thanks and great work!