How to solve CUDA error: out of memory while doing inference for my diarization model

[Ataullha]

ERROR - An error occurred: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

I'm using a 12GB NVIDIA GeForce RTX 2050 with Cuda compilation tools, release 11.8

How to Solve this or how to use batching/ batch_size while doing inference?

[Ataullha]

@kamilakesbi @sanchit-gandhi