CUDA out of memory

[integrum-aiktuck]

你好，我正在使用 T4， 16GB 的 GPU 来做 fine tuning。 但是 一直遇到 CUDA out of memory error. 请问您是用什么 parameters 来做 fine tuning 的呢？多谢！

MICRO_BATCH_SIZE = 4 # this could actually be 5 but i like powers of 2
BATCH_SIZE = 8
MAX_STEPS = None
GRADIENT_ACCUMULATION_STEPS = BATCH_SIZE // MICRO_BATCH_SIZE
EPOCHS = 3 # we don't always need 3 tbh
LEARNING_RATE = 3e-4 # the Karpathy constant
CUTOFF_LEN = 256 # 256 accounts for about 96% of the data
LORA_R = 8
LORA_ALPHA = 16
LORA_DROPOUT = 0.05
VAL_SET_SIZE = args.test_size # 2000
TARGET_MODULES = [
"q_proj",
"v_proj",
]