Loss increases during second fine-tuning phase

[jackatls]

This is the result after my first fine-tuning.

This is the second fine-tuning, where the checkpoint saved after the first fine-tuning was loaded. The loss starts lower than the loss value from the first fine-tuning but gradually increases.

Could it be an issue with the loading process, such as certain optimizer states not being properly restored? I load the saved checkpoint by
trainer.train(checkpoint=lastCheckpointDir)

I would greatly appreciate your help!