sftrainer

[thistleknot]

I'm reading that sftrainer can be used to train an llm rather than simply trainer

https://huggingface.co/docs/trl/main/en/sft_trainer

there is also this concept about NEFTune which sounds like masking and/or lora dropout

However, I did also want to ask, does this setup support masked models? I know I was able to specify when manually training using my own code, I could set masked llm for llama.

[winglian]

Axolotl is similar to sfttrainer as it is a wrapper around the hf trainer.

As for NEFTune, ymmv. We've done some experiments using the neft-v3 branch and while.ot performs better than the mt bench score that trl reported, it still performed worse than a basic fine tune.