Finetuning blogs, experience, etc.?

[jooray]

Hi,

I am playing with finetune on MacBook M1 (although finetune does not seem to use GPU, is it planned?).

I generated prompt formatted dataset with simple script.

I have a few questions:

• What are the best models to finetune? I've been playing with Mistral based models (Synthia-7b and Zephyr) and now training with NousHermes-LLaMA2-13B.
• Llama.cpp says finetuning quantized models is not recommended, but several research papers say it should be OK. My results are not very satisfactory though
• Are there any good blogs/tutorials on finetuning llama-based models with llama.cpp? There's a lot with pytorch, peft, transformers, axolotl, I wonder if anyone had success with llama.cpp.
• I have a question on the output:

train_opt_callback: iter= 11 sample=45/2308 sched=0.110000 loss=13.552320 dt=00:01:37 eta=00:31:00 |--------------->

(What does the "--->" mean? Where does it end? is it a progressbar to infinity? :)

It says iter= 11, sample 45/2308 (2308 is my sample size, which is OK). It says it will end in 31 minutes, which it does, but it only processes a few of the samples. Should I rerun the command to continue finetuning? What's the stopping criteria? I would to process all samples during finetuning, but it seems it has some preset iterations.

Should I increase --adam-iter? Should I train more than one time?

Should I lower the learning rate? (--adam-alpha ?) Seems quite large compared to other howtos.

Any discussion forum/telegram/discord/... where we could chat about this and collectively improve? Or is this a good place? I can help with dataset generation, I have done some experiments.

[Green-Sky]

you can checkout the lengthy discussions in the related PRs (eg #2632)
@PossiblyAnEngineer also made a guide here https://rentry.org/cpu-lora , but no guaranties that it is up-to-date

[Green-Sky]

the metrics must be out-of-date, since some optimizations where merged since.

[xaedes]

Llama.cpp says finetuning quantized models is not recommended, but several research papers say it should be OK. My results are not very satisfactory though

llama.cpp (i.e. main) will applying a LoRa to a quantized model in such a way that the resulting model is also quantized. This results in small differences to the lora trained model as optimized during training.

What does the "--->" mean? Where does it end? is it a progressbar to infinity?

It is a cheap progress bar with length proportional to the loss improvement over the first loss it encountered during this training run.
I found it easier to glance at than actually reading the loss numbers^^

What's the stopping criteria? Should I increase --adam-iter? Should I train more than one time?

Training stops after number of iterations (--adam-iter N) is reached or the number of epochs (--epochs N) is reached. Whichever happens first.
Processing all samples once is one epoch.
So just increase --adam-iter, for example to the number of samples, and specify --epochs 1 then it will run until all samples are processed once.
Alternatively you can also run the finetune call multiple times until all samples are processed: the training can resume from a training checkpoint (--checkpoint-in FN). I often make use of this option, to inspect intermediate results, for example by running inference to see what will actually be generated at current state of training.

[pacozaa]

@xaedes Do you have any short guide or hands on example how to fine tune with llama.cpp with cpu? I saw you are the one that pushed PR. Maybe short guide could help get the words out.

[pacozaa]

@jooray How is your fine-tuning with llama.cpp so far? I am looking for a guide on this topic as well.

[jooray]

I switched to axolotl for now

[pacozaa]

@jooray I am going for unsloth. Maybe let's compare note.