How do I enable evaluation for DPO?

[PhilipMay]

I do DPO but Axolotl never evaluates the model.
With SFT it works. DPO -> no evaluation.

Most important parts of my config are below.
Can someone please help me?

Or is evaluation simply not available for DPO?

Many thanks
Philip

rl: dpo
datasets:
  - path: argilla/ultrafeedback-binarized-preferences
    split: train
    type: chatml.argilla

val_set_size: 0.05

sequence_len: 2048
sample_packing: false
pad_to_sequence_len: true

gradient_accumulation_steps: 2
micro_batch_size: 1
num_epochs: 2
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0000002

train_on_inputs: false
group_by_length: false

gradient_checkpointing: true
logging_steps: 1

warmup_steps: 200
eval_steps: 500
saves_per_epoch: 1

[NanoCode012]

I just took a look at the source code. If I understand it correctly, you would need to use the test_datasets parameter.

https://github.com/OpenAccess-AI-Collective/axolotl/blob/a3595793712000921a94440e8272723a733d6f57/src/axolotl/utils/data.py#L960-L965