Train with multiple candidate output token?

[fzyzcjy]

Hi thanks for the library! It would be great if a output position can have multiple candidate output token. For example, at position 10, the target distribution is "token A should have 70% probability and token B be 30%".

[yukiarimo]

Isn’t it just a DPO?

[fzyzcjy]

I guess maybe no, b/c maybe the target distribution is "A has 50%, B has 30%, C has 20%"