You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I'm trying to do supervised learning given labeled data of an input token sequence mapping to a probability output. Does Nemo support a simple way to do this? For example, via soft targets (predicting one of two tokens each with a certain probability that sums to 1) as opposed to 1-hot vectors, or otherwise doing regression for a continuous value?
Describe the solution you'd like
Some modification to standard SFT training for example.
Describe alternatives you've considered
Have tried looking at distillation as a proxy, where the probabilities imitate the logits produced by a teacher model. But this is quite cumbersome.
Additional context
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
I'm trying to do supervised learning given labeled data of an input token sequence mapping to a probability output. Does Nemo support a simple way to do this? For example, via soft targets (predicting one of two tokens each with a certain probability that sums to 1) as opposed to 1-hot vectors, or otherwise doing regression for a continuous value?
Describe the solution you'd like
Some modification to standard SFT training for example.
Describe alternatives you've considered
Have tried looking at distillation as a proxy, where the probabilities imitate the logits produced by a teacher model. But this is quite cumbersome.
Additional context
The text was updated successfully, but these errors were encountered: