The difference between 'vm' and 'prm' in 'reward_model_type'?

[Aurora-slz]

What is the difference between 'vm' and 'prm' in 'reward_model_type'? How should I choose one, for example when initializing prm training data?

Thansk a lot!

[girotte-tao]

I think 'vm' is to evaluate the vm, which is called from eval_vm.py.
While 'prm' is to sample data (generate dataset)

[zhoubiansining]

We only use this PRM implementation for comparison in the research process. You can simply use the vm for initialization, sampling and evaluation.