ETH-UCY results, train/ test/ val sets, best point picked up on the test set?

[abduallahmohamed]

Hi,

Thanks for your work. I was looking into the ETH-UCY notebooks/ code that is provided in the code. What I see that the test set where used to pickup the best point and then the results where reported on the test set. Yet, all prior works - I'm aware of- used the validation set to pickup the best training point and then reported the ADE/FDE metrics on the test set.

I just want to confirm this because the results will change dramatically due this. For example, the ETH scenarios validation set doesn't contain cases that you find it in the test set [ For example a pedestrian doing more than delta 1 meter between the 2.5 fps] , so technically it's much more difficult to generalize from the training set to these situations.

Best,
Abduallah

[zoeyliu1999]

Hi Abduallah,

I am playing with the Ynet code for helping my research and curious about the dramatic results change you mentioned. Like you said that the prior works used the validation set to pick up the best training point. I checked the reference paper such as AgentFormer, Introvert and Trajectron++, but found they all evaluated forecasting with the minADE and adopted a leave-one-out split strategy. Could you kindly remind me of the prior works using validation set? Or could you please provide the code for ETH training and testing?

Thank you very much.
Best,
Zoey