About the result

[dongZheX]

Hello, thanks for the code.
In the paper, you follow the evlataion strategy used in 《How Powerful are Graph Neural Networks?》. In your code, I can't find how you calculate the final result.
Because I find several other papers that use the same split strategy but the method of computing final result is different.
so, could you please tell me that whether you follow the method:
"The cross-validation in our paper only uses training and validation sets (no test set) due to small dataset size. Specifically, after obtaining 10 validation curves corresponding to 10 folds, we first took average of validation curves across the 10 folds (thus, we obtain an averaged validation curve), and then selected a single epoch that achieved the maximum averaged validation accuracy. Finally, the standard devision over the 10 folds was computed at the selected epoch."
to calculate the result reported in your paper.

Looking forward to your reply.

Thanks

[xiaominUMD]

Thanks for the code too! Your codes look decent.

I am also curious about the result. and as dongZheX mentioned "selected a single epoch that achieved the maximum averaged validation accuracy.", I think selecting best result from one epoch may give bias to the final result, as we are tuning the train process based on test data (future unseen data). But this is from 《How Powerful are Graph Neural Networks?》. I am wondering if you are using the same strategy?

Thanks!

[dongZheX]

Hi, I use the strategy in《How Powerful are Graph Neural Networks?》 that adopt by many other papers at present. But I don't think this strategy is correct, the method in 《A Fair Comparison of Graph Neural Networks for Graph Classification》， but i don't have enough time to evalute all methods under this strategy by myself. | | 董喆 | | ***@***.*** | On 3/5/2022 ***@***.***> wrote： Thanks for the code too! Your codes look decent. I am also curious about the result. and as dongZheX mentioned "selected a single epoch that achieved the maximum averaged validation accuracy.", I think selecting best result from one epoch may give bias to the final result, as we are tuning the train process based on test data (future unseen data). But this is from 《How Powerful are Graph Neural Networks?》,which I don't quite agree with. I am wondering if you are using the same strategy? Thanks! — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>