Question about WebNLG (v3.0) Test Data Size Discrepancy

[Qing1Zhong]

Thank you for sharing your work. I have a question regarding the use of the WebNLG (v3.0) dataset in your work. In your paper, it is mentioned that the test data contains 1,165 text-triplet pairs. However, when I checked the official WebNLG repository in the file semantic-parsing-test-data-without-refs-en.txt, I found 2,155 text entries.

Could you please clarify the reason for this difference? Did you apply any specific filtering or processing to the dataset that resulted in the smaller test set? Thank you for your time and help.

[bzhangj13zzz]

Thank you for raising the issue and sorry for the late reply.

The 2155-entry file as you mentioned, if I remember correctly, is the split without references. So I was not able to evaluate with that set. The one with references should have a size of 1165. I can double check it later.

[Qing1Zhong]

Thank you for your reply @bzhangj13zzz , but I still don't understand your explanation. I checked both files (semantic-parsing-test-data-with-refs-en.xml and semantic-parsing-test-data-without-refs-en.xml), and I found 2155 entries in both files. Could you please confirm if there might be other factors contributing to this discrepancy? Or could you verify whether there are different versions of the test set? Thank you for your assistance!