Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question on AlignScore, G-Eval, and AMRFact prediction outputs #1

Open
cs329yangzhong opened this issue Apr 5, 2024 · 4 comments
Open

Comments

@cs329yangzhong
Copy link

cs329yangzhong commented Apr 5, 2024

Dear authors,

Congratulations on your paper's acceptance at NAACL! Your work's application of AMR-driven data augmentation is particularly fascinating.

However, in my attempt to replicate your experimental results, I've noticed that my results differ from those you've reported. Could you kindly share the output data for AlignScore, G-Eval (the best baseline according to your report) on AggreFact, and the AMRFact prediction labels for a cross-check?

Thank you very much!

@cs329yangzhong
Copy link
Author

Hi. Do you know if we have any updates on this thread? @ihungalexhsu @derekmma @haoyiq114

@cs329yangzhong
Copy link
Author

Dear authors, could you please clarify for me some specific information about my prior inquiry? Thank you very much for your time and consideration!

@haoyiq114
Copy link
Collaborator

Hi @cs329yangzhong, sorry for replying late. You can check out this Google Drive link for the results: https://drive.google.com/drive/folders/1f58D8dZxJAeCRTuHkOoaC8dRb3V-U1iJ?usp=drive_link. The G-Eval result is already provided in the drive, and we will upload the others very soon. Thanks for your patience!!

@cs329yangzhong
Copy link
Author

Thanks! One follow-up question is that using your G-eval predictions, the balanced accuracy on the Xsum dataset did not match with the paper-reported values. I am using the labels and predicted scores from your g-eval_only.csv file and tested on the test split. Is there something wrong with the uploaded file?
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants