Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possibly confusing mistake in test_nlgeval.py documentation #125

Open
NirmalManoj opened this issue Oct 24, 2021 · 1 comment
Open

Possibly confusing mistake in test_nlgeval.py documentation #125

NirmalManoj opened this issue Oct 24, 2021 · 1 comment

Comments

@NirmalManoj
Copy link

In line 37 below, shouldn't it be "this is one reference sentence for sentence2". Instead, it says this is a reference sentence for sentence2 which was generated by your model. If I understand correctly from other documentation available, it is not supposed to be a model-generated sentence but a reference sentence. It would be great if someone could clarify this.

scores = n.compute_metrics(ref_list=[
[
"this is one reference sentence for sentence1",
"this is a reference sentence for sentence2 which was generated by your model"
],
[
"this is one more reference sentence for sentence1",
"this is the second reference sentence for sentence2"
],
],
hyp_list=[
"this is the model generated sentence1 which seems good enough",
"this is sentence2 which has been generated by your model"
]
)

@juharris
Copy link
Member

Oops! good catch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants