Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics #3

Open
MarcosFP97 opened this issue Apr 19, 2021 · 0 comments
Open

Metrics #3

MarcosFP97 opened this issue Apr 19, 2021 · 0 comments

Comments

@MarcosFP97
Copy link

Hi,

First of all, my apologies since I am not sure if this is the appropriate forum to consult this. I am trying to compute some metrics over my rankings using the trec_eval and compatibility measures from TREC 2020 Health Misinfo. However, I have some doubts:
Is there a way to compute the individual NDCG for correctness? Since when generating the derived-qrels it only outputs a file called binary.useful-correct.
Is there a way to compute the combined harmful and helpful compatibility measures, as used in the overview paper? Since the gen-derived-qrels only outputs harmful.only and helpful.only files.
I would appreciate any help.
Best,
Marcos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant