You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What problem does your feature solve?
A method to do analysis of annotations (namely for the application of looking at differences between gold and predicted annotations).
Describe the solution you'd like
The Annotation class should be given some static methods like Annotation.diff(ann_object_1, ann_object_2) will output the difference between to annotation objects. Maybe some parameter for leniency to deal with fuzzy annotation matching.
Interface sklearn to compute various evaluation metrics between two annotation files (assuming one is gold and one is predicted).
Additional context
This would be very useful for result analysis and guiding the building of pipelines.
The text was updated successfully, but these errors were encountered:
Currently pull request #68 begins preliminary work mentioned above.
Ideas for further improvements:
Method in Dataset that will allow to compare gold and predicted over a whole corpus by utilizing the diff functionality implemented in Added Annotations.diff(); unit tests #68 .
Give the diff method optional fuzzy parameters that will highlight model predictions that are almost correct (maybe off by a few characters)
What problem does your feature solve?
A method to do analysis of annotations (namely for the application of looking at differences between gold and predicted annotations).
Describe the solution you'd like
The Annotation class should be given some static methods like
Annotation.diff(ann_object_1, ann_object_2)
will output the difference between to annotation objects. Maybe some parameter for leniency to deal with fuzzy annotation matching.Interface sklearn to compute various evaluation metrics between two annotation files (assuming one is gold and one is predicted).
Additional context
This would be very useful for result analysis and guiding the building of pipelines.
The text was updated successfully, but these errors were encountered: