Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: DIA-1523: test coverage for LabelStudioSkill #247

Draft
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

matt-bernstein
Copy link
Contributor

@matt-bernstein matt-bernstein commented Nov 5, 2024

Fuzz testing using a bunch of different label configs and models and making sure predictions are always valid

Uncovered and fixed 2 bugs:

  • NER postprocessing was only done for the first tag in a label config
  • text field failed to generate sometimes for NER entities

@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 5, 2024 17:43 Destroyed
@matt-bernstein
Copy link
Contributor Author

black made a bunch of formatting changes, ignore the ones in existing tests, there's just one big new test at the bottom

@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 5, 2024 22:33 Destroyed
@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 5, 2024 22:58 Destroyed
@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 6, 2024 00:33 Destroyed
@robot-ci-heartex robot-ci-heartex marked this pull request as draft November 6, 2024 11:05
@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 6, 2024 18:51 Destroyed
@matt-bernstein matt-bernstein marked this pull request as ready for review November 6, 2024 18:52
@matt-bernstein matt-bernstein changed the title test coverage for LabelStudioSkill fix: DIA-1523: test coverage for LabelStudioSkill Nov 6, 2024
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.33%. Comparing base (882ca68) to head (64ab9b3).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #247      +/-   ##
==========================================
+ Coverage   65.71%   66.33%   +0.62%     
==========================================
  Files          47       47              
  Lines        2424     2439      +15     
==========================================
+ Hits         1593     1618      +25     
+ Misses        831      821      -10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 6, 2024 18:54 Destroyed
@robot-ci-heartex robot-ci-heartex temporarily deployed to fb-dia-1523-test-coverage November 6, 2024 20:00 Destroyed
input_field_name = ner_tag.objects[0].value.lstrip('$')
output_field_name = ner_tag.name
quote_string_field_name = 'text'
output = extract_indices(pd.concat([input, output], axis=1), input_field_name, output_field_name, quote_string_field_name)
df = pd.concat([input, output], axis=1)
output = validate_output_format_for_ner_tag(df, input_field_name, output_field_name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, now we can take out this additional call to validate_output_format_for_ner_tag, but other than that lgtm 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still need it, I added the call to EntityExtraction.extract_indices not the standalone extract_indices

@robot-ci-heartex robot-ci-heartex marked this pull request as draft November 7, 2024 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants