Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MetadataAnnotations class for user-facing handling of annotations #503

Merged
merged 10 commits into from
Aug 26, 2024

Conversation

kbolashev
Copy link
Member

Scope of this PR:

  • Add a new dependency: https://github.com/DagsHub/dagshub-annotation-converter/
  • Plus additionally add ultralytics into dev dependencies
  • Add MetadataAnnotations class. Objects of this class will get created for any annotation field in the QueryResult, so you can use them like this:
dp = ds.all()[0]
anns: MetadataAnnotations = dp["exported_annotations"]
anns.add_image_bbox("person", 0.1, 0.1, 0.1, 0.1)
anns.add_image_bbox("cat", 0.2, 0.2, 0.1, 0.1)
dp.save()
  • Autoload annotations on ds.all() and ds.head()
  • Add an ability to turn off this autoload functionality as load_documents and load_annotations kwargs in these functions

@kbolashev kbolashev added the enhancement New feature or request label Jul 18, 2024
@kbolashev kbolashev self-assigned this Jul 18, 2024
Copy link

dagshub bot commented Jul 18, 2024

Copy link
Contributor

@simonlsk simonlsk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All very cool features, added some comments and questions.
Thanks!

setup.py Outdated
Comment on lines 46 to 48
"dagshub-annotation-converter @ "
+ "git+https://github.com/DagsHub/"
+ "dagshub-annotation-converter@refactor/static-annotations#egg=dagshub-annotation-converter",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Remember

dagshub/py.typed Outdated Show resolved Hide resolved
dagshub/data_engine/model/query_result.py Show resolved Hide resolved
Comment on lines 437 to 439
if "load_into_memory" in kwargs:
del kwargs["load_into_memory"]
self.get_blob_fields(*self.annotation_fields, load_into_memory=True, **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit is unclear to me. You discard load_into_memory and force it to be true. Is there a common scenario where you pass load_into_memory=False explicitly to get_annotations, and expect this to happen?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, however I pass the kwargs to the downloading functions, and loading annotations requires them to be already in memory, so load_into_memory=False will break things.
Actually I should probably handle this particular edge case better in get_blob_fields and either skip loading if load_into_memory=False (explicitly respecting the user, even if it doesn't make sense), or force-opening.

dagshub/data_engine/model/query_result.py Show resolved Hide resolved
dagshub/data_engine/model/query_result.py Show resolved Hide resolved
dagshub/data_engine/annotation/metadata.py Show resolved Hide resolved
dagshub/data_engine/annotation/metadata.py Outdated Show resolved Hide resolved
@kbolashev kbolashev merged commit b859c2a into master Aug 26, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants