Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for confidence score to Annotations #16

Open
mathieupichon-bib opened this issue Mar 29, 2023 · 3 comments · May be fixed by #22
Open

Add support for confidence score to Annotations #16

mathieupichon-bib opened this issue Mar 29, 2023 · 3 comments · May be fixed by #22
Labels
enhancement New feature or request

Comments

@mathieupichon-bib
Copy link
Collaborator

mathieupichon-bib commented Mar 29, 2023

Context

Add information around the confidence and threshold for classifications

Solution 1

add flat keys to the Annotation class such as

  • confidence : float
  • threshold : float
  • verdict : bool ?

This option would not specify which part of the annotation (bbox or classification, etc) would be responsible for the values and is unfit for polygon or mask annotation

Solution 2

Major rework of the Annotation and Category classes to have properties with subclasses for each annotation type having its own confidence implementation

ex:

class Annotation:
    # polygon: List[List[float]] --> Polygon property today
    polygon: Polygon # --> the new Polygon implementation

class Polygon:
    points: List[List[float]
    confidence: List[float] # could be included as a third member for each point
    ...

Would propably induce breaking changes

@SullyP
Copy link

SullyP commented Mar 29, 2023

To fine tuning our heuristics we need more information for each detections in our yarrows.

We want to add some metadata for each Annotations for :

  • confidence value
  • bool to known if the detection is filtered by heuristics
  • bool to known if the detection is filtered because we have multiple occurence of it
  • classification level
  • tire side
  • and maybe more in future

@mathieupichon-bib
Copy link
Collaborator Author

To fine tuning our heuristics we need more information for each detections in our yarrows.

We want to add some metadata for each Annotations for :

* confidence value

* bool to known if the detection is filtered by heuristics

* bool to known if the detection is filtered because we have multiple occurence of it

* classification level

* tire side

* and maybe more in future
  • confidence value --> it will be a float
  • bool to known if the detection is filtered by heuristics --> this is too use case specific, put it in the meta key of annotation
  • bool to known if the detection is filtered because we have multiple occurence of it --> this is too use case specific, put it in the meta key of annotation
  • classification level --> this either has to be added as a Category or in the meta key
  • tire side --> very use case specific, should go to either Image meta or Annotation meta

Regarding the suggestion for "bool to know if detection is filtered" I think we can add a threshold key at someplaces to indicate if the detection should generate an "invalid" state of be silenced

@mathieupichon-bib mathieupichon-bib added the enhancement New feature or request label Mar 30, 2023
@mathieupichon-bib mathieupichon-bib linked a pull request Apr 3, 2023 that will close this issue
@SullyP
Copy link

SullyP commented Apr 18, 2023

Okey for your solution but Annotation meta doesn't exist yet ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants