Automatic segmentation settings

[aloboa]

Where could I find documentation on the settings for Automatic segmentation?
I find little change in the results when I modify the values. Also, no difference if selecting
"with background" or not.

[anwai98]

Hi @aloboa,

Thanks for your interest in micro-sam.

Could you help us with some context of your images and what is your desired task?

In any case, we have some documentation on Annotator Tools and brief explanations on using the automatic segmentation with the 2d, 3d and image series annotator.

[constantinpape]

In addition to the points raised by @anwai98 can you also explain which model you're using?
Also, it would be very helpful if you can share an example screenshot with the image, the segmentation result and the corresponding paramters.

[aloboa]

Context:

We are dealing with SEM images of petrographic thin sections
of volcanic rocks.
We want to:

1. Segment everything in the SEM image to use an object-based approach
   to identify different geologic material materials
   by stacking images of different sources in a similar way as done with multi-spectral imagery.
2. Define an efficient interactive labeling method to label the most important structures.

My question here concerns mainly 1, although the
current results could also be taken as an initial step for interactive labeling.

Example: input DI17_2.tif

Annotator2d
Model: vit_b
Automatic Segmentation Settings
pred_iou_thresh: .88
stability_score_thresh: .95
box_nms_thresh: .70
min_object_size: 100

Results:

• autoseg_01.png

  • with background
    

• autoseg_02.png

  • without background
    

• autoseg_03.png

  • with background
  • min_object_size: 1
    

• autoseg_04.png

  • with background
  • pred_iou_thresh: .10
  • min_object_size: 100
    

The most surprising is the small difference between selecting/unselecting
with_background.

Note that having at least an approximation of the area covered by lighter grey and darker grey
of what has not been segmented in any of these results ("background"), is important for us. Perhaps I can address this using more traditional methods such as an Otsu threshold.

I would like to run a set of tests with different models and different settings,
but I need to understand the settings first.

[anwai98]

Hi @aloboa,

Thanks for sharing the details, I'll try to address the aforementioned discussion point-wise:

1. Segment everything in the SEM image to use an object-based approach to identify different geologic material materials by stacking images of different sources in a similar way as done with multi-spectral imagery.

I tried performing automatic segmentation using the default Segment Anything model (vit_b), and looks like the model has a tendency of segmenting the darker holes on the surface of volcanic rocks (similar results to the ones you posted).

2. Define an efficient interactive labeling method to label the most important structures.

I tried annotating a few structures around with boxes and points, I think both work fine to annotate different types of structures in your images. As a minor mention, I would recommend to focus on segmenting one structure as one object at a time, and continue with the next object for segmentation.

My question here concerns mainly 1, although the current results could also be taken as an initial step for interactive labeling.

I would propose the same, you can take the results from the automatic segmentation as a starting point, and use interactive annotation for labelling all other structures you are interested in.
Once you have labelled all the relevant structures, you can finetune Segment Anything on your data to attain a better approach to segment everything in SEM images using automatic segmentation.

Re: Annotator2d (with different parameters)

You can adjust the parameters as you like by reducing them from the defaults to capture more objects, however remember that it might also lead to capturing unnecessary artefacts / undesired objects.

One question: are you using the tiling option in micro-sam? (if yes, that's fine. If not, I just tried it, and using tiling-based segmentation option does help you capture more relevant objects with the default parameters. Here's a hint: FAQ -> Usage questions -> Point 3 on it's usage)

• The most surprising is the small difference between selecting/unselecting with_background.

  Hmm, I tried it with vit_b model with the default parameters, and I do see the issue you mentioned (a bit suspicious). However, once I reduce the stabillity_score_thresh to 0.90 and leave other parameters to default, I observe the difference between using with_background. For now, I would say you can leave the with_background parameter as selected (as it does not remove your desired objects) either ways.

• Note that having at least an approximation of the area covered by lighter grey and darker grey of what has not been segmented in any of these results ("background"), is important for us. Perhaps I can address this using more traditional methods such as an Otsu threshold.

  I would say traditional thresholding methods would do a better job in segmenting abstract sparsely distributed regions in a specific contrast range (say separating the different gray tone regions distinctly, as you mentioned) (please correct me if I missed the point).

Hope this answers your questions!

[aloboa]

As you say, reducing the value of stabillity_score_thresh makes it possible to observe differences between with and without background, but the results are the inverse of what I expected:

with_background not selected

with_background selected

One question: are you using the tiling option in micro-sam?

I can't seem to find it, where is this option?

Before I start to run more tests, is there any place where I can read the definitions of the settings?

[anwai98]

Hi @aloboa,

Apologies for not being clear about the feature. When you select / add check to the with_background feature, this means that you are choosing to remove the large background area, and vice-versa while not selected / removed check (i.e. you are choosing to keep the large background area). There is a tooltip to most of the features, you will see the message pop-up when you hover over the menu options.

I can't seem to find it, where is this option?

This feature can be activated using either the napari GUI or the command line. Here's the hint to guide you better for using the tiling-based feature: FAQ -> Usage questions -> Point 3.

Before I start to run more tests, is there any place where I can read the definitions of the settings?

Most settings' menu options have a tooltip (i.e. when you hover over the menu option, you will find a brief description). If you want more detailed information on some specific features, let us know.

[constantinpape]

We have added more documentation on this and there are tooltips for these settings.

[aloboa]

We have added more documentation on this and there are tooltips for these settings.

Excellent, thanks. Could you post the link to the definitions of

• pred_iou_thresh
• stability_score_thresh
• box_nms_thresh
• min_object_size
• with_background

Also, I think that a glossary would be most useful for users. In particular the difference between AMG and AIS.

[anwai98]

Hi @aloboa,

We plan to describe this in detail (there is a separate issue open for this: #791). I recommend following it for details on this, coming soon (we do plan to document all relevant parameters as much as we can).