Add GUI for extending 2d instances to 3d?

[constantinpape]

We may consider adding a GUI element for prompting volumetric segmentation with the results of 2d automatic segmentation.

Currently not implemented in the UI but we have the underlying functionality for it. The function segment_mask_in_volume takes a (2D) mask as input and then uses it as prompt for 3D segmentation, so what you want is quite easy to implement (just outlining the idea):

from micro_sam.multi_dimensional_segmentation import segment_mask_in_volume

# assuming we have the array 'segmentation' and unique ids 'seg_ids' 
for seg_id in seg_ids:
  mask = segmentation == seg_id
  mask_3d = segment_mask_in_volume(
    segmentation=mask,
    predictor=predictor,  # this is the sam model
    image_embeddings=image_embeddings,  # these are the precomputed 3d image embeddings - important to make this efficcient
    segmented_slices=[z],  # the slice id for which you ran auto segmentation, i.e. which contains the mask
   stop_lower=False, stop__upper=False,  # these both have to be set to False for this use-case
   iou_threshold=0.5,  # when to stop segmentation depending on IOU of adjacent slices
   projection="single_point",  # different projection options might affect the result, single_point is the simplest and often works well. 
  )
  ...  # do something to keep track of the 3d masks

I am also not quite sure what would be the best way to integrate this into the GUI to avoid more options and complexity for the user.

cc @psobolewskiPhD