Segmentation Performance

[cyrilzakka]

I greatly enjoyed reading your paper but I'm curious about the segmentation performance of such models. Do MSNs share the same segmentation properties as DINO?

[MidoAssran]

Hi @cyrilzakka ,

Great question! We didn't check segmentation properties, but just released the pre-trained models so you're welcome to check!

My thoughts:
In general, MSN introduces an additional mask invariance, which helps the model discard a lot of instance-level information and produce more abstract representations. This property is helpful for low-shot semantic abstraction tasks, but I imagine could hurt performance on low-level tasks like segmentation. In short, I would expect performance to be similar to DINO on segmentation, although I would be a little surprised if it was better by any significant margin. Having said this, I have not personally checked and would be curious to learn about your findings if you try this.