Training / Tuning the model

[paul-mdt]

Hello !
Amazing work, I was wondering if it was possible to continue training the model on a different dataset (for domain transfer for example)?

[mranzinger]

Yep, it typically works really well too.

[paul-mdt]

Thanks ! What would you recommend ? Is there a way to train it in a self-supervised way without too much ressources ? Or should it only be fine-tuned on downstream tasks ?

[mranzinger]

I think I might be a bit confused by your question. The resource demand would likely depend on what application you have in mind, aside from you being able to now choose between a ViT-B, L, or H variant, which would affect resources. Could you elaborate on what you're wanting to accomplish?