Modifying models with pretrained weights ?

[vikashg]

Hi all,
I have been thinking about a use case. I will explain everything with regards to Inception V3 but this might apply to other models as well.

As we know that many such models use transfer learning and we need to use pre-trained weights. Now specifically in case of Inception-V3 if we load the architecture without the aux_logits, it is not possible to load the pretrained weights. But if we use the aux_logits we run into errors from the previous discussion.
The other option is to use the full model and load the pre-trained weights into it and then turn off the branch of the network that we dont need as we do here. An example code snippet is here

from torchvision import models
from torchviz import make_dot
from monai.networks.nets import DenseNet121
import torch.nn as nn
import torch
class ModelDefinition():
	def __init__(self, num_class: int, pretrained_flag=True, dropout_ratio=0.5, fc_nodes=1024):
		"""
		Model Definition
		:param num_class:
		:param pretrained_flag:
		:param dropout_ratio:
		:param fc_nodes:
		"""
		self.num_class = num_class
		self.pretrain_flag = pretrained_flag
		self.dropout_ratio=dropout_ratio
		self.fc_nodes = fc_nodes

	def inception_v3(self, device='cpu'):
		model = models.inception_v3(pretrained=self.pretrain_flag)
		model.fc = nn.Sequential(nn.Linear(model.fc.in_features, self.num_class))
		model.aux_logits = False
		model.to(device)
		return model

However the torchvisionFC in monai does not allow me to do that once we load the pretrained weights. We cannot do something like model.aux_logits=False. Even if I were able to do it, I will need to create a custom function where the loaded model needs to be further processed before actually starting training.

So, do you think that it is wise to try to do all these things in a monai bundle or we can just train as we normally do and use monai bundle for inference. In other words, would you say if there is a "complicated" model train it using the regular pipeline and use monai bundle just for inference.

I would like to know your thoughts

Thanks so much for all the help in the last few days.
Vikash

[yiheng-wang-nv]

Hi @vikashg , sorry for the late reply.
Only adding inference script is fine, for example, this bundle only contains the inference script.

[vikashg]

Hi @yiheng-wang-nv, Sorry I have been out of loop because of MICCAI and IHE Connectathon over the last 2 weeks. Yes the bundle is ready and I will send the pull request soon

Thanks
vikash

[yiheng-wang-nv]

Sounds good! Thanks @vikashg !