Model Inferer and postprocessing

[vikashg]

Hi @Nic-Ma and @yiheng-wang-nv,
I have a question regarding the model infererer and I will post the relevant code below for context. So, in general when we are using a SimpleInferer the output is a tensor and not a dictionary as it is the case in SupervisedTrainer. My understanding is as follows, the data should flow from
SimpleInferer ---> Postprocessing

Howeever the post processing section is a list of transforms generally starts with Activationsd which requires a "pred" key. If I pass the output of inferer to postprocessing I get the following error

  File "/raid/Vikash/Tools/MONAI-DEPLOY/model-zoo/models/breast-model/scripts/script.py", line 36, in <module>
    x = postprocessing(a)
  File "/raid/Vikash/Tools/MONAI-DEPLOY/MONAI/monai/transforms/compose.py", line 173, in __call__
    input_ = apply_transform(_transform, input_, self.map_items, self.unpack_items, self.log_stats)
  File "/raid/Vikash/Tools/MONAI-DEPLOY/MONAI/monai/transforms/transform.py", line 113, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.post.dictionary.Activationsd object at 0x7f142ff00100>

The relevant code is

inferer = parser.get_parsed_content("inferer")
preprocessing = parser.get_parsed_content("preprocessing")
postprocessing = parser.get_parsed_content("postprocessing")

for _item in dataloader:
   img = _item['image']
   a = inferer(inputs=img, network=network)
   print(type(a))
   x = postprocessing(a)
   # print(a.shape)

TLDR: The output of the infererer is a torch but the post processor needs a dictionary.

[yiheng-wang-nv]

Hi @vikashg , in this case, could you please use Activations rather than Activationsd transform? In MONAI, we have array based transforms and dictionary based transforms, and here is an example of using array based transforms for training:

https://github.com/Project-MONAI/tutorials/blob/main/3d_segmentation/torch/unet_training_array.py#L81

[vikashg]

Yes I can but I think there is some issue with that. Because I believe that the post processing section between SupervisedTrainer and SimpleInferer should be common, the output of SupervisedTrainer is a dictionary so the postprocessing works. In addition, I am following the spleen ct segmentation example In that example people use only one postprocessing field with Activationsd So, I am not able to resolve this discrepancy.
Attaching the config file here for reference.
train2.txt

[Nic-Ma]

Hi @vikashg ,

I think @yiheng-wang-nv maybe misunderstood your question, the confusion thing you have is the postprocessing logic.
Actually, it doesn't perform on the output of SimpleInferer, it performs on the output of SupervisedTrainer, which is a dict:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/engines/workflow.py#L217
It's registered to the trainer as a handler for the event MODEL_COMPLETED.
That's why we apply dict based transforms.

Thanks.

[Nic-Ma]

Hi @vikashg ,

Most of the inference config in the model-zoo use SupervisedEvaluator, which is very similar to the SupervisedTrainer, the output is also a dict:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/engines/evaluator.py#L297

If you still have some issue, let's have a short meeting.

Thanks.

[vikashg]

Thanks @Nic-Ma,
So looking at the code again. Because I was testing things on each function, I was getting the error. So here is what my understanding is the SimpleInferer only outputs a tensor but this inferer when encapsulated inside a SupervisedEvaluator it outputs a dictionary and that is what passed into the loss where it can find the keys.
That is my understanding looking at the code and everything.

I believe that keys like pred and label are not hard coded. So suppose I have multiple predictions, with keys like pred or some random key for my prediction like pred234. What is the right way to establish under what key name I am storing the prediction values ?

Sorry for all the questions but I really am trying to understand all the awesome work the team has put in

Thanks

[Nic-Ma]

Hi @vikashg ,

Thanks very much for your valuable feedback.
If you have customized logic for infererer and prediction, you can easily write a callback function to do your iteration logic and set to the evaluator or trainer:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/engines/evaluator.py#L189

Thanks.

[vikashg]

Awesome.
Thanks @Nic-Ma

[vikashg]

So happy to report that I have a basic code that is working and so most of the things are resolved I just need to clean it up and post it.

Thanks once again for answering all my questions.