MONAI_Design_Philosophy

MONAI Design Pages

• Design philosophy - this page
• Design
• Design discussions

Introduction

This page is the place to learn about the MONAI design philosophy. If you are intending to contribute to the MONAI code base, it is important that you get an understanding of the design and design philosophy of MONAI so that we can ensure that MONAI is a success into the future.

Our design philosophy is to build a framework that is, above all other things, unopinionated. This is a central tenet to the philosophy of MONAI, and gives easy access to MONAI for new users coming to the framework from various different directions.

What does 'unopinionated' mean in this context?

We want a design for Monai that closely follows the philosophy of pytorch and ignite, the two deep learning (DL) packages on which Monai is being built. When we use the term 'unopinionated', we mean the following:

1. The design should not apply radically different design decisions to those made by pytorch and ignite, except where there is a very strong reason to do so
2. The design should provide useful extensions to pytorch and ignite such that they should appear to be the natural ways that pytorch / ignite would provide such functionality
3. The fundamental nature of pytorch / ignite should not be hidden from the MONAI user
4. An existing code-base should be extensible using MONAI without requiring a rewrite to conform to the 'MONAI way'

Vanilla and specialised

Everything that we add in terms of layers, networks, applications and so forth should be as if they were written for those frameworks by those teams. This is the vanilla version of all functionality, and all functionality that we add should be present in vanilla form. This way, there is no impedance mismatch to be overcome by someone who wants to dip their toe into MONAI on a project.

Specialised functionality is provided to make a user's life easier by providing useful types, abstractions and capabilities that go above and beyond pytorch / ignite philosophy. All such functionality should be opt-in and, in general, wrap the vanilla functionality to maximise code-reuse. One such example is the creation of preprocessing wrappers that are specialised to given types.

---

Routes to using MONAI

We expect users to arrive at Monai through one of three main routes. Our goal is to make it easy for them to use MONAI whichever way they choose to access it, and for them to be able to gradually adopt use of MONAI without ever facing a sudden, steep learning curve.

Configuration route

Note: at this point in time, the initial release of MONAI will not have a configuration-based route to usage. This decision may change according to feedback *The user is initially using networks through configuration files and not interacting with the python API. Everything is done through modification of modular, intuitive configuration options:

1. The user adopts a 'MONAIRun' configured network as the starting point and makes changes to the configuration file to use with their particular dataset, potentially with a set of pre-trained weights
2. The user eventually wants to tweak the existing application and make new changes that are not reflected in the config file*

Existing network route

The user has written networks in pytorch and wants to make use of MONAI functionality so that they don't have to write that code themselves. They want to use MONAI because it doesn't force them to rewrite their code-base:

1. The user adds some piece of missing functionality. Likely candidates are:
   1. Pre-processing functionality, including use of asynchronous IO
   2. Adaptive learning rate functionality
   3. Network telemetry to their output of choice
   4. Encapsulated trainers / evaluators
2. The user gradually adopts more and more elements of MONAI as they evolve their network sophistication, looking through the set of MONAI applications and reducing their line count as they adopt functionality

New network from scratch

The user wants to write a network from scratch and wishes to use MONAI to do so:

1. The user looks at existing applications and adapts one to their liking, or
2. The user looks at existing applications and writes a new application from the ground up

---

How MONAI adds value

Monai adds value by making simple things easy and hard things possible. It does so in a way that is entirely compatible and in keeping with the design of pytorch itself, the ignite framework that sits on top of pytorch, and other extensions to pytorch such as torchvision. While Monai should always represent a relatively thin, unopinionated wrapper around pytorch / ignite, there are particular areas where Monai can easily add value.

Pre-processing steps

Pre-processing is an important area where both Clara Train and NiftyNet have provided functionality that is conspicuously absent from tensorflow / pytorch. Sophisticated pre-processing pipelines are essential to achieving ultimate model performance and hard to write, especially once we take into account complexities such as extension of augmentation to 3D and 4D datasets, the complexity of implementing non-linear deformations, for example, is often outside the skill set of otherwise skilled deep learning practitioners. The same can be said of using asynchronicity to minimise the overhead of IO completion when reading source data from disk.

Preprocessing is also gateway functionality to a more comprehensive use of the Monai framework by people who have existing pytorch codebases; it is largely additive in its usage. The underlying functionality should be exposed in a completely vanilla fashion, and then wrapped by more sophisticated handlers that result in natural looking sequences of function calls. Under the hood, we can make use of generators to provide caching, asynchrony, and other mechanisms to limit IO latency. We can provide typed wrappers that reduce the noise by using types such as MedicalImage, that bundle together related pieces of information.

Engines

One of ignite's key value propositions is that it provides engines; these are training and evaluation loops with a flexible set of hooks that allow a user to build applications in only a few lines of code, while having the ability to customise the behaviour of the engine through event handlers. These are accessed by natural looking factory functions, and this is a good starting point on which to build a broader set of medical application specific training and evaluation loops.

One of the design goals for MONAI is that, as we add to the set of engines that are available for training and evaluation, we continuously refactor towards a useful set of composable components by which new engine training and evaluation loops are built.

Engine helpers

- TODO: whole paragraph is vague, refine and add content

There are several techniques in deep learning that can be considered as higher-order deep learning techniques. Ensembling and hyperparameter searches, for example, can be implemented either as pre-processing style functionality or as engine helpers that iteratively run engines themselves.

Mainstream features such as adaptive learning rates are also candidate pieces of composable engine functionality.