Conceptual Question about DeepONets Structure and Independent Variables

[IBo-96]

Dear community,

I have one conceptual question regarding the structure of DeepONets.

I have read the main articles from professor Lu Lu and team (DeepONet publication in Nature, MIONet, Fourier-MIONet, etc.). In these papers, the independent variables (dimensions) from the ODE/PDEs are usually time and space. These two inputs go to the Trank net, whereas the input functions defined in the partial derivatives (e.g. u(x) or u(t)) go to the branch net.

I'm trying to implement DeepONets in a fermentation process, where the increase in Cell Density (X) over time is defined by the specific growth rate (µ) as the following ODE:

$\frac{dX}{dt} = \mu X$

Where µ depends on time (e.g. cells get old/accumulate stress) as well as on other process parameters such as pH. The evolution of mu over time and process parameters is not completely understood (there is no mechanistic equation describing the relationship). Process parameters might change over fermentation time, however, their profile does not depend on time, in the sense that they are controlled process parameters/independent process variables.

For example, in the case of pH, it's the operator who decides which pH profile to execute during fermentation. Due to these process parameters change over time but are time-independent, my question is if they should go to the branch or trunk net.

In this example with pH, I have two possible scenarious in mind which actually both make sense to me due to several reasons:

Scenario 1

• Branch net input: µ's values over time and pH
• Trunk net inputs: time + pH
• Output: VCD values at different time and pH values.

(Note: µ by definition is the increase in Cell Density over time divided by Cell Density (that's why it's called specific growth rate). This means the matrix of mu values would be calculated from the dataset by simply isolating it from the equation above:

$ \mu= \frac{dX}{dt} / X$

This scenario makes sense to me because I'm using as input to the Branch net a function (mu) which is defined in the ODE equation, as it happens in the DeepONet publications, while I'm also using the two independent variables that define this function (time and pH, since µ(t, x)) in the trunk net.

Scenario 2

• Branch net input: pH values (over time)
• Trunk net inputs: time
• Output: VCD values at different times.

This on the one hand makes sense to me because the DeepONet would directly predict how X would evolve over time for a given pH profile. On the other hand, however, it is not respecting the ODE equation we know from process knowledge (After having implemented the DeepONet, I would like to introduce a physic loss to make sure $\frac{dX}{dt} = \mu X$ is respected).

Thank you very much for your support, and if you need any further information about this specific case application please let me know.

[vl-dud]

I think both scenarios are correct. Personally, I would use the second one and make µ an additional output.
But it would be interesting to listen to other community members.

[IBo-96]

Thank you for your answer.

After reading a bit more literature I'm also starting to believe Option 2 makes a bit more sense.
However, I'm still concerned about how to compute the physic loss for this option.
I like your idea of introducing µ as an additional output in the DeepONet for the loss. Have you ever tried something similar to this, or have you ever seen it in a publication? I'm just asking because the other option I had actually in mind was to predict mu in a separate parallel DeepONet based on pH and time inputs as well, and then apply the predicted mu's to the physics loss. Similar to how this example is using two parallel NNs for the two unknown functions: https://deepxde.readthedocs.io/en/latest/demos/pinn_inverse/elliptic.inverse.field.html
However, I don't know if this is still valid for DeepONets. I also assume this would have a rather bigger computing cost.

Has anyone from the community ever tried something like this? Or does anyone know about a publication describing a similar problem? Getting inspiration from other engineering fields would be highly useful!!

[vl-dud]

I like your idea of introducing µ as an additional output in the DeepONet for the loss. Have you ever tried something similar to this, or have you ever seen it in a publication?

Yes, I have defined one of inner parameters of Navier–Stokes equations as a network output. But I have not seen anything like this in published articles.

I'm just asking because the other option I had actually in mind was to predict mu in a separate parallel DeepONet based on pH and time inputs as well, and then apply the predicted mu's to the physics loss. Similar to how this example is using two parallel NNs for the two unknown functions

Check DeepM&Mnet https://arxiv.org/abs/2009.12935

[IBo-96]

Hey @vl-dud!
I was thinking a bit more about your answer and a question came to my mind:

When you mentioned:

Yes, I have defined one of inner parameters of Navier–Stokes equations as a network output. But I have not seen anything like this in published articles.

Were you referring to the situation where one parameter of your loss function is being independently predicted by a 2nd model (either DeepONet or PINN)? or did you really want to mean that the same model is predicting both your final response as well as the inner parameter?

I'm asking this because of the following:
In PINNs, you can define several outputs by just changing the width of your last (output) layer. However, in the case of DeepONets , they can only handle one output per model by definition, is this correct? is it actually possible to build a multioutput DeepONet?
@lululxvi

Finally, regarding the DeepM&Mnet publication. Did you ever find some open source library to implement such architecture? In https://github.com/lululxvi/deepxde/blob/master/README.md it's mentioned that DeepXDE could handle that, but actually there is no class for that. Also in the publication you shared there are no code for the examples mentioned.

Thanks a lot for your support!

[vl-dud]

Hey @IBo-96 !

or did you really want to mean that the same model is predicting both your final response as well as the inner parameter?

I meant it.

However, in the case of DeepONets , they can only handle one output per model by definition, is this correct? is it actually possible to build a multioutput DeepONet?

You can build a multioutput DeepONet. Check num_outputs and multi_output_strategy in https://github.com/lululxvi/deepxde/blob/master/deepxde/nn/tensorflow_compat_v1/deeponet.py

Did you ever find some open source library to implement such architecture?

I'm guessing it could be implemented based on DeepXDE but it would require some customization.