[Feature] Add training strategies based on learning Tendencies

[Rilwan-Adewoyin]

Is your feature request related to a problem? Please describe.

@Rilwan-Adewoyin > Allow optional training with optimisation towards tendency as opposed to state/residual

@jakob-schloer > While the deterministic AIFS is currently trained on the weighted mean square error of absolute values, GraphCast is trained to predict the normalized tendency. This might be beneficial for variables changing on different timescales.

Describe the solution you'd like

Instead of predicting x(t+1) directly, the model should predict the normalized tendency y(t) = x(t+1) - x(t).

Describe alternatives you've considered

NA

Additional context

Has been shown to improve results with a diffusion model targeting atmosphere - this will be useful for investigating whether or not improvements extend to other models and other domains such as ocean and wave.

Dependent on a issue/branch on anemoi-models repository: #39

@jakob-schloer
@da-ewanp
@ssmmnn11

Organisation

ECMWF