Pre and post processing: denormalize

[jdeschamps]

In the specs, the same normalization functions are defined for pre and post processing:

PreprocessingDescr = Annotated[
    Union[
        BinarizeDescr,
        ClipDescr,
        EnsureDtypeDescr,
        ScaleLinearDescr,
        SigmoidDescr,
        FixedZeroMeanUnitVarianceDescr,
        ZeroMeanUnitVarianceDescr,
        ScaleRangeDescr,
    ],
    Discriminator("id"),
]
PostprocessingDescr = Annotated[
    Union[
        BinarizeDescr,
        ClipDescr,
        EnsureDtypeDescr,
        ScaleLinearDescr,
        SigmoidDescr,
        FixedZeroMeanUnitVarianceDescr,
        ZeroMeanUnitVarianceDescr,
        ScaleRangeDescr,
        ScaleMeanVarianceDescr,
    ],
    Discriminator("id"),
]

That means that if one wants to define a normalization (pre-processing) and a denormalization (post-processing) with FixedZeroMeanUnitVarianceDescr, the post-processing is a bit awkward as we need to invert the normalization:

eps = 1e-6
inv_mean = - mean / (std + eps)
inv_std = 1 / (std + eps) - eps

# pseudo code
FixedZeroMeanUnitVarianceDescr(
    kwargs={
       "mean": inv_mean,
       "std": inv_std,
   }
)

# proof for reference (because I don't want to calculate it again):
# normalization: y = (x-mean) / (std+eps)
# denormalization: x = y * (std + eps) + mean
# post-processing normalization: x = (y-mean') / (std'+eps) = y * (1 / (std' + eps)) + (- mean' / (std' + eps))
# which leads to:
# std + eps = 1 / (std' + eps) <=> std' = 1/(std + eps) - eps
# mean = - mean' / (std' + eps) <=> mean' = -mean*(std'+eps) <=> mean' = -mean / (std + eps)

Wouldn't it make sense to define a UndoFixedZeroMeanUnitVarianceDescr (with a better name) that would just do:

output = sample * (std + eps) + mean

[FynnBe]

For this operation ScaleLinearDescr could be used instead.
FixedZeroMeanUnitVarianceDescr could be replaced as well (we used to allow fixed values for the ZeroMeanUnitVarianceDescr op...) So rather than adding more convenience functions that are alternative forms of ScaleLinearDescr, I'd remove FixedZeroMeanUnitVarianceDescr instead.