Init the refactoring

[nhuet]

Full refactoring of the library

The main goal is to ease the introduction of custom layer by users, and thus the implementation of new decomon layers.

Main features:

• backward and forward layers are merged into a single DecomonLayer
• To implement a new decomon layer, one has only to implement
  • linear (or affine) case: get_affine_representation() -> w,b so that layer(x) can be written as x * w + b
  • generic case:
    • forward_ibp_propagate(): propagation of constant bounds through the layer
    • get_affine_relaxation() -> w_l, b_l, w_u, b_u such that x* w_l + b_l <= layer(x) <= x* w_u + b_u
      See the tutorial tutorials/z_Advanced/custom_layer.ipynb for more details.
• a keras model can now have a multidimensional input (before needed to be flattened), but is supposed to have a single input (but potentially several outputs)
• a keras model can include submodels which must also have single input but also single output. The backward propagation is now managed so that this does not decrease the tightness of the bound as the resulting model will be unwrapped as if a single model.
• The affine bounds can have several representation:
  • diagonal: the weights w are represented (batch by batch) by their "diagonal" (potentially multid) so that, for a given batch,
    w_generic = reshape(diag(ravel(w_diag)), w_diag.shape + w_diag.shape)
    In that case, w.shape == b.shape
  • from_linear: this means that the affine bounds come from a linear layer/model and thus
    • they are independent of the batch => no batch axis
    • upper bound == lower bound
  • a mix of both
  • generic: batchsize + full shape
• a dedicated function batch_multi_dot manage the product x * w. This is a batch per batch dot product of tensors allowing x to be multi-dimensional. This function is responsible for managing all above mentioned representations for w
  • if x is flattened and w generic, this is the keras batch_dot() or Dot layer call
  • if w is from a linear layer, this is keras tensordot()
  • if w is diagonal and from a linear layer, this is a mere product element by element.
  • the same function is also used to merge affine bounds and "multiply" weights tensors between them
• ForwardMode is replaced by booleans ibp and affine (to unify the existing mix)
• All operations needed to create inputs, merge bounds, convert format are implemented as dedicated layers to keep a clean graph for decomon models =>Fuse, ReduceCrownBounds, ConvertOutput, ForwardInput, BackwardInput, DecomonOracle, ...
• forward layers do not pass perturbation domain input in output anymore, and they take it in input only if in affine mode:
  • if affine and ibp bounds computed => to get tightest bound between ibp and affine turned into constant bounds
  • if affine for a non-linear layer => to get the forward oracle (constant bounds on kerasinputs) from previous layer
• backward layers do not take directly forward layer output as input but rather:
  • the previous backward layers outputs (bounds to propagate)
  • • oracle bounds if needed (i.e. constant bounds on keras layer input).
      These oracle bounds come from dedicated DecomonOracle bounds that convert the output of a forward layer from a first forward conversion, or the affine bounds from a subcrown launch starting from the layer of interest. In the future, other oracle could be used to inject external information on keras layers inputs bounds.
• decomon layers inputs and outputs are flat (to be consistent with decomon models which follow keras model api) but then internal methods use the split inputs/outputs. This conversions are managed by InputsOutputsSpecs which is also responsible for detecting the representation of affine bounds.
• backward conversion for merge layers is now handled
• keras models with multiple outputs are now handled and tested

Goodies:

• a dedicated plot_model() allows to show the graph of a decomon model. This is a modified version of original keras plot_model() (and thus still working on plain keras models) that
  • for each DecomonLayer, displays
    • the name of the underlying keras layer
    • the sense of propagation with a distinct color
  • displays other utilitary decomon layers in a separate color
  • allows argument to specify other attributes to show and choose their colors => see tests/visu-decomon.ipynb for more details

To be continued:

• the core architecture has been set
• some layers have been implemented: Dense, Activation with linear, relu or softsign activation function, Add
• other layers are still to be reimplemented