Questions about layer input/output quantization in FINN

[hleblevec]

Hi,
I have noticed that some quantization options available in Brevitas are not compatible with the export to FINN onnx format. Mainly, the "input_quant" and "output-quant" options make the export fail.
I have several questions about this :

• Is there a specific reason that this does not work when exporting to FINN, or is it just that the feature hasn't been implemented yet?
• How to chose the quantization of the activation values when using FINN? Is there a step in the FINN transformations flow that quantize the inputs and outputs of the layers? (I don't seem to see it when viewing the models in Netron)

Hoping you can help me understand it better.

[maltanar]

Hi Hugo,

Brevitas is intended to be a general-purpose research tool for quantization-aware training, and there are many networks/quantization configurations that can be represented in Brevitas which won't export to or otherwise go through the FINN compiler. We don't have full documentation on what is and isn't supported in the FINN compiler, see also related FAQ item here: https://finn.readthedocs.io/en/latest/faq.html#can-i-deploy-custom-nns-with-arbitrary-precisions-and-layers-using-finn

• Regarding quantizing layer inputs/outputs, we typically do this with an explicit quantized activation layer (e.g. QuantIdentity, QuantReLU that comes before/after the layer itself.
• Regarding choosing the activation quantization, please note that the FINN compiler does not do any extra quantization (at least not yet). All quantization must be done in Brevitas prior to exporting the network.

A helpful source of information here can be some of the Brevitas examples that go through FINN. If you choose layer & quantization patterns similar to these when designing your own networks, this should export and go through the compiler without issues:

• https://github.com/Xilinx/brevitas/tree/master/src/brevitas_examples/bnn_pynq
• https://github.com/Xilinx/brevitas/blob/master/src/brevitas_examples/imagenet_classification/models/mobilenetv1.py
• https://github.com/Xilinx/finn/blob/master/notebooks/end2end_example/cybersecurity/1-train-mlp-with-brevitas.ipynb

We also have a QAT guidelines shared document here that may come in handy.

[hleblevec]

Hi Yaman,
Thanks for your reply! What I'm allowed to do with Brevitas in order to make the model compatible with FINN is still a bit obscure to me but I think I'll figure things out with the resources you gave.