Determining Quantization Level By Using Exported PTH & ONNX Files

[Ba1tu3han]

Hello,

I have trained a NN model with Brevitas & Torch by using QAT for W1A1, W2A2, W4A4, W8,A8 variations. Then I saved them as PTH and ONNX formats. I want to determine quantization level of the models by just using the exported files. Is it possible? Is there any direct method instead of comparing each other?

Thanks in advance,

[Giuseppe5]

For pth, we currently don't store bit width as part of the state dict with the majority of the pre-defined quantizers (although it is possible to do it if you create a custom one).
For ONNX, the Clip node in the graph is a hint with respect to the quantization format you're using, e.g. Clip between -4 and 3 represents INT3 quantization.

I am not sure I understand what you mean with comparing each other though.

[Ba1tu3han]

Hello Giuseppe,

Thank you for your reply.

Creating a custom quantizer is too hard for me right now. However, checking the "clip" node is a good idea thank you. For example 2 bit quantization should I see clip between -4 and 2 ? what is the calculation here?

I was comparing accuracy results and HW utilization of different level quantized models in FINN to double check the quantization is applied. I meant the comparing like that.

[JPPalacios]

Hey @Ba1tu3han,

Do you mind describing your quantization setup? I am also looking to train the exact same n-w, m-a widths. Did you have a quant_common.py for binary/ternary quantization?

Thank you

[Giuseppe5]

The calculation for the clip node depends on:

• bitwidth
• signedness
• narrow range
  The code that is used to compute min/max value can be found here:
  

  brevitas/src/brevitas/export/common/handler/base.py

  Line 44 in 2004568

  def int_clip_symbolic_kwargs(cls, narrow, signed, bit_width):