A question about instancing brevitas quantizer to act layer

[RyougiKukoc]

I want to quantized an unimplemented active layer in brevitas (such as nn.GELU), after taking a glance at QuantReLU and QuantTanh, I wonder if it is right to write as follow?

from brevitas.nn.quant_layer import QuantNonLinearActLayer
from brevitas.quant import Int8ActPerTensorFloat
from torch import nn
quant_gelu = QuantNonLinearActLayer(
    act_impl=nn.GELU, 
    passthrough_act=True, 
    input_quant=None, 
    act_quant=Int8ActPerTensorFloat,
    return_quant_tensor=True
)

[Giuseppe5]

This works, and it would be basically equivalent to having nn.GeLU followed by a QuantIdentity(act_quant=Int8ActPerTensorFloat).

The thing I would point out is that passthrough_act in this case should be False.

[RyougiKukoc]

Thanks!
Well I cannot figure out what does passthrough_act mean since QuantNonLinearActLayer is a hidden class, would you explain somehow?

[Giuseppe5]

You can check the differences between passthrough_act=True (like in QuantReLU) and other quant activations in our tutorial: https://github.com/Xilinx/brevitas/blob/master/notebooks/02_quant_activation_overview.ipynb

[RyougiKukoc]

Oh, so it means passing through the input of act layer, like x>0 for RELU(x)?

[Giuseppe5]

If a quantized activation (e.g., QuantReLU) has its quantization disabled (for any reason), there could be two main behaviours:

• Pass-through activation: If the input is a QuantTensor, the output will be a QuantTensor with the same quantization metadata (e.g, scale, bitwidth, etc.) of the original input tensor.
• Non pass-through: Whether the input is a QuantTensor or not, the output will be a PyTorch's tensor (dequantized).

QuantReLU behaves as pass-through quantized activation. However I believe that GeLU should not.

As you can see it's a subtle difference that applies in very specific cases, and if your quant activation has its own quantizer specified and enabled, the flag is not used.

[RyougiKukoc]

Thank you very much. You've been a great help.