Which quantization mode is faster at runtime, W8A8 or W4A16?

[cocosci]

Hi group,

Great to know TRT-LLM supports INT4 for Llama now! However, it's only for weights with activations in 16-bit. So, here's the question, compared to INT8 SQ where both W and A are in 8-bit, which is faster?

Also, will W4A4 or W4A8 be supported in the near future by TRT-LLM?

Thanks!