Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MatMulInteger op is partially supported #159

Closed
3 tasks done
gyulaz-htec opened this issue Dec 1, 2023 · 1 comment
Closed
3 tasks done

MatMulInteger op is partially supported #159

gyulaz-htec opened this issue Dec 1, 2023 · 1 comment

Comments

@gyulaz-htec
Copy link

gyulaz-htec commented Dec 1, 2023

MatMulInteger op can accept different input types for T1 and T2 inputs, but MIGraphX only accepts in8 input tensors.
MIGraphx handles MatMulInteger in two files: parse_matmul.cpp and in quant_dot.hpp. In the second file there is a restriction about that T1 and T2 should be the same type, but it's not the case.

I came across this when tried to compile BERT-Squad int8 onnx zoo model:
migraphx-driver compile text/machine_comprehension/bert-squad/model/bertsquad-12-int8.onnx with the changes from this PR
Got the following error:
what(): /code/AMDMIGraphX/src/include/migraphx/check_shapes.hpp:210: same_type: quant_dot: Types do not match
Also the a_zero_point and b_zero_point optional inputs are not handled in MIGraphX.
The list of actions required:

  • Update MatMulInteger implementation to accept different types for T1 and T2 inputs
  • Update MatMulInteger implementation to process a_zero_point and b_zero_point
  • Enable test_matmulinteger_cpu in onnx_backend:test.py
@gyulaz-htec gyulaz-htec converted this from a draft issue Dec 1, 2023
@gyulaz-htec gyulaz-htec changed the title MatMulInteger op is not properly supported MatMulInteger op is partially supported Dec 1, 2023
@gyulaz-htec
Copy link
Author

Fixed in ROCm#2513

@github-project-automation github-project-automation bot moved this from 🔖 Ready to 🎉 Done in MIGraphX ONNX support Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🎉 Done
Development

No branches or pull requests

1 participant