Is there any plan to change convolution calculation method on ROCM provider?

[twoapples1]

I would like to inquire if you have any plans to change the convolutional calculation method on the ROCM provider? In the previous version, onnxruntime still uses miopen as the convolutional application acceleration library on ROCM provider.However, due to the optimization mechanism of miopen, it exhibits poor performance in dynamic model inference.When Miopen encounters shapes and sizes of convolutions that have not been met before, it will first search for the optimal implementation algorithm and kernel function, and then add the searched strategy to the cache and generate the cache file. This process is quite time-consuming, and we are unlikely to warmup all shapes and sizes of convolutions when reasoning with a large amount of dynamic data. Therefore, I think miopen is not very suitable as a convolution acceleration library for dynamic model applications, and perhaps CK would be more suitable.What do you think of this?

[twoapples1]

I once tried to change the calculation method of convolution to im2col+gemm, which can indeed solve the bottleneck of dynamic application in the first frame, but the calculation performance is not very good

[JehandadKhan]

Hello @twoapples1, MIOpen has implemented AI based heuristics for selecting optimally performing kernels without bench marking, this feature is slated to be release with ROCm 5.7 (merged into our develop branch via PR # 2107). This feature does require the use of our immediate mode API which would be a change in the ROCm EP. This method provides best performance both for known problems as well as dynamic models without the overhead of bench marking.

Since MIOpen uses CK as a kernel backend, you get all the CK kernels automatically, with the added advantage of heuristics and tuning.

[twoapples1]

Okay, thank you