Warnings displayed as errors during TensorRT optimization.

[BengtGustafsson]

Describe the issue

This output appears on log level warning, but on log level error, despite it announcing itself as an error. This happens in the ort::Session constructor for one of our models. I would say towards the end of the time (maybe 20 s) it uses for the optimization, all in one go as fast as the terminal can print.

E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0xaa15e43058248292 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0xb4fe42a3eed4e892 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0xe57a5f08bb65a719 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0xbe93c53dd404a388 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0x2b78a94a2ec1dff6 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0x63718a7552e7f5ad due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0x5c1d65bb3bbd1490 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:43 ERROR] Error Code: 9: Skipping tactic 0xf1fc7e0537458603 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0xaa15e43058248292 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0xb4fe42a3eed4e892 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0xe57a5f08bb65a719 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0xbe93c53dd404a388 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0x2b78a94a2ec1dff6 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0x63718a7552e7f5ad due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0x5c1d65bb3bbd1490 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]
E onnxruntime: [2024-09-20 12:15:44 ERROR] Error Code: 9: Skipping tactic 0xf1fc7e0537458603 due to exception canImplement1 [tensorrt_execution_provider.h:88 onnxruntime::TensorrtLogger::log]

To reproduce

Unfortunately I can't provide the model. We do use embedded engine caching for TensorRT but I think we got the same warnings with regular caching. We don't get these warnings when reading back from the cache.

Urgency

No response

Platform

Windows

OS Version

Windows 11

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.19.2

ONNX Runtime API

C++

Architecture

X64

Execution Provider

TensorRT

Execution Provider Library Version

TensorRT 10.4.0.26 on CUDA 11.6

[chilo-ms]

This "error" message is from TRT during engine build phase. So that's why you don't get these messages when reading back from the cache.
TRT will run many tactics during kernel selection to pick the optimal one, even though running "canImplement1" tactic throws exception, TRT can still choose other tactic to proceed without exiting the program.

i assume if you run trtexec, you will get the same "error" message. There is not much ORT TRT can do here.

[BengtGustafsson]

Yeah, I know that, I just posted this issue because you guys asked me to.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[BengtGustafsson]

This system of making issues stale because you didn't react to them is ridiculous!