don't check max_batch_size for cpu

[yufenglee]

No description provided.

[wangyems]

for cpu models, we don't expect users to execute params.try_use_cuda_graph_with_max_batch_size(1). I think we can add a check to conditionally pass try_use_cuda_graph...() in the model-qa.py

[yufenglee]

for cpu models, we don't expect users to execute params.try_use_cuda_graph_with_max_batch_size(1). I think we can add a check to conditionally pass try_use_cuda_graph...() in the model-qa.py

if user called it, it should not crash. I think we can add a warning later instead of quit.

[baijumeswani]

if user called it, it should not crash. I think we can add a warning later instead of quit.

It would still crash (throw an exception) since you're checking for cuda graph enabled in cpu ep, no?

[yufenglee]

h (throw an exception) since you're checking for cuda graph enabled in cpu ep, no?
The PR prevents quit if user calls: params.try_use_cuda_graph_with_max_batch_size(1)

onnxruntime-genai/examples/python/model-qa.py

Line 31 in 352c1ec

params.try_use_cuda_graph_with_max_batch_size(1)

It will still crash if user set to enable cuda_graph for cpu

[baijumeswani]

How is that any different from what it was earlier? It appears that max_batch_size_ is only set when try_use_cuda_graph_with_max_batch_size is called.

[yufenglee]

How is that any different from what it was earlier? It appears that max_batch_size_ is only set when try_use_cuda_graph_with_max_batch_size is called.

It will not crash with this PR if user call try_use_cuda_graph_with_max_batch_size for CPU device

[baijumeswani]

Sorry, I didn't check the definition of IsCudaGraphEnabled. PR looks good.