Python config API docs #22951
Python config API docs #22951
Conversation
docs/genai/api/python.md
Outdated
| @@ -28,6 +28,57 @@ pip install onnxruntime-genai | |||
| import onnxruntime_genai | |||
| ``` | |||
|
|
|||
| ## Config class | |||
|
|
|||
| If the config needs to be modified at runtime to change providers/set provider options, this object can be first created and used, then a model can be created from this config object. | |||
There was a problem hiding this comment.
It is perhaps obvious but maybe worth stating that if the user does not create the object, then the config is used directly from the file in the model folder
docs/genai/api/python.md
Outdated
|
|
||
| ### Add a provider to the list | ||
|
|
||
| If the provider isn't already in the list of providers, this adds it to the end of the list. |
There was a problem hiding this comment.
What does adding to the end of the list mean?
There was a problem hiding this comment.
I tried to find onnxruntime docs about how the provider list works, but couldn't find anything obvious. I don't think we should be explaining it here, do you know where it is?
|
|
||
| ### Clear list of providers | ||
|
|
||
| Clears the list of providers. This is the only way to remove existing providers to enforce using only a specific provider. |
There was a problem hiding this comment.
Once the list is cleared, what provider is used? CPU? Can you state that?
| onnxruntime_genai.Config.set_provider_option(provider_name: str, option_name: str, option_value: str) | ||
| ``` | ||
|
|
||
| #### Parameters |
There was a problem hiding this comment.
Please add an example of setting a new provider, as that is the scenario we are addressing with this change
Description
Motivation and Context