Skip to content

Commit

Permalink
Added docs
Browse files Browse the repository at this point in the history
  • Loading branch information
oandreeva-nv committed Nov 2, 2023
1 parent 09e5ab8 commit 5174313
Showing 1 changed file with 20 additions and 1 deletion.
21 changes: 20 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,9 @@ class TritonPythonModel:
inputs = [{
'name': 'INPUT0',
'data_type': 'TYPE_FP32',
'dims': [4]
'dims': [4],
# this parameter will set `INPUT0 as an optional input`
'optional': True
}, {
'name': 'INPUT1',
'data_type': 'TYPE_FP32',
Expand Down Expand Up @@ -394,6 +396,23 @@ function to gain read-only access to the `pb_utils.ModelConfig` object.
The `pb_utils.ModelConfig` object being returned from here will be used as the
final configuration for the model.

In addition to minimal properties, you can also set [model_transaction_policy](
https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#model-transaction-policy)
through `auto_complete_config` using `set_model_transaction_policy`.
For example,
```python
import triton_python_backend_utils as pb_utils


class TritonPythonModel:
@staticmethod
def auto_complete_config(auto_complete_model_config):
...
transaction_policy = {"decoupled": True}
auto_complete_model_config.set_model_transaction_policy(transaction_policy)
...
```

Note: The Python interpreter used to invoke this function will be destroyed
upon returning from this function and as a result none of the objects
created here will be available in the `initialize`, `execute`, or `finalize`
Expand Down

0 comments on commit 5174313

Please sign in to comment.