Skip to content

Commit

Permalink
docs(a): add documentation for max price (#639)
Browse files Browse the repository at this point in the history
* add documentation for maxPricePerUnit, ignoreMaxPriceIfNeeded and maxPricePerCapability

* refactor(ai): create seperate gateway maxPrice config section

This commit restructures the pull request so that the maxPrice
configuration for the gateway has its own section.

---------

Co-authored-by: Rick Staa <[email protected]>
  • Loading branch information
ad-astra-video and rickstaa authored Sep 17, 2024
1 parent 320c733 commit c4f8640
Showing 1 changed file with 57 additions and 0 deletions.
57 changes: 57 additions & 0 deletions ai/gateways/onchain.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,63 @@ for more information. Here’s a summary of the key steps:
- `-ethPassword=/root/.lpData/.eth_secret`: Sets the path to the Ethereum keystore password file.
- `-ethOrchAddr=<MAIN_ORCH_ETH_ADDRESS>`: Sets the Ethereum address of the Mainnet Transcoding Network Gateway.
- `-maxTotalEV=100000000000000`: Ensures the AI Gateway complies with the max ticket value limits in `go-livepeer`.
- `-maxPricePerUnit=<MAX PRICE in WEI or USD>`: This defaults to 0. Set to acceptable max price in wei (or USD, e.g. 0.02USD) willing to pay. Note: this can impact amount of Orchestrators available to process the work.
- `-ignoreMaxPriceIfNeeded=<true or false>`: This defaults to false. Set to true if want to process requests if no Orchestrators are under maxPricePerUnit or do not want to use a max price.
- `-maxPricePerCapability=/path/to/maxPrices.json`: This flag sets the max price per unit for one or many pipeline/models. Refer to the [Set Max Price Per Pipeline and Model](#set-max-price-per-pipeline-and-model-optional) section for more details.

</Step>
</Steps>

## Set Max Price Per Pipeline and Model (Optional)

To avoid overpaying for AI inference jobs, you can set a maximum price per unit
for each pipeline and model. This feature is optional and can be configured
using the `-maxPricePerCapability` flag. If a price is not set for a specific
pipeline/model, the `-maxPricePerUnit` price will be used. The flag input should
be a JSON file with the specified max prices. The example below demonstrates the
following configurations:

- Set the price for the `image-to-image` pipeline and model
`ByteDance/SDXL-Lightning` to 1,700,000 wei per unit (pixels).
- Set the price for the `text-to-image` pipeline and model
`stabilityai/stable-diffusion-3-medium-diffusers` to 4,768,371 wei per unit
(pixels).
- Set the price for the `upscale` pipeline for all models to 4,768,371 wei per
unit. The `pixels_per_unit` defaults to `1` if not specified.
- Set the price for the `image-to-video` pipeline for all models to 3,390,842
wei per unit (pixels).
- Set the price for the `audio-to-text` pipeline for all models to 12,882,811
wei per unit (milliseconds of audio).

```json
{
"capabilities-prices": [
{
"pipeline": "image-to-image",
"model_id": "ByteDance/SDXL-Lightning",
"price_per_unit": 1700000,
"pixels_per_unit": 1
},
{
"pipeline": "text-to-image",
"model_id": "stabilityai/stable-diffusion-3-medium-diffusers",
"price_per_unit": 4768371,
"pixels_per_unit": 1
},
{
"pipeline": "upscale",
"model_id": "default",
"price_per_unit": 4768371
},
{
"pipeline": "image-to-video",
"price_per_unit": 3390842,
"pixels_per_unit": 1
},
{
"pipeline": "audio-to-text",
"price_per_unit": 12882811,
}
]
}
```

0 comments on commit c4f8640

Please sign in to comment.