Skip to content

Commit

Permalink
Add docs for Carton export (#336)
Browse files Browse the repository at this point in the history
  • Loading branch information
VivekPanyam authored Jan 5, 2024
1 parent 4292df5 commit 6544e5e
Show file tree
Hide file tree
Showing 2 changed files with 65 additions and 0 deletions.
38 changes: 38 additions & 0 deletions docs/user_guide/command_line_interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Ludwig provides several functions through its command line interface.
| [`collect_weights`](#collect_weights) | Collects tensors containing a pretrained model weights |
| [`collect_activations`](#collect_activations) | Collects tensors for each datapoint using a pretrained model |
| [`export_torchscript`](#export_torchscript) | Exports Ludwig models to Torchscript |
| [`export_carton`](#export_carton) | Exports Ludwig models to Carton |
| [`export_neuropod`](#export_neuropod) | Exports Ludwig models to Neuropod |
| [`export_mlflow`](#export_mlflow) | Exports Ludwig models to MLflow |
| [`preprocess`](#preprocess) | Preprocess data and saves it into HDF5 and JSON format |
Expand Down Expand Up @@ -973,6 +974,43 @@ optional arguments:
For more information, see [TorchScript Export](/user_guide/model_export/#torchscript-export)
# export_carton
A Ludwig model can be exported as a [Carton](https://carton.run/), which allows it to be run from several programming languages (including C, C++, Rust, and more).
In order to export a Ludwig model to Carton, first make sure the `cartonml` package is installed in your environment (`pip install cartonml-nightly`), then run the following command:
```bash
ludwig export_carton [options]
```
or with:
```bash
python -m ludwig.export carton [options]
```
These are the available arguments:
```
usage: ludwig export_carton [options]
This script loads a pretrained model and saves it as a Carton.
options:
-h, --help show this help message and exit
-m MODEL_PATH, --model_path MODEL_PATH
model to load
-mn MODEL_NAME, --model_name MODEL_NAME
model name
-od OUTPUT_PATH, --output_path OUTPUT_PATH
path where to save the export model
-l {critical,error,warning,info,debug,notset}, --logging_level {critical,error,warning,info,debug,notset}
the level of logging to use
```
For more information, see [Carton Export](/user_guide/model_export/#carton-export)
# export_neuropod
A Ludwig model can be exported as a [Neuropod](https://github.com/uber/neuropod), a mechanism that allows it to be executed in a framework agnostic way.
Expand Down
27 changes: 27 additions & 0 deletions docs/user_guide/model_export.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,33 @@ For the majority of features, NaNs are handled by the preprocessor in the same w

HuggingFace models are not yet supported for TorchScript export, though we are working on it!

## Carton Export

[Carton](https://carton.run/) is a library that allows users to efficiently run ML models from several programming languages (including C, C++, Rust, and more).

A subset of Ludwig Models can be exported to Carton. In addition to the model itself, the preprocessing and postprocessing steps are included in the exported model as well, ensuring that the model can be used for inference in a production environment out-of-the-box.

To get started, simply run the [`export_carton`](/latest/user_guide/command_line_interface/#export_carton) command:

```
ludwig export_carton -m=results/experiment_run/model
```
This will produce a file that can be loaded by Carton from any supported programming language.

For example, from Python, you can load and run the model as follows:

```py
import cartonml as carton

async def main():
model = await carton.load("/path/to/model.carton")
output = await model.infer({
"x": np.zeros(5)
})
```

See the [Carton quickstart guide](https://carton.run/quickstart) for usage from other programming languages.

## Triton Export

Coming soon...
Expand Down

0 comments on commit 6544e5e

Please sign in to comment.