Skip to content

Commit

Permalink
Update the documentation for the Anthropic integration
Browse files Browse the repository at this point in the history
  • Loading branch information
rlouf committed Dec 16, 2024
1 parent fffa05a commit 6084fd5
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 0 deletions.
45 changes: 45 additions & 0 deletions docs/reference/models/anthropic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Anthropic

!!! Installation

You need to install the `anthropic` library to be able to use the Anthropic API in Outlines. Or alternatively you can run:

```bash
pip install "outlines[anthropic]"
```

## Anthropic models

Outlines supports models available via the Anthropic API, e.g. Claude 3.5 Haiku or Claude 3.5 Sonner. You can initialize the model by passing the model name to `outlines.models.Anthropic`:

```python
from outlines import models

model = models.Anthropic("claude-3-5-haiku-latest")
model = models.Anthropic("claude-3-5-sonnet-latest")
```

Check the [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models) for an up-to-date list of available models. You can pass any paramater you would pass to the Anthropic SDK as keyword arguments:

```python
model = models.Anthropic(
"claude-3.5-haiku-latest",
api_key="<my api key>"
)
```

## Text generation

To generate text using an Anthropic model you need to build a `Generator` object, possibly with the desired output type. You can then call the model by calling the `Generator`. The method accepts every argument that you could pass to the `client.completions.create` function, as keyword arguments:

```python
from outlines import models, Generator

model = models.Anthropic("claude-3-5-haiku-latest")
generator = Generator(model)
result = generator("Prompt", max_tokens=1024)
```

See the [Anthropic SDK documentation](https://github.com/anthropics/anthropic-sdk-python/blob/main/src/anthropic/resources/messages.py) for the list of available arguments.

The Anthropic API currently does not support structured generation.
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ vllm = ["vllm", "transformers", "numpy2"]
transformers = ["transformers", "accelerate", "datasets", "numpy<2"]
mlxlm = ["mlx-lm", "datasets"]
openai = ["openai"]
anthropic = ["anthropic"]
gemini = ["google-generativeai"]
llamacpp = ["llama-cpp-python", "transformers", "datasets", "numpy<2"]
exllamav2 = ["exllamav2"]
Expand Down

0 comments on commit 6084fd5

Please sign in to comment.