diff --git a/docs/reference/openai_text_generation.md b/docs/reference/openai_text_generation.md index 5845545fa..0e7209de3 100644 --- a/docs/reference/openai_text_generation.md +++ b/docs/reference/openai_text_generation.md @@ -1,38 +1,85 @@ # Generate text with the OpenAI API -Outlines is focused on 🔓 models, but includes an OpenAI integration nevertheless. You can instantiate a model very simply by calling the [outlines.models.openai][] function, with either a chat or non chat model: +Outlines supports models available via the OpenAI Chat API, e.g. ChatGPT and GPT-4. The following models can be used with Outlines: ```python from outlines import models -model = models.openai("text-davinci-003") -model = models.openai("gpt4") +model = models.openai("gpt-3.5-turbo") +model = models.openai("gpt-4") print(type(model)) -# OpenAIAPI +# OpenAI ``` -!!! note +It is possible to pass a system message to the model when initializing it: + +```python +from outlines import models - It is currently not possible to pass a system message to the model. If that is something you need, please [open an Issue](https://github.com/outlines-dev/outlines/issues) or, better, [submit a Pull Request](https://github.com/outlines-dev/outlines/pulls). +model = models.openai("gpt-4", system_prompt="You are a useful assistant") +``` -The OpenAI integration supports the following features: +## Usage -- The ability to stop the generation when a specified sequence is found [🔗](#stop-when-a-sequence-is-found) -- The ability to choose between different choices [🔗](#multiple-choices) -- Vectorization, i.e. the ability to pass an array of prompts and execute all requests concurrently [🔗](#vectorized-calls) +### Call the model -## Stop when a sequence is found +OpenAI models can be directly called with a prompt: + +```python +from outlines import models + +model = models.openai("gpt-3.5-turbo") +result = model("Say something", temperature=0) +``` + +!!! warning + + This syntax will soon be deprecated and one will be able to generate text with OpenAI models with the same syntax used to generate text with Open Source models. + +### Stop when a sequence is found The OpenAI API tends to be chatty and it can be useful to stop the generation once a given sequence has been found, instead of paying for the extra tokens and needing to post-process the output. For instance if you only to generate a single sentence: ```python from outlines import models -model = models.openai("text-davinci-003") +model = models.openai("gpt-4") response = model("Write a sentence", stop_at=['.']) ``` +### Choose between multiple choices + +Outlines allows you to force OpenAI models to generate a string chosen among multiple choices, and does so in the most efficient way possible: + +```python +from outlines import models + +model = models.openai("gpt-3.5-turbo") +result = model.generate_choice("Red or blue?", ["red", "blue"]) +``` + +!!! warning + + This syntax will soon be deprecated and one will be able to generate text with OpenAI models with the same syntax used to generate text with Open Source models. + +## Monitoring API use + +It is important to be able to track your API usage when working with OpenAI's API. The number of prompt tokens and completion tokens is directly accessible via the model instance: + +```python +import outlines.models + +model = models.openai("gpt-4") + +print(model.prompt_tokens) +# 0 + +print(model.completion_tokens) +# 0 +``` + + ## Multiple choices It can be difficult to deal with a classification problem with the OpenAI API. However well you prompt the model, chances are you are going to have to post-process the output anyway. Sometimes the model will even make up choices. Outlines allows you to *guarantee* that the output of the model will be within a set of choices you specify: @@ -165,3 +212,21 @@ You may find this useful, e.g., to implement [Tree of Thoughts](https://arxiv.or !!! note Outlines provides an `@outlines.vectorize` decorator that you can use on any `async` python function. This can be useful for instance when you call a remote API within your workflow. + + +## Advanced usage + +It is possible to specify the values for `seed`, `presence_penalty`, `frequence_penalty`, `top_p` by passing an instance of `OpenAIConfig` when initializing the model: + +```python +from outlines.models.openai import OpenAIConfig +from outlines import models + +config = OpenAIConfig( + presence_penalty=1., + frequence_penalty=1., + top_p=.95, + seed=0, +) +model = models.openai("gpt-4", config=config) +``` diff --git a/outlines/models/openai.py b/outlines/models/openai.py index f87a26f7e..cb5bddae6 100644 --- a/outlines/models/openai.py +++ b/outlines/models/openai.py @@ -58,7 +58,7 @@ class OpenAIConfig: """ - model: str + model: str = "" frequency_penalty: float = 0 logit_bias: Dict[int, int] = field(default_factory=dict) max_tokens: Optional[int] = None