CFGGuide
+
+
+
+ Bases: Guide
Guide to generate text that is in the language of a context-free Lark grammar.
+ +outlines/fsm/guide.py316 +317 +318 +319 +320 +321 +322 +323 +324 +325 +326 +327 +328 +329 +330 +331 +332 +333 +334 +335 +336 +337 +338 +339 +340 +341 +342 +343 +344 +345 +346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 +361 +362 +363 +364 +365 +366 +367 +368 +369 +370 +371 +372 +373 +374 +375 +376 +377 +378 +379 +380 +381 +382 +383 +384 +385 +386 +387 +388 +389 +390 +391 +392 +393 +394 +395 +396 +397 +398 +399 +400 +401 +402 +403 +404 +405 +406 +407 +408 +409 +410 +411 +412 +413 +414 +415 +416 +417 +418 +419 +420 +421 +422 +423 +424 +425 +426 +427 +428 +429 +430 +431 +432 +433 +434 +435 +436 +437 +438 +439 +440 +441 +442 +443 +444 +445 +446 +447 +448 +449 +450 +451 +452 +453 +454 +455 +456 +457 +458 +459 +460 +461 +462 +463 +464 +465 +466 +467 +468 +469 +470 +471 +472 +473 +474 +475 +476 +477 +478 +479 +480 +481 +482 +483 +484 +485 +486 +487 +488 +489 | |
__init__(cfg_string, tokenizer)
+
+Construct the PartialLark parser and set the empty initial_state (PartialParserState)
+ +outlines/fsm/guide.pycan_terminate_state(state)
+
+Generation is allowed to terminate
+ +outlines/fsm/guide.pycopy()
+
+get_next_instruction(state)
+
+Return the next instruction for guided generation.
+Current lazy approach: +- For each token in the vocabulary + - create a copy of the parsers state + - add the tokens to the parsers input text + - if valid, add token to returned tokens
+Further refinements are necessary for performant text processing.
+state + The guides current PartialParserState, or None if complete
+A Generate instance that contains the model and the allowed token ids.
outlines/fsm/guide.pyget_next_state(state, token_id)
+
+Update the state of the guide. +Decode the token_id, and calculate the new parser_state with the token applied.
+state + The guides current PartialParserState, or None if complete +token_id + The id of the token that was just generated.
+The guides new PartialParserState
+ +outlines/fsm/guide.pyiter_valid_token_ids(state, candidate_token_ids)
+
+Iterate over the given token_ids and yield those that are valid for the current parser state.
+parser_state + The current state of the parser, or None if complete. +token_ids + The list of token ids to check for validity.
+int + Valid token ids.
+ +outlines/fsm/guide.pymust_terminate_state(state)
+
+Generation must terminate, no legal continuations
+ + +Generate
+
+
+
+ dataclass
+
+
+Generate instruction
+tokens
+ The tokens that lead to a valid completion if generated. A value
+ of None indicates that all tokens are allowed.
outlines/fsm/guide.pyGuide
+
+
+
+ Bases: Protocol
Base definition of a generation guide.
+A generation guide defines the behavior of a finite-state machine that guides
+a text generation procedure. Unlike the DFAs built from regular expressions
+guides can also emit a Write instructions which tells the model that it can
+append a sequence of tokens (or token word) instead of generating it.
outlines/fsm/guide.pyRegexGuide
+
+
+
+ Bases: Guide
Guide to generate text in the language of a regular expression.
+ +outlines/fsm/guide.py184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 | |
get_next_instruction(state)
+
+Return the next instruction for guided generation.
+The initialization of the guide builds an index which maps FSM states to a +map from authorized tokens to the state in which the guide needs to move +if said token is generated. Therefore the authorized tokens at the +current state are the keys of the map returned by the value of the index +for current state.
+If the current state is not contained in the end this means that we are +in a final state of the guide. We only authorize EOS tokens in the final +state.
+state + The current state of the guide.
+A Generate instance that contains the model and the allowed token ids.
outlines/fsm/guide.pyget_next_state(state, token_id)
+
+Update the state of the guide.
+We use the index to determine to which state the guide should transition +given the token that was just generated.
+state + The current state of the guide. +token_id + The id of the token that was just generated.
+The new state of the guide.
+ +outlines/fsm/guide.pyis_final_state(state)
+
+StopAtEOSGuide
+
+
+
+ Bases: Guide
Guide to generate tokens until the EOS token has been generated.
+ +outlines/fsm/guide.py__init__(tokenizer)
+
+Initialize the generation guide.
+model + The logit generator used to generate the next token.
+ +outlines/fsm/guide.pyWrite
+
+
+
+ dataclass
+
+
+Write instruction.
+tokens + The sequence of tokens to be added to the current sequence by the + generation process.
+ +outlines/fsm/guide.pycreate_states_mapping(regex_string, tokenizer, regex_parser=interegular.parse_pattern, frozen_tokens=[])
+
+Create the variables related to the mapping between states and tokens +The parameters of the function are used for caching purpose.
+regex_string: (str):
+ The regular expression string to generate a states mapping for.
+tokenizer: (Tokenizer):
+ The model's tokenizer.
+regex_parser: (Callable[[str], interegular.Pattern], optional):
+ A function that parses a regex string into an interegular Pattern object.
+frozen_tokens: (List[str], optional):
+ A list of tokens that should be kept as-is when expanding the token-level FSM
+ into a byte-level FSM. Defaults to an empty list.
states_to_token_maps: (Dict[int, Dict[int, int]]):
+ A mapping from states to a mapping from token ids originating from that state
+ to the next state to transition to given that token. The structure is as follows:
+ (origin_state -> (token_id -> next_state))
+empty_token_ids: (Set[int]):
+ A set of token ids that correspond to empty strings.
+final_states: (set):
+ A set of final states in the FSM.
outlines/fsm/guide.pybuild_regex_from_schema(schema, whitespace_pattern=None)
+
+Turn a JSON schema into a regex that matches any JSON object that follows + this schema.
+JSON Schema is a declarative language that allows to annotate JSON documents + with types and descriptions. These schemas can be generated from any Python + datastructure that has type annotation: namedtuples, dataclasses, Pydantic + models. And by ensuring that the generation respects the schema we ensure + that the output can be parsed into these objects. + This function parses the provided schema and builds a generation schedule which + mixes deterministic generation (fixed strings), and sampling with constraints.
+Parameters
+schema
+ A string that represents a JSON Schema.
+ whitespace_pattern
+ Pattern to use for JSON syntactic whitespace (doesn't impact string literals)
+ Example: allow only a single space or newline with whitespace_pattern=r"[
+]?"
Returns
+A generation schedule. A list of strings that represent the JSON + schema's structure and regular expression that define the structure of + the fields.
+References
+.. [0] JSON Schema. https://json-schema.org/
+ +outlines/fsm/json_schema.pyconvert_json_schema_to_str(json_schema)
+
+Convert a JSON schema to a string.
+json_schema + The JSON schema.
+str + The JSON schema converted to a string.
+ValueError + If the schema is not a dictionary, a string or a Pydantic class.
+ +outlines/fsm/json_schema.pyget_schema_from_signature(fn)
+
+Turn a function signature into a JSON schema.
+Every JSON object valid to the output JSON Schema can be passed
+to fn using the ** unpacking syntax.
outlines/fsm/json_schema.pyto_regex(resolver, instance, whitespace_pattern=None)
+
+Translate a JSON Schema instance into a regex that validates the schema.
+Note
+Many features of JSON schema are missing:
+ - Handle additionalProperties keyword
+ - Handle types defined as a list
+ - Handle constraints on numbers
+ - Handle special patterns: date, uri, etc.
This does not support recursive definitions.
+Parameters
+resolver
+ An object that resolves references to other instances within a schema
+ instance
+ The instance to translate
+ whitespace_pattern
+ Pattern to use for JSON syntactic whitespace (doesn't impact string literals)
+ Example: allow only a single space or newline with whitespace_pattern=r"[
+]?"
outlines/fsm/json_schema.py175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 +322 +323 +324 +325 +326 +327 +328 +329 +330 +331 +332 +333 +334 +335 +336 +337 +338 +339 +340 +341 +342 +343 +344 +345 +346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 +361 +362 +363 +364 +365 +366 +367 +368 +369 +370 +371 +372 +373 +374 +375 +376 +377 +378 +379 +380 +381 +382 +383 +384 +385 +386 +387 +388 +389 +390 +391 +392 +393 +394 +395 +396 +397 +398 +399 +400 +401 +402 +403 +404 +405 +406 +407 +408 +409 +410 +411 +412 +413 +414 +415 +416 +417 +418 +419 +420 +421 +422 +423 +424 +425 +426 +427 +428 +429 +430 +431 +432 +433 +434 +435 +436 +437 +438 +439 +440 +441 +442 +443 +444 +445 +446 +447 +448 +449 +450 +451 +452 +453 +454 +455 +456 +457 +458 +459 +460 +461 +462 +463 +464 +465 +466 +467 +468 +469 +470 +471 +472 +473 +474 +475 +476 +477 +478 +479 +480 +481 +482 +483 +484 +485 +486 +487 +488 +489 +490 +491 +492 +493 +494 +495 +496 +497 +498 +499 +500 +501 +502 +503 +504 +505 +506 +507 +508 +509 +510 +511 +512 +513 +514 +515 +516 +517 +518 +519 +520 +521 +522 +523 +524 | |
validate_quantifiers(min_bound, max_bound, start_offset=0)
+
+Ensures that the bounds of a number are valid. Bounds are used as quantifiers in the regex.
+min_bound + The minimum value that the number can take. +max_bound + The maximum value that the number can take. +start_offset + Number of elements that are already present in the regex but still need to be counted. + ex: if the regex is already "(-)?(0|[1-9][0-9])", we will always have at least 1 digit, so the start_offset is 1.
+min_bound + The minimum value that the number can take. +max_bound + The maximum value that the number can take.
+ValueError + If the minimum bound is greater than the maximum bound.
+TypeError or ValueError + If the minimum bound is not an integer or None. + or + If the maximum bound is not an integer or None.
+ +outlines/fsm/json_schema.pyTransformerTokenizer
+
+
+
+ Bases: Tokenizer
Represents a tokenizer for models in the transformers library.
outlines/models/transformers.py64 + 65 + 66 + 67 + 68 + 69 + 70 + 71 + 72 + 73 + 74 + 75 + 76 + 77 + 78 + 79 + 80 + 81 + 82 + 83 + 84 + 85 + 86 + 87 + 88 + 89 + 90 + 91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 | |
Transformers
+
+
+Represents a transformers model.
outlines/models/transformers.py129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 +322 +323 +324 +325 +326 +327 +328 +329 +330 +331 +332 +333 +334 +335 +336 +337 +338 +339 +340 +341 +342 +343 +344 +345 +346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 +361 +362 +363 +364 +365 +366 +367 +368 +369 +370 +371 +372 +373 +374 +375 +376 +377 +378 +379 +380 +381 +382 +383 | |
forward(input_ids, attention_mask, past_key_values=None)
+
+Compute a forward pass through the transformer model.
+input_ids + The input token ids. Must be one or two dimensional. +attention_mask + The attention mask. Must be one or two dimensional. +past_key_values + A tuple of tuples containing the cached key and value tensors for each + attention head.
+The computed logits and the new cached key and value tensors.
+ +outlines/models/transformers.pygenerate(prompts, generation_parameters, logits_processor, sampling_parameters)
+
+Generate text using transformers.
prompts
+ A prompt or list of prompts.
+generation_parameters
+ An instance of GenerationParameters that contains the prompt,
+ the maximum number of tokens, stop sequences and seed. All the
+ arguments to SequenceGeneratorAdapter's __cal__ method.
+logits_processor
+ The logits processor to use when generating text.
+sampling_parameters
+ An instance of SamplingParameters, a dataclass that contains
+ the name of the sampler to use and related parameters as available
+ in Outlines.
The generated text
+ +outlines/models/transformers.pystream(prompts, generation_parameters, logits_processor, sampling_parameters)
+
+Temporary stream stand-in which implements stream() signature +and equivalent behaviour but isn't yielded until generation completes.
+TODO: implement following completion of https://github.com/huggingface/transformers/issues/30810
+ +outlines/models/transformers.pyget_llama_tokenizer_types()
+
+Get all the Llama tokenizer types/classes that need work-arounds.
+When they can't be imported, a dummy class is created.
+ +outlines/models/transformers.pytransformers(model_name, device=None, model_kwargs={}, tokenizer_kwargs={}, model_class=None, tokenizer_class=None)
+
+Instantiate a model from the transformers library and its tokenizer.
model_name
+ The name of the model as listed on Hugging Face's model page.
+device
+ The device(s) on which the model should be loaded. This overrides
+ the device_map entry in model_kwargs when provided.
+model_kwargs
+ A dictionary that contains the keyword arguments to pass to the
+ from_pretrained method when loading the model.
+tokenizer_kwargs
+ A dictionary that contains the keyword arguments to pass to the
+ from_pretrained method when loading the tokenizer.
A TransformersModel model instance.
outlines/models/transformers.pyIntegration with OpenAI's API.
+ + + +OpenAI
+
+
+An object that represents the OpenAI API.
+ +outlines/models/openai.py69 + 70 + 71 + 72 + 73 + 74 + 75 + 76 + 77 + 78 + 79 + 80 + 81 + 82 + 83 + 84 + 85 + 86 + 87 + 88 + 89 + 90 + 91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 | |
__call__(prompt, max_tokens=None, stop_at=None, *, system_prompt=None, temperature=None, samples=None)
+
+Call the OpenAI API to generate text.
+prompt + A string or list of strings that will be used to prompt the model +max_tokens + The maximum number of tokens to generate +stop_at + A string or array of strings which, such that the generation stops + when they are generated. +system_prompt + The content of the system message that precedes the user's prompt. +temperature + The value of the temperature used to sample tokens +samples + The number of completions to generate for each prompt +stop_at + Up to 4 words where the API will stop the completion.
+ +outlines/models/openai.py__init__(client, config, system_prompt=None)
+
+Create an OpenAI instance.
This class supports the standard OpenAI API, the Azure OpeanAI API as +well as compatible APIs that rely on the OpenAI client.
+client
+ An instance of the API's async client.
+config
+ An instance of OpenAIConfig. Can be useful to specify some
+ parameters that cannot be set by calling this class' methods.
outlines/models/openai.pyOpenAIConfig
+
+
+
+ dataclass
+
+
+Represents the parameters of the OpenAI API.
+The information was last fetched on 2023/11/20. We document below the +properties that are specific to the OpenAI API. Not all these properties are +supported by Outlines.
+model
+ The name of the model. Available models can be found on OpenAI's website.
+frequence_penalty
+ Number between 2.0 and -2.0. Positive values penalize new tokens based on
+ their existing frequency in the text,
+logit_bias
+ Modifies the likelihood of specified tokens to appear in the completion.
+ Number between -100 (forbid) and +100 (only allows).
+n
+ The number of completions to return for each prompt.
+presence_penalty
+ Similar to frequency penalty.
+response_format
+ Specifies the format the model must output. {"type": "json_object"}
+ enables JSON mode.
+seed
+ Two completions with the same seed value should return the same
+ completion. This is however not guaranteed.
+stop
+ Up to 4 words where the API will stop the completion.
+temperature
+ Number between 0 and 2. Higher values make the output more random, while
+ lower values make it more deterministic.
+top_p
+ Number between 0 and 1. Parameter for nucleus sampling.
+user
+ A unique identifier for the end-user.
outlines/models/openai.pyerror_handler(api_call_fn)
+
+Handle OpenAI API errors and missing API key.
+ +outlines/models/openai.pygenerate_chat(prompt, system_prompt, client, config)
+
+
+ async
+
+
+Call OpenAI's Chat Completion API.
+prompt
+ The prompt we use to start the generation. Passed to the model
+ with the "user" role.
+system_prompt
+ The system prompt, passed to the model with the "system" role
+ before the prompt.
+client
+ The API client
+config
+ An OpenAIConfig instance.
A tuple that contains the model's response(s) and usage statistics.
+ +outlines/models/openai.pyPartialIndenter
+
+
+
+ Bases: Indenter
An Indenter that doesn't reset its state every time process is called.
outlines/fsm/parsing.pyPartialParserState
+
+
+
+ Bases: ParserState
outlines/fsm/parsing.py346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 +361 +362 +363 +364 +365 +366 +367 +368 +369 +370 +371 +372 +373 +374 +375 +376 +377 +378 +379 +380 +381 +382 +383 +384 +385 +386 +387 +388 +389 +390 +391 +392 +393 +394 +395 +396 +397 +398 +399 +400 +401 +402 +403 +404 +405 +406 +407 +408 +409 +410 +411 +412 +413 +414 +415 +416 +417 +418 +419 +420 +421 +422 +423 +424 +425 +426 +427 +428 +429 +430 +431 +432 +433 +434 +435 +436 +437 +438 +439 +440 +441 +442 +443 +444 +445 +446 +447 +448 +449 +450 +451 +452 +453 +454 +455 +456 +457 +458 +459 +460 +461 +462 +463 +464 +465 +466 +467 +468 +469 +470 +471 +472 +473 +474 +475 +476 +477 +478 +479 +480 +481 +482 +483 +484 +485 +486 +487 +488 +489 +490 +491 +492 +493 +494 +495 +496 +497 +498 +499 +500 +501 +502 +503 | |
accepts()
+
+Adapted from https://github.com/lark-parser/lark/blob/be542c2ff6d968817df019b8bf03f37b3111c08c/lark/parsers/lalr_interactive_parser.py#L95 +Returns the set of possible tokens that will advance the parser into a new valid state.
+ +outlines/fsm/parsing.pyfeed_token_no_stack(token, is_end=False)
+
+This is a copy of ParserState.feed_token with all the value stack
+steps removed. Since we're not exactly parsing in order to obtain a
+CST or anything similar, we can avoid the growing expense of tracking
+the parse tree.
outlines/fsm/parsing.pyPartialParsingFrontend
+
+
+
+ Bases: ParsingFrontend
outlines/fsm/parsing.py162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 | |
PartialScanner
+
+
+
+ Bases: Scanner
outlines/fsm/parsing.py550 +551 +552 +553 +554 +555 +556 +557 +558 +559 +560 +561 +562 +563 +564 +565 +566 +567 +568 +569 +570 +571 +572 +573 +574 +575 +576 +577 +578 +579 +580 +581 +582 +583 +584 +585 +586 +587 +588 +589 +590 +591 +592 +593 +594 +595 +596 +597 +598 +599 +600 +601 +602 +603 +604 +605 +606 +607 +608 +609 +610 +611 +612 +613 +614 +615 +616 +617 +618 +619 +620 +621 +622 +623 +624 +625 +626 +627 +628 +629 +630 +631 +632 | |
get_terminals_info(fsm_state_seq)
+
+Get the possible terminal symbols for an FSM state sequence.
+ +outlines/fsm/parsing.pymatch(text, pos, last_fsm_state_seq=None)
+
+Determine an FSM match over text starting at pos and continuing last_fsm_state_seq.
outlines/fsm/parsing.pyterminals_to_fsms(lp)
+
+Construct a dict mapping terminal symbol names to their finite state machines.
outlines/fsm/parsing.pyPrompt
+
+
+
+ dataclass
+
+
+Represents a prompt function.
+We return a Prompt class instead of a simple function so the
+template defined in prompt functions can be accessed.
outlines/prompts.py__call__(*args, **kwargs)
+
+Render and return the template.
+The rendered template as a Python str.
outlines/prompts.pyget_fn_args(fn)
+
+Returns the arguments of a function with annotations and default values if provided.
+ +outlines/prompts.pyget_fn_description(fn)
+
+Returns the first line of a callable's docstring.
+ +outlines/prompts.pyget_fn_name(fn)
+
+Returns the name of a callable.
+ +outlines/prompts.pyget_fn_signature(fn)
+
+Return the signature of a callable.
+ +outlines/prompts.pyget_fn_source(fn)
+
+Return the source code of a callable.
+ +outlines/prompts.pyget_schema_dict(model)
+
+get_schema_pydantic(model)
+
+Return the schema of a Pydantic model.
+ +outlines/prompts.pyparse_pydantic_schema(raw_schema, definitions)
+
+Parse the output of Basemodel.[schema|model_json_schema]().
This recursively follows the references to other schemas in case +of nested models. Other schemas are stored under the "definitions" +key in the schema of the top-level model.
+ +outlines/prompts.pyprompt(fn)
+
+Decorate a function that contains a prompt template.
+This allows to define prompts in the docstring of a function and simplify their
+manipulation by providing some degree of encapsulation. It uses the render
+function internally to render templates.
++++++import outlines
+@outlines.prompt +def build_prompt(question): +... "I have a ${question}" +... +prompt = build_prompt("How are you?")
+
This API can also be helpful in an "agent" context where parts of the prompt +are set when the agent is initialized and never modified later. In this situation +we can partially apply the prompt function at initialization.
+++++++import outlines +import functools as ft +... +@outlines.prompt +... def solve_task(name: str, objective: str, task: str): +... '''Your name is {{name}}. +.. Your overall objective is to {{objective}}. +... Please solve the following task: {{task}} +... ''' +... +hal = ft.partial(solve_task, "HAL", "Travel to Jupiter")
+
A Prompt callable class which will render the template when called.
outlines/prompts.pyrender(template, **values)
+
+Parse a Jinaj2 template and translate it into an Outlines graph.
+This function removes extra whitespaces and linebreaks from templates to +allow users to enter prompts more naturally than if they used Python's +constructs directly. See the examples for a detailed explanation.
+Outlines follow Jinja2's syntax
+++++++import outlines +outline = outlines.render("I like {{food}} and {{sport}}", food="tomatoes", sport="tennis") +I like tomatoes and tennis
+
If the first line of the template is empty, render removes it
++++++from outlines import render
+tpl = ''' +... A new string''' +tpl +... '\nA new string' +render(tpl) +... 'a new string'
+
Similarly, render ignores linebreaks introduced by placing the closing quotes
+underneath the text:
++++++tpl = ''' +... A new string +... ''' +tpl +... '\nA new string\n' +render(tpl) +... 'A new string'
+
If you want to insert a linebreak at the end of the rendered template, you will +need to leave an empty line at the end of the template:
+++++++tpl = ''' +... A new string +... +... ''' +tpl +... '\nA new string\n\n' +render(tpl) +... 'A new string\n'
+
render removes the identation in docstrings. This is particularly important
+when using prompt functions
++++++tpl = ''' +... a string +... and another string''' +tpl +... '\n a string\n and another string' +render(tpl) +... 'a string\nand another string'
+
The indentation of the first line is assumed to be the same as the second line's
+++++++tpl = '''a string +... and another''' +tpl +... 'a string\n and another' +render(tpl) +... 'a string\nand another'
+
To get a different indentation for the first and the second line, we can start the +prompt on the string's second line:
+++++++tpl = ''' +... First line +... Second line''' +render(tpl) +... 'First Line\n Second Line'
+
template + A string that contains a template written with the Jinja2 syntax. +**values + Map from the variables in the template to their value.
+A string that contains the rendered template.
+ +outlines/prompts.py94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 | |
regex(model, regex_str, sampler=multinomial())
+
+Generate structured text in the language of a regular expression.
+model:
+ An instance of Transformer that represents a model from the
+ transformers library.
+regex_str:
+ The regular expression that the output must follow.
+sampler:
+ The sampling algorithm to use to generate token ids from the logits
+ distribution.
A SequenceGeneratorAdapter instance that generates text constrained by the
+regular expression.
outlines/generate/regex.pyBeamSearchSampler
+
+
+Beam Search sampling algorithm.
+samples + The number of samples taken for each input sequence. Equivalent to the + number of beams.
+ +outlines/samplers.py247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 | |
__call__(next_token_logits, sequence_weights, _)
+
+Call the beam search sampler.
+next_token_logits
+ A tensor of shape (n_seqs, vocab_size,) that represents the
+ probability distribution of the next token over the vocabulary.
+sequence_weights
+ A tensor of shape (n_seqs,) that represents the cumulative
+ weight of each sequence.
+rng
+ A random number generator.
A tuple with an array that contains the ids of the sampled tokens of
+shape (n_seqs, 1), an array that contains the ancestors of each
+sampled id of shape (n_seqs,) and an array that contains the updated
+cumulative weights of each sequence of shape (n_seqs,).
outlines/samplers.py260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 | |
GreedySampler
+
+
+Greedy Sampling algorithm.
+Greedy sampling consists in choosing the token with the largest +likelihood at every step.
+We don't allow more than one sample. We could attribute this a meaning, for +instance the k-th sample represents the k-th most likely token. In which +case it would be equivalent to beam search without the sequence weights.
+samples + The number of samples taken for each input sequence.
+ +outlines/samplers.py__call__(next_token_logits, sequence_weights, _)
+
+Call the greedy sampler.
+next_token_logits
+ A tensor of shape (n_seqs, vocab_size,) that represents the
+ probability distribution of the next token over the vocabulary.
+sequence_weights
+ A tensor of shape (n_seqs,) that represents the cumulative
+ weight of each sequence.
+rng
+ A random number generator.
A tuple with an array that contains the ids of the sampled tokens of
+shape (n_seqs, 1), an array that contains the ancestors of each
+sampled id of shape (n_seqs,) and an array that contains the updated
+cumulative weights of each sequence of shape (n_seqs,).
outlines/samplers.pyMultinomialSampler
+
+
+Multinomial sampling algorithm.
+Multinomial sampling consists in randomly sampling the next token assuming +its distribution is a Categorical distribution parametrized by the +next-token logits.
+samples + The number of samples taken for each input sequence.
+ +outlines/samplers.py83 + 84 + 85 + 86 + 87 + 88 + 89 + 90 + 91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 | |
__call__(next_token_logits, sequence_weights, rng)
+
+Call the multinomial sampler.
+next_token_logits
+ A tensor of shape (n_seqs, vocab_size,) that represents the
+ probability distribution of the next token over the vocabulary.
+sequence_weights
+ A tensor of shape (n_seqs,) that represents the cumulative
+ weight of each sequence.
+rng
+ A random number generator.
A tuple with an array that contains the ids of the sampled tokens of
+shape (n_seqs, 1), an array that contains the ancestors of each
+sampled id of shape (n_seqs,) and an array that contains the updated
+cumulative weights of each sequence of shape (n_seqs,).
outlines/samplers.pykeep_top_k_logits(k)
+
+Build a function that masks logits values smaller than the top k ones.
k
+ The ranking below which logit values are replaced by -math.inf.
outlines/samplers.pykeep_top_p_logits(p)
+
+Build a function that masks the lowest probability tokens whose +cumulative probability is below a certain threshold.
+p
+ The value of the threshold. We keep the highest probability tokens whose
+ cumulative distribution is greater than or equal to p and mask the
+ others. Its value must be between 0 (excluded) and 1 (included).
outlines/samplers.pyrescale_logits(temperature)
+
+Build a function that rescales the token probabilities exponentially.
+temperature + The value by which we rescale the logits.
+ +outlines/samplers.pySCw?g1AZdtb{ZFl
z%_QRDDvV9d$zFv<9N43+R}q<=aJ<0cQRvq6Y>g57Nj0o)^pEM>={?FLD{Wf!R|_)5
ztd0|kul@rbYGAM>3Z1u}!9swXgv5kEC5*^a#?}+RI799q)hq(IB`%O7Pv^R?zQA$G
zpD&k0hs++Df%%M2pEM^E(-Ys)*>i-2VB?^tHt>@K5lWt+_fChi$l$&+TXpJ1hA#=%
zxagg4NtpL!(+*z77R?40Qehrg$dSzC`T!nNP!lKcZUltkGt(IYw8M{1l~0}pk05B9
zg{JfC!<=NMi9^JUhq(%mpz2x&uRA_&O6Q~skMR2k{*c3;em43u9kO8riQ9*?NY+IY
zDiI4fO%Ckqz2TDiCsltfo8{fd%$mVE>;pGCawuW3xPv!Gck?KeV6B|&oKj3eYOHXe
z2<-Qy8Co7!0JxFzRKx4D_G;-7qt<)V<4qn5i9cXUS2zh%T$;~i0g>;wM*-l+{l=EF
zY1oZ_#NLkhBW>>CkO#%(>$02{_D&%kFtb34cV` n)eKl*BiAN1Tp5@I|=#iLr)t
zVpgJ_)=S(njLO`vyp1<6s