This sample shows simple ways to interact with Azure OpenAI & GPT-4 model to build an interactive using Azure Functions Azure Open AI Triggers and Bindings extension. You can issue simple prompts and receive completions using the ask function, and you can send messages and perform a stateful session with a friendly ChatBot using the chats function. The app deploys easily to Azure Functions Flex Consumption hosting plan using azd up.
The easiest way to install Azurite is using a Docker container or the support built into Visual Studio:
docker run -d -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite- Once you have your Azure subscription, run the following in a new terminal window to create Azure OpenAI and other resources needed:
azd provisionTake note of the value of AZURE_OPENAI_ENDPOINT which can be found in ./.azure/<env name from azd provision>/.env. It will look something like:
AZURE_OPENAI_ENDPOINT="https://cog-<unique string>.openai.azure.com/"Alternatively you can create an OpenAI resource in the Azure portal to get your key and endpoint. After it deploys, click Go to resource and view the Endpoint value. You will also need to deploy a model, e.g. with name chat and model gpt-4o.
- Add this
local.settings.jsonfile to the root of the repo folder to simplify local development. ReplaceAZURE_OPENAI_ENDPOINTwith your value from step 4. Optionally you can choose a different model deployment inCHAT_MODEL_DEPLOYMENT_NAME. This file will be gitignored to protect secrets from committing to your repo, however by default the sample uses Entra identity (user identity and mananaged identity) so it is secretless.
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "python",
"AZURE_OPENAI_ENDPOINT": "https://cog-<unique string>.openai.azure.com/",
"CHAT_MODEL_DEPLOYMENT_NAME": "chat",
"AzureWebJobsFeatureFlags": "EnableWorkerIndexing",
"PYTHON_ISOLATE_WORKER_DEPENDENCIES": "1"
}
}- Open a new terminal and do the following:
pip3 install -r requirements.txt
func start- Using your favorite REST client, e.g. RestClient in VS Code, PostMan, curl, make a post. test.http has been provided to run this quickly.
Terminal:
curl -i -X POST http://localhost:7071/api/ask/ \
-H "Content-Type: text/json" \
--data-binary "@testdata.json"testdata.json
{
"prompt": "Write a poem about Azure Functions. Include two reasons why users love them."
}### Simple Ask Completion
POST http://localhost:7071/api/ask HTTP/1.1
content-type: application/json
{
"prompt": "Tell me two most popular programming features of Azure Functions"
}
### Simple Whois Completion
GET http://localhost:7071/api/whois/Turing HTTP/1.1We will use the test.http file again now focusing on the Chat function. We need to start the chat with chats and send messages with PostChat. We can also get state at any time with GetChatState.
### Stateful Chatbot
### CreateChatBot
PUT http://localhost:7071/api/chats/abc123
Content-Type: application/json
{
"name": "Sample ChatBot",
"description": "This is a sample chatbot."
}
### PostChat
POST http://localhost:7071/api/chats/abc123
Content-Type: application/json
{
"message": "Hello, how can I assist you today?"
}
### PostChat
POST http://localhost:7071/api/chats/abc123
Content-Type: application/json
{
"message": "Need help with directions from Redmond to SeaTac?"
}
### GetChatState
GET http://localhost:7071/api/chats/abc123?timestampUTC=2024-01-15T22:00:00
Content-Type: application/jsonThe easiest way to deploy this app is using the Azure Developer CLI. If you open this repo in GitHub CodeSpaces the AZD tooling is already preinstalled.
To provision and deploy:
azd upThe key code that makes the prompting and completion work is as follows in function_app.py. The /api/ask function and route expects a prompt to come in the POST body. The templating pattern is used to define the input binding for prompt and the underlying parameters for OpenAI models like the maxTokens and the AI model to use for chat.
The whois function expects a name to be sent in the route /api/whois/<name> and you get to see a different example of a route and parameter coming in via http GET.
app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)
# Simple ask http POST function that returns the completion based on prompt
# This OpenAI completion input requires a {prompt} value in json POST body
@app.function_name("ask")
@app.route(route="ask", methods=["POST"])
@app.text_completion_input(arg_name="response", prompt="{prompt}",
model="%CHAT_MODEL_DEPLOYMENT_NAME%")
def ask(req: func.HttpRequest, response: str) -> func.HttpResponse:
response_json = json.loads(response)
return func.HttpResponse(response_json["content"], status_code=200)
# Simple WhoIs http GET function that returns the completion based on name
# This OpenAI completion input requires a {name} binding value.
@app.function_name("whois")
@app.route(route="whois/{name}", methods=["GET"])
@app.text_completion_input(arg_name="response", prompt="Who is {name}?",
max_tokens="100",
model="%CHAT_MODEL_DEPLOYMENT_NAME%")
def whois(req: func.HttpRequest, response: str) -> func.HttpResponse:
response_json = json.loads(response)
return func.HttpResponse(response_json["content"], status_code=200)The stateful chatbot is shown in function_app.py routing to /api/chats. This is a stateful function meaning you can create or ask for a session by and continue where you left off with the same context and memories stored by the function binding (backed Table storage). This makes use of the Assistants feature of the Azure Functions OpenAI extension that has a set of inputs and outputs for this case.
To create or look up a session we have the CreateChatBot as an http PUT function. Note how the code will reuse your AzureWebJobStorage connection. The output binding of assistantCreate will actually kick off the create.
# http PUT function to start ChatBot conversation based on a chatID
@app.function_name("CreateChatBot")
@app.route(route="chats/{chatId}", methods=["PUT"])
@app.assistant_create_output(arg_name="requests")
def create_chat_bot(req: func.HttpRequest,
requests: func.Out[str]) -> func.HttpResponse:
chatId = req.route_params.get("chatId")
input_json = req.get_json()
logging.info(
f"Creating chat ${chatId} from input parameters " +
"${json.dumps(input_json)}")
create_request = {
"id": chatId,
"instructions": input_json.get("instructions"),
"chatStorageConnectionSetting": CHAT_STORAGE_CONNECTION,
"collectionName": COLLECTION_NAME
}
requests.set(json.dumps(create_request))
response_json = {"chatId": chatId}
return func.HttpResponse(json.dumps(response_json), status_code=202,
mimetype="application/json")Subsequent chat messages are sent to the chat as http POST, being careful to use the same chatId. This makes use of the inputAssistant input binding to take message as input and do the completion, while also automatically pulling context and memories for the session, and also saving your new state.
# http POST function for user to send a message to ChatBot with chatID
@app.function_name("PostUserResponse")
@app.route(route="chats/{chatId}", methods=["POST"])
@app.assistant_post_input(
arg_name="state", id="{chatId}",
user_message="{message}",
model="%CHAT_MODEL_DEPLOYMENT_NAME%",
chat_storage_connection_setting=CHAT_STORAGE_CONNECTION,
collection_name=COLLECTION_NAME
)
def post_user_response(req: func.HttpRequest, state: str) -> func.HttpResponse:
# Parse the JSON string into a dictionary
data = json.loads(state)
# Extract the content of the recentMessage
recent_message_content = data['recentMessages'][0]['content']
return func.HttpResponse(recent_message_content, status_code=200,
mimetype="text/plain")The test.http file is helpful to see how clients and APIs should call these functions, and to learn the typical flow.
You can customize this or learn more using Open AI Triggers and Bindings extension.