diff --git a/articles/ai-services/content-safety/how-to/improve-performance.md b/articles/ai-services/content-safety/how-to/improve-performance.md new file mode 100644 index 00000000000..9fa95859bf1 --- /dev/null +++ b/articles/ai-services/content-safety/how-to/improve-performance.md @@ -0,0 +1,86 @@ +--- +title: "Mitigate false results in Azure AI Content Safety" +titleSuffix: Azure AI services +description: Learn techniques to improve the performance of Azure AI Content Safety models by handling false positives and false negatives. +#services: cognitive-services +author: PatrickFarley +manager: nitinme +ms.service: azure-ai-content-safety +ms.topic: how-to +ms.date: 09/18/2024 +ms.author: pafarley +#customer intent: As a user, I want to improve the performance of Azure AI Content Safety so that I can ensure accurate content moderation. +--- + +# Mitigate false results in Azure AI Content Safety + +This guide provides a step-by-step process for handling false positives and false negatives from Azure AI Content Safety models. + +False positives are when the system incorrectly flags non-harmful content as harmful; false negatives are when harmful content is not flagged as harmful. Address these instances to ensure the integrity and reliability of your content moderation process, including responsible generative AI deployment. + +## Prerequisites + +* An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/) +* Once you have your Azure subscription, create a Content Safety resource in the Azure portal to get your key and endpoint. Enter a unique name for your resource, select your subscription, and select a resource group, supported region (see [Region availability](/azure/ai-services/content-safety/overview#region-availability)), and supported pricing tier. Then select **Create**. + +## Review and verification + +Conduct an initial assessment to confirm that the flagged content is really a false positive or false negative. This can involve: +- Checking the context of the flagged content. +- Comparing the flagged content against the content safety risk categories and severity definitions: + - If you're using content safety in Azure OpenAI, see the [Azure OpenAI content filtering doc](/azure/ai-services/openai/concepts/content-filter). + - If you're using the Azure AI Content Safety standalone API, see the [Harm categories doc](/azure/ai-services/content-safety/concepts/harm-categories?tabs=warning) or the [Prompt Shields doc](/azure/ai-services/content-safety/concepts/jailbreak-detection), depending on which API you're using. + +## Customize your severity settings + +If your assessment confirms that you found a false positive or false negative, you can try customizing your severity settings to mitigate the issue. The settings depend on which platform you're using. + +#### [Content Safety standalone API](#tab/standalone-api) + +If you're using the Azure AI Content Safety standalone API directly, try experimenting by setting the severity threshold at different levels for [harm categories](/azure/ai-services/content-safety/concepts/harm-categories?tabs=definitions) based on API output. Alternatively, if you prefer the no-code approach, you can try out those settings in [Content Safety Studio](https://contentsafety.cognitive.azure.com/) or Azure AI Studio’s [Content Safety page](https://ai.azure.com/explore/contentsafety). Instructions can be found [here](/azure/ai-studio/quickstarts/content-safety?tabs=moderate-text-content). + +In addition to adjusting the severity levels for false negatives, you can also use blocklists. More information on using blocklists for text moderation can be found in [Use blocklists for text moderation](/azure/ai-services/content-safety/how-to/use-blocklist?tabs=windows%2Crest). + + +#### [Azure OpenAI](#tab/azure-openai-studio) + +Read the [Configurability](/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new#configurability-preview) documentation, as some content filtering configurations may require approval through the process mentioned there. + +Follow the steps in the documentation to update configurations to handle false positives or negatives: [How to use content filters (preview) with Azure OpenAI Service](/azure/ai-services/openai/how-to/content-filters). + +In addition to adjusting the severity levels for false negatives, you can also use blocklists. Detailed instruction can be found in [How to use blocklists with Azure OpenAI Service](/azure/ai-services/openai/how-to/use-blocklists). + +#### [Azure AI Studio](#tab/azure-ai-studio) + +Read the [Configurability](/azure/ai-studio/concepts/content-filtering#configurability-preview) documentation, as some content filtering configurations may require approval through the process mentioned there. + +Follow the steps in the documentation to update configurations to handle false positives or negatives: [Azure AI Studio content filtering](/azure/ai-studio/concepts/content-filtering#create-a-content-filter). + +In addition to adjusting the severity levels for false negatives, you can also use blocklists. Detailed instruction can be found in [Azure AI Studio content filtering](/azure/ai-studio/concepts/content-filtering#use-a-blocklist-as-a-filter). + +--- + +## Create a custom category based on your own RAI policy + +Sometimes you might need to create a custom category to ensure the filtering aligns with your specific Responsible AI policy, as prebuilt categories or content filtering may not be enough. + +Refer to the [Custom categories documentation](/azure/ai-services/content-safety/concepts/custom-categories) to build your own categories with the Azure AI Content Safety API. + +## Document issues and send feedback to Azure + +If, after you’ve tried all the steps mentioned above, Azure AI Content Safety still can't resolve the false positives or negatives, there is likely a policy definition or model issue that needs further attention. + +Document the details of the false positives and/or false negatives by providing the following information to the [Content safety support team](mailto:contentsafetysupport@microsoft.com): +- Description of the flagged content. +- Context in which the content was posted. +- Reason given by Azure AI Content Safety for the flagging (if positive). +- Explanation of why the content is a false positive or negative. +- Any adjustments already attempted by adjusting severity settings or using custom categories. +- Screenshots or logs of the flagged content and system responses. + +This documentation helps in escalating the issue to the appropriate teams for resolution. + +## Related content + +- [Azure AI Content Safety overview](/azure/ai-services/content-safety/overview) +- [Harm categories](/azure/ai-services/content-safety/concepts/harm-categories?tabs=warning) \ No newline at end of file diff --git a/articles/ai-services/content-safety/toc.yml b/articles/ai-services/content-safety/toc.yml index 0ad9cbb3564..361b0184211 100644 --- a/articles/ai-services/content-safety/toc.yml +++ b/articles/ai-services/content-safety/toc.yml @@ -63,6 +63,8 @@ items: href: how-to/custom-categories-rapid.md - name: Use a blocklist href: how-to/use-blocklist.md + - name: Mitigate false results + href: how-to/improve-performance.md - name: Encryption of data at rest href: how-to/encrypt-data-at-rest.md - name: Migrate from public preview to GA diff --git a/articles/ai-services/openai/assistants-quickstart.md b/articles/ai-services/openai/assistants-quickstart.md index bc47d4b69ee..4fc98f0f10f 100644 --- a/articles/ai-services/openai/assistants-quickstart.md +++ b/articles/ai-services/openai/assistants-quickstart.md @@ -6,9 +6,9 @@ manager: nitinme ms.service: azure-ai-openai ms.custom: devx-track-python, devx-track-dotnet, devx-track-extended-java, devx-track-js ms.topic: quickstart -author: mrbullwinkle -ms.author: mbullwin -ms.date: 07/18/2024 +author: aahill +ms.author: aahi +ms.date: 09/18/2024 zone_pivot_groups: openai-quickstart-assistants recommendations: false --- @@ -18,15 +18,9 @@ recommendations: false Azure OpenAI Assistants (Preview) allows you to create AI assistants tailored to your needs through custom instructions and augmented by advanced tools like code interpreter, and custom functions. -::: zone pivot="programming-language-studio" - -[!INCLUDE [Azure OpenAI Studio quickstart](includes/assistants-studio.md)] - -::: zone-end - ::: zone pivot="programming-language-ai-studio" -[!INCLUDE [AI Studio (Preview)](includes/assistants-ai-studio.md)] +[!INCLUDE [AI Studio](includes/assistants-ai-studio.md)] ::: zone-end diff --git a/articles/ai-services/openai/how-to/quota.md b/articles/ai-services/openai/how-to/quota.md index c96a5c5a1dd..5d9a51369d9 100644 --- a/articles/ai-services/openai/how-to/quota.md +++ b/articles/ai-services/openai/how-to/quota.md @@ -44,13 +44,11 @@ The flexibility to distribute TPM globally within a subscription and region has When you create a model deployment, you have the option to assign Tokens-Per-Minute (TPM) to that deployment. TPM can be modified in increments of 1,000, and will map to the TPM and RPM rate limits enforced on your deployment, as discussed above. -To create a new deployment from within the Azure AI Studio under **Management** select **Deployments** > **Create new deployment**. - -The option to set the TPM is under the **Advanced options** drop-down: +To create a new deployment from within the Azure AI Studio under **Shared Resources** select **Deployments** > **Deploy model** > **Deploy base model** > **Select Model** > **Confirm**. :::image type="content" source="../media/quota/deployment.png" alt-text="Screenshot of the deployment UI of Azure AI Studio" lightbox="../media/quota/deployment.png"::: -Post deployment you can adjust your TPM allocation by selecting **Edit deployment** under **Management** > **Deployments** in Azure AI Studio. You can also modify this selection within the new quota management experience under **Management** > **Quotas**. +Post deployment you can adjust your TPM allocation by selecting **Edit** under **Shared resources** > **Deployments** in Azure OpenAI Studio. You can also modify this selection within the new quota management experience under **Management** > **Quotas**. > [!IMPORTANT] > Quotas and limits are subject to change, for the most up-date-information consult our [quotas and limits article](../quotas-limits.md). @@ -70,10 +68,10 @@ For an all up view of your quota allocations across deployments in a given regio :::image type="content" source="../media/quota/quota.png" alt-text="Screenshot of the quota UI of Azure AI Studio" lightbox="../media/quota/quota.png"::: -- **Quota Name**: There's one quota value per region for each model type. The quota covers all versions of that model. The quota name can be expanded in the UI to show the deployments that are using the quota. - **Deployment**: Model deployments divided by model class. -- **Usage/Limit**: For the quota name, this shows how much quota is used by deployments and the total quota approved for this subscription and region. This amount of quota used is also represented in the bar graph. -- **Request Quota**: The icon in this field navigates to a form where requests to increase quota can be submitted. +- **Quota type**: There's one quota value per region for each model type. The quota covers all versions of that model. +- **Quota allocation**: For the quota name, this shows how much quota is used by deployments and the total quota approved for this subscription and region. This amount of quota used is also represented in the bar graph. +- **Request Quota**: The icon navigates to a form where requests to increase quota can be submitted. ## Migrating existing deployments diff --git a/articles/ai-services/openai/how-to/working-with-models.md b/articles/ai-services/openai/how-to/working-with-models.md index 63794ed9e27..29dc6413682 100644 --- a/articles/ai-services/openai/how-to/working-with-models.md +++ b/articles/ai-services/openai/how-to/working-with-models.md @@ -20,7 +20,7 @@ You can get a list of models that are available for both inference and fine-tuni ## Model updates -Azure OpenAI now supports automatic updates for select model deployments. On models where automatic update support is available, a model version drop-down is visible in Azure OpenAI Studio under **Create new deployment** and **Edit deployment**: +Azure OpenAI now supports automatic updates for select model deployments. On models where automatic update support is available, a model version drop-down is visible in Azure OpenAI Studio under **Deployments** and **Edit**: :::image type="content" source="../media/models/auto-update.png" alt-text="Screenshot of the deploy model UI of Azure OpenAI Studio." lightbox="../media/models/auto-update.png"::: @@ -44,16 +44,10 @@ For currently deployed models, from Azure OpenAI Studio select **Deployments**: :::image type="content" source="../media/models/deployments.png" alt-text="Screenshot of the deployment UI of Azure OpenAI Studio." lightbox="../media/models/deployments.png"::: -To view retirement dates for all available models in a given region from Azure OpenAI Studio, select **Models** > **Column options** > Select **Deprecation fine tune** and **Deprecation inference**: - -:::image type="content" source="../media/models/column-options.png" alt-text="Screenshot of the models UI of Azure OpenAI Studio." lightbox="../media/models/column-options.png"::: - ## Model deployment upgrade configuration You can check what model upgrade options are set for previously deployed models in [Azure OpenAI Studio](https://oai.azure.com). Select **Deployments** > Under the deployment name column select one of the deployment names that are highlighted in blue. -:::image type="content" source="../media/how-to/working-with-models/deployments.png" alt-text="Screenshot of the deployments pane with a deployment name highlighted." lightbox="../media/how-to/working-with-models/deployments.png"::: - Selecting a deployment name opens the **Properties** for the model deployment. You can view what upgrade options are set for your deployment under **Version update policy**: :::image type="content" source="../media/how-to/working-with-models/update-policy.png" alt-text="Screenshot of the model deployments property UI." lightbox="../media/how-to/working-with-models/update-policy.png"::: diff --git a/articles/ai-services/openai/includes/assistants-ai-studio.md b/articles/ai-services/openai/includes/assistants-ai-studio.md index af766e2ef19..9e08c8b912c 100644 --- a/articles/ai-services/openai/includes/assistants-ai-studio.md +++ b/articles/ai-services/openai/includes/assistants-ai-studio.md @@ -22,6 +22,9 @@ author: mrbullwinkle ## Go to the Azure AI Studio (Preview) +> [!NOTE] +> While you can use [Azure OpenAI Studio](https://oai.azure.com/), we recommend using the steps here to use [Azure AI Studio](https://ai.azure.com). AI Studio lets you use Assistants v2 which provides several upgrades such as the [file search](../how-to/file-search.md) tool which is faster and supports more files. + 1. Sign in to [Azure AI Studio](https://ai.azure.com). 1. Go to your project or [create a new project](../../../ai-studio//how-to/create-projects.md) in Azure AI Studio. 1. From your project overview, select **Assistants**, located under **Project playground**. diff --git a/articles/ai-services/openai/includes/chatgpt-javascript.md b/articles/ai-services/openai/includes/chatgpt-javascript.md index c8ad17b9115..1b0573c117d 100644 --- a/articles/ai-services/openai/includes/chatgpt-javascript.md +++ b/articles/ai-services/openai/includes/chatgpt-javascript.md @@ -11,13 +11,25 @@ ms.author: mbullwin ms.date: 05/21/2024 --- -[Source code](https://github.com/openai/openai-node) | [Package (npm)](https://www.npmjs.com/package/openai) +[Source code](https://github.com/openai/openai-node) | [Package (npm)](https://www.npmjs.com/package/openai) | [Samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai/samples) > [!NOTE] > This article has been updated to use the [latest OpenAI npm package](https://www.npmjs.com/package/openai) which now fully supports Azure OpenAI. If you are looking for code examples for the legacy Azure OpenAI JavaScript SDK they are currently still [available in this repo](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai/samples/v2-beta/javascript). ## Prerequisites +## [**TypeScript**](#tab/typescript) + +- An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services?azure-portal=true) +- [LTS versions of Node.js](https://github.com/nodejs/release#release-schedule) +- [TypeScript](https://www.typescriptlang.org/download/) +- An Azure OpenAI Service resource with a `gpt-35-turbo` or `gpt-4` series models deployed. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). + +> [!div class="nextstepaction"] +> [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=JAVASCRIPT&Pillar=AOAI&Product=Chatgpt&Page=quickstart&Section=Prerequisites) + +## [**JavaScript**](#tab/javascript) + - An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services?azure-portal=true) - [LTS versions of Node.js](https://github.com/nodejs/release#release-schedule) - An Azure OpenAI Service resource with either a `gpt-35-turbo` or `gpt-4` series models deployed. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). @@ -25,6 +37,8 @@ ms.date: 05/21/2024 > [!div class="nextstepaction"] > [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=JAVASCRIPT&Pillar=AOAI&Product=Chatgpt&Page=quickstart&Section=Prerequisites) +--- + ## Set up [!INCLUDE [get-key-endpoint](get-key-endpoint.md)] @@ -52,6 +66,81 @@ Your app's _package.json_ file will be updated with the dependencies. Open a command prompt where you want the new project, and create a new file named ChatCompletion.js. Copy the following code into the ChatCompletion.js file. +## [**TypeScript**](#tab/typescript) + +```typescript +import "dotenv/config"; +import { AzureOpenAI } from "openai"; +import type { + ChatCompletion, + ChatCompletionCreateParamsNonStreaming, +} from "openai/resources/index"; + +// You will need to set these environment variables or edit the following values +const endpoint = process.env["AZURE_OPENAI_ENDPOINT"] || ""; +const apiKey = process.env["AZURE_OPENAI_API_KEY"] || ""; + +// Required Azure OpenAI deployment name and API version +const apiVersion = "2024-08-01-preview"; +const deploymentName = "gpt-4o-mini"; //This must match your deployment name. + +function getClient(): AzureOpenAI { + return new AzureOpenAI({ + endpoint, + apiKey, + apiVersion, + deployment: deploymentName, + }); +} + +function createMessages(): ChatCompletionCreateParamsNonStreaming { + return { + messages: [ + { role: "system", content: "You are a helpful assistant." }, + { + role: "user", + content: "Does Azure OpenAI support customer managed keys?", + }, + { + role: "assistant", + content: "Yes, customer managed keys are supported by Azure OpenAI?", + }, + { role: "user", content: "Do other Azure AI services support this too?" }, + ], + model: "", + }; +} +async function printChoices(completion: ChatCompletion): Promise { + for (const choice of completion.choices) { + console.log(choice.message); + } +} +export async function main() { + const client = getClient(); + const messages = createMessages(); + const result = await client.chat.completions.create(messages); + await printChoices(result); +} + +main().catch((err) => { + console.error("The sample encountered an error:", err); +}); +``` + +Build the script with the following command: + +```cmd +tsc +``` + +Run the script with the following command: + +```cmd +node.exe Completion.js +``` + +## [**JavaScript**](#tab/javascript) + ```javascript const { AzureOpenAI } = require("openai"); @@ -97,6 +186,8 @@ Run the script with the following command: node.exe ChatCompletion.js ``` +--- + ## Output ```output @@ -112,6 +203,78 @@ node.exe ChatCompletion.js > [!IMPORTANT] > In the previous example we are demonstrating key-based authentication. Once you have tested with key-based authentication successfully, we recommend using the more secure [Microsoft Entra ID](/entra/fundamentals/whatis) for authentication which is demonstrated in the next code sample. Getting started with [Microsoft Entra ID] will require some additional [prerequisites](https://www.npmjs.com/package/@azure/identity). +## [**TypeScript**](#tab/typescript) + +```typescript +import { + DefaultAzureCredential, + getBearerTokenProvider, +} from "@azure/identity"; +import "dotenv/config"; +import { AzureOpenAI } from "openai"; +import type { + ChatCompletion, + ChatCompletionCreateParamsNonStreaming, +} from "openai/resources/index"; + +// You will need to set these environment variables or edit the following values +const endpoint = process.env["AZURE_OPENAI_ENDPOINT"] || ""; + +// Required Azure OpenAI deployment name and API version +const apiVersion = "2024-08-01-preview"; +const deploymentName = "gpt-4o-mini"; //This must match your deployment name. + +function getClient(): AzureOpenAI { + const scope = "https://cognitiveservices.azure.com/.default"; + const azureADTokenProvider = getBearerTokenProvider( + new DefaultAzureCredential(), + scope + ); + return new AzureOpenAI({ + endpoint, + azureADTokenProvider, + deployment: deploymentName, + apiVersion, + }); +} + +function createMessages(): ChatCompletionCreateParamsNonStreaming { + return { + messages: [ + { role: "system", content: "You are a helpful assistant." }, + { + role: "user", + content: "Does Azure OpenAI support customer managed keys?", + }, + { + role: "assistant", + content: "Yes, customer managed keys are supported by Azure OpenAI?", + }, + { role: "user", content: "Do other Azure AI services support this too?" }, + ], + model: "", + }; +} +async function printChoices(completion: ChatCompletion): Promise { + for (const choice of completion.choices) { + console.log(choice.message); + } +} +export async function main() { + const client = getClient(); + const messages = createMessages(); + const result = await client.chat.completions.create(messages); + await printChoices(result); +} + +main().catch((err) => { + console.error("The sample encountered an error:", err); +}); +``` + + +## [**JavaScript**](#tab/javascript) + ```javascript const { AzureOpenAI } = require("openai"); const { DefaultAzureCredential, getBearerTokenProvider } = require("@azure/identity"); @@ -150,6 +313,9 @@ main().catch((err) => { module.exports = { main }; ``` + +--- + > [!NOTE] > If your receive the error: *Error occurred: OpenAIError: The `apiKey` and `azureADTokenProvider` arguments are mutually exclusive; only one can be passed at a time.* You may need to remove a pre-existing environment variable for the API key from your system. Even though the Microsoft Entra ID code sample is not explicitly referencing the API key environment variable, if one is present on the system executing this sample, this error will still be generated. diff --git a/articles/ai-services/openai/includes/chatgpt-studio.md b/articles/ai-services/openai/includes/chatgpt-studio.md index 394d07079cc..4755af64788 100644 --- a/articles/ai-services/openai/includes/chatgpt-studio.md +++ b/articles/ai-services/openai/includes/chatgpt-studio.md @@ -6,19 +6,13 @@ description: Walkthrough on how to get started with Azure OpenAI and make your f manager: nitinme ms.service: azure-ai-openai ms.topic: include -ms.date: 03/01/2023 +ms.date: 09/19/2024 --- ## Prerequisites - An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services?azure-portal=true). -- An Azure OpenAI Service resource with either the `gpt-35-turbo` or the `gpt-4` models deployed. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). - -> [!div class="nextstepaction"] -> [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=STUDIO&Pillar=AOAI&Product=Chatgpt&Page=quickstart&Section=Prerequisites) - -> [!TIP] -> Try out the new unified [Azure AI Studio (preview)](../../../ai-studio/what-is-ai-studio.md) which brings together capabilities from across multiple Azure AI services. +- An Azure OpenAI Service resource with either `gpt-4o` or the `gpt-4o-mini` models deployed. We recommend using standard or global standard model [deployment tpyes](../how-to/deployment-types.md) for initial exploration. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). ## Go to Azure OpenAI Studio @@ -34,82 +28,43 @@ Start exploring OpenAI capabilities with a no-code approach through the Azure Op :::image type="content" source="../media/quickstarts/chatgpt-playground-load.png" alt-text="Screenshot of the Chat playground page." lightbox="../media/quickstarts/chatgpt-playground-load.png"::: -### Assistant setup +### Setup -You can use the **Assistant setup** dropdown to select a few pre-loaded **System message** examples to get started. +You can use the **Prompt samples* dropdown to select a few pre-loaded **System message** examples to get started. **System messages** give the model instructions about how it should behave and any context it should reference when generating a response. You can describe the assistant's personality, tell it what it should and shouldn't answer, and tell it how to format responses. -**Add few-shot examples** allows you to provide conversational examples that are used by the model for [in-context learning](../concepts/prompt-engineering.md#basics). - At any time while using the Chat playground you can select **View code** to see Python, curl, and json code samples pre-populated based on your current chat session and settings selections. You can then take this code and write an application to complete the same task you're currently performing with the playground. ### Chat session -Selecting the **Send** button sends the entered text to the completions API and the results are returned back to the text box. +Selecting the **Enter** button or selecting the right arrow icon sends the entered text to the chat completions API and the results are returned back to the text box. Select the **Clear chat** button to delete the current conversation history. -### Settings +### Key settings | **Name** | **Description** | |:--------------------|:-------------------------------------------------------------------------------| | Deployments | Your deployment name that is associated with a specific model. | +| Add your data | +| Parameters | Custom parameters that alter the model responses. When you are starting out we recommend to use the defaults for most parameters | | Temperature | Controls randomness. Lowering the temperature means that the model produces more repetitive and deterministic responses. Increasing the temperature results in more unexpected or creative responses. Try adjusting temperature or Top P but not both. | -| Max length (tokens) | Set a limit on the number of tokens per model response. The API supports a maximum of 4096 tokens shared between the prompt (including system message, examples, message history, and user query) and the model's response. One token is roughly four characters for typical English text.| -| Top probabilities | Similar to temperature, this controls randomness but uses a different method. Lowering Top P narrows the model’s token selection to likelier tokens. Increasing Top P lets the model choose from tokens with both high and low likelihood. Try adjusting temperature or Top P but not both.| -| Multi-turn conversations | Select the number of past messages to include in each new API request. This helps give the model context for new user queries. Setting this number to 10 results in five user queries and five system responses.| +| Max response (tokens) | Set a limit on the number of tokens per model response. The API on the latest models supports a maximum of 128,000 tokens shared between the prompt (including system message, examples, message history, and user query) and the model's response. One token is roughly four characters for typical English text.| +| Top p | Similar to temperature, this controls randomness but uses a different method. Lowering Top P narrows the model’s token selection to likelier tokens. Increasing Top P lets the model choose from tokens with both high and low likelihood. Try adjusting temperature or Top P but not both.| | Stop sequences | Stop sequence make the model end its response at a desired point. The model response ends before the specified sequence, so it won't contain the stop sequence text. For GPT-35-Turbo, using `<|im_end|>` ensures that the model response doesn't generate a follow-up user query. You can include as many as four stop sequences.| -### Show panels - -By default there are three panels: assistant setup, chat session, and settings. **Show panels** allows you to add, remove, and rearrange the panels. If you ever close a panel and need to get it back, use **Show panels** to restore the lost panel. - -## Start a chat session - -1. From the assistant setup drop-down, select **Xbox customer support agent** -2. You'll be prompted asking if you want to update the system message, select **Continue**. -3. In the chat session pane, enter the following question: "I'm interested in buying a new Xbox", and select **Send**. -4. You'll receive a response similar to: +## View code - :::image type="content" source="../media/quickstarts/xbox.png" alt-text="Screenshot of a first question and answer in playground." lightbox="../media/quickstarts/xbox.png"::: +Once you have experimented with chatting with the model select the ** View Code** button. This will give you a replay of the code behind your entire conversation so far: -5. Enter a follow-up question like: "which models support 4K?" - - :::image type="content" source="../media/quickstarts/models.png" alt-text="Screenshot of follow-up question and answer in playground." lightbox="../media/quickstarts/models.png"::: - -6. Now that you have a basic conversation select **View code** from under **Assistant setup** and you'll have a replay of the code behind the entire conversation so far: - -```python -#Note: The openai-python library support for Azure OpenAI is in preview. -import os -import openai -openai.api_type = "azure" -openai.api_base = "https://docs-test-001.openai.azure.com/" -openai.api_version = "2024-02-01" -openai.api_key = os.getenv("OPENAI_API_KEY") - -response = openai.ChatCompletion.create( - engine="gpt-35-turbo", # replace this value with the deployment name you chose when you deployed the associated model. - messages = [{"role":"system","content":"You are an Xbox customer support agent whose primary goal is to help users with issues they are experiencing with their Xbox devices. You are friendly and concise. You only provide factual answers to queries, and do not provide answers that are not related to Xbox."},{"role":"user","content":"How much is a PS5?"},{"role":"assistant","content":"I apologize, but I do not have information about the prices of other gaming devices such as the PS5. My primary focus is to assist with issues regarding Xbox devices. Is there a specific issue you are having with your Xbox device that I may be able to help with?"},{"role":"user","content":"I am interested in buying an Xbox?"},{"role":"assistant","content":"Great! There are several Xbox models available in the market, including the Xbox Series X, Xbox Series S, and Xbox One. The Xbox Series X is the latest and most powerful console, while the Xbox Series S is a more affordable option that still offers great performance. The Xbox One is the previous generation console, but it still has a large library of games available to play.\n\nBefore purchasing an Xbox, you may want to consider factors such as your budget, the types of games you want to play, and whether you want to play games in 4K resolution. Once you have decided on a console, you can purchase it from a retailer or directly from the Xbox website. Let me know if you have any other questions or concerns!"},{"role":"user","content":"Which models are 4K?"},{"role":"assistant","content":"Both the Xbox Series X and Xbox One X support 4K resolution gaming. The Xbox Series S, on the other hand, supports up to 1440p resolution gaming, but it can upscale games to 4K. It's worth noting that in order to take advantage of 4K gaming, you'll need a 4K TV or monitor that supports HDR (High Dynamic Range) and HDMI 2.0 or higher."}], - temperature=0, - max_tokens=350, - top_p=0.95, - frequency_penalty=0, - presence_penalty=0, - stop=None) -``` - -> [!div class="nextstepaction"] -> [I ran into an issue with the playground.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=STUDIO&Pillar=AOAI&Product=Chatgpt&Page=quickstart&Section=Set-up) +:::image type="content" source="../media/quickstarts/chat-view-code.png" alt-text="Screenshot of view code experience." lightbox="../media/quickstarts/chat-view-code.png"::: ### Understanding the prompt structure -If you examine the sample from **View code** you'll notice some unique tokens that weren't part of a typical GPT completion call. GPT-35-Turbo was trained to use special tokens to delineate different parts of the prompt. Content is provided to the model in between `<|im_start|>` and `<|im_end|>` tokens. The prompt begins with a system message that can be used to prime the model by including context or instructions for the model. After that, the prompt contains a series of messages between the user and the assistant. - -The assistant's response to the prompt will then be returned below the `<|im_start|>assistant` token and will end with `<|im_end|>` denoting that the assistant has finished its response. You can also use the **Show raw syntax** toggle button to display these tokens within the chat session panel. +If you examine the sample from **View code** you'll notice that the conversation is broken into three distinct roles `system`, `user`, `assistant`. Each time you message the model the entire conversation history up to that point is resent. When using the chat completions API the model has no true memory of what you have sent to it in the past so you provide the conversation history for context to allow the model to respond properly. -The [GPT-35-Turbo & GPT-4 how-to guide](../how-to/chatgpt.md) provides an in-depth introduction into the new prompt structure and how to use the `gpt-35-turbo` model effectively. +The [Chat completions how-to guide](../how-to/chatgpt.md) provides an in-depth introduction into the new prompt structure and how to use chat completions models effectively. [!INCLUDE [deploy-web-app](deploy-web-app.md)] diff --git a/articles/ai-services/openai/includes/connect-your-data-studio.md b/articles/ai-services/openai/includes/connect-your-data-studio.md index 1fb53c5a9de..9b44eba63a9 100644 --- a/articles/ai-services/openai/includes/connect-your-data-studio.md +++ b/articles/ai-services/openai/includes/connect-your-data-studio.md @@ -21,6 +21,10 @@ Navigate to [Azure OpenAI Studio](https://oai.azure.com/) and sign-in with crede :::image type="content" source="../media/use-your-data/bring-your-data-card.png" alt-text="A screenshot of the Azure OpenAI Studio landing page." lightbox="../media/use-your-data/bring-your-data-card.png"::: +1. In the **Chat playground**, Select **Add your data** and then **Add a data source** + + :::image type="content" source="../media/use-your-data/chat-playground.png" alt-text="A screenshot of the chat playground in OpenAI Studio." lightbox="../media/use-your-data/chat-playground.png"::: + 1. In the pane that appears, select **Upload files (preview)** under **Select data source**. Azure OpenAI needs both a storage resource and a search resource to access and index your data. > [!TIP] diff --git a/articles/ai-services/openai/includes/dall-e-studio.md b/articles/ai-services/openai/includes/dall-e-studio.md index 19b208e8306..22e84e5fabc 100644 --- a/articles/ai-services/openai/includes/dall-e-studio.md +++ b/articles/ai-services/openai/includes/dall-e-studio.md @@ -35,24 +35,24 @@ Browse to [Azure OpenAI Studio](https://oai.azure.com/) and sign in with the cre #### [DALL-E 3](#tab/dalle3) -From the Azure OpenAI Studio landing page, select **DALL·E playground (Preview)** to use the image generation APIs. Select **Settings** near the top of the page and confirm that the **Deployment** dropdown has your DALL-E 3 deployment selected. +From the Azure OpenAI Studio landing page, select **Images playground** to use the image generation APIs. Select **Settings** near the top of the page and confirm that the **Deployment** dropdown has your DALL-E 3 deployment selected. #### [DALL-E 2 (preview)](#tab/dalle2) -From the Azure OpenAI Studio landing page, select **DALL·E playground (Preview)** to use the image generation APIs. Select **Settings** near the top of the page and confirm that the **Deployment** dropdown has the default **DALL-E 2** choice selected. +From the Azure OpenAI Studio landing page, select **Images playground** to use the image generation APIs. Select **Settings** near the top of the page and confirm that the **Deployment** dropdown has the default **DALL-E 2** choice selected. --- ## Try out image generation -Start exploring Azure OpenAI capabilities with a no-code approach through the **DALL·E playground (Preview)**. Enter your image prompt into the text box and select **Generate**. When the AI-generated image is ready, it appears on the page. +Start exploring Azure OpenAI capabilities with a no-code approach through the **Images playground**. Enter your image prompt into the text box and select **Generate**. When the AI-generated image is ready, it appears on the page. > [!NOTE] > The image generation APIs come with a content moderation filter. If Azure OpenAI recognizes your prompt as harmful content, it doesn't return a generated image. For more information, see [Content filtering](../concepts/content-filter.md). :::image type="content" source="../media/quickstarts/dall-e-studio.png" alt-text="Screenshot of the Azure OpenAI Studio landing page showing the DALL-E playground (Preview) with generated images of polar bears." lightbox="../media/quickstarts/dall-e-studio.png"::: -In the **DALL·E playground (Preview)**, you can also view Python and cURL code samples, which are prefilled according to your settings. Select **View code** near the top of the page. You can use this code to write an application that completes the same task. +In the **Images playground**, you can also view Python and cURL code samples, which are prefilled according to your settings. Select **View code** near the top of the page. You can use this code to write an application that completes the same task. ## Clean up resources diff --git a/articles/ai-services/openai/includes/javascript.md b/articles/ai-services/openai/includes/javascript.md index c8a423f9171..7b4714dd135 100644 --- a/articles/ai-services/openai/includes/javascript.md +++ b/articles/ai-services/openai/includes/javascript.md @@ -11,20 +11,34 @@ ms.author: mbullwin ms.date: 05/20/2024 --- -[Source code](https://github.com/openai/openai-node) | [Package (npm)](https://www.npmjs.com/package/openai) +[Source code](https://github.com/openai/openai-node) | [Package (npm)](https://www.npmjs.com/package/openai) | [Samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai/samples) > [!NOTE] > This article has been updated to use the [latest OpenAI npm package](https://www.npmjs.com/package/openai) which now fully supports Azure OpenAI. If you are looking for code examples for the legacy Azure OpenAI JavaScript SDK they are currently still [available in this repo](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai/samples/v2-beta/javascript). ## Prerequisites +## [**TypeScript**](#tab/typescript) + - An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services?azure-portal=true) - [LTS versions of Node.js](https://github.com/nodejs/release#release-schedule) +- [TypeScript](https://www.typescriptlang.org/download/) - An Azure OpenAI Service resource with the `gpt-35-turbo-instruct` model deployed. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). > [!div class="nextstepaction"] > [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=JAVASCRIPT&Pillar=AOAI&&Product=gpt&Page=quickstart&Section=Prerequisites) +## [**JavaScript**](#tab/javascript) + +- An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services?azure-portal=true) +- [LTS versions of Node.js](https://github.com/nodejs/release#release-schedule) +- An Azure OpenAI Service resource with the `gpt-35-turbo-instruct` model deployed. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). + +> [!div class="nextstepaction"] +> [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=JAVASCRIPT&Pillar=AOAI&&Product=gpt&Page=quickstart&Section=Prerequisites) + +--- + ## Set up [!INCLUDE [get-key-endpoint](get-key-endpoint.md)] @@ -50,6 +64,76 @@ Your app's _package.json_ file will be updated with the dependencies. Open a command prompt where you created the new project, and create a new file named Completion.js. Copy the following code into the Completion.js file. +## [**TypeScript**](#tab/typescript) + +```typescript +import "dotenv/config"; +import { AzureOpenAI } from "openai"; +import { type Completion } from "openai/resources/index"; + +// You will need to set these environment variables or edit the following values +const endpoint = process.env["AZURE_OPENAI_ENDPOINT"] || ""; +const apiKey = process.env["AZURE_OPENAI_API_KEY"] || ""; + +// Required Azure OpenAI deployment name and API version +const apiVersion = "2024-08-01-preview"; +const deploymentName = "gpt-35-turbo-instruct"; + +// Chat prompt and max tokens +const prompt = ["When was Microsoft founded?"]; +const maxTokens = 128; + +function getClient(): AzureOpenAI { + return new AzureOpenAI({ + endpoint, + apiKey, + apiVersion, + deployment: deploymentName, + }); +} +async function getCompletion( + client: AzureOpenAI, + prompt: string[], + max_tokens: number +): Promise { + return client.completions.create({ + prompt, + model: "", + max_tokens, + }); +} +async function printChoices(completion: Completion): Promise { + for (const choice of completion.choices) { + console.log(choice.text); + } +} +export async function main() { + console.log("== Get completions Sample =="); + + const client = getClient(); + const completion = await getCompletion(client, prompt, maxTokens); + await printChoices(completion); +} + +main().catch((err) => { + console.error("Error occurred:", err); +}); +``` + +Build the script with the following command: + +```cmd +tsc +``` + +Run the script with the following command: + +```cmd +node.exe Completion.js +``` + +## [**JavaScript**](#tab/javascript) + ```javascript const { AzureOpenAI } = require("openai"); @@ -90,6 +174,8 @@ Run the script with the following command: node.exe Completion.js ``` +--- + ## Output ```output @@ -103,6 +189,86 @@ Microsoft was founded on April 4, 1975. > [!IMPORTANT] > In the previous example we are demonstrating key-based authentication. Once you have tested with key-based authentication successfully, we recommend using the more secure [Microsoft Entra ID](/entra/fundamentals/whatis) for authentication which is demonstrated in the next code sample. Getting started with [Microsoft Entra ID] will require some additional [prerequisites](https://www.npmjs.com/package/@azure/identity). +## [**TypeScript**](#tab/typescript) + +```typescript +import { + DefaultAzureCredential, + getBearerTokenProvider, +} from "@azure/identity"; +import "dotenv/config"; +import { AzureOpenAI } from "openai"; +import { type Completion } from "openai/resources/index"; + +// You will need to set these environment variables or edit the following values +const endpoint = process.env["AZURE_OPENAI_ENDPOINT"] || ""; + +// Required Azure OpenAI deployment name and API version +const apiVersion = "2024-08-01-preview"; +const deploymentName = "gpt-35-turbo-instruct"; + +// Chat prompt and max tokens +const prompt = ["When was Microsoft founded?"]; +const maxTokens = 128; + +function getClient(): AzureOpenAI { + const scope = "https://cognitiveservices.azure.com/.default"; + const azureADTokenProvider = getBearerTokenProvider( + new DefaultAzureCredential(), + scope + ); + return new AzureOpenAI({ + endpoint, + azureADTokenProvider, + deployment: deploymentName, + apiVersion, + }); +} +async function getCompletion( + client: AzureOpenAI, + prompt: string[], + max_tokens: number +): Promise { + return client.completions.create({ + prompt, + model: "", + max_tokens, + }); +} +async function printChoices(completion: Completion): Promise { + for (const choice of completion.choices) { + console.log(choice.text); + } +} +export async function main() { + console.log("== Get completions Sample =="); + + const client = getClient(); + const completion = await getCompletion(client, prompt, maxTokens); + await printChoices(completion); +} + +main().catch((err) => { + console.error("Error occurred:", err); +}); + +``` + +Build the script with the following command: + +```cmd +tsc +``` + +Run the script with the following command: + +```cmd +node.exe Completion.js +``` + + +## [**JavaScript**](#tab/javascript) + ```javascript const { AzureOpenAI } = require("openai"); const { DefaultAzureCredential, getBearerTokenProvider } = require("@azure/identity"); @@ -136,6 +302,8 @@ main().catch((err) => { module.exports = { main }; ``` +--- + > [!NOTE] > If your receive the error: *Error occurred: OpenAIError: The `apiKey` and `azureADTokenProvider` arguments are mutually exclusive; only one can be passed at a time.* You may need to remove a pre-existing environment variable for the API key from your system. Even though the Microsoft Entra ID code sample is not explicitly referencing the API key environment variable, if one is present on the system executing this sample, this error will still be generated. diff --git a/articles/ai-services/openai/includes/studio.md b/articles/ai-services/openai/includes/studio.md index 569e0f0a621..718f87191dc 100644 --- a/articles/ai-services/openai/includes/studio.md +++ b/articles/ai-services/openai/includes/studio.md @@ -5,7 +5,7 @@ description: Walkthrough on how to get started with Azure OpenAI and make your f manager: nitinme ms.service: azure-ai-openai ms.topic: include -ms.date: 11/15/2023 +ms.date: 09/19/2023 --- ## Prerequisites @@ -13,12 +13,6 @@ ms.date: 11/15/2023 - An Azure subscription - Create one for free. - An Azure OpenAI resource with a model deployed. For more information about model deployment, see the [resource deployment guide](../how-to/create-resource.md). -> [!div class="nextstepaction"] -> [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=STUDIO&Pillar=AOAI&Product=gpt&Page=quickstart&Section=Prerequisites) - -> [!TIP] -> Try out the new unified [Azure AI Studio (preview)](../../../ai-studio/what-is-ai-studio.md) which brings together capabilities from across multiple Azure AI services. - ## Go to the Azure OpenAI Studio Navigate to Azure OpenAI Studio at https://oai.azure.com/ and sign-in with credentials that have access to your Azure OpenAI resource. During or after the sign-in workflow, select the appropriate directory, Azure subscription, and Azure OpenAI resource. @@ -27,8 +21,6 @@ From the Azure OpenAI Studio landing page navigate further to explore examples f :::image type="content" source="../media/quickstarts/studio-start.png" alt-text="Screenshot of the Azure OpenAI Studio landing page." lightbox="../media/quickstarts/studio-start.png"::: -Go to the [Playground](#playground) for experimentation and fine-tuning workflow. - ## Playground Start exploring Azure OpenAI capabilities with a no-code approach through the GPT-3 Playground. It's simply a text box where you can submit a prompt to generate a completion. From this page, you can quickly iterate and experiment with the capabilities. @@ -45,30 +37,23 @@ You can experiment with the configuration settings such as temperature and pre-r Azure OpenAI also performs content moderation on the prompt inputs and generated outputs. The prompts or responses may be filtered if harmful content is detected. For more information, see the [content filter](../concepts/content-filter.md) article. -In the GPT-3 playground you can also view Python and curl code samples pre-filled according to your selected settings. Just select **View code** next to the examples dropdown. You can write an application to complete the same task with the OpenAI Python SDK, curl, or other REST API client. +In the Completions playground you can also view Python and curl code samples pre-filled according to your selected settings. Just select **View code** next to the examples dropdown. You can write an application to complete the same task with the OpenAI Python SDK, curl, or other REST API client. ### Try text summarization -To use the Azure OpenAI for text summarization in the GPT-3 Playground, follow these steps: +To use the Azure OpenAI for text summarization in the Completions playground, follow these steps: 1. Sign in to [Azure OpenAI Studio](https://oai.azure.com). 1. Select the subscription and OpenAI resource to work with. -1. Select **GPT-3 Playground** at the top of the landing page. +1. Select **Completions playground** on the landing page. 1. Select your deployment from the **Deployments** dropdown. If your resource doesn't have a deployment, select **Create a deployment** and then revisit this step. -1. Select **Summarize Text** from the **Examples** dropdown. +1. Enter a prompt for the model. - :::image type="content" source="../media/quickstarts/summarize-text.png" alt-text="Screenshot of the playground page of the Azure OpenAI Studio with the Summarize Text dropdown selection visible" lightbox="../media/quickstarts/summarize-text.png"::: + :::image type="content" source="../media/quickstarts/summarize-text.png" alt-text="Screenshot of the playground page of the Azure OpenAI Studio with a text summarization example." lightbox="../media/quickstarts/summarize-text.png"::: 1. Select `Generate`. Azure OpenAI will attempt to capture the context of text and rephrase it succinctly. You should get a result that resembles the following text: - ``` - Tl;dr A neutron star is the collapsed core of a supergiant star. These incredibly dense objects are incredibly fascinating due to their strange properties and their potential for phenomena such as extreme gravitational forces and a strong magnetic field. - ``` - -The accuracy of the response can vary per model. The Davinci based model in this example is well-suited to this type of summarization, whereas a Codex based model wouldn't perform as well at this particular task. - -> [!div class="nextstepaction"] -> [I ran into an issue with the playground.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=STUDIO&Pillar=AOAI&Product=gpt&Page=quickstart&Section=Set-up) +The accuracy of the response can vary per model. The `gpt-35-turbo-instruct` based model in this example is well-suited to this type of summarization, though in general we recommend using the alternate chat completions API unless you have a particular use case that is particularly suited to the completions API. ## Clean up resources diff --git a/articles/ai-services/openai/media/deployment-types/deploy-models.png b/articles/ai-services/openai/media/deployment-types/deploy-models.png index fb93d4d8709..2ae28949af4 100644 Binary files a/articles/ai-services/openai/media/deployment-types/deploy-models.png and b/articles/ai-services/openai/media/deployment-types/deploy-models.png differ diff --git a/articles/ai-services/openai/media/how-to/dynamic-quota/dynamic-quota.png b/articles/ai-services/openai/media/how-to/dynamic-quota/dynamic-quota.png index 89e7fc8f244..f24640dc7d2 100644 Binary files a/articles/ai-services/openai/media/how-to/dynamic-quota/dynamic-quota.png and b/articles/ai-services/openai/media/how-to/dynamic-quota/dynamic-quota.png differ diff --git a/articles/ai-services/openai/media/how-to/working-with-models/update-policy.png b/articles/ai-services/openai/media/how-to/working-with-models/update-policy.png index f6a11e7ec42..7530404849c 100644 Binary files a/articles/ai-services/openai/media/how-to/working-with-models/update-policy.png and b/articles/ai-services/openai/media/how-to/working-with-models/update-policy.png differ diff --git a/articles/ai-services/openai/media/models/auto-update.png b/articles/ai-services/openai/media/models/auto-update.png index 545ad039495..837847c56ee 100644 Binary files a/articles/ai-services/openai/media/models/auto-update.png and b/articles/ai-services/openai/media/models/auto-update.png differ diff --git a/articles/ai-services/openai/media/models/column-options.png b/articles/ai-services/openai/media/models/column-options.png deleted file mode 100644 index 85d03eff25b..00000000000 Binary files a/articles/ai-services/openai/media/models/column-options.png and /dev/null differ diff --git a/articles/ai-services/openai/media/models/deployments.png b/articles/ai-services/openai/media/models/deployments.png index d5389f2dc72..07e0090e4f2 100644 Binary files a/articles/ai-services/openai/media/models/deployments.png and b/articles/ai-services/openai/media/models/deployments.png differ diff --git a/articles/ai-services/openai/media/quickstarts/assistant-session.png b/articles/ai-services/openai/media/quickstarts/assistant-session.png deleted file mode 100644 index a9789f81da0..00000000000 Binary files a/articles/ai-services/openai/media/quickstarts/assistant-session.png and /dev/null differ diff --git a/articles/ai-services/openai/media/quickstarts/assistants-playground.png b/articles/ai-services/openai/media/quickstarts/assistants-playground.png deleted file mode 100644 index 1fdc44c869d..00000000000 Binary files a/articles/ai-services/openai/media/quickstarts/assistants-playground.png and /dev/null differ diff --git a/articles/ai-services/openai/media/quickstarts/assistants-studio.png b/articles/ai-services/openai/media/quickstarts/assistants-studio.png deleted file mode 100644 index 679e04ff599..00000000000 Binary files a/articles/ai-services/openai/media/quickstarts/assistants-studio.png and /dev/null differ diff --git a/articles/ai-services/openai/media/quickstarts/chat-playground.png b/articles/ai-services/openai/media/quickstarts/chat-playground.png index e50a1fd52a1..75c8f7e8515 100644 Binary files a/articles/ai-services/openai/media/quickstarts/chat-playground.png and b/articles/ai-services/openai/media/quickstarts/chat-playground.png differ diff --git a/articles/ai-services/openai/media/quickstarts/chat-view-code.png b/articles/ai-services/openai/media/quickstarts/chat-view-code.png new file mode 100644 index 00000000000..3573b2f6e78 Binary files /dev/null and b/articles/ai-services/openai/media/quickstarts/chat-view-code.png differ diff --git a/articles/ai-services/openai/media/quickstarts/chatgpt-playground-load.png b/articles/ai-services/openai/media/quickstarts/chatgpt-playground-load.png index dd8dedbf2e1..06f9ee2607d 100644 Binary files a/articles/ai-services/openai/media/quickstarts/chatgpt-playground-load.png and b/articles/ai-services/openai/media/quickstarts/chatgpt-playground-load.png differ diff --git a/articles/ai-services/openai/media/quickstarts/chatgpt-playground.png b/articles/ai-services/openai/media/quickstarts/chatgpt-playground.png index 93ab8ff6ba9..6ab12f084f1 100644 Binary files a/articles/ai-services/openai/media/quickstarts/chatgpt-playground.png and b/articles/ai-services/openai/media/quickstarts/chatgpt-playground.png differ diff --git a/articles/ai-services/openai/media/quickstarts/dall-e-studio.png b/articles/ai-services/openai/media/quickstarts/dall-e-studio.png index ca86ea2d1a8..07f6c01a958 100644 Binary files a/articles/ai-services/openai/media/quickstarts/dall-e-studio.png and b/articles/ai-services/openai/media/quickstarts/dall-e-studio.png differ diff --git a/articles/ai-services/openai/media/quickstarts/playground-load.png b/articles/ai-services/openai/media/quickstarts/playground-load.png index b4a6d9f3a33..b9d9428d38e 100644 Binary files a/articles/ai-services/openai/media/quickstarts/playground-load.png and b/articles/ai-services/openai/media/quickstarts/playground-load.png differ diff --git a/articles/ai-services/openai/media/quickstarts/studio-start.png b/articles/ai-services/openai/media/quickstarts/studio-start.png index 36be83dc0a4..d8959ff89d9 100644 Binary files a/articles/ai-services/openai/media/quickstarts/studio-start.png and b/articles/ai-services/openai/media/quickstarts/studio-start.png differ diff --git a/articles/ai-services/openai/media/quickstarts/summarize-text.png b/articles/ai-services/openai/media/quickstarts/summarize-text.png index d6d80acbf8d..88c79523d5f 100644 Binary files a/articles/ai-services/openai/media/quickstarts/summarize-text.png and b/articles/ai-services/openai/media/quickstarts/summarize-text.png differ diff --git a/articles/ai-services/openai/media/quota/deployment.png b/articles/ai-services/openai/media/quota/deployment.png index 6ed7c92e218..29e242a6e1d 100644 Binary files a/articles/ai-services/openai/media/quota/deployment.png and b/articles/ai-services/openai/media/quota/deployment.png differ diff --git a/articles/ai-services/openai/media/quota/quota.png b/articles/ai-services/openai/media/quota/quota.png index 6b6af2b57a0..d3e1fa65ce1 100644 Binary files a/articles/ai-services/openai/media/quota/quota.png and b/articles/ai-services/openai/media/quota/quota.png differ diff --git a/articles/ai-services/openai/media/use-your-data/bring-your-data-card.png b/articles/ai-services/openai/media/use-your-data/bring-your-data-card.png index 2ba6df5a9e6..423d53e0d90 100644 Binary files a/articles/ai-services/openai/media/use-your-data/bring-your-data-card.png and b/articles/ai-services/openai/media/use-your-data/bring-your-data-card.png differ diff --git a/articles/ai-services/openai/media/use-your-data/chat-playground.png b/articles/ai-services/openai/media/use-your-data/chat-playground.png new file mode 100644 index 00000000000..deb2435f882 Binary files /dev/null and b/articles/ai-services/openai/media/use-your-data/chat-playground.png differ diff --git a/articles/ai-services/openai/quickstart.md b/articles/ai-services/openai/quickstart.md index c9768db7a8e..2687e636728 100644 --- a/articles/ai-services/openai/quickstart.md +++ b/articles/ai-services/openai/quickstart.md @@ -5,7 +5,7 @@ description: Walkthrough on how to get started with Azure OpenAI and make your f #services: cognitive-services manager: nitinme ms.service: azure-ai-openai -ms.custom: devx-track-dotnet, devx-track-python, devx-track-extended-java, devx-track-js, devx-track-go +ms.custom: devx-track-dotnet, devx-track-python, devx-track-extended-java, devx-track-js, devx-track-go, devx-track-ts ms.topic: quickstart author: mrbullwinkle ms.author: mbullwin diff --git a/articles/ai-services/openai/use-your-data-quickstart.md b/articles/ai-services/openai/use-your-data-quickstart.md index 6e1fe636c24..c144b8d6b6d 100644 --- a/articles/ai-services/openai/use-your-data-quickstart.md +++ b/articles/ai-services/openai/use-your-data-quickstart.md @@ -16,33 +16,71 @@ zone_pivot_groups: openai-use-your-data # Quickstart: Chat with Azure OpenAI models using your own data -::: zone pivot="programming-language-spring" +In this quickstart, you can use your own data with Azure OpenAI models. Using Azure OpenAI's models on your data can provide you with a powerful conversational AI platform that enables faster and more accurate communication. -[Source code](https://github.com/spring-projects-experimental/spring-ai)| [Source code](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai) | [Sample](https://github.com/rd-1-2022/ai-azure-retrieval-augmented-generation) +::: zone pivot="programming-language-studio" -::: zone-end -::: zone pivot="programming-language-javascript" +## Prerequisites -[Reference documentation](https://platform.openai.com/docs/api-reference/chat) | [Source code](https://github.com/openai/openai-node) | [Package (npm)](https://www.npmjs.com/package/openai) | [Samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai/samples) +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. + +> [!div class="nextstepaction"] +> [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=OVERVIEW&Pillar=AOAI&Product=ownData&Page=quickstart&Section=Prerequisites) + + +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] + +[!INCLUDE [Studio quickstart](includes/use-your-data-studio.md)] ::: zone-end -::: zone pivot="programming-language-python" +::: zone pivot="programming-language-csharp" -[Reference](https://platform.openai.com/docs/api-reference?lang=python) | [Source code](https://github.com/openai/openai-python) | [Package (pypi)](https://pypi.org/project/openai/) | [Samples](https://github.com/openai/openai-cookbook/) +## Prerequisites -These links reference the OpenAI API for Python. There's no Azure-specific OpenAI Python SDK. [Learn how to switch between the OpenAI services and Azure OpenAI services](/azure/ai-services/openai/how-to/switching-endpoints). +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. +- The [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0) + +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] + +[!INCLUDE [Csharp quickstart](includes/use-your-data-dotnet.md)] ::: zone-end -::: zone pivot="programming-language-go" +::: zone pivot="programming-language-spring" -[Reference](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go) | [Source code](https://github.com/Azure/azure-sdk-for-go) | [Package (Go)](https://pkg.go.dev/github.com/azure/azure-dev) | [Samples](https://github.com/azure-samples/azure-sdk-for-go-samples) +[Source code](https://github.com/spring-projects-experimental/spring-ai)| [Source code](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai) | [Sample](https://github.com/rd-1-2022/ai-azure-retrieval-augmented-generation) + +## Prerequisites + +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. + +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] + +[!INCLUDE [Spring quickstart](includes/use-your-data-spring.md)] ::: zone-end -In this quickstart, you can use your own data with Azure OpenAI models. Using Azure OpenAI's models on your data can provide you with a powerful conversational AI platform that enables faster and more accurate communication. +::: zone pivot="programming-language-javascript" ## Prerequisites @@ -59,6 +97,7 @@ In this quickstart, you can use your own data with Azure OpenAI models. Using Az - Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. + #### [JavaScript](#tab/javascript) - An Azure subscription - Create one for free. @@ -71,57 +110,72 @@ In this quickstart, you can use your own data with Azure OpenAI models. Using Az --- -::: zone pivot="programming-language-javascript" +[Reference documentation](https://platform.openai.com/docs/api-reference/chat) | [Source code](https://github.com/openai/openai-node) | [Package (npm)](https://www.npmjs.com/package/openai) | [Samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/openai/openai/samples) +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] -- [Long Term Support (LTS) versions of Node.js](https://github.com/nodejs/release#release-schedule) +[!INCLUDE [JavaScript quickstart](includes/use-your-data-javascript.md)] ::: zone-end -> [!div class="nextstepaction"] -> [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?PLanguage=OVERVIEW&Pillar=AOAI&Product=ownData&Page=quickstart&Section=Prerequisites) - +::: zone pivot="programming-language-python" -[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] +## Prerequisites -::: zone pivot="programming-language-studio" +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. -[!INCLUDE [Studio quickstart](includes/use-your-data-studio.md)] +[Reference](https://platform.openai.com/docs/api-reference?lang=python) | [Source code](https://github.com/openai/openai-python) | [Package (pypi)](https://pypi.org/project/openai/) | [Samples](https://github.com/openai/openai-cookbook/) -::: zone-end +These links reference the OpenAI API for Python. There's no Azure-specific OpenAI Python SDK. [Learn how to switch between the OpenAI services and Azure OpenAI services](/azure/ai-services/openai/how-to/switching-endpoints). -::: zone pivot="programming-language-csharp" +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] -[!INCLUDE [Csharp quickstart](includes/use-your-data-dotnet.md)] +[!INCLUDE [Python quickstart](includes/use-your-data-python.md)] ::: zone-end -::: zone pivot="programming-language-spring" +::: zone pivot="programming-language-powershell" -[!INCLUDE [Spring quickstart](includes/use-your-data-spring.md)] -::: zone-end +## Prerequisites -::: zone pivot="programming-language-javascript" +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. -[!INCLUDE [JavaScript quickstart](includes/use-your-data-javascript.md)] +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] -::: zone-end +[!INCLUDE [PowerShell quickstart](includes/use-your-data-powershell.md)] -::: zone pivot="programming-language-python" +::: zone-end -[!INCLUDE [Python quickstart](includes/use-your-data-python.md)] +::: zone pivot="programming-language-go" -::: zone-end -::: zone pivot="programming-language-powershell" +## Prerequisites -[!INCLUDE [PowerShell quickstart](includes/use-your-data-powershell.md)] +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. -::: zone-end +[Reference](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go) | [Source code](https://github.com/Azure/azure-sdk-for-go) | [Package (Go)](https://pkg.go.dev/github.com/azure/azure-dev) | [Samples](https://github.com/azure-samples/azure-sdk-for-go-samples) -::: zone pivot="programming-language-go" +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] [!INCLUDE [Go quickstart](includes/use-your-data-go.md)] @@ -129,6 +183,19 @@ In this quickstart, you can use your own data with Azure OpenAI models. Using Az ::: zone pivot="rest-api" + +## Prerequisites + +The following resources: +- [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) +- [Azure Blob Storage](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) +- [Azure AI Search](https://portal.azure.com/#create/Microsoft.Search) +- An [Azure OpenAI resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI) deployed in a [supported region and with a supported model](./concepts/use-your-data.md#regional-availability-and-model-support). + - Be sure that you're assigned at least the [Cognitive Services Contributor](./how-to/role-based-access-control.md#cognitive-services-contributor) role for the Azure OpenAI resource. +- Download the example data from [GitHub](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/openai/contoso_benefits_document_example.pdf) if you don't have your own data. + +[!INCLUDE [Connect your data to OpenAI](includes/connect-your-data-studio.md)] + [!INCLUDE [REST API quickstart](includes/use-your-data-rest.md)] ::: zone-end diff --git a/articles/ai-services/openai/whats-new.md b/articles/ai-services/openai/whats-new.md index e8754d41ef1..57cdcec44ed 100644 --- a/articles/ai-services/openai/whats-new.md +++ b/articles/ai-services/openai/whats-new.md @@ -10,7 +10,7 @@ ms.custom: - ignite-2023 - references_regions ms.topic: whats-new -ms.date: 9/12/2024 +ms.date: 9/19/2024 recommendations: false --- @@ -20,6 +20,11 @@ This article provides a summary of the latest releases and major documentation u ## September 2024 +### Azure OpenAI Studio UX updates + +On September 19, when you access the [Azure OpenAI Studio](https://oai.azure.com/) you'll begin to no longer see the legacy studio UI by default. If needed you'll still be able to go back to the previous experience by using the **Switch to the old look** toggle in the top bar of the UI for the next couple of weeks. If you switch back to legacy Studio UI, it helps if you fill out the feedback form to let us know why. We're actively monitoring this feedback to improve the new experience. + + ### GPT-4o 2024-08-06 provisioned deployments GPT-4o 2024-08-06 is now available for provisioned deployments in East US, East US 2, North Central US, and Sweden Central. It is also available for global provisioned deployments. diff --git a/articles/ai-services/speech-service/how-to-configure-openssl-linux.md b/articles/ai-services/speech-service/how-to-configure-openssl-linux.md index a41d85b49ee..01e53b780cb 100644 --- a/articles/ai-services/speech-service/how-to-configure-openssl-linux.md +++ b/articles/ai-services/speech-service/how-to-configure-openssl-linux.md @@ -1,15 +1,17 @@ --- title: How to configure OpenSSL for Linux titleSuffix: Azure AI services -description: Learn how to configure OpenSSL for Linux. -author: jhakulin +description: In this guide, you learn how to configure OpenSSL for Linux with the Azure AI Speech SDK. +author: eric-urban +ms.author: eur manager: nitinme ms.service: azure-ai-speech ms.custom: devx-track-extended-java, devx-track-go, devx-track-python, linux-related-content ms.topic: how-to -ms.date: 1/18/2024 -ms.author: jhakulin +ms.date: 9/19/2024 +ms.reviewer: jhakulin zone_pivot_groups: programming-languages-set-three +#Customer intent: As a developer, I want to learn how to configure OpenSSL for Linux so that I can use the Speech SDK on my Linux system. --- # Configure OpenSSL for Linux @@ -19,7 +21,7 @@ With the Speech SDK, [OpenSSL](https://www.openssl.org) is dynamically configure > [!NOTE] > This article is only applicable where the Speech SDK is [supported on Linux](speech-sdk.md#supported-languages). -To ensure connectivity, verify that OpenSSL certificates have been installed in your system. Run a command: +To ensure connectivity, verify that OpenSSL certificates are installed in your system. Run a command: ```bash openssl version -d ``` @@ -29,7 +31,7 @@ The output on Ubuntu/Debian based systems should be: OPENSSLDIR: "/usr/lib/ssl" ``` -Check whether there's a `certs` subdirectory under OPENSSLDIR. In the example above, it would be `/usr/lib/ssl/certs`. +Check whether there's a `certs` subdirectory under OPENSSLDIR. In the previous example, it would be `/usr/lib/ssl/certs`. * If the `/usr/lib/ssl/certs` exists, and if it contains many individual certificate files (with `.crt` or `.pem` extension), there's no need for further actions. @@ -52,9 +54,9 @@ export SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt ## Certificate revocation checks -When the Speech SDK connects to the Speech service, it checks the Transport Layer Security (TLS/SSL) certificate. The Speech SDK verifies that the certificate reported by the remote endpoint is trusted and hasn't been revoked. This verification provides a layer of protection against attacks involving spoofing and other related vectors. The check is accomplished by retrieving a certificate revocation list (CRL) from a certificate authority (CA) used by Azure. A list of Azure CA download locations for updated TLS CRLs can be found in [this document](/azure/security/fundamentals/tls-certificate-changes). +When the Speech SDK connects to the Speech service, it checks the Transport Layer Security (TLS/SSL) certificate. The Speech SDK verifies that the certificate reported by the remote endpoint is trusted and isn't revoked. This verification provides a layer of protection against attacks involving spoofing and other related vectors. The check is accomplished by retrieving a certificate revocation list (CRL) from a certificate authority (CA) used by Azure. A list of Azure CA download locations for updated TLS CRLs can be found in [this document](/azure/security/fundamentals/tls-certificate-changes). -If a destination posing as the Speech service reports a certificate that's been revoked in a retrieved CRL, the SDK terminates the connection and reports an error via a `Canceled` event. The authenticity of a reported certificate can't be checked without an updated CRL. Therefore, the Speech SDK also treats a failure to download a CRL from an Azure CA location as an error. +If a destination posing as the Speech service reports a revoked certificate in a retrieved CRL, the SDK terminates the connection and reports an error via a `Canceled` event. The authenticity of a reported certificate can't be checked without an updated CRL. Therefore, the Speech SDK also treats a failure to download a CRL from an Azure CA location as an error. > [!WARNING] > If your solution uses proxy or firewall it should be configured to allow access to all certificate revocation list URLs used by Azure. Note that many of these URLs are outside of `microsoft.com` domain, so allowing access to `*.microsoft.com` is not enough. See [this document](/azure/security/fundamentals/tls-certificate-changes) for details. In exceptional cases you may ignore CRL failures (see [the correspondent section](#bypassing-or-ignoring-crl-failures)), but such configuration is strongly not recommended, especially for production scenarios. @@ -63,7 +65,7 @@ If a destination posing as the Speech service reports a certificate that's been One cause of CRL-related failures is the use of large CRL files. This class of error is typically only applicable to special environments with extended CA chains. Standard public endpoints shouldn't encounter this class of issue. -The default maximum CRL size used by the Speech SDK (10 MB) can be adjusted per config object. The property key for this adjustment is `CONFIG_MAX_CRL_SIZE_KB` and the value, specified as a string, is by default "10000" (10 MB). For example, when creating a `SpeechRecognizer` object (that manages a connection to the Speech service), you can set this property in its `SpeechConfig`. In the snippet below, the configuration is adjusted to permit a CRL file size up to 15 MB. +The default maximum CRL size used by the Speech SDK (10 MB) can be adjusted per config object. The property key for this adjustment is `CONFIG_MAX_CRL_SIZE_KB` and the value, specified as a string, is by default "10000" (10 MB). For example, when creating a `SpeechRecognizer` object (that manages a connection to the Speech service), you can set this property in its `SpeechConfig`. In the following code snippet, the configuration is adjusted to permit a CRL file size up to 15 MB. ::: zone pivot="programming-language-csharp" @@ -155,7 +157,7 @@ speechConfig.properties.SetPropertyByString("OPENSSL_CONTINUE_ON_CRL_DOWNLOAD_FA ::: zone-end -To turn off certificate revocation checks, set the property `"OPENSSL_DISABLE_CRL_CHECK"` to `"true"`. Then, while connecting to the Speech service, there will be no attempt to check or download a CRL and no automatic verification of a reported TLS/SSL certificate. +To turn off certificate revocation checks, set the property `"OPENSSL_DISABLE_CRL_CHECK"` to `"true"`. Then, while connecting to the Speech service, there's no attempt to check or download a CRL and no automatic verification of a reported TLS/SSL certificate. ::: zone pivot="programming-language-csharp" @@ -203,7 +205,7 @@ By default, the Speech SDK will cache a successfully downloaded CRL on disk to i Some Linux distributions don't have a `TMP` or `TMPDIR` environment variable defined, so the Speech SDK doesn't cache downloaded CRLs. Without `TMP` or `TMPDIR` environment variable defined, the Speech SDK downloads a new CRL for each connection. To improve initial connection performance in this situation, you can [create a `TMPDIR` environment variable and set it to the accessible path of a temporary directory.](https://help.ubuntu.com/community/EnvironmentVariables). -## Next steps +## Related content - [Speech SDK overview](speech-sdk.md) - [Install the Speech SDK](quickstarts/setup-platform.md) diff --git a/articles/ai-services/speech-service/how-to-control-connections.md b/articles/ai-services/speech-service/how-to-control-connections.md index 52d93e34edd..5d7ba63e270 100644 --- a/articles/ai-services/speech-service/how-to-control-connections.md +++ b/articles/ai-services/speech-service/how-to-control-connections.md @@ -2,15 +2,17 @@ title: Service connectivity how-to - Speech SDK titleSuffix: Azure AI services description: Learn how to monitor for connection status and manually connect or disconnect from the Speech service. -author: trrwilson +author: eric-urban +ms.author: eur manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/18/2024 -ms.author: travisw +ms.date: 9/19/2024 +ms.reviewer: travisw zone_pivot_groups: programming-languages-set-thirteen ms.devlang: cpp ms.custom: devx-track-csharp, devx-track-extended-java +#Customer intent: As a developer, I want to learn how to monitor and control service connections with the Speech SDK so that I can manage connections to the Speech service. --- # How to monitor and control service connections with the Speech SDK @@ -96,14 +98,14 @@ connection.disconnected.addEventListener((s, connectionEventArgs) -> { `Connection` has explicit methods to start or end a connection to the Speech service. Reasons you might want to control the connection include: -- Preconnecting to the Speech service to allow the first interaction to start as quickly as possible -- Establishing connection at a specific time in your application's logic to gracefully and predictably handle initial connection failures -- Disconnecting to clear an idle connection when you don't expect immediate reconnection but also don't want to destroy the object +- Preconnecting to the Speech service to allow the first interaction to start as quickly as possible. +- Establishing connection at a specific time in your application's logic to gracefully and predictably handle initial connection failures. +- Disconnecting to clear an idle connection when you don't expect immediate reconnection but also don't want to destroy the object. Some important notes on the behavior when manually modifying connection state: -- Trying to connect when already connected will do nothing. It will not generate an error. Monitor the `Connected` and `Disconnected` events if you want to know the current state of the connection. -- A failure to connect that originates from a problem that has no involvement with the Speech service--such as attempting to do so from an invalid state--will throw or return an error as appropriate to the programming language. Failures that require network resolution--such as authentication failures--won't throw or return an error but instead generate a `Canceled` event on the top-level object the `Connection` was created from. +- Trying to connect when already connected doesn't generate an error. Monitor the `Connected` and `Disconnected` events if you want to know the current state of the connection. +- A failure to connect that originates from a problem that has no involvement with the Speech service--such as attempting to do so from an invalid state--results in an error as appropriate to the programming language. Failures that require network resolution--such as authentication failures--doesn't result in an error but instead generate a `Canceled` event on the top-level object the `Connection` was created from. - Manually disconnecting from the Speech service during an ongoing interaction results in a connection error and loss of data for that interaction. Connection errors are surfaced on the appropriate top-level object's `Canceled` event. ::: zone pivot="programming-language-csharp" diff --git a/articles/ai-services/speech-service/how-to-custom-speech-continuous-integration-continuous-deployment.md b/articles/ai-services/speech-service/how-to-custom-speech-continuous-integration-continuous-deployment.md index 871b3b8fa7d..39ebb0d6331 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-continuous-integration-continuous-deployment.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-continuous-integration-continuous-deployment.md @@ -2,12 +2,13 @@ title: CI/CD for custom speech - Speech service titleSuffix: Azure AI services description: Apply DevOps with custom speech and CI/CD workflows. Implement an existing DevOps solution for your own project. -author: nitinme -manager: cmayomsft +author: eric-urban +ms.author: eur +manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/19/2024 -ms.author: nitinme +ms.date: 9/19/2024 +#Customer intent: As a developer, I want to learn how to apply DevOps with custom speech and CI/CD workflows so that I can implement an existing DevOps solution for my own project. --- # CI/CD for custom speech @@ -18,15 +19,15 @@ Implement automated training, testing, and release management to enable continuo [Continuous delivery](/devops/deliver/what-is-continuous-delivery) (CD) takes models from the CI process and creates an endpoint for each improved custom speech model. CD makes endpoints easily available to be integrated into solutions. -Custom CI/CD solutions are possible, but for a robust, pre-built solution, use the [Speech DevOps template repository](https://github.com/Azure-Samples/Speech-Service-DevOps-Template), which executes CI/CD workflows using GitHub Actions. +Custom CI/CD solutions are possible, but for a robust, prebuilt solution, use the [Speech DevOps template repository](https://github.com/Azure-Samples/Speech-Service-DevOps-Template), which executes CI/CD workflows using GitHub Actions. ## CI/CD workflows for custom speech -The purpose of these workflows is to ensure that each custom speech model has better recognition accuracy than the previous build. If the updates to the testing and/or training data improve the accuracy, these workflows create a new custom speech endpoint. +The purpose of these workflows is to ensure that each custom speech model has better recognition accuracy than the previous build. If the updates to the testing or training data improve the accuracy, these workflows create a new custom speech endpoint. Git servers such as GitHub and Azure DevOps can run automated workflows when specific Git events happen, such as merges or pull requests. For example, a CI workflow can be triggered when updates to testing data are pushed to the *main* branch. Different Git Servers have different tooling, but allow scripting command-line interface (CLI) commands so that they can execute on a build server. -Along the way, the workflows should name and store data, tests, test files, models, and endpoints such that they can be traced back to the commit or version they came from. It's also helpful to name these assets so that it's easy to see which were created after updating testing data versus training data. +The workflows should name and store data, tests, test files, models, and endpoints such that they can be traced back to the commit or version they came from. It's also helpful to name these assets so that it's easy to see which were created after updating testing data versus training data. ### CI workflow for testing data updates diff --git a/articles/ai-services/speech-service/how-to-custom-speech-create-project.md b/articles/ai-services/speech-service/how-to-custom-speech-create-project.md index 364c35f780a..60260525dd8 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-create-project.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-create-project.md @@ -6,9 +6,10 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 7/15/2024 +ms.date: 9/19/2024 ms.author: eur zone_pivot_groups: speech-studio-cli-rest +#Customer intent: As a developer, I want to learn how to create a project for custom speech so that I can train and deploy a custom model. --- # Create a custom speech project diff --git a/articles/ai-services/speech-service/how-to-custom-speech-deploy-model.md b/articles/ai-services/speech-service/how-to-custom-speech-deploy-model.md index a41278e5505..d2de53ac8fa 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-deploy-model.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-deploy-model.md @@ -6,9 +6,10 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 7/15/2024 +ms.date: 9/19/2024 ms.author: eur zone_pivot_groups: speech-studio-cli-rest +#Customer intent: As a developer, I want to learn how to deploy a custom speech model so that I can use it in my applications. --- # Deploy a custom speech model @@ -385,7 +386,7 @@ The locations of each log file with more details are returned in the response bo Logging data is available on Microsoft-owned storage for 30 days, and then it's removed. If your own storage account is linked to the Azure AI services subscription, the logging data isn't automatically deleted. -## Next steps +## Related content - [CI/CD for custom speech](how-to-custom-speech-continuous-integration-continuous-deployment.md) - [Custom speech model lifecycle](how-to-custom-speech-model-and-endpoint-lifecycle.md) diff --git a/articles/ai-services/speech-service/how-to-custom-speech-display-text-format.md b/articles/ai-services/speech-service/how-to-custom-speech-display-text-format.md index cbd3378662b..1b9b4395511 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-display-text-format.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-display-text-format.md @@ -6,8 +6,9 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/19/2024 +ms.date: 9/19/2024 ms.author: eur +#Customer intent: As a developer, I want to learn how to prepare display text format training data for custom speech so that I can customize the display text formatting pipeline for my specific scenarios. --- # How to prepare display text format training data for custom speech diff --git a/articles/ai-services/speech-service/how-to-custom-speech-evaluate-data.md b/articles/ai-services/speech-service/how-to-custom-speech-evaluate-data.md index 6129dedc0af..8f558e7655d 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-evaluate-data.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-evaluate-data.md @@ -6,11 +6,12 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 7/15/2024 +ms.date: 9/19/2024 ms.author: eur zone_pivot_groups: speech-studio-cli-rest show_latex: true no-loc: [$$, '\times', '\over'] +#Customer intent: As a developer, I want to test the accuracy of a custom speech model so that I can evaluate whether it meets my requirements. --- # Test accuracy of a custom speech model diff --git a/articles/ai-services/speech-service/how-to-custom-speech-human-labeled-transcriptions.md b/articles/ai-services/speech-service/how-to-custom-speech-human-labeled-transcriptions.md index 5ce20cee5e8..19f368f4633 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-human-labeled-transcriptions.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-human-labeled-transcriptions.md @@ -6,8 +6,9 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/19/2024 +ms.date: 9/19/2024 ms.author: eur +#Customer intent: As a developer, I need to understand how to create human-labeled transcriptions for my audio data so that I can improve speech recognition accuracy. --- # How to create human-labeled transcriptions diff --git a/articles/ai-services/speech-service/how-to-custom-speech-inspect-data.md b/articles/ai-services/speech-service/how-to-custom-speech-inspect-data.md index e072644f473..c57dd74b560 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-inspect-data.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-inspect-data.md @@ -6,9 +6,10 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 7/15/2024 +ms.date: 9/19/2024 ms.author: eur zone_pivot_groups: speech-studio-cli-rest +#Customer intent: As a developer, I want to test the recognition quality of a custom speech model so that I can determine if the provided recognition result is correct. --- # Test recognition quality of a custom speech model diff --git a/articles/ai-services/speech-service/how-to-custom-speech-model-and-endpoint-lifecycle.md b/articles/ai-services/speech-service/how-to-custom-speech-model-and-endpoint-lifecycle.md index 75fb73c81fd..a09493d3627 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-model-and-endpoint-lifecycle.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-model-and-endpoint-lifecycle.md @@ -2,13 +2,15 @@ title: Model lifecycle of custom speech - Speech service titleSuffix: Azure AI services description: Custom speech provides base models for training and lets you create custom models from your data. This article describes the timelines for models and for endpoints that use these models. -author: heikora -manager: dongli +author: eric-urban +manager: nitinme +ms.author: eur ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/19/2024 -ms.author: heikora +ms.date: 9/19/2024 +ms.reviewer: heikora zone_pivot_groups: speech-studio-cli-rest +#Customer intent: As a developer, I want to understand the lifecycle of custom speech models and endpoints so that I can plan for the expiration of my models. --- # Custom speech model lifecycle diff --git a/articles/ai-services/speech-service/how-to-custom-speech-test-and-train.md b/articles/ai-services/speech-service/how-to-custom-speech-test-and-train.md index a8f39868271..72559531a0b 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-test-and-train.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-test-and-train.md @@ -6,8 +6,9 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/19/2024 +ms.date: 9/19/2024 ms.author: eur +#Customer intent: As a developer, I need to understand the types of training and testing data that I can use for a custom speech project, along with how to use and manage that data. --- # Training and testing datasets diff --git a/articles/ai-services/speech-service/how-to-custom-speech-train-model.md b/articles/ai-services/speech-service/how-to-custom-speech-train-model.md index 735824c79a1..58e5ae6404f 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-train-model.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-train-model.md @@ -6,9 +6,10 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 7/15/2024 +ms.date: 9/19/2024 ms.author: eur zone_pivot_groups: speech-studio-cli-rest +#Customer intent: As a developer, I want to train a custom speech model to improve recognition accuracy for the Microsoft base model or a custom model. --- # Train a custom speech model diff --git a/articles/ai-services/speech-service/how-to-custom-speech-transcription-editor.md b/articles/ai-services/speech-service/how-to-custom-speech-transcription-editor.md index 26729fc8499..33468ff5d96 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-transcription-editor.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-transcription-editor.md @@ -6,8 +6,9 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 1/19/2024 +ms.date: 9/19/2024 ms.author: eur +#Customer intent: As a developer, I need to understand how to use the online transcription editor for custom speech so that I can create or edit audio + human-labeled transcriptions for custom speech. --- # How to use the online transcription editor diff --git a/articles/ai-services/speech-service/how-to-custom-speech-upload-data.md b/articles/ai-services/speech-service/how-to-custom-speech-upload-data.md index 24ddde00f75..b80f5f8692f 100644 --- a/articles/ai-services/speech-service/how-to-custom-speech-upload-data.md +++ b/articles/ai-services/speech-service/how-to-custom-speech-upload-data.md @@ -6,9 +6,10 @@ author: eric-urban manager: nitinme ms.service: azure-ai-speech ms.topic: how-to -ms.date: 7/15/2024 +ms.date: 9/19/2024 ms.author: eur zone_pivot_groups: speech-studio-cli-rest +#Customer intent: As a developer, I need to understand how to upload data to test or train a custom speech model so that I can improve the accuracy of speech recognition. --- # Upload training and testing datasets for custom speech diff --git a/articles/ai-studio/how-to/develop/llama-index.md b/articles/ai-studio/how-to/develop/llama-index.md new file mode 100644 index 00000000000..9f284d00791 --- /dev/null +++ b/articles/ai-studio/how-to/develop/llama-index.md @@ -0,0 +1,202 @@ +--- +title: Develop application with LlamaIndex and Azure AI studio +titleSuffix: Azure AI Studio +description: This article explains how to use LlamaIndex with models deployed in Azure AI studio to build advance intelligent applications. +manager: nitinme +ms.service: azure-ai-studio +ms.topic: how-to +ms.date: 9/14/2024 +ms.reviewer: fasantia +ms.author: eur +author: eric-urban +--- + +# Develop applications with LlamaIndex and Azure AI studio + +In this article, you learn how to use [LlamaIndex](https://github.com/run-llama/llama_index) with models deployed from the Azure AI model catalog deployed to Azure AI studio. + +Models deployed to Azure AI studio can be used with LlamaIndex in two ways: + +- **Using the Azure AI model inference API:** All models deployed to Azure AI studio support the [Azure AI model inference API](../../reference/reference-model-inference-api.md), which offers a common set of functionalities that can be used for most of the models in the catalog. The benefit of this API is that, since it's the same for all the models, changing from one to another is as simple as changing the model deployment being use. No further changes are required in the code. When working with LlamaIndex, install the extensions `llama-index-llms-azure-inference` and `llama-index-embeddings-azure-inference`. + +- **Using the model's provider specific API:** Some models, like OpenAI, Cohere, or Mistral, offer their own set of APIs and extensions for LlamaIndex. Those extensions may include specific functionalities that the model support and hence are suitable if you want to exploit them. When working with `llama-index`, install the extension specific for the model you want to use, like `llama-index-llms-openai` or `llama-index-llms-cohere`. + +In this example, we are working with the **Azure AI model inference API**. + +## Prerequisites + +To run this tutorial, you need: + +1. An [Azure subscription](https://azure.microsoft.com). +2. An Azure AI hub resource as explained at [How to create and manage an Azure AI Studio hub](../create-azure-ai-resource.md). +3. A model supporting the [Azure AI model inference API](https://aka.ms/azureai/modelinference) deployed. In this example, we use a `Mistral-Large` deployment, but use any model of your preference. For using embeddings capabilities in LlamaIndex, you need an embedding model like `cohere-embed-v3-multilingual`. + + * You can follow the instructions at [Deploy models as serverless APIs](../deploy-models-serverless.md). + +4. Python 3.8 or later installed, including pip. +5. LlamaIndex installed. You can do it with: + + ```bash + pip install llama-index + ``` + +6. In this example, we are working with the Azure AI model inference API, hence we install the following packages: + + ```bash + pip install -U llama-index-llms-azure-inference + pip install -U llama-index-embeddings-azure-inference + ``` + +## Configure the environment + +To use LLMs deployed in Azure AI studio, you need the endpoint and credentials to connect to it. The parameter `model_name` is not required for endpoints serving a single model, like Managed Online Endpoints. Follow these steps to get the information you need from the model you want to use: + +1. Go to the [Azure AI studio](https://ai.azure.com/). +2. Go to deployments and select the model you deployed as indicated in the prerequisites. +3. Copy the endpoint URL and the key. + + :::image type="content" source="../../media/how-to/inference/serverless-endpoint-url-keys.png" alt-text="Screenshot of the option to copy endpoint URI and keys from an endpoint." lightbox="../../media/how-to/inference/serverless-endpoint-url-keys.png"::: + + > [!TIP] + > If your model was deployed with Microsoft Entra ID support, you don't need a key. + +In this scenario, we placed both the endpoint URL and key in the following environment variables: + +```bash +export AZURE_INFERENCE_ENDPOINT="" +export AZURE_INFERENCE_CREDENTIAL="" +``` + +Once configured, create a client to connect to the endpoint: + +```python +import os +from llama_index.llms.azure_inference import AzureAICompletionsModel + +llm = AzureAICompletionsModel( + endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"], + credential=os.environ["AZURE_INFERENCE_CREDENTIAL"], +) +``` + +Alternatively, if your endpoint support Microsoft Entra ID, you can use the following code to create the client: + +```python +from azure.identity import DefaultAzureCredential + +llm = AzureAICompletionsModel( + endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"], + credential=DefaultAzureCredential(), +) +``` + +> [!NOTE] +> > Note: When using Microsoft Entra ID, make sure that the endpoint was deployed with that authentication method and that you have the required permissions to invoke it. + +If you are planning to use asynchronous calling, it's a best practice to use the asynchronous version for the credentials: + +```python +from azure.identity.aio import ( + DefaultAzureCredential as DefaultAzureCredentialAsync, +) + +llm = AzureAICompletionsModel( + endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"], + credential=DefaultAzureCredentialAsync(), +) +``` + +### Inference parameters + +You can configure how inference in performed for all the operations that are using this client by setting extra parameters. This helps avoid indicating them on each call you make to the model. + +```python +llm = AzureAICompletionsModel( + endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"], + credential=os.environ["AZURE_INFERENCE_CREDENTIAL"], + temperature=0.0, + model_kwargs={"top_p": 1.0}, +) +``` + +Parameters not supported in the Azure AI model inference API ([reference](../../reference/reference-model-inference-chat-completions.md)) but available in the underlying model, you can use the `model_extras` argument. In the following example, the parameter `safe_prompt`, only available for Mistral models, is being passed. + +```python +llm = AzureAICompletionsModel( + endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"], + credential=os.environ["AZURE_INFERENCE_CREDENTIAL"], + temperature=0.0, + model_kwargs={"model_extras": {"safe_prompt": True}}, +) +``` + +## Use LLMs models + +Use the `chat` endpoint for chat instruction models. The `complete` method is still available for model of type `chat-completions`. On those cases, your input text is converted to a message with `role="user"`. + +```python +from llama_index.core.llms import ChatMessage + +messages = [ + ChatMessage( + role="system", content="You are a pirate with colorful personality." + ), + ChatMessage(role="user", content="Hello"), +] + +response = llm.chat(messages) +print(response) +``` + +You can stream the outputs also: + +```python +response = llm.stream_chat(messages) +for r in response: + print(r.delta, end="") +``` + +## Use embeddings models + +In the same way you create an LLM client, you can connect to an embedding model. In the following example, we are setting again the environment variable to now point to an embeddings model: + +```bash +export AZURE_INFERENCE_ENDPOINT="" +export AZURE_INFERENCE_CREDENTIAL="" +``` + +Then create the client: + +```python +from llama_index.embeddings.azure_inference import AzureAIEmbeddingsModel + +embed_model = AzureAIEmbeddingsModel( + endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"], + credential=os.environ['AZURE_INFERENCE_CREDENTIAL'], +) +``` + +## Configure the models used by your code + +You can use the LLM or embeddings model client individually in the code you develop with LlamaIndex or you can configure the entire session using the `Settings` options. Configuring the session has the advantage of all your code using the same models for all the operations. + +```python +from llama_index.core import Settings + +Settings.llm = llm +Settings.embed_model = embed_model +``` + +However, there are scenarios where you want to use a general model for most of the operations but a specific one for a given task. On those cases, it's useful to set the LLM or embedding model you are using for each LlamaIndex construct. In the following example, we set a specific model: + +```python +from llama_index.core.evaluation import RelevancyEvaluator + +relevancy_evaluator = RelevancyEvaluator(llm=llm) +``` + +In general, you use a combination of both strategies. + +## Related content + +* [How to get started with Azure AI SDKs](sdk-overview.md) diff --git a/articles/ai-studio/how-to/develop/sdk-overview.md b/articles/ai-studio/how-to/develop/sdk-overview.md index 844caba6f59..97b143ea34b 100644 --- a/articles/ai-studio/how-to/develop/sdk-overview.md +++ b/articles/ai-studio/how-to/develop/sdk-overview.md @@ -15,7 +15,7 @@ author: eric-urban # Overview of the Azure AI SDKs -Microsoft offers a variety of packages that you can use for building generative AI applications in the cloud. In most applications, you need to use a combination of packages to manage and use various Azure services that provide AI functionality. We also offer integrations with open-source libraries like LangChain and mlflow for use with Azure. In this article we'll give an overview of the main services and SDKs you can use with Azure AI Studio. +Microsoft offers a variety of packages that you can use for building generative AI applications in the cloud. In most applications, you need to use a combination of packages to manage and use various Azure services that provide AI functionality. We also offer integrations with open-source libraries like LangChain and MLflow for use with Azure. In this article we'll give an overview of the main services and SDKs you can use with Azure AI Studio. For building generative AI applications, we recommend using the following services and SDKs: * [Azure Machine Learning](/azure/machine-learning/overview-what-is-azure-machine-learning) for the hub and project infrastructure used in AI Studio to organize your work into projects, manage project artifacts (data, evaluation runs, traces), fine-tune & deploy models, and connect to external services and resources. @@ -54,6 +54,9 @@ Azure AI services Prompt flow * [Prompt flow SDK](https://microsoft.github.io/promptflow/how-to-guides/quick-start.html) +Agentic frameworks: +* [LlamaIndex](llama-index.md) + ## Related content - [Get started building a chat app using the prompt flow SDK](../../quickstarts/get-started-code.md) diff --git a/articles/ai-studio/media/how-to/inference/serverless-endpoint-url-keys.png b/articles/ai-studio/media/how-to/inference/serverless-endpoint-url-keys.png new file mode 100644 index 00000000000..3c6a8c286f9 Binary files /dev/null and b/articles/ai-studio/media/how-to/inference/serverless-endpoint-url-keys.png differ diff --git a/articles/ai-studio/reference/reference-model-inference-chat-completions.md b/articles/ai-studio/reference/reference-model-inference-chat-completions.md index 190655b8716..e49476cca56 100644 --- a/articles/ai-studio/reference/reference-model-inference-chat-completions.md +++ b/articles/ai-studio/reference/reference-model-inference-chat-completions.md @@ -42,6 +42,7 @@ POST /chat/completions?api-version=2024-04-01-preview | Name | Required | Type | Description | | --- | --- | --- | --- | +| model | | string | The model name. This parameter is ignored if the endpoint serves only one model. | | messages | True | [ChatCompletionRequestMessage](#chatcompletionrequestmessage) | A list of messages comprising the conversation so far. Returns a 422 error if at least some of the messages can't be understood by the model. | | frequency\_penalty | | number | Helps prevent word repetitions by reducing the chance of a word being selected if it has already been used. The higher the frequency penalty, the less likely the model is to repeat the same words in its output. Return a 422 error if value or parameter is not supported by model. | | max\_tokens | | integer | The maximum number of tokens that can be generated in the chat completion.

The total length of input tokens and generated tokens is limited by the model's context length. Passing null causes the model to use its max context length. | diff --git a/articles/ai-studio/reference/reference-model-inference-embeddings.md b/articles/ai-studio/reference/reference-model-inference-embeddings.md index bd4238a7af2..6c309dd8a27 100644 --- a/articles/ai-studio/reference/reference-model-inference-embeddings.md +++ b/articles/ai-studio/reference/reference-model-inference-embeddings.md @@ -42,6 +42,7 @@ POST /embeddings?api-version=2024-04-01-preview | Name | Required | Type | Description | | --------------- | -------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| model | | string | The model name. This parameter is ignored if the endpoint serves only one model. | | input | True | string[] | Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays. | | dimensions | | integer | The number of dimensions the resulting output embeddings should have. Returns a 422 error if the model doesn't support the value or parameter. | | encoding_format | | [EmbeddingEncodingFormat](#embeddingencodingformat) | The format to return the embeddings in. Either base64, float, int8, uint8, binary, or ubinary. Returns a 422 error if the model doesn't support the value or parameter. | diff --git a/articles/ai-studio/toc.yml b/articles/ai-studio/toc.yml index f3f1eb6fafe..b471b3973d9 100644 --- a/articles/ai-studio/toc.yml +++ b/articles/ai-studio/toc.yml @@ -252,6 +252,8 @@ items: href: how-to/develop/vscode.md - name: Start with an AI template href: how-to/develop/ai-template-get-started.md + - name: Develop with LlamaIndex and Azure AI studio + href: how-to/develop/llama-index.md - name: Trace your application with prompt flow href: how-to/develop/trace-local-sdk.md displayName: code,sdk diff --git a/articles/machine-learning/overview-what-is-azure-machine-learning.md b/articles/machine-learning/overview-what-is-azure-machine-learning.md index 37165f84211..665988b1b76 100644 --- a/articles/machine-learning/overview-what-is-azure-machine-learning.md +++ b/articles/machine-learning/overview-what-is-azure-machine-learning.md @@ -8,12 +8,13 @@ ms.topic: overview author: sdgilley ms.author: sgilley ms.reviewer: saoh -ms.date: 01/29/2024 +ms.date: 09/18/2024 ms.custom: - build-2023 - build-2023-dataai - ignite-2023 adobe-target: true +# Customer intent: Learn about Azure Machine Learning --- # What is Azure Machine Learning? @@ -60,7 +61,8 @@ As you're refining the model and collaborating with others throughout the rest o [Machine Learning studio](https://ml.azure.com) offers multiple authoring experiences depending on the type of project and the level of your past ML experience, without having to install anything. -* **Notebooks**: Write and run your own code in managed Jupyter Notebook servers that are directly integrated in the studio. +* **Notebooks**: Write and run your own code in managed Jupyter Notebook servers that are directly integrated in the studio. Or, open the notebooks in [VS Code](how-to-work-in-vs-code-remote.md), on the web or on your desktop. + * **Visualize run metrics**: Analyze and optimize your experiments with visualization. :::image type="content" source="media/how-to-log-view-metrics/metrics.png" alt-text="Screenshot that shows metrics for a training run."::: @@ -69,6 +71,20 @@ As you're refining the model and collaborating with others throughout the rest o * **Automated machine learning UI**: Learn how to create [automated ML experiments](tutorial-first-experiment-automated-ml.md) with an easy-to-use interface. * **Data labeling**: Use Machine Learning data labeling to efficiently coordinate [image labeling](how-to-create-image-labeling-projects.md) or [text labeling](how-to-create-text-labeling-projects.md) projects. +## Work with LLMs and Generative AI + +Azure Machine Learning includes tools to help you build Generative AI applications powered by Large Language Models (LLMs). The solution includes a model catalog, prompt flow, and a suite of tools to streamline the development cycle of AI applications. + +Both Azure Machine Learning Studio and Azure AI Studio allow you to work with LLMs. Use this guide to determine [which studio you should use](/ai/ai-studio-experiences-overview?toc=/azure/machine-learning/toc.json&bc=/azure/machine-learning/breadcrumb/toc.json). + +### Model catalog + +The model catalog in Azure Machine Learning studio is the hub to discover and use a wide range of models that enable you to build Generative AI applications. The model catalog features hundreds of models from model providers such as Azure OpenAI service, Mistral, Meta, Cohere, Nvidia, Hugging Face, including models trained by Microsoft. Models from providers other than Microsoft are Non-Microsoft Products, as defined in [Microsoft's Product Terms](https://www.microsoft.com/licensing/terms/welcome/welcomepage), and subject to the terms provided with the model. + +### Prompt flow + +Azure Machine Learning prompt flow is a development tool designed to streamline the entire development cycle of AI applications powered by Large Language Models (LLMs). Prompt flow provides a comprehensive solution that simplifies the process of prototyping, experimenting, iterating, and deploying your AI applications. + ## Enterprise-readiness and security Machine Learning integrates with the Azure cloud platform to add security to ML projects. @@ -207,7 +223,7 @@ Machine Learning also includes features for monitoring and auditing: If you use Apache Airflow, the [airflow-provider-azure-machinelearning](https://github.com/Azure/airflow-provider-azure-machinelearning) package is a provider that enables you to submit workflows to Azure Machine Learning from Apache AirFlow. -## Next steps +## Related content Start using Azure Machine Learning: diff --git a/articles/machine-learning/reference-model-inference-images-embeddings.md b/articles/machine-learning/reference-model-inference-images-embeddings.md index 26cd8910fcf..8edaf0264b5 100644 --- a/articles/machine-learning/reference-model-inference-images-embeddings.md +++ b/articles/machine-learning/reference-model-inference-images-embeddings.md @@ -41,6 +41,7 @@ POST /images/embeddings?api-version=2024-04-01-preview | Name | Required | Type | Description | | --- | --- | --- | --- | +| model | | string | The model name. This parameter is ignored if the endpoint serves only one model. | | input | True | [EmbeddingInput](#embeddinginput)\[\] | Input image to embed. To embed multiple inputs in a single request, pass an array. The input must not exceed the max input tokens for the model. | | dimensions | | integer | The number of dimensions the resulting output embeddings should have. Passing null causes the model to use its default value. Returns a 422 error if the model doesn't support the value or parameter. | | encoding\_format | | [EmbeddingEncodingFormat](#embeddingencodingformat) | The format to return the embeddings in. Either base64, float, int8, uint8, binary, or ubinary. Returns a 422 error if the model doesn't support the value or parameter. | diff --git a/articles/machine-learning/v1/how-to-enable-data-collection.md b/articles/machine-learning/v1/how-to-enable-data-collection.md index b6f7d635327..1811aee1c54 100644 --- a/articles/machine-learning/v1/how-to-enable-data-collection.md +++ b/articles/machine-learning/v1/how-to-enable-data-collection.md @@ -161,7 +161,7 @@ You can choose a tool of your preference to analyze the data collected in your B 1. Select **Storage**. - [![Select the Storage option](./media/how-to-enable-data-collection/StorageLocation.png)](././media/how-to-enable-data-collection/StorageLocation.png#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/StorageLocation.png" alt-text="Screenshot showing selection of the storage option." lightbox="./media/how-to-enable-data-collection/StorageLocation.png"::: 1. Follow the path to the blob's output data with this syntax: @@ -176,13 +176,13 @@ You can choose a tool of your preference to analyze the data collected in your B 1. Select **Get Data** and select [**Azure Blob Storage**](/power-bi/desktop-data-sources). - [![Power BI blob setup](./media/how-to-enable-data-collection/PBIBlob.png)](././media/how-to-enable-data-collection/PBIBlob.png#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/power-bi-blob.png" alt-text="Screenshot showing the Power BI blob setup." lightbox="./media/how-to-enable-data-collection/power-bi-blob.png"::: 1. Add your storage account name and enter your storage key. You can find this information by selecting **Settings** > **Access keys** in your blob. 1. Select the **model data** container and select **Edit**. - [![Power BI Navigator](./media/how-to-enable-data-collection/pbiNavigator.png)](././media/how-to-enable-data-collection/pbiNavigator.png#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/power-bi-navigator.png" alt-text="Screenshot showing the Power BI Navigator." lightbox="./media/how-to-enable-data-collection/power-bi-navigator.png"::: 1. In the query editor, click under the **Name** column and add your storage account. @@ -194,11 +194,11 @@ You can choose a tool of your preference to analyze the data collected in your B 1. Select the downward double arrows next to the **Content** column heading to combine the files. - [![Power BI Content](./media/how-to-enable-data-collection/pbiContent.png)](././media/how-to-enable-data-collection/pbiContent.png#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/power-bi-content.png" alt-text="Screenshot showing the Power BI Content." lightbox="./media/how-to-enable-data-collection/power-bi-content.png"::: 1. Select **OK**. The data preloads. - [![Power BI Combine Files](./media/how-to-enable-data-collection/pbiCombine.png)](././media/how-to-enable-data-collection/pbiCombine.png#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/power-bi-combine.png" alt-text="Screenshot showing the Power BI Combine Files." lightbox="./media/how-to-enable-data-collection/power-bi-combine.png"::: 1. Select **Close and Apply**. @@ -214,20 +214,20 @@ You can choose a tool of your preference to analyze the data collected in your B 1. In your Databricks workspace, select **Upload Data**. - [![Selecting Databricks Upload Data option](./media/how-to-enable-data-collection/dbupload.png)](././media/how-to-enable-data-collection/dbupload.png#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/databricks-upload.png" alt-text="Screenshot showing selection of Databricks Upload Data option." lightbox="./media/how-to-enable-data-collection/databricks-upload.png"::: 1. Select **Create New Table** and select **Other Data Sources** > **Azure Blob Storage** > **Create Table in Notebook**. - [![Databricks table creation](./media/how-to-enable-data-collection/dbtable.PNG)](././media/how-to-enable-data-collection/dbtable.PNG#lightbox) + :::image type="content" source="./media/how-to-enable-data-collection/databricks-table.png" alt-text="Screenshot showing Databricks table creation." lightbox="./media/how-to-enable-data-collection/databricks-table.png"::: 1. Update the location of your data. Here is an example: ``` - file_location = "wasbs://mycontainer@storageaccountname.blob.core.windows.net/modeldata/1a2b3c4d-5e6f-7g8h-9i10-j11k12l13m14/myresourcegrp/myWorkspace/aks-w-collv9/best_model/10/inputs/2018/*/*/data.csv" + file_location = "wasbs://mycontainer@storageaccountname.blob.core.windows.net/*/*/data.csv" file_type = "csv" ``` - - [![Databricks setup](./media/how-to-enable-data-collection/dbsetup.png)](././media/how-to-enable-data-collection/dbsetup.png#lightbox) + + :::image type="content" source="./media/how-to-enable-data-collection/databricks-setup.png" alt-text="Screenshot showing the Databricks setup." lightbox="./media/how-to-enable-data-collection/databricks-setup.png"::: 1. Follow the steps on the template to view and analyze your data. diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-setup.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-setup.png new file mode 100644 index 00000000000..f62399a7244 Binary files /dev/null and b/articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-setup.png differ diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/dbtable.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-table.png similarity index 100% rename from articles/machine-learning/v1/media/how-to-enable-data-collection/dbtable.png rename to articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-table.png diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/dbupload.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-upload.png similarity index 100% rename from articles/machine-learning/v1/media/how-to-enable-data-collection/dbupload.png rename to articles/machine-learning/v1/media/how-to-enable-data-collection/databricks-upload.png diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/dbsetup.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/dbsetup.png deleted file mode 100644 index e0d644f3b31..00000000000 Binary files a/articles/machine-learning/v1/media/how-to-enable-data-collection/dbsetup.png and /dev/null differ diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/pbicombine.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/pbicombine.png deleted file mode 100644 index 45ac2a12d4e..00000000000 Binary files a/articles/machine-learning/v1/media/how-to-enable-data-collection/pbicombine.png and /dev/null differ diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/pbiblob.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-blob.png similarity index 100% rename from articles/machine-learning/v1/media/how-to-enable-data-collection/pbiblob.png rename to articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-blob.png diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-combine.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-combine.png new file mode 100644 index 00000000000..f4aae6660f2 Binary files /dev/null and b/articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-combine.png differ diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/pbicontent.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-content.png similarity index 100% rename from articles/machine-learning/v1/media/how-to-enable-data-collection/pbicontent.png rename to articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-content.png diff --git a/articles/machine-learning/v1/media/how-to-enable-data-collection/pbinavigator.png b/articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-navigator.png similarity index 100% rename from articles/machine-learning/v1/media/how-to-enable-data-collection/pbinavigator.png rename to articles/machine-learning/v1/media/how-to-enable-data-collection/power-bi-navigator.png diff --git a/articles/search/cognitive-search-common-errors-warnings.md b/articles/search/cognitive-search-common-errors-warnings.md index eab329fce38..d5721593cf4 100644 --- a/articles/search/cognitive-search-common-errors-warnings.md +++ b/articles/search/cognitive-search-common-errors-warnings.md @@ -9,8 +9,8 @@ ms.author: heidist ms.service: cognitive-search ms.custom: - ignite-2023 -ms.topic: conceptual -ms.date: 03/27/2024 +ms.topic: reference +ms.date: 09/19/2024 --- # Troubleshooting common indexer errors and warnings in Azure AI Search @@ -32,29 +32,28 @@ Warnings don't stop indexing, but they do indicate conditions that could result To verify an indexer status and identify errors in the Azure portal, follow the steps below: -1. Navigate to the Azure portal and locate your AI Search service. -1. Once you're in the AI Search service, click on the 'Indexers' tab. -1. From the list of indexers, identify the specific indexer you wish to verify. -1. Under the 'Execution History' column, click on the 'Status' hyperlink associated with the selected indexer. -1. If there's an error, hover over the error message. A pane will appear on the right side of your screen displaying detailed information about the error. +1. Sign in to the [Azure portal](https://portal.azure.com/) and [find your search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices). -## Transient errors +1. On the left, expand **Search Management** > **Indexers** and select an indexer. -For various reasons, such as transient network communication interruptions, timeouts from long-running processes, or specific document nuances, it's common to encounter transient errors or warnings during indexer runs. However, these errors are temporary and should be resolved in subsequent indexer runs. +1. Under **Execution History**, select the status. All statuses, including Success, have details about the execution. -To manage these errors effectively, it is recommended [putting your indexer on a schedule](search-howto-schedule-indexers.md), for instance, to run every five minutes. This means the next run will commence five minutes after the completion of the first run, adhering to the [maximum runtime limit](search-limits-quotas-capacity.md#indexer-limits). Regularly scheduled runs help to rectify any transient errors or warnings swiftly. +1. If there's an error, hover over the error message. A pane appears on the right side of your screen displaying detailed information about the error. -If you notice an error persisting over multiple indexer runs, it's likely not a transient issue. In such cases, refer to the list below for potential solutions. Please note, always ensure your indexing schedule aligns with the limitations outlined in our indexer limits guide. +## Transient errors +For various reasons, such as transient network communication interruptions, timeouts from long-running processes, or specific document nuances, it's common to encounter transient errors or warnings during indexer runs. However, these errors are temporary and should be resolved in subsequent indexer runs. -## Error properties +To manage these errors effectively, we recommend [putting your indexer on a schedule](search-howto-schedule-indexers.md), for instance, to run every five minutes, where the next run commences five minutes after completing the first run, adhering to the [maximum runtime limit](search-limits-quotas-capacity.md#indexer-limits) on your service. Regularly scheduled runs help rectify transient errors or warnings. -Beginning with API version `2019-05-06`, item-level Indexer errors and warnings are structured to provide increased clarity around causes and next steps. They contain the following properties: +If an error persists over multiple indexer runs, it's likely not a transient issue. In such cases, refer to the list below for potential solutions. + +## Error properties | Property | Description | Example | | --- | --- | --- | -| Key | The document ID of the document impacted by the error or warning. | `https://.blob.core.windows.net/jfk-1k/docid-32112954.pdf`| -| Name | The operation name describing where the error or warning occurred. This is generated by the following structure: `[category]`.`[subcategory]`.`[resourceType]`.`[resourceName]` | `DocumentExtraction.azureblob.myBlobContainerName` `Enrichment.WebApiSkill.mySkillName` `Projection.SearchIndex.OutputFieldMapping.myOutputFieldName` `Projection.SearchIndex.MergeOrUpload.myIndexName` `Projection.KnowledgeStore.Table.myTableName` | +| Key | The ID of the document impacted by the error or warning. | Azure Storage example, where the default ID is the metadata storage path: `https://.blob.core.windows.net/jfk-1k/docid-32112954.pdf`| +| Name | The operation causing the error or warning. This is generated by the following structure: `[category]`.`[subcategory]`.`[resourceType]`.`[resourceName]` | `DocumentExtraction.azureblob.myBlobContainerName` `Enrichment.WebApiSkill.mySkillName` `Projection.SearchIndex.OutputFieldMapping.myOutputFieldName` `Projection.SearchIndex.MergeOrUpload.myIndexName` `Projection.KnowledgeStore.Table.myTableName` | | Message | A high-level description of the error or warning. | `Could not execute skill because the Web Api request failed.` | | Details | Specific information that might be helpful in diagnosing the issue, such as the WebApi response if executing a custom skill failed. | `link-cryptonyms-list - Error processing the request record : System.ArgumentNullException: Value cannot be null. Parameter name: source at System.Linq.Enumerable.All[TSource](IEnumerable 1 source, Func 2 predicate) at Microsoft.CognitiveSearch.WebApiSkills.JfkWebApiSkills. ...rest of stack trace...` | | DocumentationLink | A link to relevant documentation with detailed information to debug and resolve the issue. This link will often point to one of the below sections on this page. | `https://go.microsoft.com/fwlink/?linkid=2106475` | @@ -95,8 +94,8 @@ Indexer read the document from the data source, but there was an issue convertin | The document key is missing | `Document key cannot be missing or empty` | Ensure all documents have valid document keys. The document key is determined by setting the 'key' property as part of the [index definition](/rest/api/searchservice/indexes/create#request-body). Indexers emit this error when the property flagged as the 'key' can't be found on a particular document. | | The document key is invalid | `Invalid document key. Keys can only contain letters, digits, underscore (_), dash (-), or equal sign (=). ` | Ensure all documents have valid document keys. Review [Indexing Blob Storage](search-howto-indexing-azure-blob-storage.md) for more details. If you're using the blob indexer, and your document key is the `metadata_storage_path` field, make sure that the indexer definition has a [base64Encode mapping function](search-indexer-field-mappings.md?tabs=rest#base64encode-function) with `parameters` equal to `null`, instead of the path in plain text. | | The document key is invalid | `Document key cannot be longer than 1024 characters` | Modify the document key to meet the validation requirements. | -| Could not apply field mapping to a field | `Could not apply mapping function 'functionName' to field 'fieldName'. Array cannot be null. Parameter name: bytes` | Double check the [field mappings](search-indexer-field-mappings.md) defined on the indexer, and compare with the data of the specified field of the failed document. It might be necessary to modify the field mappings or the document data. | -| Could not read field value | `Could not read the value of column 'fieldName' at index 'fieldIndex'. A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)` | These errors are typically due to unexpected connectivity issues with the data source's underlying service. Try running the document through your indexer again later. | +| Couldn't apply field mapping to a field | `Could not apply mapping function 'functionName' to field 'fieldName'. Array cannot be null. Parameter name: bytes` | Double check the [field mappings](search-indexer-field-mappings.md) defined on the indexer, and compare with the data of the specified field of the failed document. It might be necessary to modify the field mappings or the document data. | +| Couldn't read field value | `Could not read the value of column 'fieldName' at index 'fieldIndex'. A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)` | These errors are typically due to unexpected connectivity issues with the data source's underlying service. Try running the document through your indexer again later. | @@ -189,7 +188,7 @@ The document was read and processed, but the indexer couldn't add it to the sear | Trouble connecting to the target index (that persists after retries) because the service is under other load, such as querying or indexing. | Failed to establish connection to update index. Search service is under heavy load. | [Scale up your search service](search-capacity-planning.md) | Search service is being patched for service update, or is in the middle of a topology reconfiguration. | Failed to establish connection to update index. Search service is currently down/Search service is undergoing a transition. | Configure service with at least three replicas for 99.9% availability per [SLA documentation](https://azure.microsoft.com/support/legal/sla/search/v1_0/) | Failure in the underlying compute/networking resource (rare) | Failed to establish connection to update index. An unknown failure occurred. | Configure indexers to [run on a schedule](search-howto-schedule-indexers.md) to pick up from a failed state. -| An indexing request made to the target index wasn't acknowledged within a timeout period due to network issues. | Could not establish connection to the search index in a timely manner. | Configure indexers to [run on a schedule](search-howto-schedule-indexers.md) to pick up from a failed state. Additionally, try lowering the indexer [batch size](/rest/api/searchservice/indexers/create#indexingparameters) if this error condition persists. +| An indexing request made to the target index wasn't acknowledged within a timeout period due to network issues. | Couldn't establish connection to the search index in a timely manner. | Configure indexers to [run on a schedule](search-howto-schedule-indexers.md) to pick up from a failed state. Additionally, try lowering the indexer [batch size](/rest/api/searchservice/indexers/create#indexingparameters) if this error condition persists. @@ -225,9 +224,9 @@ This error occurs when the indexer is attempting to [project data into a knowled | Reason | Details/Example | Resolution | | --- | --- | --- | -| Could not update projection blob `'blobUri'` in container `'containerName'` |The specified container doesn't exist. | The indexer checks if the specified container has been previously created and will create it if necessary, but it only performs this check once per indexer run. This error means that something deleted the container after this step. To resolve this error, try this: leave your storage account information alone, wait for the indexer to finish, and then rerun the indexer. | -| Could not update projection blob `'blobUri'` in container `'containerName'` |Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. | This is expected to be a transient failure with Azure Storage and thus should be resolved by rerunning the indexer. If you encounter this error consistently, file a [support ticket](https://portal.azure.com/#create/Microsoft.Support) so it can be investigated further. | -| Could not update row `'projectionRow'` in table `'tableName'` | The server is busy. | This is expected to be a transient failure with Azure Storage and thus should be resolved by rerunning the indexer. If you encounter this error consistently, file a [support ticket](https://portal.azure.com/#create/Microsoft.Support) so it can be investigated further. | +| Couldn't update projection blob `'blobUri'` in container `'containerName'` |The specified container doesn't exist. | The indexer checks if the specified container has been previously created and will create it if necessary, but it only performs this check once per indexer run. This error means that something deleted the container after this step. To resolve this error, try this: leave your storage account information alone, wait for the indexer to finish, and then rerun the indexer. | +| Couldn't update projection blob `'blobUri'` in container `'containerName'` |Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. | This is expected to be a transient failure with Azure Storage and thus should be resolved by rerunning the indexer. If you encounter this error consistently, file a [support ticket](https://portal.azure.com/#create/Microsoft.Support) so it can be investigated further. | +| Couldn't update row `'projectionRow'` in table `'tableName'` | The server is busy. | This is expected to be a transient failure with Azure Storage and thus should be resolved by rerunning the indexer. If you encounter this error consistently, file a [support ticket](https://portal.azure.com/#create/Microsoft.Support) so it can be investigated further. | @@ -237,7 +236,7 @@ Skill execution failed because the call to Azure AI services was throttled. Typi ## `Error: Expected IndexAction metadata` -An 'Expected IndexAction metadata' error means when the indexer attempted to read the document to identify what action should be taken, it did not find any corresponding metadata on the document. Typically, this error occurs when the indexer has an annotation cache added or removed without resetting the indexer. To address this, you should [reset and rerun the indexer](search-howto-run-reset-indexers.md). +An 'Expected IndexAction metadata' error means when the indexer attempted to read the document to identify what action should be taken, it didn't find any corresponding metadata on the document. Typically, this error occurs when the indexer has an annotation cache added or removed without resetting the indexer. To address this, you should [reset and rerun the indexer](search-howto-run-reset-indexers.md). @@ -379,7 +378,7 @@ Output field mappings that reference non-existent/null data will produce warning | Reason | Details/Example | Resolution | | --- | --- | --- | -| Cannot iterate over non-array | "Cannot iterate over non-array `/document/normalized_images/0/imageCelebrities/0/detail/celebrities`." | This error occurs when the output isn't an array. If you think the output should be an array, check the indicated output source field path for errors. For example, you might have a missing or extra `*` in the source field name. It's also possible that the input to this skill is null, resulting in an empty array. Find similar details in [Skill Input was Invalid](cognitive-search-common-errors-warnings.md#warning-skill-input-was-invalid) section. | +| Can't iterate over non-array | "Cannot iterate over non-array `/document/normalized_images/0/imageCelebrities/0/detail/celebrities`." | This error occurs when the output isn't an array. If you think the output should be an array, check the indicated output source field path for errors. For example, you might have a missing or extra `*` in the source field name. It's also possible that the input to this skill is null, resulting in an empty array. Find similar details in [Skill Input was Invalid](cognitive-search-common-errors-warnings.md#warning-skill-input-was-invalid) section. | | Unable to select `0` in non-array | "Unable to select `0` in non-array `/document/pages`." | This could happen if the skills output doesn't produce an array and the output source field name has array index or `*` in its path. Double check the paths provided in the output source field names and the field value for the indicated field name. Find similar details in [Skill Input was Invalid](cognitive-search-common-errors-warnings.md#warning-skill-input-was-invalid) section. | diff --git a/articles/search/cognitive-search-predefined-skills.md b/articles/search/cognitive-search-predefined-skills.md index c7d75fc4910..6d338f47cb2 100644 --- a/articles/search/cognitive-search-predefined-skills.md +++ b/articles/search/cognitive-search-predefined-skills.md @@ -8,19 +8,19 @@ ms.service: cognitive-search ms.custom: - ignite-2023 - build-2024 -ms.topic: conceptual -ms.date: 10/28/2023 +ms.topic: concept-article +ms.date: 09/19/2024 --- # Skills for extra processing during indexing (Azure AI Search) -This article describes the skills provided with Azure AI Search that you can include in a [skillset](cognitive-search-working-with-skillsets.md) to access external processing. +This article describes the skills in Azure AI Search that you can include in a [skillset](cognitive-search-working-with-skillsets.md) to access external processing. A *skill* provides an atomic operation that transforms content in some way. Often, it's an operation that recognizes or extracts text, but it can also be a utility skill that reshapes the enrichments that are already created. Typically, the output is text-based so that it can be used in [full text search](search-lucene-query-architecture.md) or vectors used in [vector search](vector-search-overview.md). Skills are organized into categories: -* A *built-in skill* wraps API calls to an Azure resource, where the inputs, outputs, and processing steps are well understood. For skills that call an Azure AI resource, the connection is made over the internal network. For skills that call Azure OpenAI, you provide the connection information that the search service uses to connect to the resource. A small quantity of processing is non-billable, but at larger volumes, processing is billable. Built-in skills are based on pretrained models from Microsoft, which means you can't train the model using your own training data. +* A *built-in skill* wraps API calls to an Azure AI resource, where the inputs, outputs, and processing steps are well understood. For skills that call an Azure AI resource, the connection is made over the internal network. For skills that call Azure OpenAI, you provide the connection information that the search service uses to connect to the resource. A small quantity of processing is non-billable, but at larger volumes, processing is billable. Built-in skills are based on pretrained models from Microsoft, which means you can't train the model using your own training data. * A *custom skill* provides custom code that executes externally to the search service. It's accessed through a URI. Custom code is often made available through an Azure function app. To attach an open-source or third-party vectorization model, use a custom skill. diff --git a/articles/search/query-lucene-syntax.md b/articles/search/query-lucene-syntax.md index 64c17e985ba..ba1c3ef24fa 100644 --- a/articles/search/query-lucene-syntax.md +++ b/articles/search/query-lucene-syntax.md @@ -9,13 +9,13 @@ ms.author: beloh ms.service: cognitive-search ms.custom: - ignite-2023 -ms.topic: conceptual -ms.date: 02/22/2024 +ms.topic: concept-article +ms.date: 09/19/2024 --- # Lucene query syntax in Azure AI Search -When creating queries in Azure AI Search, you can opt for the full [Lucene Query Parser](https://lucene.apache.org/core/6_6_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html) syntax for specialized query forms: wildcard, fuzzy search, proximity search, regular expressions. Much of the Lucene Query Parser syntax is [implemented intact in Azure AI Search](search-lucene-query-architecture.md), except for *range searches, which are constructed through **`$filter`** expressions. +When creating queries in Azure AI Search, you can opt for the full [Lucene Query Parser](https://lucene.apache.org/core/6_6_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html) syntax for specialized query forms: wildcard, fuzzy search, proximity search, regular expressions. Much of the Lucene Query Parser syntax is [implemented intact in Azure AI Search](search-lucene-query-architecture.md), except for *range searches*, which are constructed through **`$filter`** expressions. To use full Lucene syntax, set the queryType to `full` and pass in a query expression patterned for wildcard, fuzzy search, or one of the other query forms supported by the full syntax. In REST, query expressions are provided in the **`search`** parameter of a [Search Documents (REST API)](/rest/api/searchservice/documents/search-post) request. diff --git a/articles/search/samples-python.md b/articles/search/samples-python.md index 8ece27e447a..146cc56725b 100644 --- a/articles/search/samples-python.md +++ b/articles/search/samples-python.md @@ -12,7 +12,7 @@ ms.custom: - devx-track-python - ignite-2023 ms.topic: conceptual -ms.date: 08/16/2024 +ms.date: 09/19/2024 --- # Python samples for Azure AI Search @@ -28,21 +28,7 @@ Learn about the Python code samples that demonstrate the functionality and workf ## SDK samples -Code samples from the Azure SDK development team demonstrate API usage. You can find these samples in [**azure-sdk-for-python/tree/main/sdk/search/azure-search-documents/samples**](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/search/azure-search-documents/samples) on GitHub. - -| Samples | Description | -|---------|-------------| -| [Authenticate](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_authentication.py) | Demonstrates how to configure a client and authenticate to the service. | -| [Index Create-Read-Update-Delete operations](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_index_crud_operations.py) | Demonstrates how to create, update, get, list, and delete [search indexes](search-what-is-an-index.md). | -| [Indexer Create-Read-Update-Delete operations](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_indexers_operations.py) | Demonstrates how to create, update, get, list, reset, and delete [indexers](search-indexer-overview.md). | -| [Search indexer data sources](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_indexer_datasource_skillset.py) | Demonstrates how to create, update, get, list, and delete indexer data sources, required for indexer-based indexing of [supported Azure data sources](search-indexer-overview.md#supported-data-sources). | -| [Synonyms](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_synonym_map_operations.py) | Demonstrates how to create, update, get, list, and delete [synonym maps](search-synonyms.md). | -| [Load documents](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_crud_operations.py) | Demonstrates how to upload or merge documents into an index in a [data import](search-what-is-data-import.md) operation. | -| [Simple query](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_simple_query.py) | Demonstrates how to set up a [basic query](search-query-overview.md). | -| [Filter query](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_filter_query.py) | Demonstrates setting up a [filter expression](search-filters.md). | -| [Facet query](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_facet_query.py) | Demonstrates working with [facets](search-faceted-navigation.md). | -| [Semantic ranking sample](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_semantic_search.py) | Shows you how to configure semantic ranking in an index and invoke semantic queries. | -| [Vector search](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/samples/sample_vector_search.py) | Demonstrates how to get embeddings from a description field and then send vector queries against the data. | +[**azure-sdk-for-python/tree/main/sdk/search/azure-search-documents/samples**](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/search/azure-search-documents/samples) on GitHub provides code samples from the Azure SDK development team, demonstrating API usage. ## Doc samples @@ -50,24 +36,18 @@ Code samples from the Azure AI Search team demonstrate features and workflows. M | Samples | Article | |---------|---------| -| [quickstart](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/Quickstart) | Source code for the Python portion of [Quickstart: Full text search using the Azure SDKs](search-get-started-text.md). This article covers the basic workflow for creating, loading, and querying a search index using sample data. | -| [quickstart-semantic-search](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/Quickstart-Semantic-Search) | Source code for the Python portion of [Quickstart: Semantic ranking using the Azure SDKs](search-get-started-semantic.md). It shows the index schema and query request for invoking semantic ranking. | +| [Tutorial-RAG](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/Tutorial-RAG) | Source code for the Python portion of [How to build a RAG solution using Azure AI Search](tutorial-rag-build-solution.md).| +| [Quickstart](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/Quickstart) | Source code for the Python portion of [Quickstart: Full text search using the Azure SDKs](search-get-started-text.md). This article covers the basic workflow for creating, loading, and querying a search index using sample data. | +| [Quickstart-RAG](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/Quickstart-RAG) | Source code for the Python portion of [Quickstart: Generative search (RAG) with grounding data from Azure AI Search](search-get-started-rag.md). | +| [Quickstart-Semantic-Search](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/Quickstart-Semantic-Search) | Source code for the Python portion of [Quickstart: Semantic ranking using the Azure SDKs](search-get-started-semantic.md). It shows the index schema and query request for invoking semantic ranking. | | [bulk-insert](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/bulk-insert) | Source code for the Python example of how to [use the push APIs](search-how-to-load-search-index.md) to upload and index documents. | -| [azure-functions](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/azure-function-search) | Source code for the Python example of an Azure function that sends queries to a search service. You can substitute this Python version of the `api` code used in the [Add search to web sites](tutorial-csharp-overview.md) C# sample. | +| [azure-function-search](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/azure-function-search) | Source code for the Python example of an Azure function that sends queries to a search service. You can substitute this Python version of the `api` code used in the [Add search to web sites](tutorial-csharp-overview.md) C# sample. | ## Demos -A demo repo provides proof-of-concept source code for examples or scenarios shown in demonstrations. Demo solutions aren't designed for adaptation by customers. +[**azure-search-vector-samples**](https://github.com/Azure/azure-search-vector-samples/blob/main/README.md) on GitHub provides a comprehensive collection of samples for vector search scenarios, organized by scenario or technology. -| Repository | Description | -|------------|-------------| -| [azure-search-vector-python-sample.ipynb](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/basic-vector-workflow/azure-search-vector-python-sample.ipynb) | Uses the **azure.search.documents** library in the Azure SDK for Python to create, load, and query a vector index. | -| [azure-search-integrated-vectorization-sample.ipynb](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/integrated-vectorization/azure-search-integrated-vectorization-sample.ipynb) | Extends the vector indexing workflow to include integrated data chunking and embedding. | -| [azure-search-custom-vectorization-sample.ipynb](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/custom-vectorizer/azure-search-custom-vectorization-sample.ipynb) | Demonstrates custom vectorization. | -| [azure-search-vector-python-huggingface-model-sample.ipynb](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/community-integration/hugging-face/azure-search-vector-python-huggingface-model-sample.ipynb) | Hugging Face integration. | -| [azure-search-vector-python-langchain-sample.ipynb](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/community-integration/langchain/azure-search-vector-python-langchain-sample.ipynb) | LangChain integration. | -| [azure-search-vector-python-llamaindex-sample.ipynb](https://github.com/Azure/azure-search-vector-samples/blob/main/demo-python/code/community-integration/llamaindex/azure-search-vector-python-llamaindex-sample.ipynb) | Llamaindex integration. | -| [azure-search-openai-demo](https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/README.md) | ChatGPT + Enterprise data with Azure OpenAI Python code showing how to use Azure AI Search with the large language models in Azure OpenAI. For background, see this Tech Community blog post: [Revolutionize your Enterprise Data with ChatGPT](https://techcommunity.microsoft.com/t5/ai-applied-ai-blog/revolutionize-your-enterprise-data-with-chatgpt-next-gen-apps-w/ba-p/3762087). | +We also recommend [**azure-search-openai-demo**](https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/README.md). This is a ChatGPT-like experience over enterprise data with Azure OpenAI Python code showing how to use Azure AI Search with the large language models in Azure OpenAI. For background, see this Tech Community blog post: [Revolutionize your Enterprise Data with ChatGPT](https://techcommunity.microsoft.com/t5/ai-applied-ai-blog/revolutionize-your-enterprise-data-with-chatgpt-next-gen-apps-w/ba-p/3762087). | ## Other samples diff --git a/articles/search/search-filters.md b/articles/search/search-filters.md index 7306ec54c6b..64a89d8f129 100644 --- a/articles/search/search-filters.md +++ b/articles/search/search-filters.md @@ -7,18 +7,20 @@ manager: nitinme author: HeidiSteen ms.author: heidist ms.service: cognitive-search -ms.topic: conceptual -ms.date: 02/22/2024 +ms.topic: concept-article +ms.date: 09/19/2024 ms.custom: - devx-track-csharp - ignite-2023 --- -# Filters in text queries +# Filters in keyword search -A *filter* provides value-based criteria for including or excluding content before query execution. For example, including or excluding documents based on dates, locations, or language. Filters are specified on individual fields. A field definition must be attributed as "filterable" if you want to use it in filter expressions. +A *filter* provides value-based criteria for including or excluding content before query execution for keyword search, or before or after query execution for vector search. Filters are applied to nonvector fields, but can be used in vector search if documents include nonvector fields. For example, for indexes organized around chunked content, you might have parent-level fields or metadata fields that can be filtered. -A filter is specified using [OData filter expression syntax](search-query-odata-filter.md). In contrast with full text search, a filter succeeds only if the match is exact. +This article explains filtering for keyword search. For more information about vectors, see [Add a filter in a vector query](vector-search-filters.md). + +A filter is specified using [OData filter expression syntax](search-query-odata-filter.md). In contrast with keyword and vector search, a filter succeeds only if the match is exact. ## When to use a filter @@ -42,7 +44,9 @@ At query time, a filter parser accepts criteria as input, converts the expressio Filtering occurs in tandem with search, qualifying which documents to include in downstream processing for document retrieval and relevance scoring. When paired with a search string, the filter effectively reduces the recall set of the subsequent search operation. When used alone (for example, when the query string is empty where `search=*`), the filter criteria is the sole input. -## Defining filters +## How filters are defined + +Filters apply to alphanumeric content on fields that are attributed as `filterable`. Filters are OData expressions, articulated in the [filter syntax](search-query-odata-filter.md) supported by Azure AI Search. @@ -112,7 +116,7 @@ The following examples illustrate several usage patterns for filter scenarios. F In the REST API, filterable is *on* by default for simple fields. Filterable fields increase index size; be sure to set `"filterable": false` for fields that you don't plan to actually use in a filter. For more information about settings for field definitions, see [Create Index](/rest/api/searchservice/indexes/create). -In the .NET SDK, the filterable is *off* by default. You can make a field filterable by setting the [IsFilterable property](/dotnet/api/azure.search.documents.indexes.models.searchfield.isfilterable) of the corresponding [SearchField](/dotnet/api/azure.search.documents.indexes.models.searchfield) object to `true`. In the next example, the attribute is set on the `Rating` property of a model class that maps to the index definition. +In the Azure SDKs, filterable is *off* by default. You can make a field filterable by setting the [IsFilterable property](/dotnet/api/azure.search.documents.indexes.models.searchfield.isfilterable) of the corresponding [SearchField](/dotnet/api/azure.search.documents.indexes.models.searchfield) object to `true`. In the next example, the attribute is set on the `Rating` property of a model class that maps to the index definition. ```csharp [SearchField(IsFilterable = true, IsSortable = true, IsFacetable = true)] diff --git a/articles/search/search-limits-quotas-capacity.md b/articles/search/search-limits-quotas-capacity.md index b788e147327..ffbe73e6e92 100644 --- a/articles/search/search-limits-quotas-capacity.md +++ b/articles/search/search-limits-quotas-capacity.md @@ -78,9 +78,14 @@ When estimating document size, remember to consider only those fields that add v ## Vector index size limits -When you index documents with vector fields, Azure AI Search constructs internal vector indexes using the algorithm parameters you provide. The size of these vector indexes is restricted by the memory reserved for vector search for your service's tier (or `SKU`). +When you index documents with vector fields, Azure AI Search constructs internal vector indexes using the algorithm parameters you provide. The size of these vector indexes is restricted by the memory reserved for vector search for your service's tier (or `SKU`). For guidance on managing and maximizing vector storage, see [Vector index size and staying under limits](vector-search-index-size.md). -Vector limits vary by [service creation date](vector-search-index-size.md#how-to-check-service-creation-date) and [tier](search-sku-tier.md). For guidance on managing and maximizing vector storage, see [Vector index size and staying under limits](vector-search-index-size.md). +Vector limits vary by: + ++ [service creation date](vector-search-index-size.md#how-to-check-service-creation-date) ++ [region](search-region-support.md) + +Higher vector limits from April 2024 onwards exist on *new search services* in regions providing the extra capacity, which is most of them. This table shows the progression of vector quota increases in GB over time. The quota is per partition, so if you scale a new Standard (S1) service to 6 partitions, total vector quota is 35 multiplied by 6. @@ -99,7 +104,7 @@ This table shows the progression of vector quota increases in GB over time. The 4 Higher vector quota for more tiers and regions based on partition size updates. -The service enforces a vector index size quota **for every partition** in your search service. Each extra partition increases the available vector index size quota. This quota is a hard limit to ensure your service remains healthy, which means that further indexing attempts once the limit is exceeded results in failure. You can resume indexing once you free up available quota by either deleting some vector documents or by scaling up in partitions. +The service enforces a vector index size quota *for every partition* in your search service. Each extra partition increases the available vector index size quota. This quota is a hard limit to ensure your service remains healthy, which means that further indexing attempts once the limit is exceeded results in failure. You can resume indexing once you free up available quota by either deleting some vector documents or by scaling up in partitions. > [!IMPORTANT] > Higher vector limits are tied to larger partition sizes. Regions that run on older infrastructure are subject to the July-April limits. Review the [regions list](search-region-support.md) for status on partition storage limits. diff --git a/articles/search/search-region-support.md b/articles/search/search-region-support.md index 8b9d2379302..18fe6847232 100644 --- a/articles/search/search-region-support.md +++ b/articles/search/search-region-support.md @@ -76,11 +76,11 @@ You can create an Azure AI Search resource in any of the following Azure public | North Europe​​ | ✅ | ✅ | ✅ | | West Europe​​ 1, 2| ✅ | ✅ | ✅ | | France Central​​ | ✅ | ✅ | ✅ | -| Germany West Central​ 2 ​| ✅ | | ✅ | +| Germany West Central​ ​| ✅ | | ✅ | | Italy North​​ | | | ✅ | | Norway East​​ | ✅ | | ✅ | | Poland Central​​ | | | | -| Spain Central | | | ✅ | +| Spain Central 2 | | | ✅ | | Sweden Central​​ | ✅ | | ✅ | | Switzerland North​ | ✅ | ✅ | ✅ | | Switzerland West​ | ✅ | ✅ | ✅ | @@ -107,9 +107,7 @@ You can create an Azure AI Search resource in any of the following Azure public | Region | AI integration | Semantic ranking | Availability zones | |--|--|--|--| -| South Africa North​ 1 | ✅ | | ✅ | - -1 This region runs on older infrastructure that has lower storage limits per partition at every tier. Choose a different region if you want [higher limits](search-limits-quotas-capacity.md#service-limits). +| South Africa North​ | ✅ | | ✅ | ### Asia Pacific diff --git a/articles/search/search-what-is-data-import.md b/articles/search/search-what-is-data-import.md index c8985a852ce..64f29f5cad1 100644 --- a/articles/search/search-what-is-data-import.md +++ b/articles/search/search-what-is-data-import.md @@ -9,9 +9,10 @@ ms.author: heidist ms.service: cognitive-search ms.custom: - ignite-2023 -ms.topic: conceptual -ms.date: 01/17/2024 +ms.topic: concept-article +ms.date: 09/17/2024 --- + # Data import in Azure AI Search In Azure AI Search, queries execute over user-owned content that's loaded into a [search index](search-what-is-an-index.md). This article describes the two basic workflows for populating an index: *push* your data into the index programmatically, or *pull* in the data using a [search indexer](search-indexer-overview.md). @@ -19,7 +20,7 @@ In Azure AI Search, queries execute over user-owned content that's loaded into a Both approaches load documents from an external data source. Although you can create an empty index, it's not queryable until you add the content. > [!NOTE] -> If [AI enrichment](cognitive-search-concept-intro.md) is a solution requirement, you must use the pull model (indexers) to load an index. Skillsets are attached to an indexer and don't run independently. +> If [AI enrichment](cognitive-search-concept-intro.md) or [integrated vectorization](vector-search-integrated-vectorization.md) are solution requirements, you must use the pull model (indexers) to load an index. Skillsets are attached to indexers and don't run independently. ## Pushing data to an index @@ -77,7 +78,8 @@ The pull model uses *indexers* connecting to a supported data source, automatica + [Azure Files (preview)](search-file-storage-integration.md) + [Azure Cosmos DB](search-howto-index-cosmosdb.md) + [Azure SQL Database, SQL Managed Instance, and SQL Server on Azure VMs](search-howto-connecting-azure-sql-database-to-azure-search-using-indexers.md) -+ [SharePoint in Microsoft 365 (preview)](search-howto-index-sharepoint-online.md) ++ [OneLake files and shortcuts](search-how-to-index-onelake-files.md) ++ [SharePoint Online (preview)](search-howto-index-sharepoint-online.md) You can use third-party connectors, developed and maintained by Microsoft partners. For more information and links, see [Data source gallery](search-data-sources-gallery.md). @@ -87,7 +89,7 @@ Indexers connect an index to a data source (usually a table, view, or equivalent Use the following tools and APIs for indexer-based indexing: -+ [Import data wizard in the Azure portal](search-import-data-portal.md) ++ [Import data wizard or Import and vectorize data wizard](search-import-data-portal.md) + REST APIs: [Create Indexer (REST)](/rest/api/searchservice/indexers/create), [Create Data Source (REST)](/rest/api/searchservice/data-sources/create), [Create Index (REST)](/rest/api/searchservice/indexes/create) + Azure SDK for .NET: [SearchIndexer](/dotnet/api/azure.search.documents.indexes.models.searchindexer), [SearchIndexerDataSourceConnection](/dotnet/api/azure.search.documents.indexes.models.searchindexerdatasourceconnection), [SearchIndex](/dotnet/api/azure.search.documents.indexes.models.searchindex), + Azure SDK for Python: [SearchIndexer](/python/api/azure-search-documents/azure.search.documents.indexes.models.searchindexer), [SearchIndexerDataSourceConnection](/python/api/azure-search-documents/azure.search.documents.indexes.models.searchindexerdatasourceconnection), [SearchIndex](/python/api/azure-search-documents/azure.search.documents.indexes.models.searchindex), diff --git a/articles/search/vector-search-how-to-generate-embeddings.md b/articles/search/vector-search-how-to-generate-embeddings.md index 34cd4f93637..97e47cd7a47 100644 --- a/articles/search/vector-search-how-to-generate-embeddings.md +++ b/articles/search/vector-search-how-to-generate-embeddings.md @@ -9,12 +9,12 @@ ms.service: cognitive-search ms.custom: - ignite-2023 ms.topic: how-to -ms.date: 08/05/2024 +ms.date: 09/19/2024 --- # Generate embeddings for search queries and documents -Azure AI Search doesn't host vectorization models, so one of your challenges is creating embeddings for query inputs and outputs. You can use any embedding model, but this article assumes Azure OpenAI embedding models. +Azure AI Search doesn't host vectorization models, so one of your challenges is creating embeddings for query inputs and outputs. You can use any supported embedding model, but this article assumes Azure OpenAI embedding models for the steps. We recommend [integrated vectorization](vector-search-integrated-vectorization.md), which provides built-in data chunking and vectorization. Integrated vectorization takes a dependency on indexers, skillsets, and built-in or custom skills that point to a model that executes externally from Azure AI Search. @@ -30,13 +30,13 @@ If you want to handle data chunking and vectorization yourself, we provide demos ## Create resources in the same region -If you want resources in the same region, start with: +Integrated vectorization requires resources to be in the same region: -1. [Check regions for a text embedding model](/azure/ai-services/openai/concepts/models#standard-and-global-standard-deployment-model-quota). +1. [Check regions for a text embedding model](/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability). 1. [Find the same region for Azure AI Search](search-region-support.md). -1. To support hybrid queries that include [semantic ranking](semantic-how-to-query-request.md), or if you want to try machine learning model integration using a [custom skill](cognitive-search-custom-skill-interface.md) in an [AI enrichment pipeline](cognitive-search-concept-intro.md), note the Azure AI Search regions that provide those features. +1. To support hybrid queries that include [semantic ranking](semantic-how-to-query-request.md), or if you want to try machine learning model integration using a [custom skill](cognitive-search-custom-skill-interface.md) in an [AI enrichment pipeline](cognitive-search-concept-intro.md), select an Azure AI Search region that provides those features. ## Generate an embedding for an improvised query @@ -64,12 +64,18 @@ Output is a vector array of 1,536 dimensions. ## Tips and recommendations for embedding model integration -+ **Identify use cases**: Evaluate the specific use cases where embedding model integration for vector search features can add value to your search solution. This can include matching image content with text content, cross-lingual searches, or finding similar documents. ++ **Identify use cases**: Evaluate the specific use cases where embedding model integration for vector search features can add value to your search solution. This can include multimodal or matching image content with text content, multilingual search, or similarity search. + + **Design a chunking strategy**: Embedding models have limits on the number of tokens they can accept, which introduces a data chunking requirement for large files. For more information, see [Chunk large documents for vector search solutions](vector-search-how-to-chunk-documents.md). -+ **Optimize cost and performance**: Vector search can be resource-intensive and is subject to maximum limits, so consider only vectorizing the fields that contain semantic meaning. [Reduce vector size]() so that you can store more vectors for the same price. + ++ **Optimize cost and performance**: Vector search can be resource-intensive and is subject to maximum limits, so consider only vectorizing the fields that contain semantic meaning. [Reduce vector size](vector-search-how-to-configure-compression-storage.md) so that you can store more vectors for the same price. + + **Choose the right embedding model:** Select an appropriate model for your specific use case, such as word embeddings for text-based searches or image embeddings for visual searches. Consider using pretrained models like **text-embedding-ada-002** from OpenAI or **Image Retrieval** REST API from [Azure AI Computer Vision](/azure/ai-services/computer-vision/how-to/image-retrieval). + + **Normalize Vector lengths**: Ensure that the vector lengths are normalized before storing them in the search index to improve the accuracy and performance of similarity search. Most pretrained models already are normalized but not all. + + **Fine-tune the model**: If needed, fine-tune the selected model on your domain-specific data to improve its performance and relevance to your search application. + + **Test and iterate**: Continuously test and refine your embedding model integration to achieve the desired search performance and user satisfaction. ## Next steps @@ -77,3 +83,4 @@ Output is a vector array of 1,536 dimensions. + [Understanding embeddings in Azure OpenAI Service](/azure/ai-services/openai/concepts/understand-embeddings) + [Learn how to generate embeddings](/azure/ai-services/openai/how-to/embeddings?tabs=console) + [Tutorial: Explore Azure OpenAI Service embeddings and document search](/azure/ai-services/openai/tutorials/embeddings?tabs=command-line) ++ [Tutorial: Choose a model (RAG solutions in Azure AI Search)](tutorial-rag-build-solution-models.md) diff --git a/articles/search/vector-search-ranking.md b/articles/search/vector-search-ranking.md index 8514d8e22fa..c182384df05 100644 --- a/articles/search/vector-search-ranking.md +++ b/articles/search/vector-search-ranking.md @@ -8,8 +8,8 @@ ms.author: jlembicz ms.service: cognitive-search ms.custom: - ignite-2023 -ms.topic: conceptual -ms.date: 08/05/2024 +ms.topic: concept-article +ms.date: 09/19/2024 --- # Relevance in vector search diff --git a/articles/search/vector-store.md b/articles/search/vector-store.md index f9b51460cd9..3e6fbf39703 100644 --- a/articles/search/vector-store.md +++ b/articles/search/vector-store.md @@ -8,15 +8,15 @@ ms.author: robertlee ms.service: cognitive-search ms.custom: - ignite-2023 -ms.topic: conceptual -ms.date: 02/14/2024 +ms.topic: concept-article +ms.date: 09/19/2024 --- # Vector storage in Azure AI Search Azure AI Search provides vector storage and configurations for [vector search](vector-search-overview.md) and [hybrid search](hybrid-search-overview.md). Support is implemented at the field level, which means you can combine vector and nonvector fields in the same search corpus. -Vectors are stored in a search index. Use the [Create Index REST API](/rest/api/searchservice/indexes/create-or-update) or an equivalent Azure SDK method to [create the vector store](vector-search-how-to-create-index.md). +Vectors are stored in a search index. Use the [Create Index REST API](/rest/api/searchservice/indexes/create) or an equivalent Azure SDK method to [create the vector store](vector-search-how-to-create-index.md). Considerations for vector storage include the following points: @@ -54,7 +54,7 @@ Vector fields are distinguished by their data type and vector-specific propertie } ``` -Vector fields are of type `Collection(Edm.Single)`. +Vector fields have [specific data types](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields). Currently, `Collection(Edm.Single)` is the most common, but using narrow data types can save on storage. Vector fields must be searchable and retrievable, but they can't be filterable, facetable, or sortable, or have analyzers, normalizers, or synonym map assignments. @@ -146,7 +146,7 @@ Here's a screenshot showing search results in [Search Explorer](search-explorer. ## Physical structure and size -In Azure AI Search, the physical structure of an index is largely an internal implementation. You can access its schema, load and query its content, monitor its size, and manage capacity, but the clusters themselves (inverted and vector indexes), and other files and folders) are managed internally by Microsoft. +In Azure AI Search, the physical structure of an index is largely an internal implementation. You can access its schema, load and query its content, monitor its size, and manage capacity, but the clusters themselves (inverted and vector indexes), and other files and folders are managed internally by Microsoft. The size and substance of an index is determined by: diff --git a/zone-pivots/zone-pivot-groups.yml b/zone-pivots/zone-pivot-groups.yml index 4b7d1d854cd..8622cf1ce28 100644 --- a/zone-pivots/zone-pivot-groups.yml +++ b/zone-pivots/zone-pivot-groups.yml @@ -119,8 +119,6 @@ groups: title: Programming languages prompt: Choose your preferred usage method pivots: - - id: programming-language-studio - title: Azure OpenAI Studio - id: programming-language-ai-studio title: AI Studio (Preview) - id: programming-language-csharp