LLMs Deployed on AzureMLStudio

[Manikanta5112]

Hi,

I deployed llama_2, gpt-3.5 models on Azure ML studio. Now, I want to use those models with llama index.
with langchain it was quite easy, can some help me how to work those with llama index

[dosubot[bot]]

🤖

Hi,

To integrate the llama_2 and gpt-3.5 models deployed on Azure ML Studio with the LlamaIndex, you can create an instance of the AzureOpenAI class for each model. Here's an example of how you can do this:

from llama_index.llms.azure_openai import AzureOpenAI

# Create an instance of AzureOpenAI for the llama_2 model
llama_2 = AzureOpenAI(
    model="llama_2",
    engine="your_llama_2_deployment_name",
    api_key="your_api_key",
    api_base="https://YOUR_RESOURCE_NAME.openai.azure.com/",
    api_type="azure",
    api_version="2023-05-15"
)

# Create an instance of AzureOpenAI for the gpt-3.5 model
gpt_35 = AzureOpenAI(
    model="gpt-3.5",
    engine="your_gpt_3.5_deployment_name",
    api_key="your_api_key",
    api_base="https://YOUR_RESOURCE_NAME.openai.azure.com/",
    api_type="azure",
    api_version="2023-05-15"
)

Please replace "your_llama_2_deployment_name", "your_gpt_3.5_deployment_name", "your_api_key", and "YOUR_RESOURCE_NAME" with your actual Azure deployment names, API key, and resource name respectively.

Remember to set the environment variables OPENAI_API_TYPE, OPENAI_API_VERSION, OPENAI_API_BASE, and OPENAI_API_KEY as mentioned in the class docstring.

The AzureOpenAI class in the LlamaIndex codebase is designed to interact with models deployed on Azure OpenAI. It requires the specification of an engine parameter to identify the deployment (also known as "model deployment name" in Azure portal). The class also requires certain environment variables to be set, such as OPENAI_API_TYPE, OPENAI_API_VERSION, OPENAI_API_BASE, and OPENAI_API_KEY.

You can find more information about the AzureOpenAI class in the llama_index/llms/azure_openai.py file.

Let me know if you have any other questions.

Sources

• llama_index/llms/azure_openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Manikanta5112]

is it same for chat based models like llama_13_B_chat and foundational models like llama_13_B ?