Standardize access via LiteLLM

[345ishaan]

I want to try out latest Gemini models and seems like there are two ways to access them.

1. Google AI API
2. Vertex AI API

I think I created my API key using the 1) and when i run:

from litellm import completion
import os

os.environ['GEMINI_API_KEY'] = ""
response = completion(
    model="gemini-1.5-flash", 
    messages=[{"role": "user", "content": "write code for saying hi from LiteLLM"}]
)

I get the following error:

APIConnectionError: litellm.APIConnectionError: Your default credentials were not found. To set up Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information.

I am able to use my API key though if i run like this:

import google.generativeai as google_genai


google_genai.configure(api_key="")

llm = google_genai.GenerativeModel("gemini-1.5-flash")
response = llm.generate_content(["write code for saying hi from LiteLLM"])

[holtskinner]

Ok, you are using Google AI Studio (Also known as the Gemini Developer API).

And the LiteLLM documentation shows this code to use that API.

from litellm import completion
import os

os.environ['GEMINI_API_KEY'] = ""
response = completion(
    model="gemini/gemini-pro", 
    messages=[{"role": "user", "content": "write code for saying hi from LiteLLM"}]
)

Yours is slightly different, you don't have the gemini/ prefix to your model name.

If you have further issues with LiteLLM, I recommend filing an issue in their repository here: https://github.com/BerriAI/litellm