Can Azure OpenAI Service be supported?

[d5ve]

Thanks for smartcat, it looks like it will fit perfectly into my workflow (work from a laptop ssh-d into a remote linux server).

We have loads of credits for the Azure-hosted OpenAI service, and I was wondering how difficult it would be to support those API endpoints.

See https://learn.microsoft.com/en-us/azure/ai-services/openai/reference for some info about how the API hostnames are composed.

POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2024-06-01

POST https://{endpoint}/openai/deployments/{deployment-id}/completions?api-version=2024-06-01

I took a look at src/config/api.rs and perhaps the config could be changed (a) support a new Azure type, and (b) to allow a user to enter the URL in the config file. The API key stuff looks the same between Azure and regular OpenAI.

Cheers,

Dave

[efugier]

Hey, I'd definitely want to add support for more apis!

You can actually already specify an url for all APIs, if the protocol is the same as one of them then it should work! (the way messages are sent, field names etc)

[openai]
api_key = "your api key"
# api_key_command = "pass openai/api_key"
url = "https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2024-06-01"
default_model = "gpt-4o"

But reading the doc I'm afraid they have a custom way to encode them (prompt field in the request) so it likely won't work. Why can't people agree on a standard 😅

I'll get to it when I find some time, in the meantime contributions are welcomed 🙂

[d5ve]

Cheers @efugier!

I gave it a try today using your suggestion, but ran into auth errors.

My config was like:

[openai]
api_key = "badf00ddeadbeef"
url = "https://myexample.openai.azure.com/openai/deployments/my-deployment/chat/completions?api-version=2024-06-01"
version = "2024-06-01"
default_model = "gpt-4o"

Then I ran sc and got the following error:

$ sc --api openai --model gpt-4o "say hi"
thread 'main' panicked at src/utils.rs:12:9:
API request failed with status 401 Unauthorized: { "statusCode": 401, "message": "Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired." }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Abort trap: 6

The following curl does work for my Azure deployment:

$ curl -s "https://myexample.openai.azure.com/openai/deployments/my-deployment/chat/completions?api-version=2024-06-01" \
    -H "Content-Type: application/json" \
    -H "api-key: badf00ddeadbeef" \
    -d "$payload_json"

[efugier]

Yeah I guess auth method isn't the same, I can make it work given time but I'm a bit under heavy load atm.

In the meantime if you want to try smartcat you can use Groq which provides free api access with great models!

Otherwise it remains pretty cheap given the services provided, I personally use Claude 3.5 sonnet and it only costs me about 2 to 3$ a month for a regular usage.

[d5ve]

No worries @efugier, I'll take a look at the auth myself. I'll let you know if I work something out.

[d5ve]

My minimum-effort too-lazy-to-learn-rust dirty hack below gets things working for my Azure-hosted deployment.

git diff
diff --git a/src/text/api_call.rs b/src/text/api_call.rs
index a4354d5..93ef4fe 100644
--- a/src/text/api_call.rs
+++ b/src/text/api_call.rs
@@ -64,8 +64,7 @@ pub fn post_prompt_and_get_answer(
     // Add auth if necessary
     let request = match prompt.api {
         Api::Openai | Api::Mistral | Api::Groq => request.header(
-            "Authorization",
-            &format!("Bearer {}", &api_config.get_api_key()),
+            "api-key", &api_config.get_api_key()
         ),
         Api::Anthropic => request
             .header("x-api-key", &api_config.get_api_key())

I'll just use this for now thanks!

[d5ve]

I opened #35 to more cleanly add Azure OpenAI support.

[efugier]

Support is now available starting from 2.1.0, thanks a lot @d5ve!