Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAI response compatibility #1103

Closed
xkaraman opened this issue Sep 25, 2023 · 1 comment · Fixed by #1164
Closed

OpenAI response compatibility #1103

xkaraman opened this issue Sep 25, 2023 · 1 comment · Fixed by #1164
Labels
bug Something isn't working good first issue Good for newcomers roadmap

Comments

@xkaraman
Copy link

LocalAI version:
latest docker container with `docker compose up -d --pull always'

Environment, CPU architecture, OS, and Version:

Linux ***** 5.4.0-150-generic #167~18.04.1-Ubuntu SMP Wed May 24 00:51:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug
Response object should define an id and created keys to be fully compatible as per the official OpenAI documentation if it's going to be a drop in replacement. ChatDev for example read this key and it crashes with KeyError.

To Reproduce
docker compose up -d --pull always

 curl http://localhost:8081/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "gpt-3.5-turbo-16k-0613",
     "messages": [{"role": "user", "content": "What is an alpaca?"}],
     "temperature": 0.2
   }'

Response:

{"object":"chat.completion","model":"gpt-3.5-turbo-16k-0613","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Alpacas are domesticated animals native to South America, specifically the Andean region. They belong to the camelid family, which also includes llamas and vicuñas. Alpacas are known for their soft, luxurious fleece, which is made up of 80% protein and is considered one of the warmest and lightest natural fibers in the world.\n\nAlpacas are small animals, typically standing between 36 and 45 inches (90-114 cm) tall at the shoulder and weighing between 100 and 200 pounds (45-90 kg). They have a distinctive appearance, with a long, flat face, large ears, and a bushy tail. Their fleece is their most notable feature, as it is incredibly soft, lightweight, and warm. Alpacas are also known for their gentle nature and intelligence, making them popular as pets and working animals.\n\nAlpacas have been bred for thousands of years in South America, primarily for their fleece, which was highly valued by the Incas and other ancient civilizations. Today, alpacas are raised worldwide for their fleece, as well as for their meat and milk. In addition to their practical uses, alpacas have also become popular as pets and show animals, with many breeders and owners dedicating themselves to preserving and improving the breed."}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}%

Expected behavior
Response object should include id and created keys along with object, model, choices,usage

@xkaraman xkaraman added the bug Something isn't working label Sep 25, 2023
@mudler mudler added the roadmap label Sep 30, 2023
@mudler mudler added the good first issue Good for newcomers label Oct 9, 2023
@mudler mudler removed their assignment Oct 9, 2023
@jespino
Copy link
Contributor

jespino commented Oct 11, 2023

I'm taking a look to this and I have a solution. I'll submit the PR soon.

mudler pushed a commit that referenced this issue Oct 12, 2023
…onses (#1164)

Adding the extra ID and Created fields to any request to the OpenAI
Compatible API to improve the compatibility.

This PR fixes #1103
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers roadmap
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants