Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LmStudioPromptDriver and LmStudioEmbeddingDriver #955

Closed
wants to merge 94 commits into from

Conversation

collindutter
Copy link
Member

@collindutter collindutter commented Jul 10, 2024

Describe your changes

Adds Prompt Driver and Embedding Driver for LM Studio

Issue ticket number and link

Closes #890


📚 Documentation preview 📚: https://griptape--955.org.readthedocs.build//955/

@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch from 4c18c4e to 8f8995c Compare July 10, 2024 16:46
client: Optionally provide custom `openai.OpenAI` client.
"""

base_url: str = field(default="http://localhost:1234/v1", kw_only=True, metadata={"serializable": True})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been getting feedback from users that localhost doesn't always resolve for them (hit this with lmstudio and ollama). I had to change my default to 127.0.0.1 in order to resolve it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! Even the LM Studio code snippets use localhost. Updated though.

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ 
    "model": "lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF",
    "messages": [ 
      { "role": "system", "content": "Always answer in rhymes." },
      { "role": "user", "content": "Introduce yourself." }
    ], 
    "temperature": 0.7, 
    "max_tokens": -1,
    "stream": true
}'



@define
class OpenAiStructureConfig(StructureConfig):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
class OpenAiStructureConfig(StructureConfig):
class LmStudioStructureConfig(StructureConfig):

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vachillo this was an accidental check in. I'm actually not sure what a Structure Config would look like for LM Studio since there are no default models.

Suggestions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah i see. i would say either make it all the same as the OpenAi default models or just remove it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah removed for now.

@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch 2 times, most recently from f7ff4d9 to 2a7df61 Compare July 10, 2024 17:26
@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch from 2a7df61 to d9f483f Compare July 10, 2024 17:30
@collindutter collindutter mentioned this pull request Jul 10, 2024
1 task
@collindutter collindutter force-pushed the feature/native-functions branch from a06442c to 9c44a62 Compare July 10, 2024 20:30
@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch 2 times, most recently from d029a69 to 577c7fb Compare July 10, 2024 21:02
Base automatically changed from feature/native-functions to dev July 10, 2024 22:00
@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch from 577c7fb to 4ddaa95 Compare July 10, 2024 22:03
@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch from 4ddaa95 to fd1c558 Compare July 10, 2024 22:04
@collindutter collindutter requested review from shhlife and vachillo July 10, 2024 22:05
Copy link

codecov bot commented Jul 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch from f87bd01 to 4fa3bf3 Compare July 11, 2024 00:53
@collindutter collindutter force-pushed the feature/lm-studio-prompt-driver branch from 67d7287 to 2d4d889 Compare July 11, 2024 17:27
@collindutter collindutter deleted the feature/lm-studio-prompt-driver branch August 21, 2024 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a driver for LMStudio
3 participants