-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add LmStudioPromptDriver and LmStudioEmbeddingDriver #955
Conversation
Test fixes
4c18c4e
to
8f8995c
Compare
client: Optionally provide custom `openai.OpenAI` client. | ||
""" | ||
|
||
base_url: str = field(default="http://localhost:1234/v1", kw_only=True, metadata={"serializable": True}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been getting feedback from users that localhost doesn't always resolve for them (hit this with lmstudio and ollama). I had to change my default to 127.0.0.1 in order to resolve it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting! Even the LM Studio code snippets use localhost. Updated though.
curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF",
"messages": [
{ "role": "system", "content": "Always answer in rhymes." },
{ "role": "user", "content": "Introduce yourself." }
],
"temperature": 0.7,
"max_tokens": -1,
"stream": true
}'
|
||
|
||
@define | ||
class OpenAiStructureConfig(StructureConfig): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
class OpenAiStructureConfig(StructureConfig): | |
class LmStudioStructureConfig(StructureConfig): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vachillo this was an accidental check in. I'm actually not sure what a Structure Config would look like for LM Studio since there are no default models.
Suggestions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah i see. i would say either make it all the same as the OpenAi default models or just remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah removed for now.
f7ff4d9
to
2a7df61
Compare
2a7df61
to
d9f483f
Compare
a06442c
to
9c44a62
Compare
d029a69
to
577c7fb
Compare
577c7fb
to
4ddaa95
Compare
4ddaa95
to
fd1c558
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
f87bd01
to
4fa3bf3
Compare
67d7287
to
2d4d889
Compare
Describe your changes
Adds Prompt Driver and Embedding Driver for LM Studio
Issue ticket number and link
Closes #890
📚 Documentation preview 📚: https://griptape--955.org.readthedocs.build//955/