-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Difficulty getting started! #352
Comments
for now, getting this error: |
Hello @mcchung52 I had similar issues. Please check this issue: #350 |
Hi, Does it help ? |
LMQL looks very promising (having played w/ Guidance) so I want to make this work but having issues from get go, trying to run it locally. I'm really hoping I can get some help.
IMMEDIATE GOAL: what is the simplest way to make this work?
Context:
I have several gguf models in my comp that I want to run in my Macbook pro (pre-M, intel), basically via CPU which I ran previously many times in python code, though slow.
I want to:
1.run model directly in python code
2a.run model by exposing via api, like localhost:8081
2b.(can't in my mac but can in pc) run gguf via LM Studio and expose ip:port in PC and have python code in mac tap into it
Code:
Thanks in advance.
The text was updated successfully, but these errors were encountered: