Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(grammar): mark grammar disabled by default #2919

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mudler
Copy link
Owner

@mudler mudler commented Jul 19, 2024

Description

This PR disable grammar by default and make that opt-in. More and more models now support function calling, and grammar support is better to be enabled on these model that fits this logic, or alternatively enabled when want to constrain models that doesn't support function calls directly.

Copy link

netlify bot commented Jul 19, 2024

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 4ebf3c7
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/669a15832e6d9c00088c95c5
😎 Deploy Preview https://deploy-preview-2919--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Repository owner deleted a comment from localai-bot Jul 19, 2024
@localai-bot
Copy link
Contributor

This PR diff introduces a few changes:

  1. In core/http/endpoints/openai/chat.go, there is a change in the condition for using a grammar function. Previously, the condition was !config.FunctionsConfig.GrammarConfig.NoGrammar && shouldUseFn. Now, the condition has been updated to config.FunctionsConfig.GrammarConfig.EnableGrammar && shouldUseFn. This change allows the grammar parsing to be enabled or disabled based on the EnableGrammar field in the configuration.

  2. In the gallery/mudler.yaml file, there is an addition of the grammar field under the function section. The grammar field has a sub-field enable which is set to true. This enables the grammar parsing for the Mudler gallery.

  3. In pkg/functions/parse.go, there is a change in the GrammarConfig struct. The NoGrammar field has been renamed to EnableGrammar, and its type has been updated to a boolean. This change aligns with the changes made in the chat.go file.

These changes seem to be related to enabling or disabling grammar parsing in the OpenAI chat functionality. The logic for grammar parsing has been updated to consider an EnableGrammar boolean flag in the configuration, allowing the feature to be toggled on or off depending on the specific use-case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants