Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🛡️ feat: Google Reverse Proxy support, CIVIC_INTEGRITY harm category #4869

Closed
wants to merge 2 commits into from

Conversation

KiGamji
Copy link
Contributor

@KiGamji KiGamji commented Dec 5, 2024

Summary

This PR implements reverse proxy support for Google endpoint, with the ability to pass the API key in Authorization header, as some reverse proxies don't properly handle x-goog-api-key. This is controlled by the GOOGLE_AUTH_HEADER variable, which defaults to false and is commented out by default. I have tested it, and it works perfectly fine, though it won't work with oai-reverse-proxy, as it is broken and passes some weird info data in chunks which makes the google-genai library crash. Further changes could include sanitization for chunks starting with ':', for anyone willing to implement proper support for this reverse proxy.

Change Type

Please delete any irrelevant options.

  • New feature (non-breaking change which adds functionality)

@KiGamji
Copy link
Contributor Author

KiGamji commented Dec 5, 2024

it won't work with oai-reverse-proxy

also it gives this error, when the request gets sent from oai-reverse-proxy to the upstream API: "issues":"systemInstruction.role: Invalid enum value. Expected 'user' | 'model', received 'system'". pretty sure this comes from it's middleware. adding and setting it to 'user' from LibreChat seems to work, but that's a dirty solution too

@danny-avila
Copy link
Owner

Have you tested with Vertex AI and Gemini API?

@KiGamji
Copy link
Contributor Author

KiGamji commented Dec 11, 2024

Have you tested with Vertex AI and Gemini API?

Only Gemini API, shouldn't break Vertex AI though

@KiGamji
Copy link
Contributor Author

KiGamji commented Dec 11, 2024

the new request options are only set for GoogleGenerativeAI if created, and the Authorization header is toggleable in .env, being switched off by default.

@danny-avila
Copy link
Owner

this does break Vertex AI, will try to see if just adding what we had is fine

@danny-avila
Copy link
Owner

replaced by #5037

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants