Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support code_interpreter and vision for file uploads #179

Open
bruffridge opened this issue Aug 13, 2024 · 3 comments
Open

Support code_interpreter and vision for file uploads #179

bruffridge opened this issue Aug 13, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@bruffridge
Copy link
Member

Deep-chat 2.0 added support for v2 Assistants API. It also changed how uploaded files are handled. Now there is a files_tool_type parameter that determines which tool operates on file uploads. So instead of letting OpenAI decide all uploaded files are handled by the tool set by the parameter. This means ALL uploaded files can only be handled by one tool. We want it to be based on the file and what the user asks. For example, uploaded images should be passed to the gpt-4o for vision. Other files such as excel and csv the user might want visualized via code_interpreter. PDFs may want summarized via file_search.

OvidijusParsiunas/deep-chat#190 (comment)

@bruffridge bruffridge added the bug Something isn't working label Aug 13, 2024
@bruffridge
Copy link
Member Author

Can workaround by defining a function for files_tool_type that returns the tool based on the file type. Waiting for OvidijusParsiunas/deep-chat#281

@bruffridge bruffridge added the waiting for External party has to do something before this can be worked on. label Oct 11, 2024
@OvidijusParsiunas
Copy link

Hi @bruffridge.
The issue mentioned above should already be taken care of. Hence you should be able to use the files_tool_type property as a function without any problems.

@bruffridge
Copy link
Member Author

Fixed in the azure branch.

@bruffridge bruffridge removed the waiting for External party has to do something before this can be worked on. label Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants