Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrated Gradio Demo App #6026

Closed
Ankur-singh opened this issue Aug 11, 2023 · 13 comments · May be fixed by #6034
Closed

Integrated Gradio Demo App #6026

Ankur-singh opened this issue Aug 11, 2023 · 13 comments · May be fixed by #6034
Assignees
@Ankur-singh
Labels
Stale

Comments

@Ankur-singh
Copy link

Describe the feature
The Jina project is an incredible tool that simplifies the process of deploying machine learning models by just writing an executor. It enables users to get a complete working service with ease. However, during the development and testing stages, it would be extremely beneficial to have an integrated Gradio demo app. This will provide an interactive user interface that allows users to easily interact and test the model without the need for using CURL, Postman, or programming languages.

Your proposal

  • The integrated Gradio demo app can be accessible at the /demo endpoint.
  • Users can interact with the model using the Gradio interface, providing input and viewing the model's output in real-time.
  • The Gradio demo app should automatically generate the appropriate input components based on the model's input types (e.g., string, integer, float, boolean, file, path) and utilize the docArray model's additional information for description, default values, minimum, maximum, and choices, when available.
  • The Gradio integration should be optional, allowing users to enable or disable it based on their use-case. This will enable developers to use the interactive demo during development and prototyping and easily disable it for production deployment.

Benefits:

  • Eases the testing and interaction with the model during development and prototyping stages.
  • Provides a user-friendly interface for non-technical stakeholders to interact with the model and understand its capabilities.
  • Improves user experience by eliminating the need for external tools (e.g., CURL, Postman) or custom code for testing the model.

Implementation Approach:

  • Utilize Gradio's Python library to create the interactive demo app.
  • Mount the Gradio demo app using FastAPI Gateway.
  • Leverage the existing docArray documents information to generate appropriate input and output components with optional descriptions.

This feature will further enhance the usability and user experience of the Jina project, making it an even more valuable tool for deploying machine learning models.

I am willing to take on this feature request as it presents an excellent opportunity to learn more about Jina internals.
@JoanFM
Copy link
Member

JoanFM commented Aug 11, 2023

This looks like an amazing feature.

@Ankur-singh
Copy link
Author

Perfect, I will start working on it this weekend

@alaeddine-13
Copy link
Contributor

Amazing feature @Ankur-singh indeed.
Here is how I imagine the feature being added to jina:

  • for simplicity, suppose for now the feature is only added to Deployments that use the HTTP protocol
  • at the gateway level, the gradio app needs to be mounted to the fastapi server under /demo. You can create 1 page per executor endpoint. Note that the gateway already fetches the schema definitions of the executor server and you can find it in the gateway class in the field self.streamer._endpoints_models_map.
  • at the executor level, if the executor uses http, the gradio app can be mounted to the executor server. Executor endpoint definitions are also already parsed and you can find them in the Executor field self.requests.

@Ankur-singh
Copy link
Author

Thank you so much @alaeddine-13 for all the insights.

@Clarasia
Copy link

I found this issue on https://ovio.org/projects and would like to contribute!
Please guide on how to contribute and work on the project.
Thanks

@Clarasia
Copy link

I have a minimal knowledge on python and would like to contribute to this open-source project.

@Ankur-singh
Copy link
Author

Hey @Clarasia, I have already halfway through it. But still feel free to go ahead and give it a try. It will be a good learning experience.

@Clarasia
Copy link

Clarasia commented Aug 22, 2023 via email

@JoanFM
Copy link
Member

JoanFM commented Aug 23, 2023

Hey @Ankur-singh ,

I suggest that if you already have some work in progress, you can open a draft PR so that we can perhaps start giving early feedback to the development, design, etc ...

@Ankur-singh
Copy link
Author

I have create a Github gist (link) which demonstrates how to add a Gradio gateway to a Jina Flow.

I wrote a function that takes Pydantic model as input and returns Gradio interface generated using the input model. The Gradio app is build dynamically and is mounted the FastAPI app object at /demo endpoint.

I have all the pieces but I will need to some help to add it to the flow/base.py file. It would be great if the Gradio app (/demo) can be places in the "HTTP extension" section when flow is executed.

@JoanFM
Copy link
Member

JoanFM commented Aug 25, 2023

You need to apply this in 2 places:

One in https://github.com/jina-ai/jina/blob/master/jina/serve/runtimes/worker/http_fastapi_app.py to be used when Deployment is used.

The other;

https://github.com/jina-ai/jina/blob/master/jina/serve/runtimes/gateway/http_fastapi_app_docarrayv2.py

to be used when Flow is used.

Just one thing I think it shlould not be under /demo perhaps but under some private reserved name like _jina_gradio_demo to avoid affecting when people may have a /demo endpoint defined

@Ankur-singh
Copy link
Author

Ankur-singh commented Aug 25, 2023 •
edited
Loading

Thanks @JoanFM, these resources are really helpful. I agree with the private reserved name _jina_gradio_demo for the endpoint.

@Ankur-singh Ankur-singh mentioned this issue Aug 27, 2023
Open
3 tasks
@jina-bot
Copy link
Member

@jina-ai/product This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 14 days

@jina-bot jina-bot added the Stale label Nov 24, 2023
@jina-bot jina-bot closed this as completed Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Ankur-singh Ankur-singh

Labels
Stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Integrating gradio demo Ankur-singh/jina
5 participants
@Ankur-singh @alaeddine-13 @JoanFM @jina-bot @Clarasia