Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segment anything pipeline image #3131

Merged
merged 15 commits into from
Sep 4, 2024

Conversation

pschroedl
Copy link
Collaborator

@pschroedl pschroedl commented Aug 15, 2024

What does this pull request do? Explain your changes. (required)

Adds support for the ai-worker pipeline which implements Segment Anything 2 from facebook/meta
(https://github.com/facebookresearch/segment-anything-2)

Corresponding AI-Worker PR:
livepeer/ai-worker#185

Specific updates (required)

* Adds a mapping of pipelines to a tag corresponding to the pipeline-specific docker image
Mapping of pipelines to a tag corresponding to the pipeline-specific docker image has been pushed down to the ai-worker

  • Launches a container with the correct image for pipelines specified warm == True in aiModels.json
  • Pricing for semgent-anything-2 requests

How did you test each of these updates (required)

E2E testing gateway + orchestrator via curl and jupyter notebook

Example usage:
https://colab.research.google.com/drive/12TC3NW7BpO77WfKHQyok18ebOtbT9F4-?usp=sharing

Does this pull request close any open issues?

Checklist:

@pschroedl pschroedl force-pushed the segment-anything-pipeline-image branch from 0a439d8 to 7d01c0a Compare August 27, 2024 18:42
Copy link
Member

@rickstaa rickstaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pschroedll Please check out my requested changed. If those are dealt with it is ready for release.

cmd/livepeer/starter/starter.go Outdated Show resolved Hide resolved
cmd/livepeer/starter/starter.go Outdated Show resolved Hide resolved
This commit applies some small comment changes.
This commit fixes the orch latency score at the Gateway for the SAM2 pipeline
which was commented out in a previous commit.
server/ai_process.go Outdated Show resolved Hide resolved
Copy link
Member

@rickstaa rickstaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 !

@rickstaa
Copy link
Member

rickstaa commented Sep 3, 2024

@pschroedl, only thing left is updating the worker version before merge.

@rickstaa
Copy link
Member

rickstaa commented Sep 4, 2024

@pschroedl although some tests pass these are the same as the ones on the ai-video main branch. We still need to fix those and write new tests for the AI stuff. Let's prioritize this after the public launch.

@rickstaa rickstaa merged commit 3316e35 into livepeer:ai-video Sep 4, 2024
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants