Add Control API

[leszko]

Add Control API for AI Live Inference. Related AI Worker PR: livepeer/ai-worker#279

Comments

1. I've updated how we the path for starting the pipeline to /live/video-to-video/{stream}/start(so the stream name is passed as a path param instead of the form param). @victorges @mjh1 @thomshutt let me know if you prefer the form param.
2. There is a slight difference between the Control API design doc (the reason for that is that during the implementation I realized it's actually simpler to use Trickle directly from Gateway)
   • Design Doc: User --HTTP--> Gateway --HTTP--> Orchestrator --Trickle--> Runner
   • Implementation User --HTTP--> Gateway --Trickle--> Orchestrator --Trickle--> Runner
3. There is no retry logic and no unit tests, I think that it's ok for this first version, but something to keep in mind that we'll need to address this tech debt after the 12/13 deadline. It's related not only to this PR, but to all AI Inference code we added.

How to test it?

Start MediaMTX with the following config

runOnReady: curl http://localhost:5936/live/video-to-video/$MTX_PATH/start -F source_id=$MTX_SOURCE_ID -F source_type=$MTX_SOURCE_TYPE -F query=$MTX_QUERY

Start Gateway

./livepeer -gateway -rtmpAddr :1936 -httpAddr :5936 -orchAddr localhost:8935 -v 6

Start Orchestrator

./livepeer -orchestrator -aiWorker -aiModels 'live-video-to-video:cumulo-autumn/stream-diffusion:false' -serviceAddr localhost:8935 -transcoder -v 6

Start streaming by accessing the following URL from the browser: http://localhost:8889/stream/publish?video-codec=h264%2F90000

Start Runner and then execute the following HTTP request to the runner to start inference

curl 'http://<runner-ip-and-port>/live-video-to-video/' \
--header 'Content-Type: application/json' \
--data '{
  "subscribe_url": "https://host.docker.internal:8935/ai/trickle/<mid>",
  "publish_url": "https://host.docker.internal:8935/ai/trickle/<mid>-out",
  "control_url": "https://host.docker.internal:8935/ai/trickle/<mid>-control"
}'

Now, you can send Control API request, e.g.

curl --location 'http://localhost:5936/live/video-to-video/stream/update' \
--header 'Content-Type: application/json' \
--data '{
  "some-param": "some-param-value"
}'

[codecov]

Codecov Report

Attention: Patch coverage is 7.14286% with 65 lines in your changes missing coverage. Please review.

Project coverage is 34.65535%. Comparing base (6a8fa52) to head (aff8951).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
server/ai_mediaserver.go	6.52174%	42 Missing and 1 partial ⚠️
server/ai_live_video.go	0.00000%	9 Missing ⚠️
server/ai_http.go	0.00000%	7 Missing ⚠️
server/ai_process.go	0.00000%	6 Missing ⚠️

Additional details and impacted files

@@                 Coverage Diff                 @@
##              master       #3255         +/-   ##
===================================================
- Coverage   34.70278%   34.65535%   -0.04743%     
===================================================
  Files            136         136                 
  Lines          36219       36283         +64     
===================================================
+ Hits           12569       12574          +5     
- Misses         22938       22996         +58     
- Partials         712         713          +1     

Files with missing lines	Coverage Δ
core/livepeernode.go	74.01575% <100.00000%> (+0.41575%)	⬆️
server/ai_process.go	0.61082% <0.00000%> (-0.00268%)	⬇️
server/ai_http.go	12.21374% <0.00000%> (-0.09395%)	⬇️
server/ai_live_video.go	0.00000% <0.00000%> (ø)
server/ai_mediaserver.go	8.49515% <6.52174%> (-0.47224%)	⬇️

... and 1 file with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a8fa52...aff8951. Read the comment docs.

Files with missing lines	Coverage Δ
core/livepeernode.go	74.01575% <100.00000%> (+0.41575%)	⬆️
server/ai_process.go	0.61082% <0.00000%> (-0.00268%)	⬇️
server/ai_http.go	12.21374% <0.00000%> (-0.09395%)	⬇️
server/ai_live_video.go	0.00000% <0.00000%> (ø)
server/ai_mediaserver.go	8.49515% <6.52174%> (-0.47224%)	⬇️

... and 1 file with indirect coverage changes

---- 🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[j0sh]

Looks like a good start!

[j0sh]

We could also make the /start path above this take the stream name too, for consistency. (If we change this we should do it soon, before the official release. No real preference though.)

[leszko]

I updated to use path everywhere. I slightly prefer it. Let's see what @mjh1 @thomshutt and @victorges say. I'm also ok to keep everywhere the form param.

[victorges]

Looks good!

[j0sh]

Looks great!