-
Notifications
You must be signed in to change notification settings - Fork 3
Home
Develop software/tools to run on the A/V capture computer to handle audiovideo streaming and archiving for the Southern California Linux Expo (SCaLE). We are investigating the use of GStreamer. (See GStreamer notes)
At SCaLE, we will have a PTZ (Pan-Tilt-Zoom) dome camera in various conference rooms, streaming video via RTSP (Real Time Streaming Protocol). Audio will be hooked up from handheld or beltpack microphones into the camera. We potentially may have scan converters and video encoders to capture what is being projected to the screen.
We need to be able to stream the video to Youtube and/or ?. We also need to save the files to disk to later upload to Youtube.
Due to the odd sampling rate and format of the audio output from the camera, we would need to filter the RTSP stream
- Keep the video stream as-is.
- Upsample the audio and re-encode it in 48kHz aac-lc
- Multiplex (Mux) the video and audio
Capture slides in a separate stream using a scan converter and video encoder; display both the speaker and the slides using picture-in-picture
- @omwah, from an archival point of view, thinks that should write video stream, slide stream, and then the picture-in-picture to a file.
- @mproctor13 is concerned that there is no way to sync the slides to the video. The easiest way is to put them in the same stream.
- A possible solution is to add timecodes, but the way it's done professionally is to have an external clock on the cameras; if we do it via GStreamer, it is too late.
- @jbermudes suggestedjust start/stop the recording for each session. But @mproctor is concerned about start/stop, because there is a strong possibility that the operator will forget to start and video won't be captured.
- An issue with the SCaLE videos on YouTube is that the audio seems to be quite a bit lower than other videos on YouTube, maybe 50% quieter or more.
To cut down on the time and workload involved in getting the videos ready to be posted online, we should figure out a way to cut the video at SCaLE. A way to partly automate this is:
- Develop a mobile app that can be used to just record the start/stop times for prepopulated sessions in various conference rooms. (Ideally, the timestamp comes from the video capture machine.)
- Use the output from above to automate a solution where ffmpeg can be used to slice up the room recording into appropriately names session files.
- Have volunteers check over the video and/or do further editing before uploading.
- Can the video capture machine handle the load of managing all the video streams?
- Will the video capture machine hold up throughout the day, i.e., will processes start crashing at some point?
PTZ cameras:
@mproctor13 thinks that when we read RTSP streams using the standard library--most are based on Live555, we get separate audio and video streams.
There are firmware updates for the camera, which may have an impact on this software development. (See Firmware update notes)
For the scan converter and video encoder:
-
Generic PC-to-TV converter has VGA input + loop-thru; it has S-Video/RGB output ports. There is a front panel which allows control of pan/zoom via on screen display. The supported resolutions/refresh rates: 640x480 @ 60/72/75/85 Hz, 800x600 @ 60/75 Hz, 1024x768 @ 60/75 Hz, and 120x1024@60 Hz.
-
Axis M7001 video encoder Amazon link can stream both H.264 and MJPEG over RTSP. Can control PTZ for certain PTZ devices. Can be powered over Ethernet (POE) or a DC adapter.
This is what Youtube requires for live streaming:
- Protocol: RTMP Flash Streaming
- Video codec: H.264, Main 4.1 (surprisingly they wont accept webm)
- Frame rate: 30 fps
- Keyframe frequency: 2 seconds
- Audio codec: AAC-LC (Audio required)
- Audio sample rate: 44.1 KHz
- Audio bitrate: 128 Kbps stereo
Notes from @irabinovitch:
- Youtube can do transcoding for us on the fly. So if we push higher bit rates they'll create versions as needed for lower bandwidth environments and for mobile.
- I dont believe our cameras support 44.1KHZ audio, we'll need to take care of that either in whatever tooling we use to transcode.
- Each live stream can last up to 4 hours, so we'll need to create a "new" event every 4 hours. This may mean needing to update the embed code on the site through out the day.
- If we want to do the video editing / clipping in youtube, we'll need to break our "live events" into 2 hours. Youtube doesn't let you edit anything
We will investigate using GStreamer to capture video. We primarily need to figure out the GStreamer pipeline to do what we need, and then we need to learn how to work with GStreamer programmatically.
- Capture from RTSP
- Save the stream to file (record)
- Deal with audio -- (resample and re-encode)
- Multiplex the video stream with reencoded audio stream
- Put into appropriate transport stream
We will also investigate gst-switch and speaker-track to see they might meet some of our needs.
- Python 3 (Has bindings for GStreamer1)
- GStreamer1 (NOTE: Comes with Ubuntu. But if you have to install it, you may have to install some plugins.)