-
Notifications
You must be signed in to change notification settings - Fork 3
Home
Develop software/tools to run on the A/V capture computer to handle audiovideo streaming and archiving for the Southern California Linux Expo (SCaLE). We are investigating the use of GStreamer. (See GStreamer notes)
At SCaLE, we will have a PTZ (Pan-Tilt-Zoom) dome camera in various conference rooms, streaming video via RTSP (Real Time Streaming Protocol). Audio will be hooked up from handheld or beltpack microphones into the camera. We potentially may have scan converters and video encoders to capture what is being projected to the screen.
We need to be able to stream the video to Youtube and/or ?. We also need to save the files to disk to later upload to Youtube.
Due to the odd sampling rate and format of the audio output from the camera, we would need to filter the RTSP stream
- Keep the video stream as-is.
- Upsample the audio and re-encode it in 48kHz aac-lc
- Multiplex (Mux) the video and audio
- Capture slides in a separate stream using a scan converter and video encoder; display both the speaker and the slides using picture-in-picture
** @Omwah, from an archival point of view, thinks that should write video stream, slide stream, and then the picture-in-picture to a file.
** @mproctor is concerned that there is no way to sync the slides to the video. The easiest way is to put them in the same stream.
** A possible solution is to add timecodes, but the way it's done professionally is to have an external clock on the cameras; if we do it via GStreamer, it is too late.
** Another possible solution is to just start/stop the recording for each session. *** @mproctor is concerned about start/stop, because there is a strong possibility that the operator will forget to start and video won't be captured. - Adjust/boost the volume on the audio ** An issue with the SCaLE videos on YouTube is that the audio seems to be quite a bit lower than other videos on YouTube, maybe 50% quieter or more.
PTZ cameras:
@mproctor thinks that when we read RTSP streams using the standard library--most are based on Live555, we get separate audio and video streams.
There are firmware updates for the camera, which may have an impact on this software development. (See Firmware update notes)
For the scan converter and video encoder:
- Generic PC-to-TV converter has VGA input + loop-thru; it has S-Video/RGB output ports. There is a front panel which allows control of pan/zoom via on screen display.
- Axis M7001 video encoder Amazon link can stream both H.264 and MJPEG over RTSP. Can control PTZ for certain PTZ devices. Can be powered over Ethernet (POE) or a DC adapter.
This is what Youtube requires for live streaming:
- Protocol: RTMP Flash Streaming
- Video codec: H.264, Main 4.1 (surprisingly they wont accept webm)
- Frame rate: 30 fps
- Keyframe frequency: 2 seconds
- Audio codec: AAC-LC (Audio required)
- Audio sample rate: 44.1 KHz
- Audio bitrate: 128 Kbps stereo
Notes from Ilan:
- Youtube can do transcoding for us on the fly. So if we push higher bit rates they'll create versions as needed for lower bandwidth environments and for mobile.
- I dont believe our cameras support 44.1KHZ audio, we'll need to take care of that either in whatever tooling we use to transcode.
- Each live stream can last up to 4 hours, so we'll need to create a "new" event every 4 hours. This may mean needing to update the embed code on the site through out the day.
- If we want to do the video editing / clipping in youtube, we'll need to break our "live events" into 2 hours. Youtube doesn't let you edit anything
We will investigate using GStreamer to capture video.
- Figure out the GStreamer pipeline to do what we need.
- Capture from RTSP
- Save the stream to file (record)
- Deal with audio -- (re-encode)
- Remux and
- Put into appropriate transport stream
- Figure out how to work programmatically with GStreamer
We will also investigate gst-switch to see if it might meet some of our needs.
- Python 3 (Has bindings for GStreamer1) (TBC)
- GStreamer1 (NOTE: Comes with Ubuntu. But if you have to install it, you may have to install some plugins.)