How to Encode video at different resolutions

[mdsohelmia]

[MadMakz]

Same; There's absolutely no documentation about adaptive streaming

[Rokamun]

Heya, Mistserver by default does not have an encoder or anything built in. It's a repackager, so if you want adaptive streaming or multiple resolutions you need to have a source with multiple resolutions or you need to add an encoder. If your stream has multiple video tracks with different resolutions included you would probably not have asked this question, so let's assume you need an encoder. This can be tackled at two places, either at the source or at the server. Now keep in mind that any manipulation of video on this level will be encoding/transcoding and this will always use up resources. So if you try to do this for multiple streams you could end up using all your available CPU/GPU quite fast. *Source* This is when you add additional video tracks to the output of whatever you're using to push into MistServer, I'm talking about applications such as OBS. The easiest method of sending multiple qualities is to set it up and choosing a protocol that can do this, such as SRT. You also have the guarantee that all tracks arrive at the same timing, so you won't have to worry about certain audio or video tracks running ahead or behind each other. *Server* This is when you want to add the qualities at the server side, meaning you receive a single bitrate quality and you want to add additional to it. This can be done through stream processes. Currently there's 3 types. We're not completely happy with the interface and we're planning to add more later on. Which is the reason it's not super well documented yet. *Encoder:FFmpeg* This uses your installed ffmpeg to work, so you will need to install ffmpeg yourself, but luckily it is in almost every Linux distribution. *MKV-Exec* This tells MistServer to run whatever you fill in as if it's a command in bash. Meaning you can feed it a script, but also simply call for an application to do a task. The standard in will receive a Matroska stream from MistServer, and MistServer expects to receive Matroska stream back from the standard out. You could use this to write your own FFmpeg or Gstreamer script for example or you can use it in combination with other hardware. An example would be: ffmpeg -loglevel quiet -hide_banner -fflags nobuffer -i - -c:v libx264 -an -s 1280x720 -force_key_frames source -bf 0 -cluster_time_limit 0 -f matroska - for adding a 720p quality. or ffmpeg -fflags nobuffer -hide_banner -loglevel warning -i - -an -map v:0 -map v:0 -c:v:0 h264 -b:v:0 4000k -s:v:0 1280x720 -c:v:1 h264 -b:v:1 1200k -s:v:1 640x360 -force_key_frames source -bf 0 -cluster_time_limit 0 -f matroska - for adding 720p and a 640x360 quality to a stream *Livepeer* This is using Livepeers encoding network to generate extra feeds. In order to set it up you will need a Livepeerkey. *VoDs* If we're not talking live streams at all, the best solution is to simply create another video with the additional qualities in there, a very simple ffmpeg script to do this would be: ffmpeg -i INPUT_FILENAME_HERE -map a:0 -map v:0 -map v:0 -c:a:0 copy -c:v:0 copy -c:v:1 h264 -b:v:1 4000k -s:v:1 1280x720 OUTPUT_FILENAME_HERE To explain all of those options in more detail: "-map a:0" selects the first available audio track from the source "-map v:0" selects the first available video track from the source "-map v:0" selects the first available video track from the source a second time "-c:a:0 copy" tells ffmpeg to copy the audio track without re-encoding it "-c:v:0 copy" tells ffmpeg to copy the video track without re-encoding it "-c:v:1 h264" tells ffmpeg to *also* re-encode the video track in h264 "-b:v:1 4000k" tells ffmpeg that this second video track should be 4000kbps "-s:v:1 1280x720" tells ffmpeg that this second video track should be at 720p resolution You can use this same concept to just add another video track by mapping an extra video with `-map v:0` and then setting the quality by setting the options for c/b/s for v:2. Or just use whatever app you would prefer to generate a video with multiple video tracks, I'm just using ffmpeg as an example because I happen to know it and it's easy to run from the terminal. With kind regards, Balder Viëtor Head of Testing MistServer

…

On Tue, Jul 2, 2024 at 3:16 PM MadMakz ***@***.***> wrote: Same; There's absolutely no documentation about adaptive streaming — Reply to this email directly, view it on GitHub <#191 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN2RJFKJ6MR7ZOWS3OOYSLZKKRZPAVCNFSM6AAAAABFDZTMISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBTGEZTAOBXGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[MadMakz]

Thanks for your great reply!

I experimented with the nginx-rtmp-module a bunch of years back when OBS Studio wasn't even in a usable state (So OBS1 times) so i'm aware that there has to be some reencoding for adaptive bitrate. I just got the wrong idea out of your project that it does this "natively" or via "one-click".

My use case would be serverside and as docker.

So if i understand correct,

1.) your docker image doesn't come with ffmpeg (that's no problem since i can dockerize it with ffmpeg myself)

2.) it IS possible then within mistserver but i fail to understand.

My guess is that "Stream processes" with optional parameters is involved in that process? If so, i'm not understanding input & target stream correctly i think.

[Rokamun]

Heya, Correct, MistServer does not have an encoder out of the box. We've early on put our focus on repackaging and not encoding. It is something we are considering on adding, but current versions of MistServer would require an external encoder. We do however have the stream process to instruct said encoders. 1). Yeah the docker image does not come with ffmpeg, as it's just the MistServer binaries and I believe we also insert a very small Alpine Linux environment around it. If you want to add ffmpeg it might be easier to build a docker yourself, and pull in & build the generic linux build. That would give you slightly better performance as well. 2) You set it up at the stream processes when creating a stream. It will only work for live streams, for VoD we recommend doing the encode before serving the stream. I'll see if I can just document this in reasonable time. It's on the list and honestly it's overdue. To give you a bit of a start: [image: image.png] *Target stream:* The stream internally within MistServer to send whatever the encoder creates towards. By default and keeping this empty will send it towards the stream this is being set up for. So if you use the stream "livestream" and you do not fill this in, the extra video/audio track will appear in that same "livestream" stream. If you fill in "livestream2" it will push it towards the stream "livestream2". *Source track mask:* This determines what MistServer will do with the track that gets transcoded. What you select are the type of processes/viewers that can use the track. For example setting the source to not allow viewers would mean that no one that watches can see the original video track, but you can record and use it for processes. *Output track mask*: This determines how the output track can be used. For example only allowing viewers on this process would ensure that viewers can see a lower quality, but you do not record or send it out to other servers. *Track inhibitors:* This determines when the process will not start at all. For example if you've set up a process to create a 720p stream. You could have a selector on "video=<1280x720" to avoid scaling up. Not selecting audio should keep that on none, same for meta. And again, this is selecting when the process does NOT activate, so it's a bit of an annoyance thought wise. *Source selectors*: This determines what track MistServer should feed the process, by default all available audio/video tracks will be given. That does mean any other transcoded tracks as well unless you hide those. You can use the selectors here again to select specific tracks or remove tracks. For example adding "audio=none" would make sure only video data gets passed to this process. For the track selectors I'd recommend reading: https://docs.mistserver.org/mistserver/concepts/track_selectors Due to all the configuration, and kind of hard to understand settings we consider the stream processes to be in a bit of an "advanced configuration" stage. We are planning on making a far simplified profile where you just say "make me a lower quality stream" and that's it, however that's definitely a future thing and not available right now. With kind regards, Balder Viëtor Head of Testing MistServer

…

On Tue, Jul 2, 2024 at 6:13 PM MadMakz ***@***.***> wrote: Thanks for your great reply! I experimented with the nginx-rtmp-module a bunch of years back when OBS Studio wasn't even in a public github (So OBS1 times) so i'm aware that there has to be some reencoding for adaptive bitrate. I just got the wrong idea out of your project that it does this "natively" or via "one-click". My use case would be serverside and as docker. So if i understand correct, 1.) your docker image doesn't come with ffmpeg (that's no problem since i can dockerize it with ffmpeg myself) 2.) it IS possible then within mistserver but i fail to understand. My guess is that "Stream processes" with optional parameters is involved in that process? If so, i'm not understanding input & target stream correctly i think. — Reply to this email directly, view it on GitHub <#191 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN2RJGDTL7O75CFHNBNRCDZKLGRRAVCNFSM6AAAAABFDZTMISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBTG4ZDANZRGU> . You are receiving this because you commented.Message ID: ***@***.***>