Media Source Extensions - Feature requests and discussion

[dsilhavy]

Description

This discussion thread collects ideas and feature requests for future editions of the Media Source Extensions.

[dsilhavy]

Feedback by Thasso and Daniel:

• API to request buffer capacity to overcome QUOTA_EXCEEDED error. Currently, players need to try and error. Schedule download based on current capacity instead of discarding downloaded content and retry later.
• Box parsing API: A media player should be able to register for specific ISOBMFF boxes and get a notification with the box payload. Potential candidates: EMSG, PRFT, ELST. Avoid mp4 box parsing in JavaScript. Potential implications for low latency streaming with CTE.
• Read the current codec string from the SourceBuffer based on the initialization value or the last value that was provided via changeType()
• Related to MPD insertion / interstitials: Internally, one buffer shall be used. Overwriting/overlapping segments/content in the buffer, shifting segments. As an alternative, have multiple buffers per type.
• In some browsers/platforms, it is not possible to remove buffer and create a new one (on the same MSE object). Check if this is a bug on the platform or if it is intended that way.

[thasso]

There is also an implementation for w3c/media-source#357 now in Webkit apparently that support detaching and re-attaching source buffers. I think this is a step in the right direction though I still wonder why we can not have more than one buffer for a given media type directly and then "pick" the active one.

[thasso]

Generally I think the ability to remove and then re-create a source buffer would be a very useful. For example, the user want to turn off audio. Today we can just mute but we still need to feed data to the buffer. Similar for video we can not remove a video buffer let alone later re-create it.
This is slightly different from "changing" the buffer to another one. This use case would be for turn off audio or video completely at least for a period of time.