Roadmap

[mickel8]

0.1.0 (01.2024)

Media reading/writing:

• IVF reader/writer - Add IVF reader #29 Add IVF writer #44
• Ogg reader/writer - Ogg reader #43 Ogg writer #50
• VP8 RTP packetizer/depacketizer - Initial version of sending from a file #30 Add VP8 depayloader #45
• Opus RTP packetize/depacketizer - Add Opus payloader #47 Restructure Media module #53
• Sending from a file (without RTCP) - Initial version of sending from a file #30 Update config on receiving remote SDP #37 Correctly generate SSRC for RTPSender #39

PeerConnection API:

• Closing PeerConnection Stop transports on closing PeerConnection #40
• Renegotiation - Add stop_transceiver/2 #55 Add test for adding and removing tracks in a loop #59
• addTrack Add add_track/2 function to API #38
• removeTrack Add removeTrack #49
• Stopping transceivers Add stop_transceiver/2 #55
• Tests for failing negotiation because of unsupported codecs - Reject SDP offer when there are no matching codecs #48 Fix no supported codecs scenario #61

Documentation:

• Documentation and typespec improvements - Improve API and API docs #57
• Transceiver guide - Add Transceiver guide #56

0.2.0 (04.2024)

• Initial benchmarks Add WebRTC Benchmark bench#1

RTCP related:

• RTCP Sender and Receiver Reports - Add support for RTCP Receiver Reports #77 Add support for RTCP Sender Reports #78 Sending RTCP Reports #81
• TWCC Feedbacks - Add support for TWCC feedbacks #63
• NACKs and RTX - Generate NACKs and handle incoming RTX packets #90 Add outbound RTX #101

ICE features:

• ICE mDNS (client-side) Add DNS message encoder/decoder ex_ice#28 Add MDNS resolver ex_ice#27
• ICE relay candidates Add support for relay candidates ex_ice#33

0.3.0 (07.2024)

• Inbound Simulcast Inbound Simulcast #111
• Benchmarks and profiling

UX improvements/features:

• User-friendly media/codec/extension configuration Improve UX of Configuration #120
• ICE port range configuration Allow for port range configuration ex_ice#43
• Better logs

Integrations:

Fishjam:

• Experimental WebRTC Endpoint using Elixir WebRTC

Broadcaster improvements:

• Viewer count [Broadcaster] Add viewer count and backend chat stub apps#12
• Client-side ICE restarts
• Server-side stats [Broadcaster] Port broadcaster to Phoenix apps#11
• Simple chat [Broadcaster] Add chat apps#13

0.4.0 (08.2024)

Website

• A new website A new website website#2
• 3 demos deployed 24/h under elixir-webrtc.org [Reco] Unify layout to be similar to the Broadcaster apps#23 [Recognizer] Add docker image apps#27 [Recognizer] Setup CD apps#29 [Broadcaster] Add Docker image apps#30 [Broadcaster] Setup CD apps#31 [Nexus] Add docker image and set up CD apps#32 [Broadcaster] Fix issues with headless client apps#37
• Videoroom demo [Nexus] Add grid layout, fix RCs, improve codebase, remove chat apps#41 [Nexus] Nexus initial commit apps#25

Broadcaster

• Chat moderation [Broadcaster] Add chat moderation capabilities apps#43 [Broadcaster] Improve the UX and styling of admin panel apps#44
• Recordings
• Simulcast [Broadcaster] Add support for Simulcast input apps#22
• [Broadcaster] Audio/video desynchronization apps#34
• [Broadcaster] Video sometimes doesn't load apps#35
• [Broadcaster] Video tile resizes when switching between simulcast layers apps#36

ExWebRTC

• Support for all 3 browsers - Chromium/Chrome, Firefox, Safari Firefox can't establish two simultanous peer connections with Elixir WebRTC  #127 :missing_rtcp_mux when streaming to firefox #149
• ICE fixes - Add retransmissions ex_ice#7 Firefox sometimes can't move to the connected state ex_ice#19 Don't add srflx candidates to the checklist ex_ice#47 Re-schedule conn-check when faild pair receives data  ex_ice#48 Use binding-requests as keepalives ex_ice#49 Don't block data received on a failed pair ex_ice#50 Clean up state when connection is failed or completed. Ignore data frome stale sockets, timers, etc.  ex_ice#55
• Examples sometimes don't load Examples sometimes don't load in Chromium #132
• DTLS transport crash on message :retransmit #137
• Unable to apply answer with disabled mlines #144

UX

• ex_webrtc_dashboard - metrics per second Improve inbound/outbound RTP stats #134 Add per-second stats, UX improvements ex_webrtc_dashboard#5
• Revisit Xav - elixir-webrtc/xav@v0.4.0...v0.5.0
• Basic tutorials Add Introduction to Elixir WebRTC tutorial #136 Further improvements to Introduction to Elixir WebRTC tutorial #141
  • I ran echo example but I can't see video
  • What are RTP packets and depayloaders?
  • How to request a keyframe and when to do this?
  • etc.
• Add section about debugging/observability in tutorials #142
• Add simulcast tutorial #151
• Create a common behaviour for payloaders and depayloaders #143

0.5.0 (09.2024)

• DataChannels! 🎉

0.6.0 (10.2024)

• Jitter buffer Add jitter buffer #158
• Support for Fly.io Add support for Fly.io deployments #167 Update Fly.io instructions #169

Others

If you have any questions or need help with a specific topic, please create a new discussion!

• setParameters on RTPSender
• Support for AV1 codec, see AV1 payloader #35
• TCP ICE
• Support for Windows
• ex_webrtc_dashboard - performance optimizations
• bandwidth estimation/probing
• telemetry

[davydog187]

Very interesting project! Would you consider supporting AV1 RTP encoding as well?

[mickel8]

Very interesting project! Would you consider supporting AV1 RTP encoding as well?

@davydog187 of course! When it comes to the echo example we already support it! It's as easy as telling PeerConnection to negotiate AV1. Just change this line to:

        {:ok, pc} =
          PeerConnection.start_link(
            ice_servers: @ice_servers,
            video_codecs: [
              %RTPCodecParameters{
                payload_type: 45,
                mime_type: "video/AV1",
                clock_rate: 90_000
              }
            ]
          )

Besides this, we will of course add AV1 RTP payloader and depayloader so that you can pack/unpack AV1 into/from RTP packets 🙂

[mickel8]

#33 also adds AV1 to the default video codecs

[lovebes]

This is really cool! How would this interact with Membrane Framework?

[LVala]

@lovebes at some point we will wrap our ExWebRTC.PeerConnection in a Membrane sink/source, then you'll be able to send/receive media from WebRTC and pass it further into a Membrane pipeline.

[ris-work]

Thanks for this, the code looks very friendly to use, like membrane.framework! Any idea about DataChannels on the roadmap?

I work a lot with webrtc-rs and I will try to port my codebase to Elixir too, if possible. For the client side deployments, relay support is sometimes really necessary from what I have noticed, but not for servers and those who have SNAT or some properly working CGNATs. Hopefully IPv6 support will make them redundant but IPv6 is only deployed where I live, in the fiber deployments and only in new routers. My usecase only relies on DataChannels and would love to have it in a memory-safe language like Elixir or Rust.

I will eventually port everything to Elixir as it seems like things are easier to manage here over tokio and async rust, given better or equal CPU usage (BEAM is especially suited for soft-RT systems), at least on *nix.

[mickel8]

Hi @ris-work!
Datachannels are of course on our radar but unfortunately not in 0.2. Maybe in 0.3 or 0.4. But any contributions are welcomed!

[ris-work]

Hi @ris-work! Datachannels are of course on our radar but unfortunately not in 0.2. Maybe in 0.3 or 0.4. But any contributions are welcomed!

Thank you very much for the response! I will be eagerly waiting. Unfortunately, I am neither an expert in WebRTC nor Elixir (or SCTP/RTP), but thank you for considering it for an upcoming version!

[type1fool]

I am also excited to see this package developing so quickly.

There are a couple use cases I have in mind that would depend on data channels. For one, it would be cool to deploy gaming app with elixir-desktop, where two peers could connect directly without an external signaling server.

It's still early in my WebRTC journey, but if I can find time to support implementation, I'd be happy to do so.

[mickel8]

Hi guys, support for DataChannels just landed in version 0.5.0! We will publish a blog post next week but I wanted to let you know right now 🙂

cc @ris-work @type1fool

[ris-work]

Thanks @mickel8 and @LVala for this and thanks for the notification! I will start using this soon!

[mickel8]

And the blog post on data channels: https://blog.swmansion.com/data-channels-in-elixir-webrtc-0853c7d0e256 :)