Adding multitrack play into iOS SDK

[yashtandon113]

We are publishing multiple sub-tracks to the one main track. Now we want to play this main track in the iOS SDK so that all other subtracks can be played.

In the Android SDK, there is a specific sample application. Could you please share the solution for the iOS SDK?

[mekya]

Hi Yash,

Thank you for the question.

Here is the basic instructions how to have multitrack play in iOS SDK.
The idea is that ConferenceViewController is actually a multitrack player. Just disable the publish part and it become a multitrack player

• Open the [ConferenceViewController] (https://github.com/ant-media/WebRTC-iOS-SDK/blob/master/WebRTC-Sample-App/ConferenceViewController.swift) in Xcode
• Comment out the publish part

  //self.conferenceClient?.publish(streamId: self.publisherStreamId, token: "", mainTrackId: roomId);

• Enable the play part in viewWillAppear

   self.conferenceClient?.play(streamId: self.roomId);

• The result viewViewAppear should look like below

   open override func viewWillAppear(_ animated: Bool)
   {
       
       //init renderers because front end manage the viewers
       initRenderer(view: remoteView0)
       initRenderer(view: remoteView1)
       initRenderer(view: remoteView2)
       initRenderer(view: remoteView3)
      
       
       AntMediaClient.setDebug(true)
       self.conferenceClient =  AntMediaClient.init();
       self.conferenceClient?.delegate = self
       self.conferenceClient?.setWebSocketServerUrl(url: self.clientUrl)
       self.conferenceClient?.setLocalView(container: self.localView)
       
       //this publishes stream to the room
       //self.publisherStreamId = generateRandomAlphanumericString(length: 10);
       //self.conferenceClient?.publish(streamId: self.publisherStreamId, token: "", mainTrackId: roomId);
       
       //this plays the streams in the room
       self.conferenceClient?.play(streamId: self.roomId);
     
   }

Then enter the stream Id in the mainscreen, choose the Conference and Click Connect Button. Then it will play the multitrack stream

Let me know if it works for you or I can help with anything

Cheers
Oguz

[deependra486]

Hi @mekya,

we are trying to achieve a dynamic multi view, so we can easily update multiple views for ex, 7 -10 views (Container) at a time and if we want to remove some of the view container than it can remove dynamically. (Ex. If one streamer camera if closed, then that view should be removed form our multi view list and vice-versa.
But as we checked in the link that you have shared it seems it is a fixed containers and if we want to add more container view than we need to update the code and re publish but in dynamic pattern like GRIDVIEW with array list it can easily achievable.

Regards
Deependra

[deependra486]

Hello AntMedia team,

Thanks for your help and support!

We are working as mobile app developer (iOS) and we are trying to integrate Ant media streaming in our project, we have some query regarding below points and for your reference I have attached a screenshot from the app.

1. Please let us know from where we will get the array list to achieve a dynamic grid Views as attached in this mail.

As we can see in conference controller, we have fixed views
@IBOutlet var remoteView0: UIView!
@IBOutlet var remoteView1: UIView!
@IBOutlet var remoteView2: UIView!
@IBOutlet var remoteView3: UIView!

2. Do we have any API or option or methods to get the Resolution details of video streams.
3. Do we have any API for the Audio controlling from where we can Mute/Unmute the stream.

Looking forward to hearing from you on the above query.

Regards
Deependra

[mekya]

Hi @deependra486,

Thank you for your questions. Let me try to answer your questions

1. Please let us know from where we will get the array list to achieve a dynamic grid Views as attached in this mail.

Generally, our expectation is to let the developer create such a dynamic UI grid that is specific to his application logic. There are callbacks when to add/remove UI view.

trackAdded

trackRemoved

2. Do we have any API or option or methods to get the Resolution details of video streams.

You can get the resolution on the SDK by listening the videoView callback. It gives the RTCVideoRenderer that you can find out which one has changed.

Alternatively, we can log this information on the server side for analytics and we may return this information through REST API or WebSocket.

3. Do we have any API for the Audio controlling from where we can Mute/Unmute the stream.

There is a handy way to do that. You can ask the server not to send audio track for the stream. Just use the enableAudioTrack and give the trackId that you want mute/unmute

I hope my answers are answers to your questions and let me know if you have any questions.

Have a great weekend ☺️

Cheers
Oguz

[VipinTripathi17]

Hi AntMedia,

Integrating the android framework sdk into our existing code getting below issue before rendering, can you please help me on this, logs attached

FATAL Exception:

Fatal error in: ../../sdk/android/src/jni/jni_generator_helper.cc, line 58

last system error: 0

Check failed: !env->ExceptionCheck()

error during GetMethodID: getCombinedAudioVideoBwe, ()Ljava/lang/Boolean;

2024-05-28 15:51:36.104 21369-21546 libc co.player A Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 21546 (pool-13-thread-), pid 21369 (co.player)

webRTC Client = io.antmedia.webrtcandroidframework.core.WebRTCClient@7901094
2024-05-28 15:12:11.759 18509-18509 WebRTCClient co.player E Play: TechmBroadcast
2024-05-28 15:12:11.761 18509-18509 WebRTCClient co.player D Preferred video codec: VP8
2024-05-28 15:12:11.761 18509-18627 WebRTCClient co.player D Initialize WebRTC. Field trials: WebRTC-IntelVP8/Enabled/
2024-05-28 15:12:11.765 18509-18509 org.webrtc.Logging co.player I WebRTCClient: Looking for front facing cameras.
2024-05-28 15:12:11.770 18509-18509 org.webrtc.Logging co.player I WebRTCClient: Creating front facing camera capturer.
2024-05-28 15:12:11.775 18509-18509 WebRTCClient co.player D Starting the audio manager...
2024-05-28 15:12:11.798 18509-18509 WebRTCClient co.player D onAudioManagerDevicesChanged: [SPEAKER_PHONE, EARPIECE], selected: SPEAKER_PHONE
2024-05-28 15:12:11.798 18509-18509 WebRTCClient co.player I WebsocketHandler is null and creating a new instance
2024-05-28 15:12:11.803 18509-18627 WebRTCClient co.player D Factory networkIgnoreMask option: 0
2024-05-28 15:12:11.919 18509-18627 WebRTCClient co.player D Peer connection factory created.
2024-05-28 15:12:14.240 18509-18627 WebRTCClient co.player D Create peer connection.
2024-05-28 15:12:14.243 18509-18627 System.err co.player W at io.antmedia.webrtcandroidframework.core.WebRTCClient.createPeerConnectionInternal(WebRTCClient.java:1634)
2024-05-28 15:12:14.243 18509-18627 System.err co.player W at io.antmedia.webrtcandroidframework.core.WebRTCClient.lambda$createPeerConnection$32$io-antmedia-webrtcandroidframework-core-WebRTCClient(WebRTCClient.java:1458)
2024-05-28 15:12:14.243 18509-18627 System.err co.player W at io.antmedia.webrtcandroidframework.core.WebRTCClient$$ExternalSyntheticLambda7.run(Unknown Source:4)
2024-05-28 15:12:14.552 18837-18837 DEBUG pid-18837 A #29 pc 000000000000a474 [anon:dalvik-classes8.dex extracted in memory from /data/app/~~Tjg7m_l_XYSC_vFoCT0W6Q==/co.player-v3kbIlESvVE2pC4Q5Xn_4w==/base.apk!classes8.dex] (io.antmedia.webrtcandroidframework.core.WebRTCClient.createPeerConnectionInternal+0)
2024-05-28 15:12:14.552 18837-18837 DEBUG pid-18837 A #34 pc 000000000000ae24 [anon:dalvik-classes8.dex extracted in memory from /data/app/~~Tjg7m_l_XYSC_vFoCT0W6Q==/co.player-v3kbIlESvVE2pC4Q5Xn_4w==/base.apk!classes8.dex] (io.antmedia.webrtcandroidframework.core.WebRTCClient.lambda$createPeerConnection$32$io-antmedia-webrtcandroidframework-core-WebRTCClient+0)
2024-05-28 15:12:14.552 18837-18837 DEBUG pid-18837 A #39 pc 0000000000007324 [anon:dalvik-classes8.dex extracted in memory from /data/app/~~Tjg7m_l_XYSC_vFoCT0W6Q==/co.player-v3kbIlESvVE2pC4Q5Xn_4w==/base.apk!classes8.dex] (io.antmedia.webrtcandroidframework.core.WebRTCClient$$ExternalSyntheticLambda7.run+0)
---------------------------- PROCESS ENDED (18509) for package co.player ----------------------------

[yashtandon113]

@VipinTripathi17
As I am aware, my colleague @lastpeony responded to you via a support ticket for this query.