Simulcast playground

WebRTC: The missing codelab Fiddle of the month Simulcast playground

🎻 April 2021 fiddle

Want to learn more about WebRTC?

Look us up here for the WebRTC Fiddle of the Month, created… once a month.

Or just enroll to one of our excellent WebRTC training courses.

Resources

const sender = pc1.getSenders()[0];
const parameters = sender.getParameters();

// bitrates
parameters.encodings[0].maxBitrate = 50000;
parameters.encodings[1].maxBitrate = 350000;
sender.setParameters(parameters);

// disabling
parameters.encodings[1].active = false;
sender.setParameters(parameters);

// total bandwidth
await pc1.setLocalDescription()
pc1.setRemoteDescription({type: 'answer', sdp: pc1.remoteDescription.sdp + "b=AS:800\r\n"})

Transcription

Tsahi: Hi, everyone, and welcome to our monthly fiddle. This time we’re going to talk about simulcast in WebRTC. With me is Philipp, as always.

Introduction to Simulcast in WebRTC

Tsahi: I think we will start by first explaining a bit about simulcast, if that makes sense. Let me share my screen real quickly.

We’re talking about simulcast. Simulcast is here to do something very specific. What we want to do is in an SFU, a media server, we want to receive media from all the participants and then route that media to all the other participants.

So we find this guy here, the red haired guy. I am sending my media, and that media gets received by everyone else. It gets out to everyone. Now, the problem with this approach is that we have a lot of different media streams going into our machine – from the server and this number grows the more users we have in the call. So what we’re looking for here is flexibility. That flexibility today is achieved by using something called simulcast.

What simulcasts does? It allows a device, my machine, for example, to encode and send multiple streams towards the SFU. Each stream has a different bitrate. Now the SFU has three different media streams from each one of the participants at a different bitrate, different resolution, different quality.

The SFU can then decide which of these streams to forward to the users. So if he sees some user that has a very good machine, the SFU can send him the high bitrate stream. If there is a user with a poor connection or a slower CPU or a smaller device, it can send it the lower bitrate.

So what we get here is the flexibility of using something like WebRTC and simulcast.

And now, Philipp, you’ve prepared for us. A demo or a sample of that?

Our WebRTC simulcast playground sample

Philipp: Yes, it’s not a fiddle this time because it is a bit too long for fiddle. It’s based on an old webrtcHacks post that we’re going to look at.

We have a page that is similar to fiddle in the sense that it is using two local connections and no server in the middle – no SFU. And that requires some SDP manipulation tricks, which you can find in the webrtcHacks post, and it gives you a playground where you can really try the browser APIs and test the behavior.

Tsahi: OK, so what I see here is the local view. The graph is the outgoing bitrate, right?

Philipp: Yes. So we have 3 outgoing bitrates. That is something that got into Chrome 5 or 6 releases ago. So you can see the different bitrates and different frame rates which are all equal in this case.

Tsahi: OK, and then for the remote video, I see 3 separate lines, so I have an incoming stream.

Philipp: We have the first one, which is a low resolution stream. You can see it’s a bit pixilated, 320×180, which is a sixteenth of the original resolution or a fourth in each dimension. And then we have a 640×360 stream and a 720p stream.

Tsahi: OK, so 100kbps, 450-500kbps and then 2.x mbps.

Philipp: Yes.

Tsahi: OK, and we can play with them dynamically right. It’s not as if this is set in stone.

Philipp: Yes, we can, for example, enable or disable different of these spatial layers. We can turn some of them off, turn them on, or we can limit the bitrate to see how does it behave if there’s not enough bandwidth available.

Modify bitrate in WebRTC simulcast layers

Tsahi: OK, so can you. Let’s play with bitrates.

Philipp: Yes, so we’re going to get the sender and call sender getParameters().

Tsahi: OK, this is on the side.

Philipp: And we have our parameters object and we are interested in the encodings, which is a 3 elements array because we’re using simulcast. So each element has a rate, which is a RTP identifier for that stream. We can see it has an active property and several unused properties as well.

And so we want to play with the bitrate, so let’s say we go for. 50 kbps for the lower stream and 350kbps for the middle stream. I’ve done that here already. And then we call sender setParameters() with the modified parameters.

It takes a few seconds, but you can see in the graph here, it drops to 50 kbps. It is still the same resolution. And the same frame rate, but the quality is a bit worse than before.

Remove and add simulcast layers in WebRTC

Philipp: Let’s reload the page. And what shall we try next?

Tsahi: Let’s remove the second to the middle layer.

Philipp: That is done by setting the active flag to false. It is currently true. We set it and call sender setParameters(). You can see it is frozen, the middle one here. Bitrate is stopping. And if we want to get it back, we can do that dynamically as well.

Tsahi: So this means that I can now dynamically decide, for example, if I’m in a call with a media server and the media server and notices that nobody needs the middle layer or the lower layer, it can just instruct the user or the device to kill one of these layers. And the only thing the client will need to do its his end, the device would be to call active=false to remove that.

Philipp: Yes.

Tsahi: And then with active=true, you’re going to get this back for us now.

Philipp: Yes. It’s. And it’s back.

Bitrate limiting on WebRTC simulcast

Tsahi: Now if I do the calculation, that something like 2.5mbps plus 500kbps plus 100kbps, it’s going to be 3mbps at the very least. Yes, and I don’t have that much. Let’s say I have only 800kbps of bandwidth available.

Philipp: Yes. So let’s reload the page and simulate that by saying that the remote end wants to receive at most 800kbps.

Tsahi: So that would be the media server. Instruct the client that the bitrate available for the video is only 800kbps.

Philipp: Yes, this could easily be done using REMB or transport-cc feedback usually.

Tsahi: OK. But what we do is SDP.

Philipp: Yes. Because we can’t control what the browser tells the other browser. An SFU is a bit more flexible in that respect, but we can simulate it quite well. So we do this. You see this layer froze. And we see now that the browser knows it has 800kbps in total, it changes the way it allocates the bitrate to those layers. So here it goes from 450 to 700 and then one hundred.

Tsahi: OK, so we need to be aware of these things because that can change the calculations or the decisions that we’re making in the SFU, in how we allocate bitrates to the users.

Philipp: Yes. You need to always know what do you send in which scenarios and this playground is really powerful as a tool to understand and visualize it.

Tsahi: OK, so we’ve seen a nice playground that allows us to play with simulcast. Seeing three different layers, then being able to manipulate the players in terms of the bitrates that they have, activating and deactivating them and also forcing bitrates down if needed.

Thank you all for that and see you next month in our next fiddle.

Philipp: Bye

Previous Lesson

Back to Module

Next Lesson