Motion vs detail

🎻 August 2022 fiddle

Want to learn more about WebRTC?

Look us up here for the WebRTC Fiddle of the Month, created… once a month.

Or just enroll to one of our excellent WebRTC training courses.

Resources

• The fiddle: https://jsfiddle.net/fippo/89zm7nrs/3/

Transcription

Tsahi : Motion or Detail? This is exactly what we want to try and answer in this WebRTC Fiddle of the Month. With me is Philippp Hancke. What we’re going to do now is discuss the need to decide between Motion and Detail when using WebRTC encoding for video channels, right?

Philipp: Yes. I mean, video encoding is always a compromise between the available bandwidth, and how to fit the most resolution or frame rate into the budget. And there is a WebRTC option called Content Hint, which lets you influence the behavior of the encoding. In that case.

Tsahi: I’m going to tell WebRTC what my content is like so it can know better how to encode it and send it over the network in the way that would fit my content.

Philipp: Yes, typically, you want to do that, for screen sharing, where you typically want Detail and not Motion. But it also has some effect on the webcam feed. So you might want to play around with what we do in the fiddle, and decide what to do in your product.

Tsahi: Okay, so let’s turn the screen and see what we are doing.

Philipp: Yes, so the fiddle is a bit larger this time, let’s quickly walk through it. So we’re including the graph library from the samples, we have a display area for local and remote videos. And we have several input options. One is for the bandwidth, we have a range between 32 kbps, which is very low and 512 kbps, which is reasonably high bandwidth for up to 720p. We don’t go for like two Mbps, because then you don’t see a difference at all.

Tsahi: Okay, so we’re going to give the user the ability to play with the bandwidth that he wants.

Philipp: Which is to simulate the bad conditions.

Tsahi: Okay.

Philipp: And we also have the option to choose between the Content Hint, the two defined values, our Motion and Detail, and we have a way to answer it to get the default behavior.

Tsahi: Okay.

Philipp: And we even have the option to play with the resolution, which we can’t do in this fiddle because the cameras already used by another application, so we can’t change it as much as we want. But if you do it instead alone, it should work.

Tsahi: Okay, so I see it twice here that’s because we’re opening two local peer connections, one sending to the other and then on the other one, again, to see what was observed remotely, so I can see the source and what happened after the encoder, right?

Philipp: Yes, yes, I think that’s the this part here, which is doing the local pair collection, which is pretty standard. We’ve done it before. And what we have after that is basically wiring up the bandwidth change dropdown, which is here to change the encoding parameters max bitrate, we can change the competent. If it’s unset, we set it to undefined, otherwise, we set it to Motion or Detail. And same for the resolution drop down, we use apply constraints to change the bandwidth of the camera on the fly. And then we have a lot of graphs help us which do this graph stuff.

Tsahi: Okay.

Philipp: it’s pretty complicated, but not very important what we want to explain.

Tsahi: So can you guide me throughout till we’re actually seeing in the results?

Philipp: So here we have the local video, and you can see 640×480. And here is the remote video. It is 320×240 because the bandwidth is pretty low. And in the canvas here we see the set bitrate and we configure it 64 kbps.

Tsahi: Okay, and then the next number is framerate, I guess?

Philipp: Yes. And we can see there was a drop initially, but currently, it’s pretty stable at 30 frames per second.

Tsahi: As long as you don’t move.

Philipp: I can move a bit more but it is good at keeping… there you can see a small dip. Because previously I didn’t move much so the encoder didn’t have to do much, it could fit the project pretty good.

Tsahi: And then we’ve got the last graph.

Philipp: Yes, which is if received frame width.

Tsahi: Okay, so though we set it to 640, it’s on 320.

Philipp: Yes, because it can’t really do 640 or 480 at 30 frames per second.

Tsahi: It can, but the result wouldn’t be as good.

Philipp: No. So if we want to enforce the resolution, we set the Content Hint of detail. And what happens then is that the framerate drops, and you can see that the resolution is better but it is that stuttering a bit and you can see it here, the resolution increased to 640×480. But the frame rate took a big hit and it’s going down to five frames per second. If I move a bit more, it will go down again. And encoder is trying to do something but it decided to drop framerate, but keep the resolution in this case.

Tsahi: So you can see good video but stuttering a bit more.

Philipp: Yes.

Tsahi: Okay.

Philipp: Yes. And if we set it to motion, it will actually makes the movement much more fluent. Let’s do that. But as you can see, it’s blurring a bit. Again, which is the reduction to 320×240.

Tsahi: Because now the capacity decides that I’m going to do 30 frames per second or more frames. They don’t have enough bitrates for my budget. And this being the case, I’m going to reduce per frame to quality in order to be able to send or to use less bits per frame because I need more frames.

Philipp: Yes. And movement is reflected pretty well. And you see the framerate is at 30 frames per second.

Tsahi: Okay, so if I want to do 4K at 100kbps?

Philipp: Well, you’re going to have a hard time getting out even one frame per seconds.

Tsahi: Yeah, that’s true. Okay, so thank you for this and see y’all in our next fiddle next month.

Philipp: Bye.