Using playback rate to help reduce rebuffering and maintain buffer health.

[gkatsev]

Have you read the FAQ and checked for duplicate open issues?
Yes, there aren't existing issues.

Is your feature request related to a problem? Please describe.

We would like to further reduce the likelihood of rebuffering and increase buffer health by tweaking the playback rate of the video.

Describe the solution you'd like

This involves 3 changes that are all related.

• For VOD
  • streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.
• For live
  • streaming.liveSyncPanicMode: if enabled, When the player rebuffers, enter "panic" mode, which sets the player to the liveSyncMinPlaybackRate for a specified duration, say 60s by default.
  • streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Describe alternatives you've considered

It's possible to introduce other ABR algorithms, but this gives us another tool to maintain buffer health for both live and vod content and help reduce rebuffering for users.

Additional context

The first two features, dynamic playback rate for vod and panic mode for live sync, can both be implemented as extensions on top of what's currently there. Though, if there are pointers on how to approach this that would fit the codebase better, that would be great!
For example, for the panic mode, we'd need to store when the last rebuffering event happened so that we can check against it in the onTimeUpate_ method in player.js, where should that var be stored? And for the vod dynamic playback rate, should the play rate controller own this functionality?

The third feature, dynamic target latency, is a bit trickier. Currently, live sync is set up by looking at the min and max latencies (either configured or received via DASH ServiceDescriptions) and then playback rate is set up based on that. However, for this feature, we'd need to have a third value, target latency -- which the DASH LL Spec does define, actually -- but switching from a min/max latency to a target latency would potentially be a breaking change or would potentially duplicate a lot of code (since we'd want to fork the onTimeUpdate_ method depending on whether dynamic target latency is enabled or not).
While, the DASH spec, technically, says that the target latency is the only required one, I'm not sure if it's necessarily worth migrating to, given the risk just outlined. However, if y'all have thoughts on how to do that or any other alternatives, it would be nice as this feature can help reduce rebufferings.

[avelad]

I've been thinking about this and I think we need to split it into 3 PRs to make it easier to review and merge.

Some comments:

streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.

I see this functionality as perfect, when we have a high buffer level I think we should calculate it as a percentage of the bufferingGoal value that we have configured.
Example: config.vodDynamicPlaybackRateBufferPercentage * config.bufferingGoal
We have to add the vodDynamicPlaybackRateBufferPercentage configuration

streaming.liveSyncPanicMode: if enabled, When the player rebuffers, enter "panic" mode, which sets the player to the liveSyncMinPlaybackRate for a specified duration, say 60s by default.

I see the functionality correct, but the time value must be configurable.

For example, for the panic mode, we'd need to store when the last rebuffering event happened so that we can check against it in the onTimeUpate_ method in player.js, where should that var be stored?

The last rebuffering time I think should be stored in shaka.media.BufferingObserver

And for the vod dynamic playback rate, should the play rate controller own this functionality?

It doesn't need to know about this functionality because already it doesn't know anything about liveSync.

streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Regarding this functionality I have many doubts, and before I can continue moving forward with the definition, how do you know if the playback has issues? (the core part that would enable this functionality).

[gkatsev]

I've been thinking about this and I think we need to split it into 3 PRs to make it easier to review and merge.

Yeah, makes sense to have them as 3 PRs.

streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.

I see this functionality as perfect, when we have a high buffer level I think we should calculate it as a percentage of the bufferingGoal value that we have configured. Example: config.vodDynamicPlaybackRateBufferPercentage * config.bufferingGoal We have to add the vodDynamicPlaybackRateBufferPercentage configuration

Is the idea to set the playback rate to a percentage as a percentage of the bufferingGoal? so the closer we are to the bufferingGoal the closer to a playback rate of 1?

streaming.liveSyncPanicMode: if enabled, When the player rebuffers, enter "panic" mode, which sets the player to the liveSyncMinPlaybackRate for a specified duration, say 60s by default.

I see the functionality correct, but the time value must be configurable.

Yeah, definitely needs to be configurable.

For example, for the panic mode, we'd need to store when the last rebuffering event happened so that we can check against it in the onTimeUpate_ method in player.js, where should that var be stored?

The last rebuffering time I think should be stored in shaka.media.BufferingObserver

And for the vod dynamic playback rate, should the play rate controller own this functionality?

It doesn't need to know about this functionality because already it doesn't know anything about liveSync.

Sounds good!

streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Regarding this functionality I have many doubts, and before I can continue moving forward with the definition, how do you know if the playback has issues? (the core part that would enable this functionality).

Yeah, what I wrote is a bit vague. I believe the idea is also to use rebuferring events, but let me gather more info and write a more detailed explanation.

[avelad]

streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.

I see this functionality as perfect, when we have a high buffer level I think we should calculate it as a percentage of the bufferingGoal value that we have configured. Example: config.vodDynamicPlaybackRateBufferPercentage * config.bufferingGoal We have to add the vodDynamicPlaybackRateBufferPercentage configuration

Is the idea to set the playback rate to a percentage as a percentage of the bufferingGoal? so the closer we are to the bufferingGoal the closer to a playback rate of 1?

Example:
playbackRate 0.95(buffer is low), if the buffer we have is less than 25% of bufferingGoal
playbackRate 1 (buffer is high), if buffer is greater than 25% of bufferingGoal

[gkatsev]

streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Regarding this functionality I have many doubts, and before I can continue moving forward with the definition, how do you know if the playback has issues? (the core part that would enable this functionality).

Yeah, what I wrote is a bit vague. I believe the idea is also to use rebuferring events, but let me gather more info and write a more detailed explanation.

So, the approach here is like so:
Keep track of the last time we reached the target latency (if we're within some threshold from the target, I think the ll dash spec recommends 500ms, but can be configurable). Then, periodically, check to see if the last time we've updated the playback rate back to 1 was longer than some stability_thershold, we can adjust the current target time forward towards the minimum latency. The move towards the min latency is done in two passes, so, if min is 1, and target is 5, it'll first set target to 3, then the second time around it'll be set to 1.
Additionally, on rebuffer, move the current target latency back towards max target latency by an increment of latency_increase_on_rebuffer(500ms by default) per rebuffer count. So, on first rebuffer, add 500ms to the current target latency. On 5th, add 500*5, or 2500 (and clamp to max).
The current target latency will also be clamped to the min/max values (probably accounting for the threshold).

Hopefully, that makes sense.

[gkatsev]

@avelad hey, did you get a chance to think about the dynamic target latency? Thanks!

[avelad]

@gkatsev I've reviewed it and it makes sense. I'm OK with implementing that.