Expose peak bitrate in Format when available (as well as average bitrate)

[juechemparathy]

Apple spec talks about having average bandwidth and peak bandwidth in master playlist as below -
'You should include the average bit rate as well as the peak bit rate. This is especially important if your peak is more than 20% different from your average. Specifying the average allows the client to make more intelligent decisions about which variant to play.'
Ref -
https://tools.ietf.org/html/draft-pantos-http-live-streaming-23#page-28
https://developer.apple.com/library/content/technotes/tn2224/_index.html#//apple_ref/doc/uid/DTS40009745-CH1-BITRATERECOMMENDATIONS

Exoplayer seems to be expecting only average Bandwidth in the master playlist as the regex in HLSplaylistParser is "BANDWIDTH=(\d+)\b".

For supporting VBR we need a) regex fix to extract average bitrate and peak bitrate, b) data holder in Format object for peak bitrate

Sample snippet from a master .m3u8 file -
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2095000,SUBTITLES="subs",AVERAGE-BANDWIDTH=1945000,RESOLUTION=1280x720

I am looking at Exo v2.3.1.
Thanks, _jue

[ojw28]

The change above updates ExoPlayer to use AVERAGE-BITRATE to populate Format.bitrate when it's available, since that's what Format.bitrate is documented to contain.

It's somewhat unclear to me that there's a need to also include the peak bitrate in the Format object. Unless people are going to write ABR algorithms that make use of it, the need to include it is pretty hypothetical. If you do have a concrete near term plan to make use of it, please could you clarify how you intend to use it (particularly given you don't know where the peaks are).

[juechemparathy]

@ojw28 - Yes, we intend to use it in our custom ABR algorithm(esp with fmp4) and is also considered as a useful input in our VBR instrumentation data. Thanks.

[NayaneshGupte]

@ojw28 any idea when is this planned tentatively ?

[ojw28]

@julioz (and anyone else who's interested) - I've been looking at how to best merge some version of #6706. It's clear we need two (possibly three to avoid changing behavior in a subtle way) fields in Format. However we need to decide what those fields should be called, what they should be set to if one or the other is not present in the media, and how to map values specified in HLS, SmoothStreaming, DASH and regular media container formats into them. The definitions used in the various specifications are:

HLS

• AVERAGE-BANDWIDTH - The value is a decimal-integer of bits per second. It represents the average segment bit rate of the Variant Stream.
• BANDWIDTH - The value is a decimal-integer of bits per second. It represents the peak segment bit rate of the Variant Stream.

DASH

• @bandwidth - Consider a hypothetical constant bitrate channel of bandwidth with the value of this attribute in bits per second (bps). Then, if the Representation is continuously delivered at this bitrate, starting at any SAP that is indicated either by @startWithSAP or by any Segment Index box, a client can be assured of having enough data for continuous playout providing playout begins after @minBufferTime * @bandwidth bits have been received (i.e. at time @minBufferTime after the first bit is received).

SmoothStreaming

• Bitrate - The average bandwidth that is consumed by the track, in bits per second (bps). The value 0 MAY be used for tracks whose bit rate is negligible relative to other tracks in the presentation

Other containers

• Unknown and possibly container specific, but I would guess they're normally average bit-rates.

---

Some thoughts:

• The DASH one is interesting. It's actually very nicely defined, from the perspective of allowing a player to do the correct thing. My interpretation is that it's closer to being a peak bit-rate than an average bit-rate. If @minBufferTime = 0 then it's definitely a peak bit-rate. As @minBufferTime becomes larger it's less of a peak bit-rate, but we'd normally expect it to be set to something relatively small.
• My interpretation of the HLS BANDWIDTH is that it's a peak bitrate as averaged over a segment. If @minBufferTime is around one segment duration in DASH, then I would guess they end up being approximately equivalent to one another.

---

Proposal:

• Keep the current bitrate field as it is today, to avoid breaking anything (possibly to be deprecated)
• Add averageBitrate, which is populated with AVERAGE-BANDWIDTH in HLS, nothing in DASH, Bitrate in SmoothStreaming, and any values parsed from other container formats (unless we know otherwise - we can set the right one in each case). In the case a value isn't present, Format.NO_VALUE will be used.
• Add peakBitrate, which is populated with BANDWIDTH in HLS, @bandwidth in DASH, and nothing in SmoothStreaming and for other container formats (unless we know otherwise - we can set the right one in each case). In the case a value isn't present, Format.NO_VALUE will be used.

One caveat here is that code implementing adaptive algorithms will need to be careful to fallback to using the other field in the case that the one it would use ideally is set to Format.NO_VALUE. This is definitely necessary for any adaptive algorithm that wants to support both DASH and SmoothStreaming, because they set different fields. Perhaps this is fine (good?) though, since an adaptive algorithm should probably behave slightly differently depending on whether it's using a peak or average (i.e. if it's using an average then it should be more cautious).

@julioz - Any thoughts on this proposal?

[julioz]

@ojw28 The proposal sounds very reasonable to me.

It does introduce a small extra bit through the learning curve of the API: I assume most consumers read bitrate and aren't interested in whether it's an average or peak, so if we decide to @deprecate that field, we must make sure we point people to either this documentation or some other clarification of the semantic differences.

(I did spend a few moments pondering whether we should instead keep bitrate as a convenience for those callers, but I guess that could cause more confusion that aid; so I believe deprecating it is the way to go.)