Support additional passthrough audio formats

[ebr11]

I'm having trouble getting Exo (2.0.4) to play mkv files with AC-3 or DTS (DCA) audio. This is on a device that should support it (Nvidia Shield hooked to an AVR with support for those and more).

All my searches have come up with the basic answer that this error indicates an unsupported audio track but, given the above hardware, I don't understand how that could be. Is there some sort of additional configuration necessary to make pass thru of these tracks to the AVR possilble? I've just used the default everything from the example/demo app.

Thanks for any direction...

12-02 16:44:33.601 E/AudioTrack: AudioFlinger could not create track, status: -22
12-02 16:44:33.605  E/AudioTrack-JNI: Error -22 initializing AudioTrack
12-02 16:44:33.605  E/android.media.AudioTrack: Error code -20 when initializing AudioTrack.
12-02 16:44:33.620  E/ExoPlayerImplInternal: Renderer error.
                                                                      com.google.android.exoplayer2.ExoPlaybackException
                                                                          at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:345)
                                                                          at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:890)
                                                                          at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:477)
                                                                          at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:431)
                                                                          at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:265)
                                                                          at android.os.Handler.dispatchMessage(Handler.java:100)
                                                                          at android.os.Looper.loop(Looper.java:148)
                                                                          at android.os.HandlerThread.run(HandlerThread.java:61)
                                                                          at com.google.android.exoplayer2.util.PriorityHandlerThread.run(PriorityHandlerThread.java:40)
                                                                       Caused by: com.google.android.exoplayer2.audio.AudioTrack$InitializationException: AudioTrack init failed: 0, Config(48000, 252, 49152)
                                                                          at com.google.android.exoplayer2.audio.AudioTrack.checkAudioTrackInitialized(AudioTrack.java:909)
                                                                          at com.google.android.exoplayer2.audio.AudioTrack.initialize(AudioTrack.java:461)
                                                                          at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:337)
                                                                          at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:890) 
                                                                          at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:477) 
                                                                          at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:431) 
                                                                          at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:265) 
                                                                          at android.os.Handler.dispatchMessage(Handler.java:100) 
                                                                          at android.os.Looper.loop(Looper.java:148) 
                                                                          at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                          at com.google.android.exoplayer2.util.PriorityHandlerThread.run(PriorityHandlerThread.java:40)

[ebr11]

Okay, I may have answered my own question - if you guys can just confirm if this is correct and we can close this...

I was using the example in the docs to create my media source and I noticed it wasn't passing in a handler or event logger but the demo app was. I added these to my constructor call and now DD and DTS are passing through as they should.

I suspect the Handler was what was really needed. Is that correct?

[andrewlewis]

One possible explanation for passthrough not working is not passing an AudioCapabilities instance to the renderer, but that seems to be different from the change you're describing (and SimpleExoPlayer does this already, in case you're using that).

Could you paste the exact diff and we can take a look? Thanks.

[ebr11]

Yes, I am using the SimpleExoPlayer and I just changed the parameters back to null that I changed before and the audio is not throwing an error anymore so I do not have an explanation at this point.

I think Exo must have determined that the audio should work or it wouldn't even have tried to send it, correct? So I don't know why it failed with the above errors and I also don't know why it doesn't now - unless it was just some sort of HDMI handshake issue or something.

As a related question, should TrueHD and DTS-HDMA work this way as well? The AVR supports them but Exo is passing through just core DD and DTS instead of the lossless versions.

[andrewlewis]

Yes, we check the audio renderer capabilities before trying to use it, so the error is unexpected. Is it possible that some override was in place that caused ExoPlayer to try to use passthrough even if the HDMI connection didn't advertise it (e.g. an Android TV platform setting to force passthrough)? If you can find some way to reproduce the original error, please provide some instructions so we can investigate further.

We don't yet support TrueHD though we might add support soon as AudioFormat.ENCODING_DOLBY_TRUEHD was added in API 24. I'm not sure about DTS-HDMA. My understanding is that ENCODING_IEC61937 can be used to support arbitrary passthrough formats. If the original issue does not need further investigation I will mark this as an enhancement to add support for those encodings.

[ebr11]

Yes, support for passing through those HD formats is my primary goal and I cannot seem to reproduce the above error.

Given a DTS-HD-MA or other bitstream format track, how would I tell Exo to treat it as IEC61937?

Also, would the ffmpeg extension get me pass-thru on these formats or would they be software decoded by ffmpeg?

Thanks very much for your help.

[andrewlewis]

MediaCodecAudioRenderer is used for passthrough. FfmpegAudioRenderer should always decode to PCM.

Support for passthrough of other formats using IEC 61937 is an enhancement. I will use this issue to track that. Thanks!

[ebr11]

On the DTS-HD MA issue, it is partially supported in that audio plays. However, it is only sending the Core DTS stream, not the lossless MA one even though the device does report that it supports DTS_HD.

Not sure if it is relevant but I can see in the track map that there is no difference to Exo (at least reported here) between a DTS HD stream and a straight DTS one. The below map contains two audio streams. The first one is DTS HD MA and the second one is straight DTS but, in the map, they are identical and, when played, the HD track comes through as straight DTS.

Hopefully, this is helpful. Thanks.

Tracks [ Renderer:0 [ Group:0, adaptive_supported=N/A [ [X] Track:0, id=1, mimeType=video/avc, res=1920x1080, supported=YES ] ] Renderer:1 [ Group:0, adaptive_supported=N/A [ [X] Track:0, id=2, mimeType=audio/vnd.dts, channels=6, sample_rate=48000, language=eng, supported=YES ] Group:1, adaptive_supported=N/A [ [ ] Track:0, id=3, mimeType=audio/vnd.dts, channels=6, sample_rate=48000, language=eng, supported=YES ] ]

[andrewlewis]

Was the stream extracted from a Matroska file? That container uses the same codec identifier for DTS and DTS-HD. Currently we just assume DTS for Matroska tracks. To make this work properly we'd need to inspect the samples to distinguish these two formats, which is inconvenient in the current model.

[ebr11]

Yes it is in an MKV. If I use MKV ToolNix I can see the difference (that's how I confirmed I wasn't just feeding it bad data).

In our processes that inspect these tracks, for DTS it is almost always "DCA" and then you have to look at the profile value to see that it is DTS or DTS-HD.

Is there anything I can do to make it pass the actual HD track?

[andrewlewis]

Does it work if you just specify MimeTypes.AUDIO_DTS_HD in the MIME type in MatroskaExtractor.java?

I'm afraid we don't have a sensible way to do this other than forking the Extractor at the moment. You can pass your own ExtractorsFactory to ExtractorMediaSource.