DTS audio doesn't play using FFmpeg extension in 2.14.2

[moneytoo]

After updating to ExoPlayer 2.14.2, I've noticed an issue with playback backed by the FFmpeg extension. I was able to play DTS audio just fine with 2.14.1 while it's no longer working on my device with 2.14.2.

I use the FFmpeg extension with SimpleExoPlayer enabled like this:

RenderersFactory renderersFactory = new DefaultRenderersFactory(this)
        .setExtensionRendererMode(DefaultRenderersFactory.EXTENSION_RENDERER_MODE_ON);

I can reproduce it with any media containing DTS (I didn't test much any other formats), including test video encoded in FFmpeg (like this one: https://brouken.com/tmp/sync-dts.mp4).

When I hit play, I get error like this one (see longer logcat.txt):

D/AudioTrack: set(): streamType -1, sampleRate 44100, format 0xb000000, channelMask 0x3, frameCount 48000, flags #0, notificationFrames 0, sessionId 169, transferType 3, uid -1, pid -1
E/IAudioFlinger: createTrack returned error -38
E/AudioTrack: createTrack_l(0): AudioFlinger could not create track, status: -38 output 184549376
E/AudioTrack-JNI: Error -38 initializing AudioTrack
D/AudioTrack: gather(): no metrics gathered, track status=-38
E/android.media.AudioTrack: Error code -20 when initializing AudioTrack.
E/DecoderAudioRenderer: Audio sink error
      com.google.android.exoplayer2.audio.AudioSink$InitializationException: AudioTrack init failed 0 Config(44100, 12, 48000)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.buildAudioTrack(DefaultAudioSink.java:2011)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.buildAudioTrack(DefaultAudioSink.java:844)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.initializeAudioTrack(DefaultAudioSink.java:653)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.handleBuffer(DefaultAudioSink.java:726)
        at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:637)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.bypassRender(MediaCodecRenderer.java:2218)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:838)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:945)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:478)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:214)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: java.lang.UnsupportedOperationException: Cannot create AudioTrack
        at android.media.AudioTrack$Builder.build(AudioTrack.java:1024)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.createAudioTrackV29(DefaultAudioSink.java:2060)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.createAudioTrack(DefaultAudioSink.java:2038)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.buildAudioTrack(DefaultAudioSink.java:2003)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.buildAudioTrack(DefaultAudioSink.java:844) 
        at com.google.android.exoplayer2.audio.DefaultAudioSink.initializeAudioTrack(DefaultAudioSink.java:653) 
        at com.google.android.exoplayer2.audio.DefaultAudioSink.handleBuffer(DefaultAudioSink.java:726) 
        at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:637) 
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.bypassRender(MediaCodecRenderer.java:2218) 
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:838) 
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:945) 
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:478) 
        at android.os.Handler.dispatchMessage(Handler.java:103) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
E/ExoPlayerImplInternal: Playback error
      com.google.android.exoplayer2.ExoPlaybackException: MediaCodecAudioRenderer error, index=2, format=Format(2, null, null, audio/vnd.dts, null, -1, en, [-1, -1, -1.0], [2, 44100]), format_supported=YES
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:555)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:214)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: com.google.android.exoplayer2.audio.AudioSink$InitializationException: AudioTrack init failed 0 Config(44100, 12, 48000)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.buildAudioTrack(DefaultAudioSink.java:2011)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.buildAudioTrack(DefaultAudioSink.java:844)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.initializeAudioTrack(DefaultAudioSink.java:653)
        at com.google.android.exoplayer2.audio.DefaultAudioSink.handleBuffer(DefaultAudioSink.java:726)
        at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:637)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.bypassRender(MediaCodecRenderer.java:2218)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:838)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:945)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:478)
        at android.os.Handler.dispatchMessage(Handler.java:103) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
          Suppressed: com.google.android.exoplayer2.audio.AudioSink$InitializationException: AudioTrack init failed 0 Config(44100, 12, 48000)
              ... 12 more
          Caused by: java.lang.UnsupportedOperationException: Cannot create AudioTrack
        at android.media.AudioTrack$Builder.build(AudioTrack.java:1024)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.createAudioTrackV29(DefaultAudioSink.java:2060)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.createAudioTrack(DefaultAudioSink.java:2038)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.buildAudioTrack(DefaultAudioSink.java:2003)
                  ... 11 more
     Caused by: java.lang.UnsupportedOperationException: Cannot create AudioTrack
        at android.media.AudioTrack$Builder.build(AudioTrack.java:1024)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.createAudioTrackV29(DefaultAudioSink.java:2060)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.createAudioTrack(DefaultAudioSink.java:2038)
        at com.google.android.exoplayer2.audio.DefaultAudioSink$Configuration.buildAudioTrack(DefaultAudioSink.java:2003)
              ... 11 more

While with connected headphones (Samsung Galaxy Buds), the video just doesn't play.

• ExoPlayer version number: 2.14.2
• Android version: Android 10
• Android device: OnePlus 7

I can reproduce this issue only on this real device (I cannot reproduce it in emulator). Playback seems to work when I enable FFmpeg extension using EXTENSION_RENDERER_MODE_PREFER. It also starts to work when I revert 3ae4c1b.

(I can include a bugreport later, when I will have some more time.)

[christosts]

@andrewlewis can you take a look?

[andrewlewis]

It looks like the detected audio capabilities include DTS passthrough output, but it seems very strange that that would be supported on a phone and indeed initializing the audio track for encoded DTS output (I think this is 0xb000000) is failing:

D/AudioTrack: set(): streamType -1, sampleRate 44100, format 0xb000000, channelMask 0x3, frameCount 48000, flags #0, notificationFrames 0, sessionId 169, transferType 3, uid -1, pid -1
E/IAudioFlinger: createTrack returned error -38

[andrewlewis]

5e4cd12 should fix this by making it so that we only use AudioTrack.isDirectPlaybackSupported on TV devices. Thanks for reporting the issue.

[moneytoo]

@andrewlewis I'm still trying to make sense of what AudioTrack.isDirectPlaybackSupported() reports. I was about to ask people to run simple test to understand the scope of this issue. However I realized that the incorrectly reported support of DTS is only one thing. I know that my device has at least AC3 and E-AC3 decoders so shouldn't these also be in the list? (Because the aren't there.)

Here's a config file that matches the reported capabilities using the API:
OnePlus 7: https://dumps.tadiphone.dev/dumps/oneplus/oneplus7/-/blob/qssi-user-11-RKQ1.201022.002-2106290956-release-keys/vendor/etc/audio/audio_policy_configuration.xml#L141-146

<profile name="" format="AUDIO_FORMAT_DTS"
         samplingRates="32000,44100,48000"
         channelMasks="AUDIO_CHANNEL_OUT_MONO,AUDIO_CHANNEL_OUT_STEREO,AUDIO_CHANNEL_OUT_2POINT1,AUDIO_CHANNEL_OUT_QUAD,AUDIO_CHANNEL_OUT_PENTA,AUDIO_CHANNEL_OUT_5POINT1"/>
<profile name="" format="AUDIO_FORMAT_DTS_HD"
         samplingRates="32000,44100,48000,64000,88200,96000,128000,176400,192000"
         channelMasks="AUDIO_CHANNEL_OUT_MONO,AUDIO_CHANNEL_OUT_STEREO,AUDIO_CHANNEL_OUT_2POINT1,AUDIO_CHANNEL_OUT_QUAD,AUDIO_CHANNEL_OUT_PENTA,AUDIO_CHANNEL_OUT_5POINT1,AUDIO_CHANNEL_OUT_6POINT1,AUDIO_CHANNEL_OUT_7POINT1"/>

Other, newer OnePlus 9 Pro will probably report similar formats (DTS), while the specs lists AC3 like formats that aren't listed in the config:
OnePlus 9 Pro: https://dumps.tadiphone.dev/dumps/oneplus/oneplus9pro/-/blob/qssi-user-11-RKQ1.201105.002-2107082109-release-keys/vendor/etc/audio/audio_policy_configuration.xml#L152-157

Even device like Xiaomi Mi 10 Pro https://dumps.tadiphone.dev/dumps/xiaomi/cmi/-/blob/qssi-user-11-RKQ1.200826.002-V12.5.4.0.RJACNXM-release-keys/vendor/etc/audio/audio_policy_configuration.xml#L128-133 reports DTS while it misses AC3. I can't really be shure what the real capabilities of the device are but supporting AC3 makes (because of streaming content) much bigger sense than DTS.

[andrewlewis]

Maybe the AC3/E-AC3 decoders are listed as MediaCodecs rather than being advertised as supported encodings for AudioTrack?