Permanent app freeze on foreground: main thread blocks in ReplayIntegration.start while replay executor hangs in MediaCodec.dequeueOutputBuffer holding the replay lock (HyperOS 3 / Android 16)

[Adherentman]

Integration

sentry-android (via sentry-flutter)

Java Version

N/A (Flutter app)

Version

sentry-android 8.49.0 (bundled by sentry_flutter 9.25.0)

Steps to Reproduce

Device: Xiaomi 14 (houji), HyperOS 3 / Android 16, build OS3.0.303.0.WNCCNXM (latest stable).

Replay config:

options.replay.onErrorSampleRate = 1.0;  // buffer mode for all sessions
options.replay.sessionSampleRate = 0.1;

1. Use the app normally, then send it to background (screen off or app switch).
2. HyperOS 3 aggressively freezes the cached process (cgroup freezer). This appears to leave the in-flight MediaCodec session in a broken state.
3. Return to the app.

Expected Result

App resumes normally. A hung replay video encoder must never block the main thread.

Actual Result

The app is permanently frozen: no touch, no back gesture, window never gains focus. The OS records an ANR with subject "Input dispatching timed out (Application does not have a focused window)". Only force-stop recovers.

It is a classic lock-ordering hazard rather than a device quirk:

Thread 1 — SentryReplayIntegration-0 is stuck forever inside MediaCodec.dequeueOutputBuffer (native ALooper::awaitResponse never returns after the process was frozen mid-encode), while holding the replay lock:

"SentryReplayIntegration-0" daemon prio=5 tid=4 Native
  native: #03 pc 0001f2c8  libstagefright_foundation.so (android::ALooper::awaitResponse+284)
  native: #04 pc 0002432c  libstagefright_foundation.so (android::AMessage::postAndAwaitResponse+452)
  native: #05 pc 001d6390  libstagefright.so (android::MediaCodec::dequeueOutputBuffer+436)
  at android.media.MediaCodec.native_dequeueOutputBuffer(Native method)
  at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:4308)
  at io.sentry.android.replay.video.SimpleVideoEncoder.drainCodec(SimpleVideoEncoder.kt:218)
  at io.sentry.android.replay.video.SimpleVideoEncoder.release(SimpleVideoEncoder.kt:288)
  at io.sentry.android.replay.ReplayCache.createVideoOf(ReplayCache.kt:212)
  at io.sentry.android.replay.capture.CaptureStrategy$Companion.createSegment(CaptureStrategy.kt:92)
  at io.sentry.android.replay.capture.BaseCaptureStrategy.createSegmentInternal(BaseCaptureStrategy.kt:142)
  at io.sentry.android.replay.capture.BufferCaptureStrategy.createCurrentSegment$lambda$5(BufferCaptureStrategy.kt:241)
  at io.sentry.android.replay.util.ReplayExecutorService.submit$lambda$0(ReplayExecutorService.kt:26)

Thread 2 — main then blocks forever acquiring that same lock during activity start, killing the whole app:

"main" prio=5 tid=1 Waiting
  at jdk.internal.misc.Unsafe.park(Native method)
  at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:322)
  at io.sentry.util.AutoClosableReentrantLock.acquire(AutoClosableReentrantLock.java:37)
  at io.sentry.android.replay.ReplayIntegration.start(ReplayIntegration.kt:171)
  at io.sentry.android.core.LifecycleWatcher.startSession(LifecycleWatcher.java:87)
  at io.sentry.android.core.LifecycleWatcher.onForeground(LifecycleWatcher.java:62)
  at io.sentry.android.core.AppState$LifecycleObserver.onStart(AppState.java:202)
  at androidx.lifecycle.ProcessLifecycleOwner.activityStarted$lifecycle_process_release(ProcessLifecycleOwner.kt:97)
  at android.app.Activity.performStart(Activity.java:9522)

A second full dump taken on another day shows the identical pair of stacks, so this is stable, not a one-off.

Impact

Our analytics defined a "frozen resume" signature (app opened from background → zero further events → cold start within 120 s, i.e. user force-killed a dead app). Over 30 days:

• Xiaomi devices on Android 16 (HyperOS 3): 6.3 % of resumes (1785 sampled devices)
• Xiaomi on Android ≤ 15: 0.3–0.6 %
• Every other brand on Android 16: 0.3–0.8 %

The rate climbed week over week as the HyperOS 3 OTA rolled out, independent of our app releases. Battery-optimization whitelisting on the device does not prevent it. We had to disable Session Replay entirely to fix our users, confirmed by an affected user: freeze gone with a replay-disabled build.

Suggestions

• ReplayIntegration.start() (and stop/pause/resume) run on the main thread via LifecycleWatcher; acquiring a lock there that can be held across a blocking native MediaCodec call turns any codec hang into a whole-app deadlock. Either move the work off the main thread, use tryLock with a short timeout, or never hold the replay lock across encoder drain/release.
• SimpleVideoEncoder.drainCodec passes an infinite/long timeout to dequeueOutputBuffer in the endOfStream/release path; a bounded timeout with bail-out would make a dead codec merely drop a segment instead of hanging the executor while holding the lock.

Possibly related: #5529 (encoder kept running while rate-limited).

[linear-code]

JAVA-678

[markushi]

@Adherentman thanks for reporting, we'll take a look at this shortly!

[romtsn]

I think #5842 is going to address this too