chore: reproduce BundlerDownload OOM issue

[AndreiCalazans]

Summary:

The whole point of this PR is to reproduce this OOM issue we are having with large bundles on Android emulators and start a dicussion on if having this land in RN Core is somethign that interests the team before I put any effort in migrating the current Java patch to the latest Kotlin version of both BundleDownloader and MultipartStreamReader.

cc @cortinico does this need to be an issue or can we discuss on the PR?

Context

We are facing this OOM on Android emluators due to our large bundles. While we can reduce the size of the bundles this seems to keep cropping back up as it grows again or when the emulator gets bottlenecked by other memory usages. We were able to fix the problem by patching the BundleDownloader with a bit of help from LLMs.

07-16 07:33:48.691  9900 10089 E AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher

07-16 07:33:48.691  9900 10089 E AndroidRuntime: Process: com.<REDACTED>.android.development, PID: 9900

07-16 07:33:48.691  9900 10089 E AndroidRuntime: java.lang.OutOfMemoryError: Failed to allocate a 8208 byte allocation with 1982936 free bytes and 1936KB until OOM, target footprint 201326592, growth limit 201326592; giving up on allocation because <1% of heap free after GC.

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.Segment.<init>(Segment.kt:62)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.SegmentPool.take(SegmentPool.kt:90)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.Buffer.writableSegment$okio(Buffer.kt:1485)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.InputStreamSource.read(JvmOkio.kt:91)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:128)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.RealBufferedSource.read(RealBufferedSource.kt:192)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okhttp3.internal.http1.Http1ExchangeCodec$AbstractSource.read(Http1ExchangeCodec.kt:339)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.kt:420)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okhttp3.internal.connection.Exchange$ResponseBodySource.read(Exchange.kt:281)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at okio.RealBufferedSource.read(RealBufferedSource.kt:192)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at com.facebook.react.devsupport.MultipartStreamReader.readAllParts(MultipartStreamReader.java:136)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at com.facebook.react.devsupport.BundleDownloader.processMultipartResponse(BundleDownloader.java:178)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at com.facebook.react.devsupport.BundleDownloader.-$$Nest$mprocessMultipartResponse(Unknown Source:0)

07-16 07:33:48.691  9900 10089 E AndroidRuntime: at

We were able to fix it by patching the BundleDownloader and MultipartStreamReader on version 0.77.2 which was still the Java version. I didn't want to put any effort towards migrating this to Kotlin unless RN Core team says this is ok to land in RN Core.

M packages/react-native/ReactAndroid/src/main/java/com/facebook/react/devsupport/BundleDownloader.java
@@ -177,42 +177,25 @@ public class BundleDownloader {
         bodyReader.readAllParts(
             new MultipartStreamReader.ChunkListener() {
               @Override
-              public void onChunkComplete(
-                  Map<String, String> headers, Buffer body, boolean isLastChunk)
+              public void onChunkComplete(Map<String, String> headers, BufferedSource body, boolean isLastChunk)
                   throws IOException {
-                // This will get executed for every chunk of the multipart response. The last chunk
-                // (isLastChunk = true) will be the JS bundle, the other ones will be progress
-                // events
-                // encoded as JSON.
-                if (isLastChunk) {
-                  // The http status code for each separate chunk is in the X-Http-Status header.
+                if (isLastChunk || "application/javascript".equals(headers.get("Content-Type"))) {
                   int status = response.code();
                   if (headers.containsKey("X-Http-Status")) {
                     status = Integer.parseInt(headers.get("X-Http-Status"));
                   }
                   processBundleResult(
                       url, status, Headers.of(headers), body, outputFile, bundleInfo, callback);
-                } else {
-                  if (!headers.containsKey("Content-Type")
-                      || !headers.get("Content-Type").equals("application/json")) {
-                    return;
-                  }
-
+                } else if ("application/json".equals(headers.get("Content-Type"))) {
                   try {
-                    JSONObject progress = new JSONObject(body.readUtf8());
-                    String status =
-                        progress.has("status") ? progress.getString("status") : "Bundling";
-                    Integer done = null;
-                    if (progress.has("done")) {
-                      done = progress.getInt("done");
-                    }
-                    Integer total = null;
-                    if (progress.has("total")) {
-                      total = progress.getInt("total");
-                    }
+                    String progressText = body.readUtf8(); // safe because progress is small
+                    JSONObject progress = new JSONObject(progressText);
+                    String status = progress.optString("status", "Bundling");
+                    Integer done = progress.has("done") ? progress.getInt("done") : null;
+                    Integer total = progress.has("total") ? progress.getInt("total") : null;
                     callback.onProgress(status, done, total);
                   } catch (JSONException e) {
-                    FLog.e(ReactConstants.TAG, "Error parsing progress JSON. " + e.toString());
+                    FLog.e(ReactConstants.TAG, "Error parsing progress JSON: " + e);
                   }
                 }
               }
@@ -237,51 +220,41 @@ public class BundleDownloader {
   }
 
   private void processBundleResult(
-      String url,
-      int statusCode,
-      Headers headers,
-      BufferedSource body,
-      File outputFile,
-      BundleInfo bundleInfo,
-      DevBundleDownloadListener callback)
-      throws IOException {
-    // Check for server errors. If the server error has the expected form, fail with more info.
-    if (statusCode != 200) {
-      String bodyString = body.readUtf8();
-      DebugServerException debugServerException = DebugServerException.parse(url, bodyString);
-      if (debugServerException != null) {
-        callback.onFailure(debugServerException);
-      } else {
-        StringBuilder sb = new StringBuilder();
-        sb.append("The development server returned response error code: ")
-            .append(statusCode)
-            .append("\n\n")
-            .append("URL: ")
-            .append(url)
-            .append("\n\n")
-            .append("Body:\n")
-            .append(bodyString);
-        callback.onFailure(new DebugServerException(sb.toString()));
-      }
-      return;
-    }
-
-    if (bundleInfo != null) {
-      populateBundleInfo(url, headers, bundleInfo);
+    String url,
+    int statusCode,
+    Headers headers,
+    BufferedSource body,
+    File outputFile,
+    BundleInfo bundleInfo,
+    DevBundleDownloadListener callback
+) throws IOException {
+  if (statusCode != 200) {
+    String errorText = body.readUtf8(); // read small error body into memory
+    DebugServerException exception = DebugServerException.parse(url, errorText);
+    if (exception != null) {
+      callback.onFailure(exception);
+    } else {
+      callback.onFailure(new DebugServerException("Server error: " + errorText));
     }
+    return;
+  }
 
-    File tmpFile = new File(outputFile.getPath() + ".tmp");
+  if (bundleInfo != null) {
+    populateBundleInfo(url, headers, bundleInfo);
+  }
 
-    if (storePlainJSInFile(body, tmpFile)) {
-      // If we have received a new bundle from the server, move it to its final destination.
-      if (!tmpFile.renameTo(outputFile)) {
-        throw new IOException("Couldn't rename " + tmpFile + " to " + outputFile);
-      }
-    }
+  File tmpFile = new File(outputFile.getPath() + ".tmp");
+  try (Sink output = Okio.sink(tmpFile)) {
+    body.readAll(output); // stream to disk
+  }
 
-    callback.onSuccess();
+  if (!tmpFile.renameTo(outputFile)) {
+    throw new IOException("Failed to move temp file to final location");
   }
 
+  callback.onSuccess();
+}
+
   private static boolean storePlainJSInFile(BufferedSource body, File outputFile)
       throws IOException {
     Sink output = null;
M packages/react-native/ReactAndroid/src/main/java/com/facebook/react/devsupport/MultipartStreamReader.java
@@ -7,6 +7,7 @@
 
 package com.facebook.react.devsupport;
 
+import java.io.EOFException;
 import java.io.IOException;
 import java.util.HashMap;
 import java.util.Map;
@@ -16,151 +17,93 @@ import okio.ByteString;
 
 /** Utility class to parse the body of a response of type multipart/mixed. */
 public class MultipartStreamReader {
-  // Standard line separator for HTTP.
-  private static final String CRLF = "\r\n";
 
-  private final BufferedSource mSource;
-  private final String mBoundary;
-  private long mLastProgressEvent;
+  private static final String CRLF = "\r\n";
+  private final BufferedSource source;
+  private final String boundary;
 
   public interface ChunkListener {
-    /** Invoked when a chunk of a multipart response is fully downloaded. */
-    void onChunkComplete(Map<String, String> headers, Buffer body, boolean isLastChunk)
-        throws IOException;
-
-    /** Invoked as bytes of the current chunk are read. */
+    void onChunkComplete(Map<String, String> headers, BufferedSource body, boolean isLastChunk) throws IOException;
     void onChunkProgress(Map<String, String> headers, long loaded, long total) throws IOException;
   }
 
   public MultipartStreamReader(BufferedSource source, String boundary) {
-    mSource = source;
-    mBoundary = boundary;
+    this.source = source;
+    this.boundary = boundary;
   }
 
-  private Map<String, String> parseHeaders(Buffer data) {
-    Map<String, String> headers = new HashMap<>();
-
-    String text = data.readUtf8();
-    String[] lines = text.split(CRLF);
-    for (String line : lines) {
-      int indexOfSeparator = line.indexOf(":");
-      if (indexOfSeparator == -1) {
-        continue;
+  public boolean readAllParts(final ChunkListener listener) throws IOException {
+    final String boundaryMarker = "--" + boundary;
+    final String closingMarker = boundaryMarker + "--";
+
+    int partCount = 0;
+    Map<String, String> headers = null;
+    Buffer chunkBuffer = null;
+
+    while (!source.exhausted()) {
+      String line;
+      try {
+        line = source.readUtf8LineStrict();
+      } catch (EOFException eof) {
+        // emit last part if boundary was missing
+        if (headers != null && chunkBuffer != null) {
+          listener.onChunkComplete(headers, chunkBuffer, true);
+        }
+        return true;
       }
 
-      String key = line.substring(0, indexOfSeparator).trim();
-      String value = line.substring(indexOfSeparator + 1).trim();
-      headers.put(key, value);
-    }
+      if (line == null || !line.startsWith("--")) continue;
+      if (!line.equals(boundaryMarker) && !line.equals(closingMarker)) continue;
 
-    return headers;
-  }
-
-  private void emitChunk(Buffer chunk, boolean done, ChunkListener listener) throws IOException {
-    ByteString marker = ByteString.encodeUtf8(CRLF + CRLF);
-    long indexOfMarker = chunk.indexOf(marker);
-    if (indexOfMarker == -1) {
-      listener.onChunkComplete(null, chunk, done);
-    } else {
-      Buffer headers = new Buffer();
-      Buffer body = new Buffer();
-      chunk.read(headers, indexOfMarker);
-      chunk.skip(marker.size());
-      chunk.readAll(body);
-      listener.onChunkComplete(parseHeaders(headers), body, done);
-    }
-  }
-
-  private void emitProgress(
-      Map<String, String> headers, long contentLength, boolean isFinal, ChunkListener listener)
-      throws IOException {
-    if (headers == null || listener == null) {
-      return;
-    }
-
-    long currentTime = System.currentTimeMillis();
-    if (currentTime - mLastProgressEvent > 16 || isFinal) {
-      mLastProgressEvent = currentTime;
-      long headersContentLength =
-          headers.get("Content-Length") != null ? Long.parseLong(headers.get("Content-Length")) : 0;
-      listener.onChunkProgress(headers, contentLength, headersContentLength);
-    }
-  }
+      // ✅ Emit previous part before starting a new one
+      if (headers != null && chunkBuffer != null) {
+        boolean isLast = line.equals(closingMarker);
+        listener.onChunkComplete(headers, chunkBuffer, isLast);
+        headers = null;
+        chunkBuffer = null;
+        partCount++;
+      }
 
-  /**
-   * Reads all parts of the multipart response and execute the listener for each chunk received.
-   *
-   * @param listener Listener invoked when chunks are received.
-   * @return If the read was successful
-   */
-  public boolean readAllParts(ChunkListener listener) throws IOException {
-    ByteString delimiter = ByteString.encodeUtf8(CRLF + "--" + mBoundary + CRLF);
-    ByteString closeDelimiter = ByteString.encodeUtf8(CRLF + "--" + mBoundary + "--" + CRLF);
-    ByteString headersDelimiter = ByteString.encodeUtf8(CRLF + CRLF);
-
-    int bufferLen = 4 * 1024;
-    long chunkStart = 0;
-    long bytesSeen = 0;
-    Buffer content = new Buffer();
-    Map<String, String> currentHeaders = null;
-    long currentHeadersLength = 0;
-
-    while (true) {
-      boolean isCloseDelimiter = false;
-
-      // Search only a subset of chunk that we haven't seen before + few bytes
-      // to allow for the edge case when the delimiter is cut by read call.
-      long searchStart = Math.max(bytesSeen - closeDelimiter.size(), chunkStart);
-      long indexOfDelimiter = content.indexOf(delimiter, searchStart);
-      if (indexOfDelimiter == -1) {
-        isCloseDelimiter = true;
-        indexOfDelimiter = content.indexOf(closeDelimiter, searchStart);
+      if (line.equals(closingMarker)) {
+        return true;
       }
 
-      if (indexOfDelimiter == -1) {
-        bytesSeen = content.size();
-
-        if (currentHeaders == null) {
-          long indexOfHeaders = content.indexOf(headersDelimiter, searchStart);
-          if (indexOfHeaders >= 0) {
-            mSource.read(content, indexOfHeaders);
-            Buffer headers = new Buffer();
-            content.copyTo(headers, searchStart, indexOfHeaders - searchStart);
-            currentHeadersLength = headers.size() + headersDelimiter.size();
-            currentHeaders = parseHeaders(headers);
-          }
-        } else {
-          emitProgress(currentHeaders, content.size() - currentHeadersLength, false, listener);
+      headers = new HashMap<>();
+      while (true) {
+        String headerLine = source.readUtf8LineStrict();
+        if (headerLine.isEmpty()) break;
+        int index = headerLine.indexOf(':');
+        if (index != -1) {
+          headers.put(headerLine.substring(0, index).trim(), headerLine.substring(index + 1).trim());
         }
+      }
 
-        long bytesRead = mSource.read(content, bufferLen);
-        if (bytesRead <= 0) {
-          return false;
+      chunkBuffer = new Buffer();
+      while (!source.exhausted()) {
+        source.request(1);
+        String maybeBoundary;
+        try {
+          maybeBoundary = source.readUtf8LineStrict();
+        } catch (EOFException eof) {
+          listener.onChunkComplete(headers, chunkBuffer, true);
+          return true;
         }
-        continue;
-      }
 
-      long chunkEnd = indexOfDelimiter;
-      long length = chunkEnd - chunkStart;
-
-      // Ignore preamble
-      if (chunkStart > 0) {
-        Buffer chunk = new Buffer();
-        content.skip(chunkStart);
-        content.read(chunk, length);
-        emitProgress(currentHeaders, chunk.size() - currentHeadersLength, true, listener);
-        emitChunk(chunk, isCloseDelimiter, listener);
-        currentHeaders = null;
-        currentHeadersLength = 0;
-      } else {
-        content.skip(chunkEnd);
-      }
+        if (maybeBoundary.equals(boundaryMarker) || maybeBoundary.equals(closingMarker)) {
+          // boundary found: backtrack to outer loop to emit and re-parse
+          source.buffer().writeUtf8(maybeBoundary).writeUtf8("\n");
+          break;
+        }
 
-      if (isCloseDelimiter) {
-        return true;
+        chunkBuffer.writeUtf8(maybeBoundary).writeUtf8("\n");
       }
+    }
 
-      bytesSeen = chunkStart = delimiter.size();
+    // last fallback
+    if (headers != null && chunkBuffer != null) {
+      listener.onChunkComplete(headers, chunkBuffer, true);
     }
+
+    return true;
   }
 }

Changelog:

Test Plan:

[react-native-bot]

	Fails
🚫	📋 Verify Changelog Format - See Changelog format

	Warnings
⚠️	❗ JavaScript API change detected - This PR commits an update to ReactNativeApi.d.ts, indicating a change to React Native's public JavaScript API. Please include a clear changelog message. This change will be subject to extra review. This change was flagged as: BREAKING

Generated by 🚫 dangerJS against 0de75b4

[cortinico]

cc @cortinico does this need to be an issue or can we discuss on the PR?

I think that, because this is related to devsupport, we'll benefit from a separate GH issue.

[AndreiCalazans]

cc @cortinico does this need to be an issue or can we discuss on the PR?

I think that, because this is related to devsupport, we'll benefit from a separate GH issue.

Got it. I'll follow up with the issue tomorrow.

[AndreiCalazans]

Issue #52818

[react-native-bot]

This PR is stale because it has been open for 180 days with no activity. It will be closed in 7 days unless you comment on it or remove the "Stale" label.

[react-native-bot]

This PR was closed because it has been stalled for 7 days with no activity.

[AndreiCalazans]

superseded by #57043