Memory-leak in fetch API

[sraesch]

Hey,
I'm using the emscripten fetch API and it looks like that the memory for each downloaded file is never released. In order to have a reproducible example, I boiled it down to a minimal example,
where files with the following urls are downloaded:

http://localhost/data/1.bin
http://localhost/data/2.bin
...
http://localhost/data/2047.bin

Even if one waits for half an hour, the memory is not released. The sample example written
in JavaScript with XHR request does not have this issue.
Am I doing something wrong?
Thanks for any help.

// compiled with: emcc -s FETCH=1 main.cpp -o index.html
#include <emscripten/fetch.h>

#include <cstdio>
#include <cstring>
#include <vector>
#include <string>

const std::size_t NUM = 2048;
const std::string baseURL = "http://localhost/data/";

void fetchData(std::string url, int fileIdx);

void downloadSucceeded(emscripten_fetch_t *fetch)
{
    printf("Received '%s'\n", fetch->url);

    int fileIdx = int(fetch->userData) + 1;
    emscripten_fetch_close(fetch); // Free data associated with the fetch.

    if(fileIdx < NUM)
        fetchData(baseURL + std::to_string(fileIdx) + ".bin", fileIdx);
}

void downloadFailed(emscripten_fetch_t *fetch)
{
  printf("[ERROR]: %s FAILED!!!\n", fetch->url);
  emscripten_fetch_close(fetch);
}


void fetchData(std::string url, int fileIdx)
{
    emscripten_fetch_attr_t attr;
    emscripten_fetch_attr_init(&attr);
    std::strcpy(attr.requestMethod, "GET");
    attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
    attr.onsuccess = downloadSucceeded;
    attr.onerror = downloadFailed;
    attr.userData = (void*)fileIdx;
    emscripten_fetch(&attr, url.c_str());
}

int main()
{
    printf("Start...\n");
    fetchData(baseURL + "1.bin", 1);

    return 0;
}

[froydnj]

I think this is due to the various TODOs found here: https://github.com/kripken/emscripten/blob/incoming/system/lib/fetch/emscripten_fetch.cpp#L73-L79

It looks like #5619 addresses those TODOs as a side effect. Could we prioritize getting that merged?

[sraesch]

Hey,

thanks for the answer. I also looked into the code and seems that the actual memory leak issue has
been caused by:

${EM_SDK}/emscripten/incoming/src/Fetch.js:
...
Fetch.xhrs.push(xhr);
var id = Fetch.xhrs.length;
...

If I replace this by some variable which just counts the incoming packages, I fix the memory leak issue, but break some other stuff according
to the emscripten test suite.