tar -vac . |ipfs tar add - out of memory

[cchris-org]

Version information:

go-ipfs version: 0.4.3-
Repo version: 4
System version: amd64/linux
Golang version: go1.7

Type:

Bug

Priority:

P4

Description:

need tar because there are some issues with file types.
whatever, tar add works fine for small files.
but a directory with large files, eat up all my memory (44GB !!) and daemon exit with out of memory.

what i did:
cd into a directory with files 600MB - ~2GB
tar -vac . |ipfs tar add

a bad idea to read the entire tar input into ram?! :(

chris.

[whyrusleeping]

Huh, thats odd. In the code we for some reason decide to read all of the input from the reader (stdin) then turn it back into a reader and pass it into the construction code... Thats a bug. Nice find.