Git corrupts pack files after adding large (>4GB) files

[t-b]

git --version: 41980ef

Steps to reproduce:

mkdir test
cd test
truncate -s 5g largefile.bin
rm -rf .git && git init && git add . && git commit -m "changes" && git fsck

Output:

Initialized empty Git repository in E:/ttest/.git/
[master (root-commit) d19adaf] changes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 largefile.bin
Checking object directories: 100% (256/256), done.
shift is 32, bitsizeof(long) 32 // debug output added by me in sha1_file.c:unpack_object_header_buffer
error: bad object header
error: unknown object type -1 at offset 12 in .git/objects/pack/pack-25250ce5c176078ba51a42fee177c2f03f8845ca.pack
error: cannot unpack 0be2be10a4c8764f32c4bf372a98edc731a4b204 from .git/objects/pack/pack-25250ce5c176078ba51a42fee177c2f03f8845ca.pack at offset 12
Checking objects: 100% (1/1), done.

$ ls -lh .git/objects/pack/
total 5.0M
-r--r--r-- 1 thomas None 1.1K Aug  7 16:12 pack-25250ce5c176078ba51a42fee177c2f03f8845ca.idx
-r--r--r-- 1 thomas None 5.0M Aug  7 16:12 pack-25250ce5c176078ba51a42fee177c2f03f8845ca.pack

$ git config --get pack.packSizeLimit
2g

Now for me this is no showstopper at the moment. I just discovered that today while wondering that I can not set pack.packSizeLimit to someting like 10g. The reason for this is that the code uses unsigned long for various file sizes, e.g. in struct object_entry in pack-objects.h. And that is just 32bit on 64bit Windows.

For a short term solution I propose to bail_out_and_die if files with file sizes larger than 32bit can hold are added.

Any ideas on where this should be added in the codebase?

[dscho]

shift is 32, bitsizeof(long) 32 // debug output added by me in sha1_file.c:unpack_object_header_buffer

Where does this come from?

Honestly, I am a bit perplexed that you were not able to debug further, given your history in Git development...

[dscho]

@t-b are you actually planning to do anything about this problem?

[t-b]

@dscho I'm bringing this upstream as it is not restricted to windows.