[SOLVED in v2.47.0(2)] v2.47.0 Git hangs on fetch

[orgads]

UPDATE: THIS ISSUE WAS FIXED

please upgrade to v2.47.0(2) or later, it fixes this issue.

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.47.0.windows.1
cpu: x86_64
built from commit: 8e380191abeda7a995e87abf4acc9e1702fdc698
sizeof-long: 4
sizeof-size_t: 8
shell-path: F:/git-sdk-64/usr/bin/sh
feature: fsmonitor--daemon
libcurl: 8.10.1
OpenSSL: OpenSSL 3.2.3 3 Sep 2024
zlib: 1.3.1

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.19045.4355]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
> type "$env:USERPROFILE\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Rebase
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable FSMonitor: Disabled

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Git Bash

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git fetch

• What did you expect to occur after running these commands?

It should be done successfully after a while.

• What actually happened instead?

It sometimes hangs. I see the following spawned process, which doesn't use any CPU, and doesn't terminate:

git index-pack --stdin -v --fix-thin "--keep=fetch-pack 4272 on my-pc" --pack_header=2,291

The output is:

remote: Counting objects: 195, done
remote: Finding sources: 100% (292/292)
<nothing happens>
<when I press Ctrl-C>
fetch-pack: unexpected disconnect while reading sideband packet

[dscho]

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git fetch

To make this example a little more complete, could you describe from where you are fetching? Is it github.com, a self-hosted git daemon, from a network share? Is it a big fetch, i.e. have already dozens of megabytes been transferred, or is it a small fetch? Did you try to disable sideband?

[orgads]

It's a hosted gerrit 3.8 server. The fetch is small, took a few seconds after I downgraded to 2.46.

I can try to disable sideband tomorrow.

[jfcherng]

Latest update: Now it clones okay... without any local env modification... Okay again it just happened.

---

I tried to clone my https://github.com/jfcherng/copilot-node-server .

It seems to alway hang if I clone git@github.com:jfcherng/copilot-node-server.git. But it's okay if I clone https://github.com/jfcherng/copilot-node-server.git.

---

Update: It still hangs with GIT_CURL_VERBOSE=1 GIT_TRACE=1 git -c sendpack.sideband=false clone git@github.com:jfcherng/copilot-node-server.git

[jfcherng@HOME Desktop]$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git -c sendpack.sideband=false clone git@github.com:jfcherng/copilot-node-server.git
10:14:30.318887 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
10:14:30.329889 git.c:479               trace: built-in: git clone git@github.com:jfcherng/copilot-node-server.git
Cloning into 'copilot-node-server'...
10:14:30.350892 run-command.c:667       trace: run_command: unset GIT_CONFIG_PARAMETERS GIT_DIR; GIT_PROTOCOL=version=2 ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''jfcherng/copilot-node-server.git'\'''
10:14:30.350892 run-command.c:928       trace: start_command: ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''jfcherng/copilot-node-server.git'\'''
remote: Enumerating objects: 542, done.
remote: Counting objects: 100% (100/100), done.
10:14:33.230184 run-command.c:667       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 32844 on 3bc-f19' --check-self-contained-and-connected
10:14:33.230184 run-command.c:928       trace: start_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 32844 on 3bc-f19' --check-self-contained-and-connected
10:14:33.298183 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
10:14:33.311182 git.c:479               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 32844 on 3bc-f19' --check-self-contained-and-connected
remote: Compressing objects: 100% (65/65), done.
Receiving objects:   1% (6/542)

It just hangs on Receiving objects: 1% (6/542). It is always 1% (6/542) everytime I retried. Eventually it failed as the following.

[jfcherng@HOME Desktop]$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git -c sendpack.sideband=false clone git@github.com:jfcherng/copilot-node-server.git
10:16:32.600368 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
10:16:32.614373 git.c:479               trace: built-in: git clone git@github.com:jfcherng/copilot-node-server.git
Cloning into 'copilot-node-server'...
10:16:32.653368 run-command.c:667       trace: run_command: unset GIT_CONFIG_PARAMETERS GIT_DIR; GIT_PROTOCOL=version=2 ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''jfcherng/copilot-node-server.git'\'''
10:16:32.653368 run-command.c:928       trace: start_command: ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''jfcherng/copilot-node-server.git'\'''
remote: Enumerating objects: 542, done.
remote: Counting objects: 100% (100/100), done.
10:16:35.393876 run-command.c:667       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 20220 on 3bc-f19' --check-self-contained-and-connected
10:16:35.393876 run-command.c:928       trace: start_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 20220 on 3bc-f19' --check-self-contained-and-connected
10:16:35.455879 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
10:16:35.469968 git.c:479               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 20220 on 3bc-f19' --check-self-contained-and-connected
remote: Compressing objects: 100% (65/65), done.
fetch-pack: unexpected disconnect while reading sideband packet
fatal: fetch-pack: invalid index-pack output

[orgads]

Ah right, my case was ssh too.

[EemilAhonen]

We are having this same problem with SSH on multiple machines that upgraded to the latest 2.47.0. All clones over 1mb from Gitlab are failing, the Git window just freezes at random points.
Uninstalling 2.47.0 and reinstalling 2.44.0 solved the issue.

[dscho]

Does it work if you copy the usr\bin\ssh.exe over from v2.46.2?

[jfcherng]

Does it work if you copy the usr\bin\ssh.exe over from v2.46.2?

In my case, no.

[dscho]

Does it work if you copy the usr\bin\ssh.exe over from v2.46.2?

In my case, no.

How about copying over usr\bin\msys-2.0.dll?

[jfcherng]

How about copying over usr\bin\msys-2.0.dll?

Yes, that's the culprit. I tried in a fresh portable build. No need to replace ssh.exe.

[dscho]

I feared as much. There is a rather big jump in the MSYS2 runtime upgrade between Git for Windows v2.46.2 and v2.47.0. And MSYS2 runtime issues are notoriously hard to fix.

You wouldn't happen to have a good reproducer that other people might be able to use, would you? I am afraid that git fetch as an MCVE does not pass muster.

[elindoorn]

A git pull from a azure repo consistently hung for me today until i reverted to 2.46.2

git pull
remote: Azure Repos
remote: Found 308 objects to send. (757 ms)

the trace suggests it can collect branches/tags and such before it hangs

`> git pull
13:28:53.108594 exec-cmd.c:266 trace: resolved executable dir: C:/s/git/mingw64/bin
13:28:53.118594 git.c:479 trace: built-in: git pull
13:28:53.157446 run-command.c:667 trace: run_command: git merge-base --fork-point refs/remotes/origin/redacted redacted
13:28:53.157446 run-command.c:928 trace: start_command: git merge-base --fork-point refs/remotes/origin/redacted redacted
13:28:53.197744 run-command.c:667 trace: run_command: git fetch --update-head-ok
13:28:53.197744 run-command.c:928 trace: start_command: git fetch --update-head-ok
13:28:53.209496 exec-cmd.c:266 trace: resolved executable dir: C:/s/git/mingw64/libexec/git-core
13:28:53.219721 git.c:479 trace: built-in: git fetch --update-head-ok
13:28:53.223755 run-command.c:667 trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 ssh -o SendEnv=GIT_PROTOCOL git@ssh.dev.azure.com 'git-upload-pack '''v3/redacted''''
13:28:53.224873 run-command.c:928 trace: start_command: ssh -o SendEnv=GIT_PROTOCOL git@ssh.dev.azure.com 'git-upload-pack '''v3/redacted''''
13:28:53.987805 pkt-line.c:86 packet: fetch< 191d8833b443043e99cd2d3184f4b66c154d3847 HEAD\0 multi_ack thin-pack side-band side-band-64k no-progress multi_ack_detailed no-done shallow allow-tip-sha1-in-want filter symref=HEAD:refs/heads/develop
...
13:28:54.115251 pkt-line.c:86 packet: fetch< NAK
13:28:54.136540 pkt-line.c:86 packet: fetch< ACK eb10cf30356d9de23600c64368d94d664db97888
13:28:54.138227 pkt-line.c:86 packet: sideband< \2Azure Repos
remote: Azure Repos
13:28:54.139224 pkt-line.c:86 packet: sideband< \2\15Found 303 objects to send. (12 ms)
remote: Found 303 objects to send. (12 ms)

nothing after this`

[ryanmkurtz]

I'm having the same issue too with the move from 2.46.0 to 2.47.0. Both clone and fetch now hang:

$ git fetch
remote: Enumerating objects: 8510, done.
remote: Counting objects: 100% (5831/5831), done.
remote: Compressing objects: 100% (782/782), done.
<hang>

I am working with my organization's GitLab repo over ssh.

[derrickstolee]

I was able to reproduce the failing clones and have a workaround:

git -c core.sshCommand="'C:\Windows\System32\OpenSSH\ssh.exe'" clone ...

or

git config --global core.sshCommand "'C:\Windows\System32\OpenSSH\ssh.exe'"
git clone ...

This configures Git to use the SSH agent built into Windows instead of the OpenSSL-based one provided by MSYS2.

[b12k]

Same here.
Hangs on clone with 2.47.0

@derrickstolee Solution helped 👆