fix: handle heredoc/multiline commands in terminal tool execution

[maruthang]

Fixes #288896
Fixes #312260

When the Copilot agent sends heredoc or multiline commands to the terminal, normalizeCommandForExecution() collapses all newlines to spaces, destroying the heredoc structure. The shell then receives a single line instead of the multi-line heredoc, causing file write failures.

Changes

1. sendToTerminalTool.ts: Detect multiline commands (containing \n or \r) and send them with bracketed paste mode enabled, bypassing normalizeCommandForExecution. This preserves newlines so the shell treats the input as a single paste.

2. Execute strategies (basicExecuteStrategy.ts, noneExecuteStrategy.ts, richExecuteStrategy.ts): Force bracketed paste mode for multiline commands on all platforms (previously only forced on macOS).

3. Shared isMultilineCommand helper (runInTerminalHelpers.ts): Extracted into a reusable function used by both sendToTerminalTool and all three execute strategies. Uses a negative lookbehind ((?<!\\)\n) so that backslash-newline line continuations are not treated as multiline — the shell naturally joins these into a single logical line, so they can be safely normalized instead of requiring bracketed paste mode.

4. normalizeCommandForExecution applied to send_to_terminal persistent path: The same normalization (collapsing accidental newlines, trimming whitespace) is applied to the persistent/background terminal path, not just foreground terminals.

How to test

1. Open Copilot Chat and ask it to create a file using a heredoc, e.g.: "Write a hello world script to /tmp/test.sh using heredoc"
2. Verify the file is created correctly with proper content
3. Verify single-line commands still work normally
4. Try a command with a line continuation (echo hello \ + newline + world) and verify it's normalized to a single line rather than using bracketed paste mode

[maruthang]

Rebased onto latest main to clear the conflict. Resolved three overlap points with upstream:

• richExecuteStrategy.ts — combined with the new commandLineForMetadata parameter so runCommand() is called with both the multiline-aware forceBracketedPasteMode and the new metadata arg.
• runInTerminalHelpers.test.ts — updated the import to match the current helper surface (dropped sanitizeTerminalOutput, kept normalizeCommandForExecution which the new tests still use).
• sendToTerminalTool.test.ts — kept both sets of new tests (upstream's carousel confirmation coverage + this PR's heredoc/multiline coverage).

Branch should show green conflict status now. Ready for review whenever convenient.

[meganrogge]

Direction looks good — preserving newlines for heredocs and forcing BPM for multiline is the right fix. A few concerns before merge:

Must-fix

• The foreground terminal path in sendToTerminalTool.ts (~line 356) still collapses newlines; the multiline branch needs to be applied there too.
• Multiline detection differs between files (\r|\n in sendToTerminalTool.ts vs \n-only in the execute strategies). Please share one helper.

Should-fix

• Forcing bracketed paste mode on all platforms may break shells that don't support it (cmd.exe, minimal POSIX shells). Please gate on shell capability or confirm downstream handles it.
• The three execute-strategy edits are near-duplicates — extract a shared shouldForceBracketedPaste(commandLine) helper.

Nit

• The added tests cover normalizeCommandForExecution (unchanged in this PR); a test for the foreground-terminal multiline path would be more valuable.
• Briefly comment that bypassing normalizeCommandForExecution also intentionally skips .trim() so future readers don't re-add it.

[meganrogge]

The foreground terminal path in this same file (around line 356) still calls sendText(normalizeCommandForExecution(args.command), true), so heredocs/multiline commands sent to a foreground terminal will still have their newlines collapsed. Please apply the same multiline-detection + bracketed-paste-mode branch there, ideally via a small shared helper so the logic doesn't diverge.

[meganrogge]

Detection here is \n or \r, but the execute strategies only check \n. A command containing only \r would take different branches in the two layers. Please extract a shared helper (e.g. isMultilineCommand(command) using /\r|\n/.test(command)) and use it everywhere so the definition stays consistent.

Also note this path intentionally skips normalizeCommandForExecution's .trim(), which is correct for heredocs but worth a brief comment so future readers don't "fix" it.

[meganrogge]

Two concerns here (same applies to noneExecuteStrategy.ts and richExecuteStrategy.ts):

1. Previously BPM was forced only on macOS. Forcing it for any multiline command on all platforms means shells that don't support bracketed paste (e.g. cmd.exe, minimal POSIX shells) may receive literal ESC[200~…ESC[201~ markers. Can we gate this on a "shell supports BPM" check, or confirm downstream (sendText/runCommand) already handles the capability?
2. This same 2-line change is duplicated across three execute strategies. Please extract a shouldForceBracketedPaste(commandLine) helper so the rule (and any future capability check) lives in one place.

[meganrogge]

These tests exercise normalizeCommandForExecution, which wasn't changed in this PR, so they add little coverage for the fix. More valuable would be a test for the foreground-terminal path in sendToTerminalTool.ts asserting that a multiline command preserves newlines and uses bracketed paste mode (mirroring the persistent-path tests below).

[maruthang]

Thanks for the thorough review @meganrogge — addressed the must-fixes in 435d3d6. Summary:

Must-fix

• Foreground terminal path (sendToTerminalTool.ts:356) — now branches on isMultilineCommand(args.command). Multiline inputs go through instance.sendText(args.command, true, /*forceBracketedPaste*/true) verbatim; single-line inputs keep the existing normalizeCommandForExecution path. Parity with the background/persistent path.
• Unified multiline detection — extracted isMultilineCommand(command: string): boolean into runInTerminalHelpers.ts and replaced all four ad-hoc call sites (basicExecuteStrategy.ts, noneExecuteStrategy.ts, richExecuteStrategy.ts, sendToTerminalTool.ts). The helper accepts both \n and \r (previously inconsistent — the execute strategies only checked \n).

Should-fix (partial)

• Extracted the shared helper so the three execute-strategy edits are now one-liners using it — no more near-duplicates.
• On shell-capability gating for bracketed paste mode: I held off on adding a shell-capability check in this PR because the platform-specific fallback already lives in the terminal layer (ITerminalInstance.sendText), and adding a separate capability probe here would duplicate that logic and expand the PR's scope. Happy to split that into a follow-up if you'd prefer — let me know.

Nit

• Added tests for the foreground-terminal multiline path (new test('foreground terminal path preserves newlines for heredoc commands') and test('foreground terminal path normalizes single-line commands')).
• Added an inline comment on both foreground and background multiline branches explaining that normalizeCommandForExecution is intentionally skipped (which also skips its .trim() side-effect), so future readers don't try to "clean up" the input.

Local checks: tsc -p src/tsconfig.json --noEmit is clean on the touched files. Let me know if you want the shell-capability gate rolled into this PR or left as a follow-up.

[meganrogge]

@anthonykim1 can you please test on windows and see before/after this change how it behaves?

[anthonykim1]

@maruthang Having trouble to get "hang" to repro. Tried to force agent to use send_to_terminal and heredoc but it seems smart enough pre-pr to send EOF on its own.

Do you have steps to test the behavior pre-pr?

[maruthang]

Thanks for looking at this @anthonykim1! The reason you're seeing the agent sidestep it is exactly what @brendaningram's issue describes — the agent learned to avoid heredocs because they silently fail, and just uses a different strategy. The "hang" only surfaces when the agent is coaxed into using heredoc through send_to_terminal (or, more reliably, into run_in_terminal, which goes through the execute strategies).

Here's a repro that forces it deterministically on Windows/Linux/macOS:

Setup

1. Check out main at a commit prior to this PR (e.g. git checkout 11d49abcbf1^) and run VS Code from source, or install the current stable build.
2. Open the chat panel in Agent mode with run_in_terminal (or send_to_terminal) enabled.

Prompt that forces the bad path
Use this exact instruction so the agent can't work around it:

Create /tmp/heredoc-test.txt (or C:\Temp\heredoc-test.txt on Windows) with the exact
contents "line one\nline two\nline three". Use run_in_terminal with a single
heredoc command of the form:

  cat > <path> << 'EOF'
  line one
  line two
  line three
  EOF

Do not substitute echo, printf, python, Set-Content, Out-File, or editor tools.
Call the tool exactly once with the full multi-line command above.

Expected vs observed (pre-PR)

• Pre-PR: normalizeCommandForExecution collapses the newlines before sendText, so the shell receives cat > <path> << 'EOF' line one line two line three EOF as a single line. On bash/zsh the heredoc is never closed (EOF is now on the content line), the terminal sits waiting for a EOF<newline> that never comes, and the command never completes. On PowerShell you get a parse error immediately.
• Post-PR: the multi-line branch sends it verbatim with forceBracketedPasteMode=true, and the heredoc completes normally.

Windows-specific notes

• On PowerShell, the pre-PR path doesn't hang the same way — PowerShell errors out on the collapsed single-line form. That's still a regression vs. the multi-line intent, but it fails loudly instead of hanging.
• On WSL / Git Bash, behavior matches Linux — silent hang waiting for EOF.
• Windows also exercises \r\n in addition to \n — that's why isMultilineCommand now checks both (previously one path checked only \n, the other both).

Quicker manual repro (bypasses the agent)

If you'd rather not coax the model, you can unit-test the exact pre-PR behavior by calling normalizeCommandForExecution directly on the heredoc string — you'll see the newlines collapse. The new isMultilineCommand check in this PR is what prevents that for multi-line inputs.

Let me know if you still can't repro and I'll send a short screencap.

[meganrogge]

Thank you! Added a bit to this to fix benchmark issues