Skip to content

Fix shellspec test suite hangs in agents package #605

Description

@williamthorsen

Problem

pnpm -F agents test:sh (and nmr -F agents test:sh) hangs indefinitely without reporting failure, leaving developers unable to verify shell script changes. Two independent root causes — one environmental, one a test bug — combine to deadlock the suite at different points depending on the local environment. A third, related defect causes the script under test (resolve-frontmatter.sh) to silently degrade on stock macOS in a way that could hang network-bound PR lookups in real use.

Context

Discovered during an interactive debugging session on main (commit e0162cdf). Three issues compound:

  • Global git signing leaks into temp-repo tests. resolve_merge_options_test.sh spawns fresh git init temp repos and runs many git commit --allow-empty invocations. With commit.gpgsign = true in the user's global git config, each commit invokes gpg/pinentry; once the gpg-agent cache lapses, pinentry can't reach a TTY (shellspec has captured stdin/stdout) and the commit deadlocks. The spec helper did not isolate git config.
  • strict_majority test inherits parent stdin. resolve_merge_options_test.sh:46 (pre-fix) ran When call strict_majority with no piped input. strict_majority reads stdin via cat; under shellspec's in-process When call, stdin is inherited from the parent shell. In an interactive terminal stdin is a TTY → cat waits indefinitely for keyboard input. In a sandboxed or CI environment stdin is closed → cat returns EOF immediately, masking the bug.
  • run_with_timeout silently no-ops on stock macOS. resolve-frontmatter.sh:358–368 falls back to running the wrapped command with no timeout when neither timeout nor gtimeout is on PATH. macOS does not ship GNU timeout; the safety mechanism is inactive on developer machines that have not installed coreutils. The PR-lookup paths (gh pr list, curl to api.github.com) could hang indefinitely under network stalls in real use.

Solution

Three targeted fixes plus a verification sweep:

  1. Hermetic git config in spec helper. Export GIT_CONFIG_GLOBAL=/dev/null and GIT_CONFIG_SYSTEM=/dev/null inside shellspec_spec_helper_configure in packages/agents/spec/spec_helper.sh. Applies uniformly to every spec; no per-test isolation required.
  2. Sever inherited TTY stdin in the strict_majority empty-input test. Wrap the empty-input case in a local helper that pipes : to strict_majority, matching the helper pattern already used by the rest of the strict_majority Describe block.
  3. Replace run_with_timeout no-op fallback with a portable Perl wrapper. Extend the fallback chain to timeoutgtimeoutperl -e 'alarm shift; exec { $ARGV[0] } @ARGV' → loud failure. Perl is ubiquitous on macOS and mainstream Linux distros, so the loud-failure branch is a sentinel that preserves the function's contract rather than a path users will hit. Update the function's leading comment and content/skills/_data/pr-resolution.md to reflect the new fallback chain.
  4. Scan-and-document pass over remaining When call invocations. Confirm no other function under test reads stdin without an explicit pipe. Fix inline using the same : | fn helper pattern if any are found; otherwise note the scan result in the PR body.

Acceptance criteria

  • pnpm -F agents test:sh runs to completion on a developer machine with commit.gpgsign = true configured globally.
  • pnpm -F agents test:sh runs to completion when stdin is an interactive TTY.
  • run_with_timeout in resolve-frontmatter.sh enforces its documented timeout on stock macOS (no coreutils installed). When no timeout backend is available at all, the function fails loudly rather than silently no-opping.
  • The remaining When call invocations in the shellspec suite have been scanned for stdin-reading targets; any further offenders are fixed inline or the scan's negative result is recorded in the PR body.
  • New/modified behavior in run_with_timeout is covered by tests (success-within-timeout and exceed-timeout cases).
  • content/skills/_data/pr-resolution.md reflects the updated fallback chain.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions