Improve progress feedback for 'wl github import'

[SorraTheOrc]

Problem statement

When running wl github import users only see coarse progress lines and occasional debug messages; during long imports this provides little visibility into which sub-phase is running, per-phase progress, throttler status, or ETA, making it hard to estimate remaining time or debug stuck imports.

Users

• Operator running wl github import locally (interactive TTY)
  • As an operator I want periodic human-readable progress updates so I can estimate remaining time and know which import phase is currently running.
• Automation / CI user running wl github import --json or --progress=json
  • As an automation user I want structured progress events that can be parsed by tools without breaking existing JSON output.
• Support engineer debugging stuck imports
  • As a support engineer I want to see recent API retry events and throttler queue metrics to help identify rate-limiting or throttling issues.

Success criteria

• AC1: Interactive CLI prints periodic progress updates for each import phase (phase name, current/total, rate, ETA where measurable) at a readable cadence (default ~1/sec or per N items) while not flooding output.
• AC2: --progress=json emits structured progress events (phase, processed, total, rate, etaMs, lastError) at a modest rate (max 1 event/sec) to a documented stream (TBD: stderr or FIFO). This mode must be parseable without breaking other JSON output.
• AC3: Non-interactive scripts retain script-friendly behavior; --json for the main command must not be broken by periodic progress lines (provide --progress=quiet to suppress human lines).
• AC4: Unit tests cover progress formatting helpers and an integration-like test simulates a multi-phase import using mocked GH API calls asserting progress events were emitted.
• AC5: All related documentation (README, CLI docs, usage examples, comments) updated. Full project test suite must pass with the new changes.

Constraints

• Must not break existing --json output or other machine-readable outputs (backwards compatibility is required).
• Avoid adding contention or synchronous hooks that degrade import throughput or change timing significantly.
• Progress reporting should be rate-limited and optionally verbose; default behavior should be conservative to avoid flooding logs/terminals.
• Throttler access must be non-blocking; exposing metrics should not change throttler behavior.

Existing state

• The import flow is already split into several coarse phases and the codebase exposes hooks for an onProgress callback (see src/github-sync.ts). Recent commits implemented some per-phase progress for comments and saving but output remains coarse.
• Files of immediate interest: src/github-sync.ts, src/commands/github.ts (rendering), src/github-throttler.ts, src/github.ts. Tests touching import and progress exist (tests/github-sync-*.test.ts and other github-related tests).

Desired change

• Implement a small progress helper module (src/progress.ts) that supports named phases, tick/count updates, rate/ETA calculations, note setting, and emits both human-readable and JSON progress events with rate-limiting.
• Integrate the helper into src/github-sync.ts at key locations (fetch-issues, parse-markers, hierarchy, import, comments, finalize) and into src/commands/github.ts to render human-friendly lines and provide --progress=json machine output.
• Add unit tests for formatting/rate-limiter and an integration-style test with mocked GH API and throttler to assert events and throttler metrics are surfaced.

Related work

Potentially related docs (file paths):

• src/github-sync.ts — current implementation with onProgress hooks
• src/commands/github.ts — CLI rendering and existing renderProgress implementation
• src/github-throttler.ts — throttler implementation and metrics accessors
• src/github.ts — GitHub-related helpers and label parsing
• src/jsonl.ts — example of progress callback usage for export
• tests/github-sync-*.test.ts and tests/github-label-resolution.test.ts — existing tests that touch import flow and may be informative

Potentially related work items (single-line summaries):

• wl-0MM3WJQL90GKUQ62: Comments not correctly imported from GitHub / pushed to GitHub (WL-0MM3WJQL90GKUQ62)
• WL-0MM79H1JR0ZFY0W2: wl gh import: no progress feedback during comment fetch phase
• WL-0MODGQT9C006RYTH: Migrate import callsites to scheduled GH API wrapper
• WL-0MM369NX61U76OVY: Structured audit logging for import
• WL-0MM38CRQN18HDDKF: Improve duplicate error message
• WL-0MM77L16U0VXR5W3: wl gh import: status/stage changes on GitHub not propagated to worklog

Appendix: Clarifying questions & answers

• Q: "Which stream should machine-readable JSON progress events be emitted to by default: stderr, stdout, or a FIFO/file?" — Answer: OPEN QUESTION (asked to user). Rationale: preserving existing --json semantics is critical; using stderr is common but confirm preference. Final: OPEN.
• Q: "What default rate limit for structured progress events is acceptable (suggested default: 1 event/sec) ?" — Answer: OPEN QUESTION (asked to user). Suggested default: 1 event/sec. Final: OPEN.

Notes on idempotence

• This draft references existing work item WL-0MODFKH0C0006GB6 and will be used to update that work item. If this intake is re-run the same file and work item idempotently replace the description and Appendix rather than append duplicate entries.

--
Draft prepared for: WL-0MODFKH0C0006GB6