[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-07-19

[github-actions[bot]]

Executive Summary

gh-aw is genuinely multi-engine and Claude is a first-class citizen — quick-start has a dedicated Anthropic auth callout, all 15 built-in tools are engine-agnostic (0 Copilot-only / 0 Claude-only), and Claude actually leads the inline engine: claude short form (45 vs 35). A Claude Code user can adopt gh-aw. Key finding (stable across 29 daily reviews): three Copilot-favoring documentation gaps persist — no PAT-free/OAuth path for Claude, a silent Copilot default, and thinner non-Copilot secret callouts. None are hard blockers, but all add avoidable friction. Overall score: 7.5/10, 0 critical blockers.

Severity Findings (Critical → Major → Minor)

Critical Blockers: None. A Claude user can complete setup end-to-end with ANTHROPIC_API_KEY.

Major Obstacles

1. No PAT-free/OAuth path for Claude. Copilot offers an org-billing shortcut (copilot-requests: write, no PAT) — how-they-work.mdx:32. Claude has no equivalent: only ANTHROPIC_API_KEY (or WIF keyless) works, and CLAUDE_CODE_OAUTH_TOKEN is unsupported/silently ignored. This directly hits the Claude Pro/Max/Teams subscription persona. Fix: add an explicit "Claude subscription/OAuth tokens are not supported — use an API key" note to the quick-start Claude tab (quick-start.mdx:100).
2. Silent Copilot default. Omitting engine: compiles to Copilot (how-they-work.mdx:26; quick-start.mdx:75), and gh aw init emits Copilot-specific artifacts unless --engine claude is passed (cli.md:148,153). This is disclosed only mid-quick-start, not in prerequisites or the add flow. Fix: surface --engine claude in the prerequisites/add steps.
3. Non-Copilot secret callouts are thinner. Quick-start gives dedicated step-by-step callouts for Copilot + Anthropic + Codex, but Gemini is prerequisite-only with no auth block, and WIF keyless auth is never surfaced in quick-start (quick-start.mdx:100-115). Full per-engine setup is deferred to an external auth page (how-they-work.mdx:37).

Minor Confusion

• Copilot-first nudge "start there — no extra account setup" (quick-start.mdx:30) with no "why Claude vs Copilot" guidance.
• Custom engine has 1 example only (shared/genaiscript.md), no standalone workflow, no dedicated auth section.
• No default Claude model documented in the core doc set.
• web-search is the sole ambiguous tool: engine-dependent, disabled by default on Codex (reference/tools.md:67).

Engine & Tool Matrix

Engine comparison (Setup/Examples/Auth from a Claude-user lens; Score /10):

Engine	Setup	Examples	Auth	Score
Copilot (default)	init default, richest	~127 explicit + 33 implicit (~160)	PAT or org-billing copilot-requests:write	9
Claude	needs --engine claude	63 (leads inline form 45/35)	ANTHROPIC_API_KEY or WIF; no OAuth	7.5
Codex	--engine codex	14	OPENAI_API_KEY/CODEX_API_KEY	7
Custom	overlay config	1 (shared/genaiscript.md)	via engine.env base-url	5

Tool classification — 15 total, all engine-agnostic (built-in 11, MCP 6, config 2; reference/tools.md). No Copilot-only or Claude-only tools — the strongest parity signal for adoption.

Parity observations (from example scan):

• Copilot ~2:1 over Claude on explicit declarations; ~2.6x including silent defaults.
• Every engine (incl. Claude) has a dedicated smoke test — good baseline parity.
• Copilot exclusively demos feature-variant smoke tests (Azure OpenAI api-key/entra, MAI, ARM, SDK, sub-agents); these advanced paths are not demoed for Claude or Codex.
• pi (internal engine, ~20 examples) out-numbers Codex in this repo's own workflows.

Auth Gaps

• Per-engine full setup deferred to an external auth page not in the core doc set (how-they-work.mdx:37).
• README <details> setup lists no explicit secret-setup step; secrets appear only in engine-selector sub-bullets (README.md:41).
• web-search for non-Codex engines: "some engines require third-party MCP servers" — which engines / how is deferred (reference/tools.md:65).
• Gemini: no standalone auth walkthrough beyond the quick-start tab (cli.md:266).
• No single authoritative fine-grained-scope list for COPILOT_GITHUB_TOKEN (split across quick-start + cli.md).

Recommended Actions

Priority 1 — Add a one-line note to the quick-start Claude tab: "Claude subscription OAuth tokens (CLAUDE_CODE_OAUTH_TOKEN) are not supported; use an ANTHROPIC_API_KEY." Closes the single most persona-relevant gap (unresolved 29 runs).

Priority 2 — Surface --engine claude and the silent-Copilot-default behavior in prerequisites/add flow; give Gemini a quick-start auth block at parity with Codex.

Priority 3 — Add a "choosing an engine" blurb (why Claude vs Copilot), a standalone custom-engine example, and a Claude feature-variant smoke test (sub-agents) to match Copilot's coverage.

---

Cache note: 29-run trend is stable at 7.5/0-critical. Appending this run to review-history.jsonl was blocked by the sandbox (cache-memory write not permitted in this run); trend continuity is otherwise intact.

Generated by 📝 Claude Code User Documentation Review · 420.9 AIC · ⌖ 34.1 AIC · ⊞ 5.2K · ◷

• expires on Jul 20, 2026, 4:52 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-07-20T12:52:26.975Z.

Closed by Workflow