fix(attribution): deterministic why metadata + interactive viewer + honest tag legend

[suhaanthayyil]

Trail: https://entire.io/gh/entireio/cli/trails/737

What

entire why <file>[:<line>] could show three different answers on three equivalent clones of the same repo at the same commit — full metadata on one, partial on another, bare CheckpointID on a third. This makes the resolution deterministic: given the same remote state, every clone converges on the same output, and when it can't, the miss reason says exactly why and how to recover.

Root cause (what remained after #1535 / 4f435384d)

Earlier work already added MetadataMissingReason, file-level enrichment, and the fetch suggestion. Two residual defects kept the issue alive:

1. The partial-blob divergence (the issue's "some see partial data"): when the checkpoint summary read fine but the per-session records were unreadable (treeless/partial clone, git gc-pruned objects — see KNOWN_LIMITATIONS ENT-241), the resolver set MetadataMissing with no reason and no remote refresh. Worse, on multi-session checkpoints the session loop silently continued over unreadable records, so which fallback session got picked depended on which blobs a clone happened to have — different agent/model/prompt per clone, unflagged.
2. Per-miss fetch churn with swallowed diagnostics: each missing checkpoint ran its own remote-fetch chain (fetchCheckpointContext → fresh lookup → getMetadataTree) and discarded the fetch error. A file with N missing checkpoints paid up to N network timeouts, and the reason couldn't distinguish "fetch failed (offline/auth/no remote)" from "the remote genuinely lacks this checkpoint".

Fix

Restructured the resolver around a memoized once-per-resolver refresh:

• Any incomplete local read — summary unreadable or session records partially/wholly unreadable — triggers one durable fetch of entire/checkpoints/v1; on success the store is re-pointed at a fresh repo handle (post-fetch packfiles are invisible to the already-open handle) and the read retried. Retries are gated on the store having actually changed during the call (no wasted double reads), and a retry that reads worse than the local pass (an unpushed local-only checkpoint) keeps the local data.
• The refresh outcome is memoized: many misses share one fetch (or one failure) — bounding network attempts and making a single run internally consistent.
• The refresh succeeds only when a remote fetch genuinely succeeded. It deliberately does not use getMetadataTree, whose local-branch fallback reports success when every actual fetch failed — that would have made an offline run claim "the remote doesn't have this checkpoint" (found by the audit; regression-pinned by a test that fails on that seam).
• MetadataMissingReason now distinguishes all the recovery situations the issue reporters couldn't tell apart:
  • no fetch attempted (blame path) → the exact git fetch … entire/checkpoints/v1 to run;
  • refresh failed → the underlying error (auth/network/remote), plus the fetch command;
  • refresh succeeded but the checkpoint is absent → "may not have been pushed yet — its author can push entire/checkpoints/v1" (no pointless fetch suggestion);
  • refresh succeeded but only the session records are unavailable → says exactly that, without falsely claiming the just-resolved checkpoint is missing.
• The resolution path emits debug logs (summary hit/miss, sessions read x/y, refresh attempt + outcome) to .entire/logs/ for field diagnosis (issue ask Improved entire enable messaging #6).

entire blame stays local-only/fast by design (unchanged semantics; it now also gets a reason for the sessions-unreadable case). File-level and line-level why share the same resolution path (both enrich).

Audit

Adversarially audited in a fix-until-clean loop (multi-lens review; every finding independently adversarially verified before being acted on; exit = two consecutive clean passes). Eleven fix rounds landed; highlights of what the loop caught in my own cuts, all fixed and regression-pinned:

• Unsound refresh evidence (3 rounds): getMetadataTree's local-branch fallback reported success while offline (an offline run would claim "not pushed"); an origin fallback masked a configured checkpoint remote's failure; a URL-string-equality guard was evadable by token rewrites and false-positived on legitimately-equal URLs → replaced with an explicit FetchURLWithSource source signal (mirrors PushURL).
• Evidence-gated wording: "not pushed" requires the refreshed store's genuine-absence sentinel; non-absence errors (storage/cancellation) make no remote claim; a rejected-worse retry contributes its proof instead of speculating; a settings-load failure is a refresh failure, not "no checkpoint remote".
• TOCTOU: the authority is resolved exactly once and the fetch targets the verified URL — no re-resolution inside the fetch helper.
• git-refs backend (main moved under the branch): the refreshed store keeps RefFetcher capability parity with the initial store, and the strong "the remote's entire/checkpoints/v1 does not contain it" evidence is gated on the git-branch backend (the v1 refresh proves nothing about per-checkpoint refs — refs-backend misses get neutral wording).
• Consistency of guidance: the appended fetch suggestion can never contradict the failure it accompanies (no "git fetch origin" after refusing origin on policy; prose guidance renders as prose, not as a pasteable command).
• Test hermeticity: all reason-path tests pin CWD and the checkpoint-backend env (ENTIRE_CHECKPOINTS_PRIMARY), and the real-seam tests clear ENTIRE_CHECKPOINT_TOKEN so no developer/CI environment leaks in (one test provably failed under a refs-backend env before the pin).

Testing

• New attribution_refresh_test.go: unreadable-session reason on the no-fetch path; refresh restores unreadable sessions; refreshed-but-absent says "not pushed" (and does not suggest a fetch); at-most-once refresh on both the failure and success paths (fetch/reopen counted across misses); local-data-kept when the refreshed read is worse; differently-damaged clones converge on the identical context; partial multi-session reads trigger refresh and resolve the file-matching session; real-seam regression test (local metadata branch + no remote → "remote refresh failed", not "not pushed") — verified to fail on the pre-fix seam.
• Existing attribution suite (incl. JSON stability, session-fallback, prompt provenance) — green unchanged.
• mise run lint — 0 issues; mise run test:ci — unit + integration + Vogon canary green on the committed tree.

Related

• Evaluated Make entire why deterministic across clones and surface actionable checkpoint-metadata miss reasons karthik-rameshkumar/cli#1 (thanks — the fetchOnMiss-parity and reason-propagation direction was right and much of it had landed via 4f435384d); this PR fixes the deeper residual non-determinism it didn't reach: the partial-blob/session-selection divergence, the per-miss fetch churn, and the unsound refresh-success signal.

---

Part 2: interactive viewer + insider-feedback fixes

An interactive viewer for entire why <file> plus the readability fixes insiders asked for — built strictly under the new Agent-Safe CLI Fallbacks rules (#1596).

entire why <file> --tui

Master-detail Bubble Tea viewer over the already-resolved attribution (no git/network I/O inside the TUI):

• Left pane: the file's lines with [AI]/[MX]/[HU]/[??] tags and ~/? markers, windowed scrolling for large files.
• Right pane: the selected line's full explanation — authorship sentence, commit/author/time, agent · model, session (OSC 8 hyperlink to the entire.io session when origin resolves — the CLI↔UI linking idea from the summit), checkpoint, prompt (with the Session prompt caveat), intent, metadata-missing reason, candidate checkpoints, and the checkpoint explain hint (suppressed when metadata is missing, matching the text view).
• Keys: ↑/↓ line, n/p next/previous agent-attributed line (jump straight to the interesting lines), enter expands the full prompt/candidates, pgup/pgdn, g/G, q. why <file>:12 --tui opens positioned at line 12.

Agent-safe by construction

• TUI is opt-in (--tui) and TTY-gated (interactive.IsTerminalWriter) and skipped in accessible mode — the exact experts --tui pattern the guidelines cite as good.
• Non-TTY with --tui falls through to the same deterministic plain output; --json always wins over --tui so scripted callers never block. Both pinned by tests against a non-TTY writer.
• No information is TUI-only: everything the TUI shows exists in the plain/JSON paths.

Insider-feedback fixes (text output)

• Tag legend everywhere (blame, why file view, TUI footer): [AI] all agent · [MX] agent+human mixed · [HU] no agent · [??] uncommitted — wording per soph's correction (AI = 100% agent-authored checkpoint work; MX = agent + human edits; careful not to overclaim). Full sentences in the TUI detail pane.
• Fits the blame table's 80-column budget (test-enforced).

Testing

• Model-level TUI tests (the repo's established pattern — pure Update/View, color off): render + summary, navigation → prompt detail, start-line positioning, n/p agent-line jumps (including exhaustion + status message), expand toggle, missing-metadata reason + suppressed explain hint, quit keys, empty file, tiny window.
• Agent-safe fallback pins: --tui on a non-TTY writer produces the plain file summary; --tui --json produces JSON.
• Found during development: the shared keymap claims n/p for paging — the viewer rebinds paging to pgup/pgdn so n/p can do agent-line jumps (commented).
• mise run lint 0 issues; full cmd/entire/cli suite green; mise run test:ci green.

Notes

entire why stays hidden/labs. The TUI addresses the insiders' "why is hard to read" feedback and pfleidi's browse-the-file/summit-linking ideas without touching the default output contract.

---

Part 3: entire blame — what actually changes (and what doesn't)

To be explicit about the two commands, since they share the resolver:

entire blame behavior is unchanged: local-only and fast, never fetches (now pinned by seam-trap tests that fail if the blame path ever refreshes). It gains only diagnostics + the legend:

• the sessions-unreadable case now carries a MetadataMissingReason (before: flagged missing with no reason and no recovery hint);
• the tag legend (below) is printed under the table.

entire why is where the behavior changes live: remote enrichment on miss (Part 1) and the opt-in TUI (Part 2).

Tag legend — per-commit inference, spelled out

Per the internal wording guidance: the [AI]/[MX]/[HU] tags are a per-commit inference, not a per-line truth. It shows [AI] only when 100% of the commit's checkpointed work was agent-authored; otherwise [MX]; [HU] when the commit had no agent checkpoint. Shown in blame, the why file view, and the TUI footer:

per commit: [AI] all agent · [MX] mixed — line may be either · [HU] no agent
note: [AI] requires a fully agent commit; one human edit turns all lines [MX]

The note line exists because the rule is not what people infer on their own (a mostly-agent commit with one human tweak renders every line [MX], including the purely agent-written ones). [??] appears on a conditional marker line only when uncommitted lines exist.

[Copilot]

Pull request overview

This PR fixes a non-determinism bug in entire why <file>[:<line>] where equivalent clones could resolve different checkpoint context depending on which metadata/session blobs were locally available. It restructures attribution’s checkpoint metadata resolution to perform a memoized, once-per-run remote refresh when local metadata reads are incomplete, and improves miss reasons so users can distinguish “not fetched”, “fetch failed”, and “remote lacks checkpoint”.

Changes:

• Added an explicit FetchURLWithSource API so callers can reliably tell whether the fetch URL came from checkpoint_remote vs an origin fallback.
• Reworked attribution checkpoint context reads to detect incomplete local reads (including partially/fully unreadable session records), perform at-most-once durable refresh + repo reopen, and emit deterministic MetadataMissingReason.
• Updated attach/error guidance to treat fetch suggestions as either a pasteable command or prose (when settings/URL derivation prevents a safe command), and added a comprehensive regression test suite for the refresh behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
cmd/entire/cli/checkpoint/remote/util.go	Adds FetchURLWithSource and routes FetchURL through it to expose a sound “source” signal.
cmd/entire/cli/checkpoint/remote/util_test.go	Adds coverage for the new source-signal behavior, including token rewrite and “checkpoint remote equals origin” cases.
cmd/entire/cli/attribution.go	Implements memoized remote refresh + store reopen, stronger evidence-gated miss reasons, and debug logging for diagnosis.
cmd/entire/cli/attribution_test.go	Extends reader stubs and pins default backend env to keep attribution tests hermetic.
cmd/entire/cli/attribution_refresh_test.go	Adds a comprehensive suite validating determinism, refresh semantics, evidence gating, and error messaging.
cmd/entire/cli/attach.go	Improves missing-checkpoint guidance by distinguishing command vs prose suggestions and introduces branch-vs-refs refspec suggestions.