Skip to content

git_operations: stop shallow-fetching the metadata tip (root-cause fix for shallow-metadata false-disconnect) - #1443

Merged
Soph merged 5 commits into
mainfrom
soph/treeless-fetch-full-depth
Jun 19, 2026
Merged

git_operations: stop shallow-fetching the metadata tip (root-cause fix for shallow-metadata false-disconnect)#1443
Soph merged 5 commits into
mainfrom
soph/treeless-fetch-full-depth

Conversation

@Soph

@Soph Soph commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

https://entire.io/gh/entireio/cli/trails/578

Problem

FetchMetadataTreeOnly resolves the latest checkpoint on resume / explain / attach by fetching the entire/checkpoints/v1 tip with --depth=1. A depth-1 fetch adds the fetched commit to .git/shallow. Once the metadata tip is a shallow boundary, a later git merge-base against refs/remotes/origin/entire/checkpoints/v1 can't reach the real common ancestor (it lives below the boundary), so the disconnection check falsely reports "no common ancestor" — aborting git push and looping entire doctor.

This is the upstream source of the shallow-metadata false-disconnect. It's self-inflicted by the CLI's own tip-read, which runs on essentially every entire resume when checkpoints live on origin (no checkpoint_remote to short-circuit it via getMetadataTree). The user never has to clone shallowly — the CLI shallows the branch for them.

Why not "fetch the tip into a throwaway ref"?

The shallow boundary is intrinsic to --depth=1: it truncates the fetched commit, regardless of which ref it lands on, and git (2.54) opportunistically points refs/remotes/origin/<branch> at that commit anyway (verified against --refmap=, fetch-by-URL, FETCH_HEAD-only, and -c remote.origin.fetch=). You cannot read a tip via --depth=1 without creating a shallow boundary on the shared object store.

Fix

Stop shallow-fetching. FetchMetadataTreeOnly now fetches the metadata commit+tree graph at full depth, relying on --filter=blob:none (when filtered fetches are enabled) to skip blob content instead of on depth-limiting. No shallow boundary is ever created, so refs/remotes/origin/<branch> stays connected and merge-base works.

git fetch is incremental, so after the first fetch only new commits/trees travel — the cheapness now comes from blob filtering + incremental fetch rather than from truncating history.

Test

TestFetchMetadataTreeOnly_DoesNotShallowRepo seeds a 2-commit metadata branch, runs the tip-read on a clone, and asserts the repo is not shallow afterward (--is-shallow-repository=false), full history is present, and the local primary ref is advanced. Under the old --depth=1 behavior the repo would be shallow.

Notes

  • Repos with a configured checkpoint_remote were never affected (that path short-circuits before the shallow fetch and is already full-depth).
  • Complements the detection-side hardening in strategy: don't treat shallow merge-base miss as disconnected metadata #1434 (IsMetadataDisconnected shallow-awareness + doctor deepen); this PR removes the source so the boundary isn't created in the first place.
  • Unit + relevant integration tests (filtered_fetches enabled) pass.

🤖 Generated with Claude Code


Note

Medium Risk
Changes default fetch behavior on a hot path (resume/explain/attach), which can increase first-fetch work on repos without blob filtering, but fixes incorrect disconnect detection tied to git shallow state.

Overview
FetchMetadataTreeOnly (used on resume/explain/attach to read the latest checkpoint) no longer uses --depth=1. It fetches the metadata branch at full commit/tree depth and stays cheap via --filter=blob:none when filtered fetches are on, instead of truncating history.

That removes the root cause where a depth-1 fetch wrote a shallow boundary and later git merge-base / strategy.IsMetadataDisconnected falsely reported no common ancestor—blocking push and looping entire doctor.

The Shallow option is dropped from fetchMetadataOpts. TestFetchMetadataTreeOnly_DoesNotShallowRepo asserts the clone stays non-shallow, keeps two metadata commits, and advances the local primary ref to the origin tip.

Reviewed by Cursor Bugbot for commit ebe73db. Configure here.

FetchMetadataTreeOnly resolves the latest checkpoint on resume / explain /
attach. It fetched with --depth=1, which adds the fetched tip to .git/shallow.
Once the entire/checkpoints/v1 tip is a shallow boundary, a later
`git merge-base` against refs/remotes/origin/entire/checkpoints/v1 can no longer
reach the real common ancestor (it lives below the boundary), so the
disconnection check falsely reports "no common ancestor" — aborting `git push`
and looping `entire doctor`. This is the upstream source of the
shallow-metadata false-disconnect: it is self-inflicted by the CLI's own
tip-read on essentially every resume when checkpoints live on origin (no
checkpoint_remote short-circuit).

The shallow boundary is intrinsic to --depth=1 — the fetched commit is
truncated regardless of which ref it lands on, and git opportunistically points
refs/remotes/origin/<branch> at it anyway. So the fix is to not shallow at all:
fetch the metadata commit+tree graph at full depth and rely on
--filter=blob:none (when filtered fetches are enabled) to skip blob content.
git fetches incrementally, so after the first fetch only new commits/trees
travel. The remote-tracking ref stays connected and merge-base works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b85c03197b07
@Soph
Soph requested a review from a team as a code owner June 17, 2026 12:14
Copilot AI review requested due to automatic review settings June 17, 2026 12:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a root cause of the “shallow-metadata false-disconnect” issue by changing FetchMetadataTreeOnly to stop using git fetch --depth=1, preventing the CLI from creating shallow boundaries in the user’s repo that later break git merge-base connectivity checks for the metadata branch.

Changes:

  • Remove shallow (--depth=1) fetching from FetchMetadataTreeOnly, relying on incremental full-depth fetches and blob filtering when enabled.
  • Add a regression test asserting FetchMetadataTreeOnly does not leave the repository in a shallow state and preserves full metadata history.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
cmd/entire/cli/git_operations.go Switch FetchMetadataTreeOnly to full-depth fetch (no --depth=1) to avoid creating shallow boundaries.
cmd/entire/cli/treeless_fetch_full_depth_test.go Add regression test to ensure the metadata tip-read does not shallow the repo and retains full metadata history.

Comment thread cmd/entire/cli/treeless_fetch_full_depth_test.go Outdated
Comment thread cmd/entire/cli/treeless_fetch_full_depth_test.go Outdated
Soph and others added 2 commits June 17, 2026 14:41
Cleanups from /simplify review (no behavior change):
- Collapse fetchMetadataOpts to a single noFilter bool. After dropping the
  Shallow field, the struct held only NoFilter plus a dead Unshallow field
  (never set true by any caller); the two named wrappers FetchMetadataBranch /
  FetchMetadataTreeOnly document intent at the only two call sites.
- Reuse the existing gitOutput test helper instead of the duplicate gitOut, and
  drop the thin gitRevParse wrapper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 9bf304d541d6
PR review (Copilot): the plain `git clone <bareDir>` fetched all branches via
the local hardlink optimization, so refs/remotes/origin/entire/checkpoints/v1
and its history were already present before FetchMetadataTreeOnly ran —
weakening the assertions. Clone single-branch main via file:// so the metadata
branch is absent until the tip-read fetches it; the old --depth=1 behavior now
visibly shallows the repo and trips the assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 7deb7f80719f
Dropping --depth=1 stops creating new shallow boundaries, but a repo that an
older CLI already shallowed keeps the metadata tip grafted in .git/shallow
forever — so its merge-base disconnection checks stay broken until something
unshallows it.

Set Unshallow:true on the metadata fetch. remote.Fetch only adds --unshallow
when the repo is actually shallow, so this is a no-op on a normally-cloned repo
and only does work where a prior shallow boundary needs removing. --unshallow
fetches the missing ancestry, so the repo becomes genuinely complete (not a
"claims complete but missing objects" state) and composes with --filter=blob:none.

Add TestFetchMetadataTreeOnly_HealsPriorShallow (and extract the shared
seed+clone setup into a helper). Update the now-stale FetchMetadataBranch
comment that said it never unshallows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 1b1d826bd5d5
Follow-up to the --unshallow heal: --unshallow is repo-global, so on a repo
whose source tree was independently shallow-cloned it would also deepen that
unrelated history. Switch to a ref-scoped deep fetch instead.

Add FetchOptions.Depth (--depth=<N>) and fetch the metadata branch with a large
Depth (1e9). git semantics: a --depth that exceeds the branch's length fully
fetches it (healing a prior --depth=1 boundary) and is a no-op on a full repo
(no shallow introduced), while leaving an independently-shallow branch's
.git/shallow entry intact. The value stays below math.MaxInt32 (2147483647),
which git special-cases as a global unshallow.

Tests: TestFetch_Depth (remote) proves Depth heals the named branch while a
second shallow branch stays shallow; the cli heal/no-shallow tests still pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 4277ced83b37
@Soph
Soph merged commit 16af73d into main Jun 19, 2026
9 checks passed
@Soph
Soph deleted the soph/treeless-fetch-full-depth branch June 19, 2026 14:52
@Soph
Soph restored the soph/treeless-fetch-full-depth branch June 22, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants