Skip to content

feat(cli): add inspect chunks note-level retrieval view - #1249

Open
phernandez wants to merge 1 commit into
mainfrom
feat/1155-inspect-chunks
Open

feat(cli): add inspect chunks note-level retrieval view#1249
phernandez wants to merge 1 commit into
mainfrom
feat/1155-inspect-chunks

Conversation

@phernandez

Copy link
Copy Markdown
Member

Phase 1+2 PR A of #1155 (retrieval inspector; scope decision on the issue). This PR is the
note-level half — no retrieval-path changes; the execution trace + inspect query follows
as PR B.

What's new

bm inspect chunks <identifier> and POST /v2/projects/{id}/inspect/chunks: view any
note exactly as the index sees it —

  • the note's search rows (entity / observation / relation) with the vector chunks each row
    produced (chunk key, ordinal, text, source hash);
  • per-chunk status from one pure classifier over a closed union:
    ready | pending | stale | orphanedstale is derived by recomputing chunk records
    from the current search rows (pure functions from semantic_chunking, no embedding
    calls) and diffing stored hashes; orphaned means the stored
    (embedding_model, vector_index) no longer matches the configured identity — i.e. the
    rows retrieval cannot see (the Vector search silently degrades when configured index has no ready manifest rows #1236 condition, made visible per note);
  • a readiness summary + entity-fingerprint match, answering "is this a chunking problem or
    a freshness problem?" in one command.

Design per the reviewed plan: module-level service functions (no service class), public
repository read methods (plain portable SQL, both backends), POST-with-body identifier
resolution following the /knowledge/resolve precedent, and the CLI --json output being
the API Pydantic schema serialized verbatim — one machine contract, locked by a
round-trip-and-revalidate test. Auto-JSON when piped; rich/plain interactively.

Verification

  • 17 new focused tests (grouping, project isolation, staleness, orphaned identity,
    timestamp normalization) — plus the chunk-inspection suite re-run on real Postgres.
  • Directory-level sweep: 2016 passed across repository/api/cli/services.
  • just typecheck, just lint clean. Live smoke output in the review thread shape:
    entity + observation + relation rows each showing their chunks with status.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CPdSXDbYyhyZ1TwgFnpEv8

@phernandez phernandez added this to the v0.23 milestone Aug 12, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2761a1fdcc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/repository/search_repository_base.py
Comment thread src/basic_memory/services/retrieval_inspect.py Outdated
Comment thread src/basic_memory/services/retrieval_inspect.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9e7148b70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/services/retrieval_inspect.py Outdated
@phernandez

Copy link
Copy Markdown
Member Author

Amendment (94695b7): the chunks view now conveys indexing status via a derived freshness field — fresh | index_behind_rows | rows_behind_file | unknown — per the accepted proposal on the issue thread. rows_behind_file derives from Entity.checksum + note-content lineage + an on-demand file compare (readable bytes are authoritative; lineage proves divergence only in self-consistent configurations; unknown is honest, never guessed), and it takes precedence over index_behind_rows since it is the upstream divergence. Evidence checksums + file_write_status accompany any non-fresh value, which also self-disambiguates the transient DB-first window where a pending materialization briefly reads as rows_behind_file. 43 focused tests incl. per-trigger, precedence, and unknown cases; full suites + Postgres green; PR B rebased on top (d552e70) with behavior preserved (range-diff verified).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 94695b7167

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/services/retrieval_inspect.py Outdated
@phernandez
phernandez force-pushed the feat/1155-inspect-chunks branch from 94695b7 to da8cd03 Compare August 12, 2026 23:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da8cd03084

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/services/retrieval_inspect.py Outdated
@phernandez
phernandez force-pushed the feat/1155-inspect-chunks branch 2 times, most recently from cff15bf to 6e2a8a2 Compare August 13, 2026 01:55

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e2a8a26b8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/services/retrieval_inspect.py
@phernandez
phernandez force-pushed the feat/1155-inspect-chunks branch from 6e2a8a2 to 61c302c Compare August 13, 2026 02:19

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61c302c7f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/repository/sqlite_search_repository.py
@phernandez
phernandez force-pushed the feat/1155-inspect-chunks branch from 61c302c to f22d09c Compare August 13, 2026 02:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f22d09c02e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/repository/test_chunk_inspection.py Outdated
@phernandez
phernandez force-pushed the feat/1155-inspect-chunks branch from f22d09c to 5d845a0 Compare August 13, 2026 02:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d845a07b8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/services/retrieval_inspect.py Outdated
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez
phernandez force-pushed the feat/1155-inspect-chunks branch from 5d845a0 to 0177f50 Compare August 13, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant