Skip to content

feat(recall): progressive retrieval with G-document routing - #102

Merged
jeff-r2026 merged 3 commits into
Tencent:mainfrom
m0Nst3r873:feat/progressive-retrieval
Jul 2, 2026
Merged

feat(recall): progressive retrieval with G-document routing#102
jeff-r2026 merged 3 commits into
Tencent:mainfrom
m0Nst3r873:feat/progressive-retrieval

Conversation

@m0Nst3r873

Copy link
Copy Markdown
Collaborator

Summary

  • Implement three-layer progressive retrieval (route → context → lookup) replacing flat BM25 search
  • Add G5 scenario sequence diagrams and G6 multi-hop transitive dependency analysis
  • Upgrade router.md/index.md with search-anchor and question routing table
  • Optimize graph-boost with pre-computed adjacency map (O(E²) → O(degree²))
  • Fix multiple knowledge repo generation bugs (overview duplication, g2 filename, race condition, idempotency)

Progressive retrieval layers

depth Behavior Use case
route Returns routing table only, no BM25 Discover available projects/G-docs
context (default) Searches overview + modules + docs Most queries
lookup Full search including raw symbol pages Precise file:line lookups

G-document routing

Question type Document Example
Dependencies G1 relations matrix "What depends on X?"
Call chains G2 dataflow "Request path from entry to DB?"
Business flows G5 scenarios "Full login flow?"
Blast radius G6 multi-hop "If X fails, what breaks?"

Test plan

  • 130 test files, 1675 tests passing
  • TypeScript type check clean
  • New recall-progressive.test.ts verifying route/context/lookup behavior
  • CSIG + code quality + PR review findings all addressed

Depends on: #98 (merged)

🤖 Generated with Claude Code

jaelgeng added 2 commits July 2, 2026 10:46
- Remove docs/team-codebase/repos/ duplication; teamwiki evidence as
  sole knowledge carrier with AI narrative appended idempotently
- Fix deep-enrich reading call-chains.md → dependency-paths.md (g2 bug)
- Fix rebuild-wiki-index.ts still referencing old filename
- Guard deepEnrich with skipEnrich flag to prevent unwanted AI calls
- Fix router.md append link: [[code/...]] → [[evidence/code/...]]
- Eliminate graph-index write race: extractCodebase writes per-repo only
- Share Intl.Segmenter as module-level singleton (perf)
- Fix --output CLI description (team-codebase → teamwiki)
Three-layer progressive retrieval system replacing flat BM25 search:
- depth=route: returns routing suggestions only (no BM25)
- depth=context (default): searches overview + modules + docs
- depth=lookup: full search including raw symbol pages

Production-side enhancements:
- Rewrite routerTemplate with search-anchor + question routing table
- Rewrite indexTemplate with module quick-reference
- Upgrade G1 to N×N matrix with forward/reverse dependency indexes
- Upgrade G2 with entry-grouped call chains and search-anchor
- Add G5 (AI scenario sequence diagrams) and G6 (BFS multi-hop analysis)
- Add graph/README.md dispatch routing table

Consumption-side optimizations:
- Pre-computed adjacency map for graph-boost (O(E²) → O(degree²))
- Exclude navigation files from BM25 corpus in context mode
- Add MAX_RECURSION_DEPTH=10 to loadPagesRecursive
- Warn on invalid --depth values with fallback to context
- Update recall subagent to leverage G-documents and depth levels

@jeff-r2026 jeff-r2026 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — medium effort

概览

PR 实现了三层渐进检索(route/context/lookup)、G5 场景序列图和 G6 多跳依赖分析、router.md/index.md 增强、graph-boost 性能优化。设计合理,变更范围清晰。

发现

🔴 src/deep-enrich.ts:570 — G6 BFS 方向反转

buildG6Content 的 "爆炸半径" BFS 遍历了正向边(X 依赖什么),而非反向边(谁依赖 X)。

adj 仅存储 from → Set<to>,BFS 从某节点出发跟随正向边,计算的是 "X 传递依赖了什么",而输出标签声称是 "X 变更可能影响 N 个组件"。

复现:[{from:'auth', to:'db'}, {from:'api', to:'auth'}]auth 的爆炸半径应是 api(api 依赖 auth),但 BFS 跟随 adj.get('auth')={'db'},错误报告 db 为受影响组件。

建议修复: 构建反向邻接表 revAdj: to → Set<from> 用于 BFS 遍历。


🟡 src/ci/extract-mr.ts:373 — 全局 graph-index.json 被单仓库覆盖

extractCodebase 写 per-repo graph 到 srcWiki/.indices/graph-index.json,然后 fse.copy(srcWiki, teamWikiRoot, { overwrite: true }) 将其覆盖到 teamWikiRoot/.indices/graph-index.json(原本是聚合后的全局 graph),导致多仓库 graph 数据丢失。

建议修复: copy 后调用 aggregateGlobalGraph(),或 copy 时排除 .indices/graph-index.json


🟡 src/code-knowledge-recall.ts — NAVIGATION_FILES 可能误杀嵌套 index.md

NAVIGATION_FILES.has(entry.name)CONTEXT_ALLOWED_PATTERNS 白名单检查之前执行。任何 docs/index.md(如 evidence/code/project/docs/index.md)会被 index.md 匹配跳过,即使它满足 /docs\/.+\.md$/ 模式。

建议修复: 将 NAVIGATION_FILES 检查限定为相对路径的顶层文件(如检查 relativePath 而非仅 entry.name),或将其移到 pattern 匹配之后作为 fallback。


🤖 Generated with Claude Code

- P1-1: overview.md handles missing frontmatter on empty base
- P1-2: fs.copy srcGraph guarded with pathExists check
- P1-3: depth=route only loads router.md, uses actual page title
- P2-4: graphReadmeTemplate dynamically renders based on G5/G6 flags
- P2-5: CONTEXT_ALLOWED_PATTERNS comment clarifies G1-G6 coverage
@m0Nst3r873
m0Nst3r873 force-pushed the feat/progressive-retrieval branch from 8da0259 to c2b30bc Compare July 2, 2026 04:07

@jeff-r2026 jeff-r2026 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three findings from the previous review have been properly addressed:

  1. G6 BFS now uses revAdj (reverse adjacency map) — blast radius correctly computes "who depends on X"
  2. ci/extract-mr.ts no longer blanket-copies srcWiki; copies per-repo evidence only and calls aggregateGlobalGraph()
  3. NAVIGATION_FILES exclusion is now depth-gated (depthFromProject <= 1), protecting nested docs/index.md

LGTM

@jeff-r2026
jeff-r2026 merged commit 7e881b4 into Tencent:main Jul 2, 2026
7 checks passed
@hsuchifeng hsuchifeng mentioned this pull request Jul 3, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants