Skip to content

feat(codebase): wiki-engine extraction + AI enrichment + knowledge reconciler - #55

Merged
jeff-r2026 merged 1 commit into
Tencent:mainfrom
m0Nst3r873:feat/import-enrichment-v2
Jun 29, 2026
Merged

feat(codebase): wiki-engine extraction + AI enrichment + knowledge reconciler#55
jeff-r2026 merged 1 commit into
Tencent:mainfrom
m0Nst3r873:feat/import-enrichment-v2

Conversation

@m0Nst3r873

@m0Nst3r873 m0Nst3r873 commented Jun 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Full deterministic code knowledge pipeline: extract → enrich → reconcile → compile → index.

Part 1 of 2 — this PR is functionally complete. PR #56 builds on top (deep-enrich + recall + skill).

Phase 1: Wiki-engine + code extraction pipeline

  • Vendor wiki-engine modules: interface scanner (HTTP/MQ/RPC, 5 languages), dependency path tracer, code-graph overlay, doc-graph extractor
  • codebase-extract.ts: full pipeline (collect → extract facts → build graph → write evidence pages + overview.md)
  • CLI: teamai codebase --extract [path] with --incremental, --project, --max-files
  • Output: evidence pages, overview.md, graph-index.json, router.md, index.md, gaps/detected.md

Phase 2: AI enrichment + reconciler + wiki index

  • enrich-with-ai.ts: Per-module AI responsibility inference + repo-level domain classification
  • knowledge-reconciler.ts: 9-phase product↔code cross-mapping with confidence scoring
  • manifest-compiler.ts: Compile AI-enriched manifest into structured component docs with wiki-links
  • rebuild-wiki-index.ts: Domain-grouped router.md + stats-enriched index.md
  • Remove legacy AI domain classification (domains.yaml)

How it's reachable (no dead code)

  • teamai import --from-repo <url> → extractCodebase + enrichWithAI + compileFromManifest + reconcileKnowledge
  • teamai import --from-org <org> → batch import + rebuildWikiIndex
  • teamai codebase --extract . → enrichWithAI + compile docs/<slug>.md

Quality fixes

Fix Description
B1 Unified graph-index path to .indices/
B2 Fixed router.md wiki-links (evidence/code/ prefix)
B3 Added teamwiki to safeIgnore
B6 Wire compileFromManifest() into pipeline (was dead code)
B9 Unified graph schema to GraphIndex
B12 Fix reconciler stale detection
B13 Filter third-party npm imports from relation facts
B15 Key file priority sorting in collection
B16 Generate deterministic overview.md without AI
B17 Rename call-chains → dependency-paths
B18 Python extractor: only service-pattern functions as components
B19 Facts deduplication by kind:name
B20 Add stopword filtering to reconciler term matching
B21 Restrict config extraction to SCREAMING_SNAKE_CASE
B22 API path detection no longer requires /v\d*/ prefix
B23 Manifest-compiler outputs wiki-links for upstream/downstream

Test plan

  • npx tsc --noEmit — zero errors
  • npx vitest run — all tests passed
  • teamai codebase --extract . produces teamwiki/evidence/code/<project>/
  • teamai import --from-repo <url> runs full pipeline end-to-end

Dependency chain

PR 1 (this) → #56 (deep-enrich + recall + skill)

Replaces #50 and #51. Subsumes #54 (now closed).

This was referenced Jun 26, 2026
Merged
Merged
Closed
@m0Nst3r873 m0Nst3r873 changed the title feat(import): AI enrichment + knowledge reconciler + wiki index feat(codebase): wiki-engine extraction + AI enrichment + knowledge reconciler Jun 29, 2026
@m0Nst3r873
m0Nst3r873 force-pushed the feat/import-enrichment-v2 branch from f2dcc2f to 492b8f2 Compare June 29, 2026 03:04
m0Nst3r873 pushed a commit to m0Nst3r873/teamai-cli that referenced this pull request Jun 29, 2026
jaelgeng
feat(codebase): deep-enrich + graph-aware recall + team-wiki-codebase…
ca8fa3d 
… skill

Complete code knowledge pipeline with AI-powered deep generation and
graph-boosted retrieval.

Base layer (wiki-engine + enrichment, same as PR Tencent#55):
- Deterministic code extraction pipeline (5-language extractors)
- AI enrichment + knowledge reconciler + manifest compiler
- Wiki index rebuild with domain grouping

Deep-enrich (this PR's additions):
- deep-enrich.ts: background AI knowledge generation with resume support
  and --max-modules cost control
- code-knowledge-recall.ts: BM25 + graph-boost retrieval engine with
  CJK bigram segmentation and camelCase splitting
- codebase-upgrade-wiki.ts: migration from docs/team-codebase/ to teamwiki/
- codebase-wiki-lint.ts: graph health diagnostics

Recall improvements:
- --depth option (route/context/lookup) with per-level descriptions
- Graph boost via slug/title matching (fixes dead-code path mismatch)
- BM25 normalization to prevent score domination over learnings
- 2-hop neighbor extension for graph edges

Integration:
- CI: MR comment API + graph change detection
- import-iwiki.ts: iWiki document import
- team-wiki-codebase skill bundle (methodology + agent prompts)
- Security hardening (shell-quote, JSON parse guards, path validation)

Tests: restore 3 import-org tests (replace stale clusterRepos assertions)
m0Nst3r873 pushed a commit to m0Nst3r873/teamai-cli that referenced this pull request Jun 29, 2026
jaelgeng
feat(codebase): deep-enrich + graph-aware recall + team-wiki-codebase…
47c1c52 
… skill

Complete code knowledge pipeline with AI-powered deep generation and
graph-boosted retrieval.

Base layer (wiki-engine + enrichment, same as PR Tencent#55):
- Deterministic code extraction pipeline (5-language extractors)
- AI enrichment + knowledge reconciler + manifest compiler
- Wiki index rebuild with domain grouping

Deep-enrich (this PR's additions):
- deep-enrich.ts: background AI knowledge generation with resume support
  and --max-modules cost control
- code-knowledge-recall.ts: BM25 + graph-boost retrieval engine with
  CJK bigram segmentation and camelCase splitting
- codebase-upgrade-wiki.ts: migration from docs/team-codebase/ to teamwiki/
- codebase-wiki-lint.ts: graph health diagnostics

Recall improvements:
- --depth option (route/context/lookup) with per-level descriptions
- Graph boost via slug/title matching (fixes dead-code path mismatch)
- BM25 normalization to prevent score domination over learnings
- 2-hop neighbor extension for graph edges

Integration:
- CI: MR comment API + graph change detection
- import-iwiki.ts: iWiki document import
- team-wiki-codebase skill bundle (methodology + agent prompts)
- Security hardening (shell-quote, JSON parse guards, path validation)

Tests: restore 3 import-org tests (replace stale clusterRepos assertions)
jaelgeng
feat(codebase): wiki-engine extraction + AI enrichment + knowledge re…
e31c518 
…conciler

Deterministic code knowledge pipeline: extract → enrich → reconcile → compile → index.

Wiki-engine modules:
- Interface scanner (HTTP/MQ/RPC, 5 languages)
- Dependency path tracer + code-graph overlay
- Doc-graph extractor + manifest schema

Extraction pipeline (teamai codebase --extract):
- Code collector with priority sorting + incremental mode
- Per-language extractors (TS/Go/Java/Python/Rust/Config)
- Graph-index builder + evidence page renderer

AI enrichment layer:
- enrich-with-ai.ts: per-module responsibility inference
- knowledge-reconciler.ts: 9-phase product↔code cross-mapping
- manifest-compiler.ts: structured component docs with wiki-links
- rebuild-wiki-index.ts: domain-grouped router.md + stats index.md

Import integration:
- import-repo: full pipeline (extract + enrich + compile + reconcile)
- import-org: simplified whitelist-based flow (no AI clustering)
- Remove legacy domain classification (domains.yaml)

Tests: 39 new unit tests for wiki-engine modules + hook-output
@m0Nst3r873
m0Nst3r873 force-pushed the feat/import-enrichment-v2 branch from 492b8f2 to e31c518 Compare June 29, 2026 03:26
@jeff-r2026
jeff-r2026 merged commit 1291866 into Tencent:main Jun 29, 2026
7 checks passed
m0Nst3r873 pushed a commit to m0Nst3r873/teamai-cli that referenced this pull request Jun 29, 2026
jaelgeng
feat(codebase): deep-enrich + graph-aware recall + team-wiki-codebase…
52bfa84 
… skill

Complete code knowledge pipeline with AI-powered deep generation and
graph-boosted retrieval.

Base layer (wiki-engine + enrichment, same as PR Tencent#55):
- Deterministic code extraction pipeline (5-language extractors)
- AI enrichment + knowledge reconciler + manifest compiler
- Wiki index rebuild with domain grouping

Deep-enrich (this PR's additions):
- deep-enrich.ts: background AI knowledge generation with resume support
  and --max-modules cost control
- code-knowledge-recall.ts: BM25 + graph-boost retrieval engine with
  CJK bigram segmentation and camelCase splitting
- codebase-upgrade-wiki.ts: migration from docs/team-codebase/ to teamwiki/
- codebase-wiki-lint.ts: graph health diagnostics

Recall improvements:
- --depth option (route/context/lookup) with per-level descriptions
- Graph boost via slug/title matching (fixes dead-code path mismatch)
- BM25 normalization to prevent score domination over learnings
- 2-hop neighbor extension for graph edges

Integration:
- CI: MR comment API + graph change detection
- import-iwiki.ts: iWiki document import
- team-wiki-codebase skill bundle (methodology + agent prompts)
- Security hardening (shell-quote, JSON parse guards, path validation)

Tests: restore 3 import-org tests (replace stale clusterRepos assertions)
m0Nst3r873 pushed a commit to m0Nst3r873/teamai-cli that referenced this pull request Jun 29, 2026
jaelgeng
feat(codebase): deep-enrich + graph-aware recall + team-wiki-codebase…
fe43de4 
… skill

Complete code knowledge pipeline with AI-powered deep generation and
graph-boosted retrieval.

Base layer (wiki-engine + enrichment, same as PR Tencent#55):
- Deterministic code extraction pipeline (5-language extractors)
- AI enrichment + knowledge reconciler + manifest compiler
- Wiki index rebuild with domain grouping

Deep-enrich (this PR's additions):
- deep-enrich.ts: background AI knowledge generation with resume support
  and --max-modules cost control
- code-knowledge-recall.ts: BM25 + graph-boost retrieval engine with
  CJK bigram segmentation and camelCase splitting
- codebase-upgrade-wiki.ts: migration from docs/team-codebase/ to teamwiki/
- codebase-wiki-lint.ts: graph health diagnostics

Recall improvements:
- --depth option (route/context/lookup) with per-level descriptions
- Graph boost via slug/title matching (fixes dead-code path mismatch)
- BM25 normalization to prevent score domination over learnings
- 2-hop neighbor extension for graph edges

Integration:
- CI: MR comment API + graph change detection
- import-iwiki.ts: iWiki document import
- team-wiki-codebase skill bundle (methodology + agent prompts)
- Security hardening (shell-quote, JSON parse guards, path validation)

Tests: restore 3 import-org tests (replace stale clusterRepos assertions)
jeff-r2026 pushed a commit that referenced this pull request Jun 29, 2026
@m0Nst3r873 jaelgeng
feat(codebase): deep-enrich + graph-aware recall + team-wiki-codebase…
9851ffa 
… skill (#56)

Complete code knowledge pipeline with AI-powered deep generation and
graph-boosted retrieval.

Base layer (wiki-engine + enrichment, same as PR #55):
- Deterministic code extraction pipeline (5-language extractors)
- AI enrichment + knowledge reconciler + manifest compiler
- Wiki index rebuild with domain grouping

Deep-enrich (this PR's additions):
- deep-enrich.ts: background AI knowledge generation with resume support
  and --max-modules cost control
- code-knowledge-recall.ts: BM25 + graph-boost retrieval engine with
  CJK bigram segmentation and camelCase splitting
- codebase-upgrade-wiki.ts: migration from docs/team-codebase/ to teamwiki/
- codebase-wiki-lint.ts: graph health diagnostics

Recall improvements:
- --depth option (route/context/lookup) with per-level descriptions
- Graph boost via slug/title matching (fixes dead-code path mismatch)
- BM25 normalization to prevent score domination over learnings
- 2-hop neighbor extension for graph edges

Integration:
- CI: MR comment API + graph change detection
- import-iwiki.ts: iWiki document import
- team-wiki-codebase skill bundle (methodology + agent prompts)
- Security hardening (shell-quote, JSON parse guards, path validation)

Tests: restore 3 import-org tests (replace stale clusterRepos assertions)

Co-authored-by: jaelgeng <jaelgeng@tencent.com>
@m0Nst3r873
m0Nst3r873 deleted the feat/import-enrichment-v2 branch June 29, 2026 06:15
@hsuchifeng hsuchifeng mentioned this pull request Jul 3, 2026
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@m0Nst3r873 @jeff-r2026