Proposal: AST import-graph aware chunking (uses Part A output to inform Part B packing)

[jasonm4130]

A design discussion for a follow-up to #625.

Context

#625 packs files greedily into chunks within a token budget, grouped by parent directory. That's a clear improvement over the current static chunk_size=20, but it leaves obvious signal on the floor: graphify's Part A AST extraction already produces a structural graph (imports, calls, inherits) before Part B semantic extraction runs. We're not using any of it for packing.

Real codebases routinely have modules that span directories — e.g., apps/web/api/foo.ts imports packages/core/foo-service.ts imports packages/db/foo-models.ts. Directory grouping puts these in three different chunks; an LLM looking at any one chunk in isolation can't surface the cross-module relationship as a semantically_similar_to or shares_data_with edge because it can't see the other side.

Proposal

After Part A AST runs, build a coarse co-location graph from import/call edges and use it to drive Part B chunk packing:

1. Take the AST nodes/edges from Part A (already in .graphify_ast.json)
2. Build a NetworkX graph keyed by source file (one node per file)
3. Add edges between files with any AST relationship (imports, calls, inherits)
4. Find weakly-connected components, cap each at the token budget, split if needed
5. Pack the resulting components into chunks

This is using the existing AST community structure to inform extraction — same idea graphify already applies to the output graph (community detection in Step 4), but applied to chunking inputs.

Why this might help

The strongest signal for "these files share semantic context" in a code repo isn't directory; it's import. A chunk where every file is mutually-importing gives the LLM the best chance to extract cross-file calls AST already finds plus semantic aliasing AST cannot, shares_data_with between concrete classes and their callers, and semantically_similar_to where the same concept is implemented twice in adjacent modules.

Pure directory grouping captures this for repos that respect package-by-feature; it misses it for repos with package-by-layer (controllers/, services/, models/), which is most enterprise codebases. The worked/ examples in this repo include a few of each.

Tradeoffs

• The pipeline currently runs Part A and Part B in parallel per skill.md. Using AST output for Part B chunking introduces a serial dependency. Either accept the lost concurrency, or do a coarse "AST scan only" pre-pass (no LLM, deterministic, ~ms per file) before any extraction starts purely to build the import graph.
• The second option is cleaner: tree-sitter parses are deterministic and fast, so a pre-pass adds negligible time vs. the LLM-bound Part B which dominates wall-clock.

Filing this as a discussion rather than a PR because the pipeline reordering has design implications worth talking through before any code lands.

Related: #625 (token-budget chunking, direct predecessor), #623 (Kimi reasoning fix).