Summary
detect.CODE_EXTENSIONS includes .json (detect.py, the big extension set), so the AST/structural pass treats every JSON file as code and emits every JSON key as its own node. On repos where .json is data — eval fixtures, config, datasets, manifests — this floods the graph with orphan key-nodes and near-duplicate communities that swamp the real structure.
Observed (graphifyy 0.8.36, Windows, real repo)
A Python CLI repo (~200 files) with ~40 data .json files (eval cases.json/eval.json pairs, two plugin manifests, one test fixture):
|
with .json as code |
.json excluded |
| nodes |
1819 |
1170 |
| communities |
114 |
73 |
| isolated nodes |
561 |
2 |
- One repeated key (
prompt_inputs_spec) appeared as 214 occurrences in the AST extraction (0 in the LLM semantic pass).
- ~30 near-identical communities formed, one per eval suite, each holding just that file's keys (
generation, target, cases_file, num_cases, …).
- The report's "Knowledge Gaps" then flags 561 isolated nodes / 735 weakly-connected — reads like a codebase problem but is purely extraction noise.
Workaround
.graphifyignore with *.json (works — both detect() and the post-commit hook honor it). But it costs the LLM pass's concept extraction from those files too, and .graphifyignore shadowing .gitignore per-directory means re-listing everything else.
Suggested fixes (any of)
- Treat data-shaped
.json as document rather than code — or only AST-extract manifest-style json the resolver actually understands (package.json, tsconfig.json, lockfiles are already skipped).
- A config knob to reclassify extensions per-repo (e.g.
json: document / json: skip).
- Cheapest guard: warn in the report when a single
.json yields N+ key-nodes with no cross-file edges — silent flooding is the painful part.
Happy to provide the full before/after graph.json pair if useful.
Summary
detect.CODE_EXTENSIONSincludes.json(detect.py, the big extension set), so the AST/structural pass treats every JSON file as code and emits every JSON key as its own node. On repos where.jsonis data — eval fixtures, config, datasets, manifests — this floods the graph with orphan key-nodes and near-duplicate communities that swamp the real structure.Observed (graphifyy 0.8.36, Windows, real repo)
A Python CLI repo (~200 files) with ~40 data
.jsonfiles (evalcases.json/eval.jsonpairs, two plugin manifests, one test fixture):prompt_inputs_spec) appeared as 214 occurrences in the AST extraction (0 in the LLM semantic pass).generation,target,cases_file,num_cases, …).Workaround
.graphifyignorewith*.json(works — bothdetect()and the post-commit hook honor it). But it costs the LLM pass's concept extraction from those files too, and.graphifyignoreshadowing.gitignoreper-directory means re-listing everything else.Suggested fixes (any of)
.jsonasdocumentrather thancode— or only AST-extract manifest-style json the resolver actually understands (package.json,tsconfig.json, lockfiles are already skipped).json: document/json: skip)..jsonyields N+ key-nodes with no cross-file edges — silent flooding is the painful part.Happy to provide the full before/after graph.json pair if useful.