Summary
On a mid-sized Swift codebase (~382 Swift files, SwiftPM with 9 library modules + app + Vapor server + tests), supermodel audit / analyze returns a graph that is missing most declarations and the vast majority of call edges. Downstream commands (focus, blast-radius, health verdict) are consequently misleading — audit reported Status: HEALTHY with 0 circular dependencies and Transitive = 0 for every file, which is a broken-graph signature, not a real result.
This looks like the Swift parser is only capturing top-level class declarations and a fraction of func declarations, and not capturing struct / enum / protocol / actor as node labels at all.
Environment
supermodel version 0.6.10
- macOS Darwin 25.3.0 (arm64)
- API base:
https://staging-api.supermodeltools.com (same behavior expected on prod)
- Codebase: Swift 6.2.4, Xcode 26.3, macOS 26.0+ target, SwiftPM multi-module project
- Repo (private, for reference): https://github.com/cadespivey/GreyMatter — happy to reproduce on a public Swift project if helpful.
Evidence
1. Node coverage
Supermodel's reported counts vs. a naive grep of declarations in the same tree:
| Entity |
Supermodel |
grep lower bound in repo |
Capture rate |
| Files |
393 |
382 Swift + ~11 py/html/json |
~100% ✓ |
| Classes |
1,211 |
~2,152 class + ~4,583 struct + ~1,735 enum + ~480 protocol + ~36 actor ≈ 8,986 nominal types |
~13% |
| Functions |
3,865 |
~34,241 func keyword lines (even conservatively deduped ≈ 10–15K) |
~25–40% |
summary.types field |
0 |
thousands |
field unpopulated entirely |
The grep numbers are approximate (they count lines, not parsed decls), but the gap is orders of magnitude.
2. Edge coverage
Relationship counts from the cached graph JSON:
belongsTo 4037
defines_function 3865 ← one per reported func
calls 2208 ← THE call graph
declares_class 1211
imports 986
contains_file 393
defines 79
child_directory 79
partOf 10
processes_requests_from 1
triggers_analysis_in 1
coordinates_workflow_with 1
persists_data_to 1
transforms_data_for 1
validates_input_for 1
sends_events_to 1
authenticates_requests_for 1
2,208 calls edges for 3,865 reported functions ≈ 0.57 calls/function. This is ~an order of magnitude below what a real Swift application produces. It's why the audit impact table shows Transitive = 0 for every file — transitive closure has nowhere to reach.
The handful of bespoke relationship types (processes_requests_from, coordinates_workflow_with, authenticates_requests_for, etc.) each appear exactly once and read like LLM-generated domain-level prose rather than extracted edges.
3. supermodel focus smoking gun
Run against a central 4,069-line file with 174 declarations (class/struct/func/enum):
$ supermodel focus Sources/GreyMatterData/DataStoreImpl.swift
## Context: Sources/GreyMatterData/DataStoreImpl.swift
### Imports
- CoreData
- Foundation
- GreyMatterDomain
<!-- approx 19 tokens -->
Imports only. No functions, no classes, no callers, no callees — on a 4K-line file that other top-coupling files depend on.
4. Domain inference
audit returned 4 domains (GreyMatterApp, GreyMatterServer, CortexAI, GreyMatterData) whereas the SwiftPM manifest defines ~11 targets (9 library modules + app + server). Every domain reports uniformly Key files: 3 / Responsibilities: 3 / Subdomains: 2–3, which looks like default placeholders rather than computed values.
5. Health verdict is a false green
## Status: ✅ HEALTHY
| Circular Dependencies | 0 | ✅ PASS |
| High-Coupling Domains | 0 | ✅ |
Both "0"s appear to be artifacts of the sparse edge graph (too few edges to form cycles or cross domains), not real observations. An end user acting on this report could be badly misled — the report is more dangerous in its current state than if it errored out.
What I'd expect
struct, enum, protocol, actor recognized as distinct node labels (or at minimum counted in summary.types).
extension Foo { ... } declarations attributed to Foo and their members included.
- Function coverage much closer to 100% of source-order
func declarations (including methods in extensions, nested functions, and closures if they have names).
- Call-edge density roughly 3–10× function count on typical Swift codebases — enough to make
Transitive meaningful.
- When the parser can't produce a usable graph,
audit should indicate degraded confidence rather than emit a HEALTHY verdict.
Reproducer
I can share the cached graph JSON (with file/symbol names redacted if needed) and a list of per-file decl counts from the private repo. I'd also be happy to rerun against any public Swift project you suggest — a reasonable test target would be anything with 100+ files and meaningful use of struct/enum/protocol (most Swift apps). If you point at a public Swift repo you already test against, I'll send you the per-file gap numbers for it.
Ask
- Is the Swift parser supposed to capture
struct/enum/protocol/actor today? If yes, this is a regression; if no, this is the primary gap.
- Is there a known limit on the call-edge extractor for Swift (e.g. requires build-system integration / SourceKit) that explains the ~0.57 calls/func ratio?
- Would you consider gating the
HEALTHY verdict behind a minimum node/edge-coverage threshold, so users aren't handed a false green?
Thanks — happy to provide more data or test fixes.
Summary
On a mid-sized Swift codebase (~382 Swift files, SwiftPM with 9 library modules + app + Vapor server + tests),
supermodel audit/analyzereturns a graph that is missing most declarations and the vast majority of call edges. Downstream commands (focus, blast-radius, health verdict) are consequently misleading —auditreportedStatus: HEALTHYwith0 circular dependenciesandTransitive = 0for every file, which is a broken-graph signature, not a real result.This looks like the Swift parser is only capturing top-level
classdeclarations and a fraction offuncdeclarations, and not capturingstruct/enum/protocol/actoras node labels at all.Environment
supermodel version 0.6.10https://staging-api.supermodeltools.com(same behavior expected on prod)Evidence
1. Node coverage
Supermodel's reported counts vs. a naive grep of declarations in the same tree:
greplower bound in repoclass+ ~4,583struct+ ~1,735enum+ ~480protocol+ ~36actor≈ 8,986 nominal typesfunckeyword lines (even conservatively deduped ≈ 10–15K)summary.typesfieldThe
grepnumbers are approximate (they count lines, not parsed decls), but the gap is orders of magnitude.2. Edge coverage
Relationship counts from the cached graph JSON:
2,208
callsedges for 3,865 reported functions ≈ 0.57 calls/function. This is ~an order of magnitude below what a real Swift application produces. It's why theauditimpact table showsTransitive = 0for every file — transitive closure has nowhere to reach.The handful of bespoke relationship types (
processes_requests_from,coordinates_workflow_with,authenticates_requests_for, etc.) each appear exactly once and read like LLM-generated domain-level prose rather than extracted edges.3.
supermodel focussmoking gunRun against a central 4,069-line file with 174 declarations (
class/struct/func/enum):Imports only. No functions, no classes, no callers, no callees — on a 4K-line file that other top-coupling files depend on.
4. Domain inference
auditreturned 4 domains (GreyMatterApp,GreyMatterServer,CortexAI,GreyMatterData) whereas the SwiftPM manifest defines ~11 targets (9 library modules + app + server). Every domain reports uniformlyKey files: 3 / Responsibilities: 3 / Subdomains: 2–3, which looks like default placeholders rather than computed values.5. Health verdict is a false green
Both "0"s appear to be artifacts of the sparse edge graph (too few edges to form cycles or cross domains), not real observations. An end user acting on this report could be badly misled — the report is more dangerous in its current state than if it errored out.
What I'd expect
struct,enum,protocol,actorrecognized as distinct node labels (or at minimum counted insummary.types).extension Foo { ... }declarations attributed toFooand their members included.funcdeclarations (including methods in extensions, nested functions, and closures if they have names).Transitivemeaningful.auditshould indicate degraded confidence rather than emit a HEALTHY verdict.Reproducer
I can share the cached graph JSON (with file/symbol names redacted if needed) and a list of per-file decl counts from the private repo. I'd also be happy to rerun against any public Swift project you suggest — a reasonable test target would be anything with 100+ files and meaningful use of
struct/enum/protocol(most Swift apps). If you point at a public Swift repo you already test against, I'll send you the per-file gap numbers for it.Ask
struct/enum/protocol/actortoday? If yes, this is a regression; if no, this is the primary gap.HEALTHYverdict behind a minimum node/edge-coverage threshold, so users aren't handed a false green?Thanks — happy to provide more data or test fixes.