Skip to content

Add Ruby extraction via Prism, not tree-sitter - #116

Open
emmahyde wants to merge 1 commit into
mex-memory:mainfrom
emmahyde:ruby-prism-extractor
Open

Add Ruby extraction via Prism, not tree-sitter#116
emmahyde wants to merge 1 commit into
mex-memory:mainfrom
emmahyde:ruby-prism-extractor

Conversation

@emmahyde

@emmahyde emmahyde commented Aug 4, 2026

Copy link
Copy Markdown

What

Adds a Ruby language extractor so mex graph indexes .rb files. Ruby was previously silently skipped — no grammar, no extractor — so mex graph query where-defined <RailsSymbol> returned TARGET_NOT_FOUND for every Ruby symbol on a real Rails codebase.

Why Prism, not tree-sitter

Rather than vendor the community tree-sitter-ruby grammar (the pattern every other language in this repo follows), this uses @ruby/prism — Ruby core's own parser, the one CRuby 3.3+ ships. It's more semantically accurate for Ruby than a third-party tree-sitter grammar, and it's maintained by the Ruby core team.

Prism's typed AST has no tree-sitter node shape to adapt into, so this needed two small, deliberately isolated deviations from the standard extractor path (documented in docs/extractors.md under "Vendored Grammars > Ruby"):

  • src/graph/extraction/prism-runtime.ts — Ruby's parallel loader to grammars.ts's tree-sitter loader (Prism has no relationship to web-tree-sitter).
  • rubyExtractor.extract() ignores the passed tree: TSTree parameter entirely and re-parses source with Prism directly. grammars.ts#parse() still returns a placeholder TSTree for Ruby so extractFile's generic if (!tree) return null gate behaves identically across every language — no changes to the frozen LanguageExtractor interface, node identity scheme, or SQLite schema.

Coverage

rubyExtractor (src/graph/extraction/languages/ruby.ts) covers: modules, classes (with superclass extends edges), instance and singleton (def self.x) methods, module/class-scoped constants, mixins (include/extend/prependimplements), require/require_relativeimports, .newinstantiates, and general method calls.

Fixture (src/graph/__tests__/fixtures/sample.rb) and focused tests (src/graph/__tests__/extractor-ruby.test.ts, 7 tests) follow the existing extractor.test.ts/sample.ts pattern.

Verification

  • Verified against real production Rails code (STI phase base class, service objects) with dense method bodies and class-method definitions — zero crashes.
  • npx tsc --noEmit — clean.
  • npx vitest run — full suite passes except one pre-existing, unrelated failure in test/tui.test.ts (confirmed via git stash/git stash pop bisection: identical failure with zero of this PR's changes present).
  • npm run build — clean, .wasm/schema assets copy as expected.
  • Built a real graph against a ~1600-file Rails codebase (1681 files, 31,816 nodes, 63,637 edges) and confirmed where-defined/who-calls/graph scope now resolve Ruby symbols end-to-end, including cross-file call chains and STI overrides.

Docs

Updated docs/extractors.md (Vendored Grammars > Ruby, with an explicit deviation note) and docs/code-graph-support.md (added the Ruby row to the support table).

mex's code graph only indexed frontend TS/JS/Python/Rust; Ruby files were
silently skipped (no extractor, no grammar), so `mex graph query` returned
TARGET_NOT_FOUND for every Rails symbol. Rather than vendor the community
tree-sitter-ruby grammar, use @ruby/prism — Ruby core's own parser (the one
CRuby 3.3+ ships) — which is more semantically accurate for Ruby.

Prism's typed AST has no tree-sitter node shape, so `rubyExtractor.extract()`
ignores the passed `tree` and re-parses `source` with Prism directly;
`prism-runtime.ts` is Ruby's parallel loader alongside `grammars.ts`'s
tree-sitter loader. Verified against real groot Rails files (STI phase base
class, service objects) with no crashes.
@emmahyde

emmahyde commented Aug 4, 2026

Copy link
Copy Markdown
Author

Split out a related but independent fix into #117 (graph scope's exact-match scoring was picking up English stopwords from task phrasing — found while building the graph for this Ruby extractor and comparing it against grep, see #115 for the full writeup). No overlap with this PR's changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant