feat: language capability rows — assembler language-agnosticism (#619) - #643
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 36 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (11)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements support for open language vocabularies (RFC-LANGUAGE-PLUGINS §7, issue #619), allowing the assembler to learn and support new script languages entirely from manifest capability rows rather than a closed, frozen list of built-ins. Load-time validation for language requirements is relaxed to shape-only checks, while full vocabulary validation is deferred to generate-time when the scripting plugin's manifest is available. Additionally, the event consumption scan is updated to include row-declared language extensions, and a comprehensive 'quokka' litmus test suite is added to verify end-to-end agnosticism. No review comments were provided, so there is no feedback to address.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f1210adbe9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (name[0] < 'a' or name[0] > 'z') return false; | ||
| for (name[1..]) |c| { | ||
| switch (c) { | ||
| 'a'...'z', '0'...'9', '_' => {}, |
There was a problem hiding this comment.
When a manifest row uses an identifier-shaped language name that is still unsafe for the generated enum literal—e.g. a Zig keyword such as error, or a name long enough to exceed the 96-byte scripting_lang_buf used by generateBuildZig—this predicate accepts it, validateDeclaredLanguage admits the row, and generation later emits invalid build code or hits the catch unreachable while formatting , .language = .{s}. The previous closed vocabulary only contained short non-keyword names, so row-declared languages need equivalent constraints here or the build.zig emission needs quoting/dynamic allocation.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in da1a42b. isLanguageIdentifier now rejects Zig keywords (error/fn/test/…) — the generated .language = .<name> enum literal would be a syntax error otherwise. Enforced at manifest load (pointed PluginManifestInvalidLanguageRow) and in resolveProjectLanguage/requires_language. Primitives (i32) stay admissible since .i32 is a legal enum literal. Tests: keyword-named row rejected + isLanguageIdentifier keyword cases.
| if (vocab.rowFor(lang) != null) continue; | ||
| try langs.append(allocator, .{ | ||
| .name = lang, | ||
| .extensions = scriptExtensions(lang), |
There was a problem hiding this comment.
When a migrated scripting manifest includes a row for a built-in language, this skips the frozen entry and later re-adds it with only the row's extensions and dir = r.name. For TypeScript rows, which typically declare authored .ts and .transpile.emits = "js", that drops the frozen .js policy extension and the ts/ legacy/misplaced-dir handling; e.g. a TypeScript project with files under typescript/ now passes even though resolveScriptDir still reads only scripts/ or ts/, and a Lua project stops flagging stray scripts/foo.js as foreign. Keep the frozen legacy dir/emitted-extension metadata when a row shadows a built-in.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in da1a42b. Vocabulary.build now UNIONS the frozen scriptExtensions into a shadowing row (row order first, deduped), so a migrated typescript row declaring only authored .ts keeps the frozen emitted .js; scanUnitLanguageDirs keeps the built-in legacy-dir mapping for shadowed languages instead of dropping it. (The full splice/declare/transpile metadata — TSC_PLATFORMS, staging geometry — is read directly from the manifest LanguageRow by the splice, not through this policy-side projection, so it was never at risk; this fixes the extension-set loss the scans depend on.) Test: Vocabulary.build shadow-row unions frozen extensions.
| try std.fmt.allocPrint(allocator, ".{s}", .{ext}); | ||
| ext_len += 1; | ||
| } | ||
| try rows.append(allocator, .{ .name = name, .extensions = exts }); |
There was a problem hiding this comment.
When a new-language .languages row omits .extensions (the manifest type defaults it to empty), this still appends the row to the vocabulary, so validateDeclaredLanguage accepts .params.language even though scripting_splice.spliceFromRow later returns null for row.extensions.len == 0. That means a project can generate with the language declared but no scripting splice installed, and the policy scan has no extension to catch the corresponding scripts/ files, so the scripts are silently ignored instead of failing at manifest/policy validation.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in da1a42b. A .languages row with empty .extensions is now rejected at manifest load with a pointed diagnostic (PluginManifestInvalidLanguageRow) — the shared-dir and plugin-event consumption scans find a language's sources by extension, so an extensionless row would silently elide every source it declares. Test: extensionless row rejected at load.
…in metadata on shadow (codex #643 P2) Three review fixes on the capability-row vocabulary: 1. Reject language names that break build.zig emission. isLanguageIdentifier now also rejects Zig keywords (`error`/`fn`/`test`/…): the generated `.language = .<name>` enum literal would be a syntax error otherwise. Enforced at manifest load (pointed PluginManifestInvalidLanguageRow) and in resolveProjectLanguage/requires_language. (Primitives like i32 stay admissible — `.i32` is a legal enum literal; only keywords break syntax.) 2. Preserve built-in metadata when a row shadows a built-in. Vocabulary.build now UNIONS the frozen scriptExtensions into a shadowing row's extensions (row order first, deduped), so a migrated typescript row declaring only authored `.ts` keeps the frozen emitted `.js`. scanUnitLanguageDirs keeps the built-in's legacy-dir mapping for shadowed languages instead of dropping it. (Splice/declare/transpile metadata — TSC_PLATFORMS, staging geometry — is read directly from the manifest LanguageRow, unaffected.) 3. Reject rows without source extensions. A `.languages` row with empty `.extensions` is rejected at manifest load with a pointed diagnostic — the shared-dir/consumption scans find sources by extension, so an extensionless row would silently elide every source it declares. Tests: keyword-named row rejected, extensionless row rejected, well-formed row loads, shadow-row unions frozen extensions, keyword identifiers rejected. Baseline green (flow_catalog env lanes tolerated). Claude-Session: https://claude.ai/code/session_011szWvquoss1yNX7KWSKCaM
da1a42b to
cb73c86
Compare
…in metadata on shadow (codex #643 P2) Three review fixes on the capability-row vocabulary: 1. Reject language names that break build.zig emission. isLanguageIdentifier now also rejects Zig keywords (`error`/`fn`/`test`/…): the generated `.language = .<name>` enum literal would be a syntax error otherwise. Enforced at manifest load (pointed PluginManifestInvalidLanguageRow) and in resolveProjectLanguage/requires_language. (Primitives like i32 stay admissible — `.i32` is a legal enum literal; only keywords break syntax.) 2. Preserve built-in metadata when a row shadows a built-in. Vocabulary.build now UNIONS the frozen scriptExtensions into a shadowing row's extensions (row order first, deduped), so a migrated typescript row declaring only authored `.ts` keeps the frozen emitted `.js`. scanUnitLanguageDirs keeps the built-in's legacy-dir mapping for shadowed languages instead of dropping it. (Splice/declare/transpile metadata — TSC_PLATFORMS, staging geometry — is read directly from the manifest LanguageRow, unaffected.) 3. Reject rows without source extensions. A `.languages` row with empty `.extensions` is rejected at manifest load with a pointed diagnostic — the shared-dir/consumption scans find sources by extension, so an extensionless row would silently elide every source it declares. Tests: keyword-named row rejected, extensionless row rejected, well-formed row loads, shadow-row unions frozen extensions, keyword identifiers rejected. Baseline green (flow_catalog env lanes tolerated). Claude-Session: https://claude.ai/code/session_011szWvquoss1yNX7KWSKCaM
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cb73c86c54
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // that both spells a valid enum literal and cannot overflow the | ||
| // fixed buffer. Dev-mode hot reload (#637): a |
There was a problem hiding this comment.
Fresh evidence after the prior fix: isLanguageIdentifier rejects keywords but still has no length cap, while generateBuildZig formats the value into a [96]u8 buffer with catch unreachable. A row-declared language name longer than 81 bytes (or longer than 47 bytes when hot reload is appended) passes manifest/policy validation and then panics during generation instead of producing a diagnostic; the old closed table only had short names. Please cap/reject names or switch this formatting to allocation.
Useful? React with 👍 / 👎.
| for (in.extensions) |ext| { | ||
| const dotted = if (ext.len > 0 and ext[0] == '.') | ||
| try allocator.dupe(u8, ext) | ||
| else | ||
| try std.fmt.allocPrint(allocator, ".{s}", .{ext}); | ||
| errdefer allocator.free(dotted); | ||
| try exts.append(allocator, dotted); | ||
| } |
There was a problem hiding this comment.
When a row declares multiple source extensions, e.g. .extensions = .{ ".py", ".pyw" }, this loop adds all of them to the project vocabulary, so scanUnitLanguageDirs treats scripts/foo.pyw as valid for the selected language. The actual splice still derives authored = row.extensions[0] in scripting_splice.spliceFromRow, and the consumption scan only receives s.extension/the transpile source, so non-first extensions are neither registered/embedded nor scanned for plugin-event references. Either consume/thread every declared extension or reject rows with more than one extension to avoid silently dead scripts.
Useful? React with 👍 / 👎.
| // 2. `.extensions` — non-empty: the shared-dir + plugin-event | ||
| // consumption scans find a language's sources BY EXTENSION, so an | ||
| // extensionless row would silently elide every source it declares. | ||
| for (parsed.languages) |row| { |
There was a problem hiding this comment.
A new .languages row such as .kind = .native with .extensions but without .module_root or .stage_subdir passes this manifest validation and is then admitted by validateDeclaredLanguage. The selected row cannot actually drive generation: scripting_splice.spliceFromRow returns null when either native field is absent, and for a non-built-in language detect falls through to no splice, so declared scripts/*.zc files are skipped by policy but never staged or wired. Please make native rows for non-fallback languages require both fields, or fail the selected-row detection instead of degrading to no integration.
Useful? React with 👍 / 👎.
| if (declared) |d| { | ||
| if (std.mem.eql(u8, d.language, lang)) continue; | ||
| } | ||
| const matches = for (scriptExtensions(lang)) |ext| { | ||
| const matches = for (lang_entry.extensions) |ext| { | ||
| if (std.mem.endsWith(u8, entry.name, ext)) break true; |
There was a problem hiding this comment.
If the selected row reuses an extension owned by another language, the shared-dir scan skips only the declared language and then continues matching the same filename against the remaining entries. For example, a new quokka row using .rb (or two rows both using .qk) makes scripts/behavior.rb/.qk in a quokka project get bucketed as a foreign language and fail before the row-driven splice can consume it. The built-in table relies on disjoint extensions for attribution, so row extensions need the same collision/suffix-overlap validation.
Useful? React with 👍 / 👎.
| for (pmani.languages, inputs) |row, *in| { | ||
| in.* = .{ .name = row.name, .extensions = row.extensions }; |
There was a problem hiding this comment.
For a new non-built-in transpiled row such as .name = "coffee", .extensions = .{".coffee"}, .transpile = .{ .emits = "js", ... }, this projection gives the language policy only the authored extension. The splice intentionally uses the emitted extension as s.extension and probes/collects scripts/*.js, but scanUnitLanguageDirs will still classify those .js files as the built-in TypeScript language and fail them as foreign before the splice can consume them. Please union the row's emitted extension into the declared language's policy extensions, or reject emitted extensions that another language owns.
Useful? React with 👍 / 👎.
| // consumption scans find a language's sources BY EXTENSION, so an | ||
| // extensionless row would silently elide every source it declares. | ||
| for (parsed.languages) |row| { | ||
| if (!language_policy.isLanguageIdentifier(row.name)) { |
There was a problem hiding this comment.
A row named scripts passes this validation, but it makes normal scripts/*.sc projects unusable: language_policy.legacyDir("scripts") returns the same scripts/ directory as the shared convention dir, so resolveScriptDir sees both the top-level scripts probe and the legacy probe populated and raises LegacyScriptDirConflict. Since open row names can now collide with assembler-owned convention directories, reject at least scripts here or disable legacy fallback for row-declared languages.
Useful? React with 👍 / 👎.
Open the assembler's language VOCABULARY so a language plugin can add a brand-new language (the RFC's "python") through a `plugin.labelle` `.languages` capability row alone — zero assembler changes. This closes the last per-language hardcode the row-driven declare/transpile/embed/ native machinery (already on main via #621–#624) sat on top of: the closed `SUPPORTED_LANGUAGES`/`scriptExtensions` table that the policy gate, the shared-dir scan, the plugin-event consumption scan, and `requires_language` validation all consulted, and which rejected any language it had never heard of before the row machinery ever ran. What moved (RFC-LANGUAGE-PLUGINS rev 16 §7 "language-agnostic assembler"): - `language_policy.Vocabulary` — the project's OPEN language set: the frozen `SUPPORTED_LANGUAGES` built-ins ∪ the declaring scripting plugin's manifest `.languages` rows. Rows are PRIMARY (a row naming a frozen language shadows its extension set); a row language the tables never heard of is admitted exactly like a built-in. `EMPTY` (no rows) reproduces the pre-#619 closed-vocabulary behavior bit for bit. - `validateDeclaredLanguage` — the vocabulary gate, re-homed from `resolveProjectLanguage` into `generate_phases.validateLanguagePolicy` where the declaring plugin's manifest rows are in hand. Self-describing capabilities replace the closed table: a pointed error naming BOTH the built-ins and the manifest rows, no version compare. - `resolveProjectLanguage` / manifest-load `requires_language` now gate on SHAPE only (`isLanguageIdentifier` — `[a-z][a-z0-9_]*`, the one constraint the generated `.language = .<name>` enum literal genuinely imposes); the vocabulary check runs at generate where rows are visible. - `scanUnitLanguageDirs` / `checkRequiresLanguage` take the `Vocabulary` and police row languages by the row's extensions (frozen ∪ rows). - The plugin-event consumption scan gains the active splice's script extensions (`filterConsumedEvents(extra_extensions)`) so a row language's sources (`.py`) are not invisible to the scan and its subscriptions silently elided — the comptime `scanned_extensions` covers only the frozen built-ins. Migration map (hardcoded path → row-driven): - closed `SUPPORTED_LANGUAGES` vocabulary check in `resolveProjectLanguage` → `Vocabulary.isKnown` (frozen ∪ manifest rows) in `validateDeclaredLanguage` - `scriptExtensions`-only dir/event policing → `Vocabulary.extensionsOf` (row extensions primary, frozen fallback) - manifest-load closed-table `requires_language` reject → shape-only reject + generate-time vocabulary check - (already on main, unchanged) declare → `runGenericDeclarePhase` rows; embed/native/transpile → `spliceFromRow`; the frozen `DECLARE_RUNNERS` / `EMBED_LANGUAGES` / `NATIVE_LANGUAGES` / `TSC_PLATFORMS` tables stay FROZEN FALLBACKS for row-less (pre-migration) manifests. Back-compat proof: `LUA_ROW_BACKCOMPAT` generates a lua project twice — frozen fallback (no row) vs an equivalent `.languages` row — and asserts the main.zig + build.zig are BYTE-IDENTICAL. Migrating lua/ruby onto rows shifts nothing. Every existing golden/e2e suite is unchanged. Litmus (`test/quokka_litmus_tests.zig`): a fake embedded-VM language "quokka" (`.qk`) the assembler learns ENTIRELY from a manifest row generates end-to-end (embed registration, `.language = .quokka` dep arg, the four splice touchpoints), a foreign frozen-language file (`.rb`) in the same project still errors (combined vocabulary polices both ways), and a META test proves "quokka"/".qk" appear in NO `src/**/*.zig` — the mechanical proof the generate rode rows, not a hidden branch. Closes #619 Claude-Session: https://claude.ai/code/session_011szWvquoss1yNX7KWSKCaM
…in metadata on shadow (codex #643 P2) Three review fixes on the capability-row vocabulary: 1. Reject language names that break build.zig emission. isLanguageIdentifier now also rejects Zig keywords (`error`/`fn`/`test`/…): the generated `.language = .<name>` enum literal would be a syntax error otherwise. Enforced at manifest load (pointed PluginManifestInvalidLanguageRow) and in resolveProjectLanguage/requires_language. (Primitives like i32 stay admissible — `.i32` is a legal enum literal; only keywords break syntax.) 2. Preserve built-in metadata when a row shadows a built-in. Vocabulary.build now UNIONS the frozen scriptExtensions into a shadowing row's extensions (row order first, deduped), so a migrated typescript row declaring only authored `.ts` keeps the frozen emitted `.js`. scanUnitLanguageDirs keeps the built-in's legacy-dir mapping for shadowed languages instead of dropping it. (Splice/declare/transpile metadata — TSC_PLATFORMS, staging geometry — is read directly from the manifest LanguageRow, unaffected.) 3. Reject rows without source extensions. A `.languages` row with empty `.extensions` is rejected at manifest load with a pointed diagnostic — the shared-dir/consumption scans find sources by extension, so an extensionless row would silently elide every source it declares. Tests: keyword-named row rejected, extensionless row rejected, well-formed row loads, shadow-row unions frozen extensions, keyword identifiers rejected. Baseline green (flow_catalog env lanes tolerated). Claude-Session: https://claude.ai/code/session_011szWvquoss1yNX7KWSKCaM
…cisions Migrate the two `language == "csharp"` DECISIONS #644 added into capability rows, per #619's design (the module doc #644 shipped explicitly anticipated this): - New LanguageRow capabilities `runtime_output` (the link-less build outputs ARE the runtime payload — stage beside the binary + set the run step's assembly-dir env) and `probe_tools` (opt into the generate-time PATH probe of the language's steps). - scripting_csharp.stagesRuntimeOutputs now takes the resolved `runtime_output` capability instead of hardcoding `== "csharp"`; the ensureStepToolsOnPath call is gated on the resolved `probe_tools`. - The csharp knowledge is DEMOTED to a frozen fallback (scripting_csharp.frozenRuntimeOutput/frozenProbeTools), the same #619 pattern as DECLARE_RUNNERS/EMBED_LANGUAGES/NATIVE_LANGUAGES — the shipped csharp manifest (which predates these capabilities) keeps working byte-for-byte; a NEW runtime-loaded language rides its row alone. - ScriptingSplice.detect resolves both capabilities (row PRIMARY ∪ frozen) and threads them onto the splice; root.zig consumes them. csharp's EMBED compile wiring itself needed NO migration — it was already fully row-driven (csharp has no frozen-table fallback; it resolves entirely via its manifest .languages native row + plugin-declared .language_builds). Only the runtime-output-staging + toolchain-probe seams were csharp-named. Residue (documented, RFC honest boundary): the runtime assembly-dir env var NAME (LABELLE_CS_ASSEMBLY_DIR) stays an assembler-owned constant, and the dev-.csproj IDE aid stays csharp-keyed MSBuild codegen — analogous to the assembler-owned tsconfig codegen; neither is needed for the language-agnostic litmus (which adds an EMBEDDED language). Tests: runtime_output/probe_tools row parse (+ default-false), frozen fallback (csharp opts in, others out), the capability-driven stagesRuntimeOutputs. csharp_splice_tests (#644's e2e) stay green via the frozen fallback — the back-compat proof. Baseline green (flow_catalog lanes). Claude-Session: https://claude.ai/code/session_011szWvquoss1yNX7KWSKCaM
cb73c86 to
9149be1
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9149be10bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // vocabulary — bit-identical to the pre-#619 closed-table behavior. | ||
| var vocab = language_policy.Vocabulary.EMPTY; | ||
| defer vocab.deinit(); | ||
| if (declared) |d| { |
There was a problem hiding this comment.
When a row-only scripting plugin is attached but the project forgets .params.language, this branch leaves vocab empty, so the later scanUnitLanguageDirs(..., declared = null, &vocab) only knows the frozen extensions. A project with scripts/behavior.qk and a plugin manifest declaring the quokka row then generates without the MissingScriptingPlugin error that a built-in .lua file would get, silently ignoring the script. Consider loading the attached scripting plugin's rows for the no-declaration scan, or otherwise failing row-language sources when no language is declared.
Useful? React with 👍 / 👎.
| for (vocab.rows) |r| { | ||
| if (isSupportedLanguage(r.name)) continue; // already covered as a built-in above | ||
| try langs.append(allocator, .{ .name = r.name, .extensions = r.extensions, .dir = r.name }); |
There was a problem hiding this comment.
If a manifest repeats the same new language name, e.g. two quokka rows for .qk and .qq, this loop adds both entries to the scan vocabulary. walkSharedCollect skips every entry whose name matches the declared language, while PluginManifest.languageRow() and the splice use only the first row, so scripts/foo.qq passes policy but is never collected or embedded. Reject duplicate row names or merge them consistently before building the vocabulary.
Useful? React with 👍 / 👎.
| for (parsed.languages) |row| { | ||
| if (!language_policy.isLanguageIdentifier(row.name)) { |
There was a problem hiding this comment.
When a native language row supplies an escaping .stage_subdir such as ../../.., this validation still admits the manifest. For a project with native sources and the required module root, stageNativeSources later joins that value under the staged plugin package and hands the result to scanner.linkDirAbs, which reconciles the destination with deleteTree before linking; an escaping row can therefore replace paths outside .labelle/.../deps/labelle-<plugin>. Validate native staging paths as safe relative subpaths before accepting the row.
Useful? React with 👍 / 👎.
| if (if (maybe_scripting) |s| s.probe_tools else false) { | ||
| try scripting_csharp.ensureStepToolsOnPath(allocator, plugin.name, dl, lang_steps_slice); |
There was a problem hiding this comment.
When the selected language is csharp but no scripting splice is detected, this guard is now false even though the surrounding loop still loads and wires matching .language_builds for every cfg_modules plugin. For example, a non-scripting plugin declaring .params.language = "csharp" with a dotnet language build used to get the generate-time missing-tool diagnostic from the old dl == "csharp" check, but now the same missing SDK fails later as a child-spawn error during zig build. Keep the frozen csharp probe independent of maybe_scripting, or derive the probe capability from the selected language build entry.
Useful? React with 👍 / 👎.
1 participant
Opens the assembler's language vocabulary so a language plugin can add a brand-new language (the RFC's "python") through a
plugin.labelle.languagescapability row alone — zero assembler changes. This closes the last per-language hardcode the already-shipped row-driven declare/transpile/embed/native machinery (runGenericDeclarePhase,spliceFromRow; #621–#624) sat on top of.Part of labelle-toolkit/labelle-engine#237; RFC-LANGUAGE-PLUGINS rev 16 §7 "the language-agnostic assembler".
What already existed vs what this PR adds
Already on
main(not rebuilt here):runGenericDeclarePhase), embed/native/transpile splice (spliceFromRow) — ts/crystal already ride manifest rows.DECLARE_RUNNERS/EMBED_LANGUAGES/NATIVE_LANGUAGES/TSC_PLATFORMStables demoted to fallback defaults for row-less (pre-migration) manifests.LanguageRowmanifest schema (.name/.extensions/.kind/.module_root/.stage_subdir/.declare/.transpile).The gap this PR fills — the language VOCABULARY axis. Before this PR, the closed
SUPPORTED_LANGUAGES/scriptExtensionstable was consulted by the policy gate, the shared-dir scan, the plugin-event consumption scan, andrequires_languagevalidation, and it rejected any language it had never heard of before the row machinery ever ran — so the litmus was impossible. Now:language_policy.Vocabulary— the project's OPEN language set: frozen built-ins ∪ the declaring scripting plugin's manifest.languagesrows. Rows are PRIMARY (a row shadows a frozen language's extensions); a row language the tables never heard of is admitted like a built-in.EMPTY(no rows) reproduces pre-Language capability rows in plugin.labelle — make the assembler language-agnostic #619 behavior bit-for-bit.validateDeclaredLanguage— the vocabulary gate, re-homed fromresolveProjectLanguageintogenerate_phases.validateLanguagePolicywhere the manifest rows are in hand. Self-describing capabilities replace the closed table: a pointed error naming both built-ins and manifest rows, no version compare.resolveProjectLanguage/ manifest-loadrequires_languagenow gate on SHAPE only (isLanguageIdentifier—[a-z][a-z0-9_]*, the one constraint the generated.language = .<name>enum literal genuinely imposes).scanUnitLanguageDirs/checkRequiresLanguagetake theVocabularyand police row languages by the row's extensions.filterConsumedEvents(extra_extensions)) so a row language's.pysources are not invisible to the scan and its subscriptions silently elided.Migration map (hardcoded path → row-driven)
SUPPORTED_LANGUAGEScheck inresolveProjectLanguageVocabulary.isKnown(frozen ∪ rows) invalidateDeclaredLanguagescriptExtensions-only dir/event policingVocabulary.extensionsOf(row extensions primary, frozen fallback)requires_languagerejectrunGenericDeclarePhase/spliceFromRowrows (unchanged); frozen tables stay fallbacksBack-compat proof
LUA_ROW_BACKCOMPATgenerates a lua project twice — frozen fallback (no row) vs an equivalent.languagesrow — and asserts themain.zig+build.zigare byte-identical. Migrating lua/ruby onto rows shifts nothing on output. Every existing golden/e2e suite is unchanged.Litmus evidence
test/quokka_litmus_tests.zig— a fake embedded-VM language "quokka" (.qk) the assembler learns ENTIRELY from a manifest row:scripting.registerScript("behavior", @embedFile("scripts/behavior.qk")),.language = .quokkadep arg, the alias +scripting_enabledflag, staged embed source..rb) in the quokka project still errorsScriptLanguageMismatch— the combined (frozen ∪ row) vocabulary polices both ways."quokka"and".qk"appear in NOsrc/**/*.zig— the mechanical proof the generate rode rows, not a hidden branch. (The fixture is named "quokka" rather than literally "python" precisely so this substring proof is unambiguous — "python" already appears in assembler source as an illustrative RFC comment.)Coordination
Rebased over
#641(consumption-filter precision — resolved theWalk/filterConsumedEventssignature conflict, keeping both.gated_downgradeand the new.extra_extensions). Open siblings #639/#640/#642 don't touch the vocabulary/row layer. Merges LAST.Deviation
The csharp dev-
.csprojIDE aid (#617) still branches ons.language == "csharp"inroot.zig— a dev convenience (not a generate-correctness path), analogous to the RFC's assembler-owned "honest boundary" codegen residue (tsconfig, d.ts sidecar). Not row-ified here; the embedded-language litmus is unaffected. Row-ifying it would need a.dev_projectcapability — out of scope for this PR.Tests: full suite green except the pre-existing
flow_catalogenv-lane baseline (2 failures present on cleanorigin/main).https://claude.ai/code/session_011szWvquoss1yNX7KWSKCaM
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.Update — rebased onto current main + #644 csharp migration (final)
Rebased over the whole backlog sweep now on main (#640 debug, #645 showcase, #644 csharp EMBED, #642 pack-dirs watch, #639 sidecar prune). One conflict (build.zig test list) — kept both
csharp_splice_tests.zigandquokka_litmus_tests.zig.#644 (csharp EMBED) migration — what was already generic vs what I row-ified:
.languagesnative row (kind = .native,module_root/stage_subdir) + plugin-declared.language_buildsdotnet publish. The actualscripts/*.cscompile ridesspliceFromRow+stageNativeSources. Confirmed generic — left untouched.stagesRuntimeOutputsno longer hardcodeslanguage == "csharp"— it consults a newLanguageRow.runtime_outputcapability (the link-less build outputs ARE the runtime payload → stage beside the binary + set the run-step assembly-dir env).ensureStepToolsOnPathcall is gated on a newLanguageRow.probe_toolscapability instead ofdl == "csharp".ScriptingSplice.detect(row PRIMARY ∪ frozen fallback) and threaded onto the splice. The csharp knowledge is demoted to a frozen fallback (scripting_csharp.frozenRuntimeOutput/frozenProbeTools) — the same Language capability rows in plugin.labelle — make the assembler language-agnostic #619 pattern asDECLARE_RUNNERS/NATIVE_LANGUAGES. A new runtime-loaded language declares.runtime_output/.probe_toolsin its row and stages with zero assembler changes.LABELLE_CS_ASSEMBLY_DIR) stays an assembler-owned constant, and the dev-.csprojIDE aid stays csharp-keyed MSBuild codegen — analogous to the assembler-owned tsconfig codegen. Neither is needed for the language-agnostic litmus (which adds an EMBEDDED language, not a runtime-loaded native one).Back-compat proof:
#644'scsharp_splice_tests.zige2e stays green unchanged — its fixture manifest carries NO.runtime_output/.probe_tools, yet csharp still stages runtime outputs + probes the toolchain via the frozen fallback. That IS the byte-identical migration proof for csharp. The quokka litmus +LUA_ROW_BACKCOMPATbyte-identity remain green.Tests:
runtime_output/probe_toolsrow parse (+ default-false), frozen-fallback (csharp opts in, other langs out), capability-drivenstagesRuntimeOutputs. Full suite green except the pre-existingflow_catalogenv-lane baseline (2 failures on clean origin/main).