W-23530974: Add TCK conformance harness for the Node.js binding by mlischetti · Pull Request #135 · mulesoft/data-weave-cli

mlischetti · 2026-07-21T21:05:58Z

What

Fills the empty tck lane with a DataWeave conformance harness for the @dataweave/native binding. Mirrors the CLI's TCKCliTest: replays the runtime's own test corpus against the binding, in-process via run().

Structured as five commits: harness core → sourcing + driver → CI wiring → do-block separator fix → guarded output-directive replace.

Changes

tests/tck/formats.ts — extension ↔ MIME map for formats this dwlib compiles in (YAML isn't, so yaml/yml cases are unsupported), plus familyForMime() for directive comparison.
tests/tck/case-loader.ts — pure parseCase(): folder listing → scenarios or a skip decision, with the CLI's structural filters. Unit-tested.
tests/tck/compare.ts — extension-dispatched semantic comparison: structural JSON, structural XML via fast-xml-parser (ignores attribute quoting, self-closing tags, and xmlns declaration placement), EOL-normalized text, byte-compare for bin. Unit-tested.
tests/tck/transform.ts — ensureOutputDirective() normalizes the transform's output format to the expected extension: splits on the column-0 --- (so an indented do {} separator isn't mistaken for the body split) and replaces a conflicting output mime (guarded so it won't clobber output :Type json selectors or multipart/* subtypes). Unit-tested.
tests/tck/tck.test.ts — driver: discover → parse → normalize → run → compare; skips ignore-listed cases; no-ops when the corpus isn't staged (source-only runs pass).
tests/tck/ignore-list.ts — empirically-seeded skips grouped by root cause, overlapping the CLI's ignoreTests().
native-lib/build.gradle — stageTckSuites downloads the runtime + core-modules :test@zip artifacts and unzips into tests/tck/suites/ (gitignored). Not in nodeTest.
.github/workflows/main.yml — master-only TCK step, mirroring the native-cli regression gating.

Testing

npm run test:unit → 119 passed (incl. TCK core)
npm run test:integration → 35 passed
tsc --noEmit clean

Conformance coverage

Strengthening the directive normalizer recovered 20 cases with zero regressions:

npm run test:unit → 119 passed (incl. TCK core)
npm run test:integration → 35 passed
npm run test:tck → 660 passed, 57 skipped, 0 failures against the staged corpus
tsc --noEmit clean

Conformance coverage

Strengthening the directive normalizer recovered 20 cases with zero regressions:

Stage	TCK pass	skip
Initial harness	640	77
+ column-0 separator (do-block)	650	67
+ guarded directive replace	660	57

The remaining 57 skips are environmental limits, not harness gaps: modules/Java not compiled into this dwlib, dw::Runtime behavior, multipart write edges, nondeterministic timestamps, and a few residual XML/DWL serialization differences — consistent with the CLI's own ignoreTests().

Notes

Adds fast-xml-parser (devDependency) — required for semantic XML comparison; a whitespace-strip compare mis-flagged ~52 XML cases.
The harness does text-level directive normalization rather than the CLI's JVM CodeGenerator AST rewrite; the residual directive-shaped skips would need that rewriter (or a run() output-MIME override) and are left as documented skips.

🤖 Generated with Claude Code

First of three commits for the Group C conformance harness. Pure, dwlib-free logic that lands in the unit lane; the suite sourcing + driver and CI wiring follow in later commits. - tests/tck/formats.ts: extension ↔ MIME mapping limited to the formats this dwlib compiles in (json, xml, csv, txt, dw, octet-stream, properties, urlencoded, multipart). YAML is not compiled in, so yaml/yml cases are treated as unsupported. - tests/tck/case-loader.ts: parseCase() turns a case directory's file listing into runnable scenarios or a skip decision, mirroring the CLI's structural filters (single transform.dwl, no *-config.properties, no groovy/java, no bare config.properties, no _wip) and dropping unsupported-format scenarios. - tests/tck/compare.ts: compareOutput() does extension-dispatched semantic comparison ported from AssertionHelper — structural JSON deep-equal, whitespace-collapsed XML, EOL-normalized csv/txt/yaml, all-whitespace-stripped dwl, exact bytes for bin. Not naive byte equality. - 33 unit tests (tck-case-loader.test.ts, tck-compare.test.ts). Unit lane: 70 -> 103 tests passing. tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Second commit for Group C: source the conformance corpus and run it. - native-lib/build.gradle: stageTckSuites task downloads the runtime and core-modules :test@zip artifacts at the pinned weaveTestSuiteVersion and unzips each into node/tests/tck/suites/<suite>/ (gitignored). Not wired into nodeTest (PR builds); invoked on demand and by master CI. - tests/tck/transform.ts: ensureOutputDirective() appends an output directive matching the expected file's format when the header lacks one (bare-body and headered transforms both handled). do-block headers and transforms that pin a conflicting output are out of scope for text rewriting and are skipped. - tests/tck/compare.ts: XML comparison upgraded from whitespace-strip to a structural compare via fast-xml-parser — ignores attribute quote style, self-closing vs explicit-close tags, and xmlns declaration placement. This recovered ~52 XML cases that the naive string compare mis-flagged. Adds the fast-xml-parser devDependency. - tests/tck/tck.test.ts: driver — discovers cases across staged suites, parses, normalizes, runs via run(), compares. Skips ignore-listed cases with reasons; no-ops (describe.skip) when the corpus is not staged so source-only runs pass. - tests/tck/ignore-list.ts: empirically-seeded skips grouped by root cause (unresolved-module, java, do-block, output-directive-mismatch, dwl-output-format, multipart, nondeterministic, coercion/runtime, residual xml), overlapping the CLI's ignoreTests(). - Unit tests for transform.ts and the XML compare path. TCK lane: 640 cases pass, 77 skipped, 0 failures against the staged corpus. Unit lane: 103 -> 113. tsc clean; all lanes green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add a master-gated CI step that stages the DataWeave test-suite corpus and runs the Node tck vitest project against the built package. Gated with if: github.ref == 'refs/heads/master', mirroring the native-cli regression steps, so PRs keep running only the fast unit/integration lanes while the heavier conformance replay runs post-merge. ci.yml (the weekly latest-version build) is intentionally left unchanged, as it does not run the native-cli TCK regression either. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…separator ensureOutputDirective split the transform on the first `---` matched with trim(), which also matched the indented `---` inside a `do { … }` block — so the output directive was appended mid-do-block, failing with "Invalid output directive in do block". Match only a column-0 `---` (the document body separator); a do-block's indented `---` is left alone, and a transform whose only `---` is inside a do-block is treated as a bare body. - transform.ts: detect the separator with /^---\s*$/ instead of trim()==="---". - ignore-list.ts: 10 do-block cases now run (do-1, do-block, do-overload, csv-streaming, range-selector, etc.); the remaining try-handle-* and private_scope_directives fail on unresolved-module and are regrouped there. - 2 new unit tests for the column-0 vs do-block separator handling. TCK lane: 640 -> 650 passing, 77 -> 67 skipped, 0 failures. Unit lane: 115. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…recover TCK cases ensureOutputDirective now also replaces a pinned output mime when it targets a different format family than the expected file, matching what the CLI's AST rewriter does. Previously it only appended when the header had no directive, so transforms pinning e.g. `output application/json` with an expected `out.xml` were skipped as "output-directive-mismatch". The replace is guarded to avoid corrupting directives that were already correct: it only rewrites a plain `mime/type` token whose family differs from the target, leaving `output :Type json` selectors and `multipart/*` subtypes (with their boundary= options) untouched. Verified across the full corpus: +10 recovered, 0 regressions. - formats.ts: add familyForMime() (collapses multipart/* subtypes; maps known mimes to their extension key). - transform.ts: guarded mime replacement; refreshed docs. - ignore-list.ts: drop the 10 recovered cases (recursive_mapObject, xml-root-with-text, groupby-complex, tree-filter*, bad-inline, string_interpolation_selection, overload-functions, dynamic_attribute_name); regroup the dw::Runtime cases under their real cause. - 4 new/updated transform unit tests (replace, option preservation, multipart subtype guard, :Type selector guard). TCK lane: 650 -> 660 passing, 67 -> 57 skipped, 0 failures. Unit lane: 119. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

mlischetti and others added 3 commits July 21, 2026 16:41

mlischetti requested a review from a team as a code owner July 21, 2026 21:05

mlischetti and others added 2 commits July 22, 2026 08:52

martincousido approved these changes Jul 22, 2026

View reviewed changes

mlischetti merged commit c19f5d6 into master Jul 22, 2026
4 checks passed

mlischetti deleted the W-23530974-group-c-tck-harness branch July 22, 2026 18:14

mlischetti mentioned this pull request Jul 22, 2026

W-23530974: Normalize CRLF in TCK JSON string comparison #136

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

W-23530974: Add TCK conformance harness for the Node.js binding#135

W-23530974: Add TCK conformance harness for the Node.js binding#135
mlischetti merged 5 commits into
masterfrom
W-23530974-group-c-tck-harness

mlischetti commented Jul 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mlischetti commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Changes

Testing

Conformance coverage

Conformance coverage

Notes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mlischetti commented Jul 21, 2026 •

edited

Loading