clusterd-test-driver: add explain verb to assert optimized plan shape#182
Closed
antiguru wants to merge 4 commits into
Closed
clusterd-test-driver: add explain verb to assert optimized plan shape#182antiguru wants to merge 4 commits into
explain verb to assert optimized plan shape#182antiguru wants to merge 4 commits into
Conversation
…rializeInc#37083) reopening MaterializeInc#36773 ---- Adding docs for: - GCP connection - GCP Lakehouse/BigLake Iceberg Catalog Connection _(BigLake is still the name of the API and everything in the GCP console, but it lives under a Lakehouse umbrella now.)_ Small changes to docs for Iceberg sink.
Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Pranshu Maheshwari <pranshu.maheshwari@materialize.com> Co-authored-by: Pranshu Maheshwari <maheshwarip@users.noreply.github.com>
…ute tests (MaterializeInc#37008) ### Motivation Running a small compute experiment today means standing up a full `environmentd` — the SQL layer, the catalog, the coordinator — even when all you want is to hand `clusterd` a few commands and watch what it does. That is slow to set up and hard to control precisely. This adds `mz-clusterd-test-driver`: a headless frontend that speaks the compute protocol to `clusterd` directly, with no `environmentd`. A test drives it from a text script, so it controls the exact persist state, the exact commands the replica sees, and the exact timestamps. Design doc: `doc/developer/design/20260612_headless_clusterd_test_driver.md`. ### What a script looks like Each stanza is a command followed by a `----` block holding its expected output. That block *is* the assertion, and `REWRITE=1` regenerates it in place. Here is the whole lifecycle of a `count(*)` materialized view over a persist shard, read back at the end: ``` create-instance ---- ok initialization-complete ---- ok write-single-ts shard=events ts=0 count=3000 ---- wrote 3000 define-schema name=count_out count bigint ---- ok create-dataflow name=count-mv as-of=0 import source=1000 shard=events upper=1 build id=2000 Reduce aggregates=[count(*)] Get u1000 export kind=materialized-view sink=2001 on=2000 shard=mv schema=count_out ---- ok schedule id=2001 ---- ok allow-writes id=2001 ---- ok peek id=2001 schema=count_out ts=0 ---- 3000 ``` A command that fails renders as `error: <message>`, so an expected failure is just another golden block — there is no special assertion command. Because the waits are level-triggered on monotonic frontiers, the order a script waits in does not change the result, so a single sequential script stays deterministic. ### How it works The crate is a generic mechanism, a dataflow builder, and the scripting layer on top. * The mechanism hosts the persist PubSub server, connects over CTP (sending only `Hello`), and exposes a `Driver` that sends any `ComputeCommand`, submits dataflows without auto-scheduling, watches frontiers, and peeks. * `DataflowBuilder` takes generic parts — persist imports, MIR objects to build, and index/materialized-view/subscribe exports — and runs the real MIR → LIR → `RenderPlan` lowering, because a `RenderPlan` can't be hand-built outside `mz-compute-types`. A `build`'s computation is written in the `mz-expr-parser` `.spec` syntax (`Reduce aggregates=[count(*)]` over `Get u1000`) rather than a bespoke vocabulary, since `MirRelationExpr`'s own serde isn't hand-authorable (`Row` literals are opaque bytes). * `create-dataflow` is the one abstraction behind index, materialized-view, and subscribe exports (`copy-to` isn't implemented yet). A materialized view is read back by `peek`ing the sink id — that becomes a persist peek of its output shard, the same path `SELECT * FROM mv` takes — and a subscribe streams responses that the driver buffers and `await-subscribe` drains. Dataflows start read-only, so a sink needs `allow-writes` before its writes land. * The handshake is explicit (`create-instance`, optional `update-configuration`, `initialization-complete`), and `reconnect` re-runs it without `initialization-complete` to exercise reconciliation. ### Verification `mzcompose` runs each scenario script against a real `clusterd` and fails on any golden mismatch; the scenarios are index, deep-history, side-effects, multi-dataflow, reconciliation, error-behavior, reduce, materialized-view, and subscribe. Unit tests cover the direct-write round trip, the lowered dataflow structure, and the script parser, and `run-local.py` runs the same scripts on the host (with `REWRITE=1`) without docker images. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Per review (DAlperin): a script using `optimize` asserts only the result, so optimizer or lowering drift could silently change the plan under test. `explain` renders the lowered LIR plan (the `EXPLAIN PHYSICAL PLAN` form, via a no-catalog `DummyHumanizer` so ids are `u<n>` and columns `#n`) as its golden, submitting nothing. It takes the dataflow either inline (the same body as `create-dataflow`) or by reference: `explain ref=<name>` renders a dataflow a prior `create-dataflow name=<name>` declared, without repeating its body. `join.spec` declares the join, then `explain ref=join` asserts the differential-join plan alongside the count. The multi-object render separates objects with blank lines, so the `.spec` format gains the `datadriven` doubled-`----` block form, emitted automatically by `REWRITE`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
antiguru
force-pushed
the
headless-driver-explain
branch
from
1c73c26 to
3c46acd
Compare
Owner
Author
|
Superseded by MaterializeInc#37141 — retargeted to upstream/main now that MaterializeInc#37008 merged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on MaterializeInc#37008 (base is its branch; retarget to upstream/main once it merges).
Addresses DAlperin's review on
join.spec: a script usingoptimizeasserts only the result, so optimizer or lowering drift could silently change the plan under test.explaintakes the same body ascreate-dataflowbut renders the lowered LIR plan (theEXPLAIN PHYSICAL PLANform, via a no-catalogDummyHumanizerso ids areu<n>and columns#n) as its golden, submitting nothing. The multi-object render contains blank lines, so the.specformat gains thedatadrivendoubled-----block form, emitted automatically byREWRITE.join.specnow asserts the differential-join plan alongside the count.🤖 Generated with Claude Code