feat(cts): add spec-driven contract test suite for Python, Java, and Rust clients

[XuQianJin-Stars]

This PR introduces a two-layer contract test suite (CTS) for the Lance Namespace REST API:

1. A wire-level layer that exercises every generated client (Python urllib3, Java apache, Java async, Rust reqwest) against a WireMock standalone server, with both the per-client tests and the stub mappings code-generated from the OpenAPI spec.
2. A behavioural layer that drives the in-process Rust harness (lance-namespace-cts) directly against DirectoryNamespace, replaying YAML-authored contracts from docs/src/cts-contracts/.

Both layers are produced by code generators and never mutate the source spec.

Summary

End-to-end contract testing pipeline driven entirely by code generators on top of the OpenAPI spec and an authoritative behavioural-contract YAML set:

• Wire CTS — per-client test files + WireMock stub mappings are generated from the spec, then replayed against WireMock standalone to verify HTTP serialization/deserialization across all generated language clients.
• Behavioural CTS — per-operation Rust test modules are generated from cts-contracts/*.yaml and run in-process against DirectoryNamespace (no JVM, no WireMock, no Spring Boot), capability-gated so impls advertise what they actually support.

Motivation

Multiple generated clients (Python urllib3, Java apache, Java async, Rust reqwest) and a Spring Boot server are all derived from a single OpenAPI spec. Until now there was no automated mechanism to ensure that:

• Each client correctly serializes requests / deserializes responses for every operation.
• All clients remain mutually consistent in wire format.
• Spec changes don't silently break a particular language binding.
• Server implementations actually satisfy the behavioural contract (pre-conditions, error mapping, idempotency), not just the wire shape.

This PR makes both kinds of regression mechanically detectable: adding a new operation in the spec automatically yields new wire-level tests in every client on the next make gen-cts, and authoring a new entry in docs/src/cts-contracts/*.yaml automatically yields a new behavioural test module on the next make gen-cts-behavior.

Architecture

   OpenAPI spec ─────────────────────────────────────┐
        │                                            │
        ├── examples overlay ── WireMock mappings ──┐│
        │                                           ││
        └── per-client test generators              ││
                  │                                 ││
                  ▼                                 ▼▼
         tests (py / java / rust) ───►  WireMock standalone (replay)
                                                              [wire layer]

   docs/src/cts-contracts/*.yaml ──► gen_contract_tests.py
                                            │
                                            ▼
                       rust/lance-namespace-cts/tests/contracts/*.rs
                                            │
                                            ▼
                     DirectoryNamespace (in-process, capability-gated)
                                                              [behavioural layer]

The source spec is never mutated — example payloads are layered via an overlay file and applied at generation time only. The behavioural-contract YAML is the authoritative spec for everything beyond wire shape.

Generators (the heart of the pipeline)

File	LOC	Purpose
ci/cts/gen_wiremock_tests.py	2009	Generates per-operation WireMock-driven tests for Python urllib3, Java apache, Java async, and Rust reqwest clients from Mustache templates
ci/cts/gen_wiremock_mappings.py	389	Generates WireMock stub mappings from spec example payloads
ci/cts/gen_examples_overlay.py	281	Builds an OpenAPI overlay carrying example payloads for stub generation
ci/cts/apply_overlay.py	107	Applies overlays without mutating the source spec
ci/cts/gen_contract_tests.py	826	Generates the per-operation behavioural-contract Rust test modules from cts-contracts/*.yaml, post-processed with rustfmt --edition 2024
ci/cts/contract_loader.py	407	Parses, validates and resolves cts-contracts/*.yaml against the JSON Schema
ci/cts/lint_contracts.py	317	Strict structural / coverage / capability-gating lint over the contract YAML
ci/cts/capabilities.py	69	Capability registry shared by the loader, lint and the Rust harness

CI and Build Tooling

• Makefile: new targets — wire layer (gen-cts / build-cts / test-cts-wiremock), behavioural layer (gen-cts-behavior / test-cts-behavior), default test-cts runs test-spec-lint + behavioural, plus verify-spec-untouched to fence the source spec.
• ci/spectral.yaml — Spectral lint rules for the OpenAPI spec.
• ci/cts/schemathesis.toml — Schemathesis config for property-based contract testing.
• ci/cts/cts-contracts.schema.json — JSON Schema for the behavioural-contract YAML.
• .github/workflows/contract-tests.yml — CI workflow running the full CTS pipeline (253 lines) with PR-blocking limited to the behavioural job; the WireMock job runs alongside as a wire-level signal.

Generated Wire-Level Client Tests

Auto-generated — do not hand-edit. Re-run make gen-cts instead.

Client	File	LOC	Framework
Python urllib3	python/lance_namespace_urllib3_client/tests/test_wiremock.py	536	pytest + WireMock (dynamic port)
Java apache	java/lance-namespace-apache-client/.../WireMockIT.java	540	JUnit 5 IT
Java async	java/lance-namespace-async-client/.../WireMockIT.java	584	JUnit 5 IT
Rust reqwest	rust/lance-namespace-reqwest-client/tests/wiremock.rs	737	tokio async + WireMock process lifecycle

Each suite exercises every Namespace / Table / Transaction / Tag / Data API operation end-to-end. (Rename note: these files used to be test_contract.py / WireMockContractIT.java / contract.rs; they were renamed to *wiremock* so that the word "contract" can be reserved for the new behavioural layer.)

Generated Behavioural-Contract Tests

Auto-generated — do not hand-edit. Re-run make gen-cts-behavior instead.

• Source of truth: docs/src/cts-contracts/{main,index,namespace,table,tag,transaction,data}.yaml — one entry per operation describing pre-conditions, request/response shape, expected outcomes (success, 4xx, 409, …) and required capabilities.
• Harness crate: rust/lance-namespace-cts/ — Fixtures, ContractCaller, Capabilities, assert_contract_{ok,error}. Uses DirectoryNamespace via cargo path on the sibling lance repository's lance-namespace-impls crate.
• Generated tests: rust/lance-namespace-cts/tests/contracts/*.rs — 43 per-operation modules + a mod.rs wired into tests/cts.rs.
• Local result: make test-cts-behavior → 170 passed.

Dependencies

• Python: pyproject.toml / uv.lock — test deps.
• Java: parent pom.xml, apache-client/pom.xml, async-client/pom.xml — WireMock + JUnit 5.
• Rust: rust/Cargo.toml, rust/lance-namespace-reqwest-client/Cargo.toml (async test deps for the WireMock layer), rust/lance-namespace-cts/Cargo.toml (in-process harness, depends on the sibling lance repo via path).

Documentation

• AGENTS.md: top-level agent-role overview.
• CONTRIBUTING.md: refreshed contributor guide covering both CTS layers.
• New README.md files for apache-client, async-client, springboot-server, urllib3-client, and reqwest-client.
• docs/src/cts-contracts/ is wired into the docs site so the behavioural contracts ship as published reference material.

How to Run Locally

# Wire layer — generated clients vs WireMock
make gen-cts-wiremock     # regenerate stubs + per-client wire tests from the spec
make build-cts-wiremock   # build all clients, then drop generated wire tests into the trees
make test-cts-wiremock    # run the full WireMock suite (Python + Java apache + Java async + Rust)

# Behavioural layer — in-process Rust harness vs DirectoryNamespace
make gen-cts-behavior     # regenerate per-operation Rust test modules from cts-contracts/*.yaml
make test-cts-behavior    # cargo test -p lance-namespace-cts (170 tests)

# Default — what CI blocks on
make test-cts             # = test-spec-lint + test-cts-behavior + test-cts-wiremock

Quality Gates

All green locally on the tip of this branch:

• cargo fmt --check ✅
• cargo clippy -D warnings ✅
• gen_contract_tests.py --check (generator output is a fixed point) ✅
• lint_contracts.py --strict ✅
• make test-cts → 170 passed ✅
• make test-cts-wiremock → 49 (Rust) + 50 (Python) + 49 (Java) ✅

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

[jackye1995]

Thanks for putting this together! The infrastructure work here (WireMock lifecycle, CI matrix, Makefile targets) is solid and could be reusable.

That said, I think we can get more value from a CTS by focusing on behavioral contract testing rather than serialization/deserialization. The generated clients are produced by OpenAPI codegen, so serde correctness is largely the code generator's responsibility — what we really need to validate is that implementations conform to the spec's behavioral contracts.

For example, for CreateNamespace:

1. Creating the root namespace should fail — root always exists
2. Creating an already-existing namespace should fail with ALREADY_EXISTS (error code 2)
3. Creating a non-existing namespace should succeed and the namespace should then be describable

The spec already declares the expected error types for every operation in errors.md. For example, CreateNamespace can return NamespaceAlreadyExists (2), DropNamespace can return NamespaceNotFound (1) and NamespaceNotEmpty (3), etc. A CTS should codify when each of these error codes must be returned — e.g. CreateNamespace with an existing namespace path must return error code 2, DropNamespace on a namespace containing child namespaces must return error code 3. We may want to enrich the spec with these detailed per-operation error conditions first, then generate CTS tests that assert each condition against a real implementation.

Tests could also be parameterized by capability flags (supports_one_level_namespace_path, supports_two_level_namespace_path, etc.) to accommodate differences across implementations.

I'd suggest pivoting toward:

• Defining expected behaviors per operation based on the spec (success cases, error conditions, state transitions)
• Enriching the error spec with detailed conditions for when each error code should be returned
• Generating tests that assert those behavioral contracts
• Running against a reference server implementation rather than canned stubs

Happy to discuss further if you'd like to align on the direction before investing more time.