Skip to content

feat: DuckDB engine support (REF-290)#10

Merged
therecluse26 merged 4 commits into
mainfrom
feature/duckdb
Jul 20, 2026
Merged

feat: DuckDB engine support (REF-290)#10
therecluse26 merged 4 commits into
mainfrom
feature/duckdb

Conversation

@therecluse26

Copy link
Copy Markdown
Collaborator

Summary

Adds DuckDB as a fourth first-class engine with 100% feature parity with the existing engines (PostgreSQL, MySQL, SQLite), plus a full offline test suite.

Engine (src/engine/duckdb)

  • validate_connection, introspect, execute via the duckdb crate (bundled, no system dependency)
  • Read-only defense in depth: file-backed databases open with AccessMode::ReadOnly — DuckDB itself rejects writes even if the parser were bypassed (covered by a session-layer test). :memory: cannot be opened read-only and relies on the capability parser.
  • Introspection via duckdb_tables()/views()/columns()/constraints()/indexes(): databases, schemas (DuckDB supports real schemas, default main, --schema filter honored), tables, views, composite PK/FK, unique indexes, table/column comments, and row estimates
  • Type mapping: HUGEINT/DECIMAL → string (precision-preserving), DATE/TIME/TIMESTAMP → ISO-8601, BLOB → Base64, LIST/STRUCT/MAP/ENUM/UNION → recursive JSON
  • Timeouts via InterruptHandle + timer thread → QUERY_TIMEOUT
  • Structured EXPLAIN via EXPLAIN (FORMAT JSON) normalized to ExplainPlanNode

Capability layer

New DuckDB dialect validator: SELECT/CTE/SHOW/DESCRIBE/SUMMARIZE/EXPLAIN/transaction control allowed; PRAGMA allowlist (setter forms rejected); COPY/ATTACH/DETACH/INSTALL/LOAD/EXPORT and all DML/DDL rejected with CAPABILITY_VIOLATION.

Surface wiring

  • duckdb: DSN scheme (same path forms as sqlite:, incl. duckdb::memory:)
  • CLI (connect wizard + flags, introspect, query), MCP server tools/schemas, and schema-diff dispatch
  • duckdb cargo feature included in default all-engines

Output contract

Additive only: "engine": "duckdb" in envelopes. No changes to src/output.rs or existing schemas (schema_drift green).

Test evidence

  • cargo test — full suite green, including 24 new engine unit tests, 14 capability tests, 7 DSN tests, and 40 integration tests in tests/duckdb_parity.rs (mirrors the SQLite parity matrix: connect, introspect, allowed/denied queries with state-unchanged verification, max_rows/offset/timeout safety, envelope determinism)
  • cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings — clean
  • cargo build --all-targets + cargo build --release — clean
  • scripts/test-live.sh — MySQL 8.0/8.4 + PostgreSQL 16 live suites green (connection-resolution paths touched)
  • Manual CLI end-to-end: success envelope, write denial, introspect, and read-only open all verified against the compiled binary

🤖 Generated with Claude Code

therecluse26 and others added 3 commits July 19, 2026 22:09
- New src/engine/duckdb module implementing DatabaseEngine:
  validate_connection, introspect (databases/schemas/tables/views/
  indexes/table details incl. comments + row estimates via duckdb_*
  catalog functions), and capability-enforced execute
- File-backed connections open with AccessMode::ReadOnly (defense in
  depth); :memory: opens read-write with the capability parser as the
  enforcement boundary
- Rich type mapping: HUGEINT/DECIMAL stringified to preserve precision,
  DATE/TIME/TIMESTAMP as ISO-8601, BLOB as Base64, LIST/STRUCT/MAP/
  ENUM/UNION converted recursively to JSON
- Statement timeouts via InterruptHandle + timer thread -> QUERY_TIMEOUT
- Structured EXPLAIN via EXPLAIN (FORMAT JSON) normalized to
  ExplainPlanNode
- Read-only validator for the DuckDB dialect: SELECT/CTE/SHOW/DESCRIBE/
  SUMMARIZE/EXPLAIN/transaction control plus a PRAGMA allowlist;
  COPY/ATTACH/INSTALL/LOAD/EXPORT and all DML/DDL rejected
- duckdb: DSN scheme (same path forms as sqlite:)
- CLI, MCP server, and schema-diff dispatch wired for the new engine

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
40 integration tests mirroring the SQLite parity matrix (REF-278):
connect, introspection (composite PK/FK, unique indexes, views,
schemas, native types), allowed reads (SELECT/EXPLAIN/SHOW/DESCRIBE/
SUMMARIZE/PRAGMA/transactions), denied writes with state-unchanged
verification, max_rows/offset/timeout safety, and JSON envelope
determinism. Runs offline - no Docker required.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
…op safety (REF-290)

- MCP query tool's engine description now lists duckdb alongside the enum
- Timeout timer thread comment now cites the duckdb crate contract that
  interrupt() after connection drop is a no-op (verified against crate source)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
@therecluse26

Copy link
Copy Markdown
Collaborator Author

CTO review — approved. Reviewed the full diff (capability layer, DSN parsing, CLI/MCP wiring, 1500-line engine module, parity suite). No blockers or majors — read-only enforcement is properly layered (dialect validator + AccessMode::ReadOnly storage backstop for file DBs), no unsafe code, no runtime panics, max_rows/timeout enforcement matches the SQLite engine pattern, and the 40-test parity suite is deterministic with state-unchanged verification on denied writes. Verified locally: build / fmt / clippy -D warnings / full test suite all green. Pushed one review commit (5136eba) fixing a stale MCP tool description and documenting the InterruptHandle-after-drop safety contract (verified against duckdb crate source). Known parity-consistent minors (interrupt during prepare surfaces as QUERY_FAILED not QUERY_TIMEOUT; no BLOB param binding) match existing SQLite behavior — tracked in REF-290 notes. (Posting as comment: GitHub blocks formal self-approval since the branch shares my git identity.)

@therecluse26
therecluse26 merged commit c72ebba into main Jul 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant