Multi-database support: bundled SQLite default, plus PostgreSQL / MariaDB / SQL Server - #14
Merged
Conversation
Dispatch is a single-node product — one container in docker-compose, one
systemd unit, a single-VM ARM template, an OVF appliance, and a Windows
installer that bundles an entire PostgreSQL server purely to satisfy this
dependency. It never uses the database as a coordination substrate: the work
queue is the filesystem spool (SpoolWorkerPool claims .eml files via
FileSystemWatcher + per-relay SemaphoreSlim), and SQL is only touched after a
provider responds. Every writer is a thread in one process, which is the shape
SQLite's single-writer model handles well.
Engine is chosen from the connection string, so existing Postgres deployments
need no config change.
* ISqlDialect covers only what has no portable form: interval arithmetic,
date formatting, on-disk size, space reclamation, and bootstrap. Everything
else is written in the subset both engines share.
* Per-engine migrations under Migrations/{Postgres,Sqlite}. SQLite's 0007
rebuilds relay_counters, since SQLite has no ALTER TABLE DROP CONSTRAINT.
* SqliteDialect sets WAL + synchronous=NORMAL once, and busy_timeout,
foreign_keys and case_sensitive_like per connection. The last preserves
Postgres LIKE semantics so a backend swap doesn't silently widen search
results.
* SqliteTypeHandlers works around Dapper materialising records against
SQLite's row-dependent reader schema (TEXT timestamps, Int64 for every
integer, Byte[] for computed columns on empty result sets).
The 33 repository integration tests now run against both engines — DatabaseFixture
defaults to a temp SQLite file when DISPATCH_TEST_SQL is unset, so a developer
with no database runs the full suite instead of silently skipping it. CI should
run it twice, once per engine.
ConcurrentWriteTests asserts the property that actually matters: 16 unthrottled
concurrent writers commit every row with no retry loop and no lock errors.
Locally that measures 5,970 writes/sec on SQLite against 1,121 on containerised
Postgres.
Also pins SQLitePCLRaw 2.1.12; the version Microsoft.Data.Sqlite resolves by
default carries GHSA-2m69-gcr7-jv3q.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build.yml sets DISPATCH_TEST_SQL and DISPATCH_REQUIRE_SQL at job level, so the dual-engine fixture added in c8bedd3 only ever ran Postgres — the SQLite backend and its concurrency guarantee were untested in CI despite that commit claiming the suite should run twice. Clearing both variables for a second Data-only run takes DatabaseFixture down its SQLite path. This is what covers ConcurrentWriteTests on SQLite, which asserts that unthrottled concurrent writers serialise on the write lock rather than failing with SQLITE_BUSY. That depends on WAL and busy_timeout, so it needs verifying on every change. Also sets retention-days on the test-results artifact, which had no expiry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groundwork for the two-mode database story: bundled SQLite by default, or a
database server the operator already runs (PostgreSQL, MariaDB/MySQL, SQL
Server). Four engines with divergent statement shapes — MERGE vs ON CONFLICT vs
ON DUPLICATE KEY, TOP vs LIMIT vs OFFSET/FETCH — is past what the hand-rolled
ISqlDialect should carry, so schema definition moves to EF Core.
DispatchDbContext is the single model; table and column names are pinned to the
snake_case names the existing hand-written migrations created, because live
PostgreSQL deployments hold data under those names.
EF Core 9, not 10: Pomelo is the only provider with real MariaDB support (Oracle's
connector has dropped MariaDB compatibility) and has no EF Core 10 release. EF 9
runs on net10.0 and all four providers exist at 9.x.
Provider-conditional index filters, since filtered indexes are raw SQL and each
engine spells them differently:
* Postgres/SQLite "is_default", "NOT revoked", "api_key_id IS NOT NULL"
* SQL Server "[is_default] = 1", "[revoked] = 0", ...
* MySQL/MariaDB no filtered indexes; IX_relays_default is omitted and the
single-default invariant stays enforced in SqlRelayRepository
Verified by applying all four migrations to live engines — postgres:17,
mariadb:11, mssql/server:2022 (under emulation on arm64), and a SQLite file —
producing the same eight tables on each. Timestamps map natively per engine:
timestamptz / datetime2 / datetime(6) / ISO-8601 TEXT.
Additive only. The Dapper repositories are untouched and all 325 tests still
pass on both currently-wired engines; the repository port is the next step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verified by applying both to live PostgreSQL databases and diffing
information_schema. Columns were already identical (83/83); indexes were not,
and the differences were not cosmetic:
* api_keys had ONE filtered unique index on key_id instead of two separate
indexes. EF identifies an index by its property set, so the two unnamed
HasIndex calls intended as "unique across all keys" and "partial lookup for
live keys" MERGED into a single unique index filtered to NOT revoked — which
would have allowed a revoked key's id to be reissued. Fixed with the named
HasIndex overload, which keeps them distinct.
* Every logged_at index had lost its DESC ordering. The Message Log and audit
log always read newest-first, so ascending indexes would mean reading ranges
backwards or sorting. Restored via IsDescending on relay_log, audit_log and
relay_counters.
* The three covering indexes had lost their INCLUDE payloads, so the two hot
list queries would do a heap lookup per matched row. Reapplied for Postgres
and SQL Server via the provider annotations; SQLite and MySQL/MariaDB have no
equivalent and keep just the key columns.
Indexes are now 27/28 identical. The one addition is an EF-convention index on
routing_rules.relay_id, which the hand-written schema lacked — additive and
useful, so kept.
That equivalence is what makes an upgrade path possible for existing PostgreSQL
deployments: the initial migration can be recorded as already-applied rather than
re-creating tables that hold live data.
Also adds DatabaseProviderResolver. Sniffing alone cannot separate SQL Server from
MySQL — "Server=h;Database=d;User Id=u" is valid for both — so an explicit
Database:Provider setting always wins and a genuinely ambiguous string throws with
instructions rather than guessing and failing later at an opaque syntax error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SQLite is the default backend, so its behaviour at the volume a busy relay
reaches in a month is a product question, not a curiosity. ConcurrentWriteTests
already covers whether parallel writers serialise gracefully; this covers what
only appears at size.
Opt-in via DISPATCH_VOLUME_TEST (writes hundreds of thousands of rows, takes
minutes). DISPATCH_VOLUME_ROWS sets the target, default 250,000.
Assertions are deliberately loose — orders of magnitude, not milliseconds — so it
fails on a collapse (a scan where a seek was intended) rather than on a slow CI
agent. The printed numbers are the deliverable.
Measured locally at 250k and 2M rows:
250k 2M scaling
bulk load 60,724/s 54,877/s flat
file size 180 MiB 1,487 MiB linear, ~780 bytes/row
first page 16ms 17ms FLAT
page 20 0ms 0ms FLAT (keyset cursor holds)
indexed filter 0ms 0ms FLAT
subject LIKE 72ms 755ms linear (unindexed by design, see 0005)
count(*) 4ms 1,343ms linear
purge 1.27M rows - 164,711ms
The indexed read paths are flat across an 8x volume increase, which is the result
that matters: the Message Log stays instant regardless of history size.
Two characteristics worth knowing rather than fixing:
* ~780 bytes/row means 10M rows is roughly 7.4 GB. The size-pressure cap is the
control for this, and it is off by default.
* Purge is dominated by its own inter-batch delay: 1,267 batches x 100ms is 127s
of the 165s. That delay is not waste on SQLite — it is what lets ingest take
the single write lock while a purge runs — but it does cap purge at ~10k
rows/sec, so a large backlog clears in minutes, not seconds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ng databases
DatabaseInitializer now applies EF migrations instead of the hand-written .sql
runner, and handles three cases: a database already on EF migrations, an empty
one, and — the risky one — a pre-EF database created by the old runner. Those hold
live mail history, so re-creating their tables would be destructive; the baseline
migration is recorded as already-applied instead. LegacySchemaAdoptionTests builds
a genuine pre-EF database from the original .sql scripts (kept embedded for that
purpose), puts a row in it, and asserts the row survives, the schema reads as
current, and a second run is a no-op.
Two bugs found by extending the schema diff to cover column defaults, which the
earlier comparison had missed entirely:
* Every timestamp column had lost its DEFAULT, so any insert not naming
logged_at/created_at/updated_at failed on NOT NULL.
* The obvious fix would have been wrong. CURRENT_TIMESTAMP returns LOCAL server
time on SQL Server and MySQL/MariaDB, and UTC only on Postgres and SQLite.
Dispatch stores UTC throughout, so the portable-looking spelling would have
written skewed timestamps on two of four engines — plausible-looking values,
wrong ordering and retention, no error anywhere. Now per-provider: now(),
CURRENT_TIMESTAMP, SYSUTCDATETIME(), UTC_TIMESTAMP().
Columns, defaults and indexes now match the hand-written PostgreSQL schema exactly,
apart from one FK index EF adds by convention.
Scope, stated plainly: the schema layer supports all four engines and their
migrations apply cleanly to live servers. The REPOSITORIES do not — they still run
on Dapper via SqlConnectionFactory, which only implements PostgreSQL and SQLite.
Resolving a SQL Server or MySQL connection string there now throws with that
explanation rather than building an Npgsql connection from it and failing later
with "Couldn't set data source". Porting the repositories is the remaining work.
Also adds MariaDB collation handling: databases are created utf8mb4_bin, because
MySQL and MariaDB default to a case-insensitive collation that would silently widen
every LIKE in the Message Log, tag matching and audit search relative to the other
three engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removes the legacy-schema adoption branch from DatabaseInitializer, the test that exercised it, and the 14 hand-written .sql migrations that only survived as its fixture. Dead recovery code is worse than none — it reads as a tested safety net when nothing can reach it. This also retires a latent bug rather than leaving it: the SQL Server branch of the legacy table probe filtered by table_catalog but not by schema, so a relay_log in any schema would have been read as an existing Dispatch install. EF migrations themselves stay. They cost nothing now and are the only mechanism for changing the schema of an install that already holds mail history, which every deployment becomes the moment it is running. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The one existing install runs the bundled PostgreSQL and needs to land on bundled SQLite in 0.7. That is a data migration, not a schema one, so it is a different problem from the pre-EF adoption path removed in 60518e2 — which stays removed. Both ends share DispatchDbContext, so the copy is entity-by-entity and each provider handles its own type mapping. That makes this engine-agnostic rather than a PostgreSQL-to-SQLite script, at no extra cost. Three things it is careful about, each because getting them wrong fails silently: * Primary keys are preserved. relay_log.id is half the Message Log's keyset cursor and the target of every foreign key; renumbering would break routing-rule and API-key attribution on historical mail with nothing raised. A separate KeyPreservingDbContext switches key generation off — a distinct context type rather than a flag, since EF caches one compiled model per type. * Encrypted config crosses as ciphertext, never decrypted in transit. The key lives in DISPATCH_KEY_DIR and is NOT in the database, so the CLI documents that migrating to another host means carrying that key too. * Row counts are verified against the source afterwards. The copy cannot be one transaction across two engines, so this is what makes "it worked" evidence rather than an assumption. Guarded both ways: refuses a target that already holds data (no sane resolution for two rows claiming one relay_log.id), and checks the SOURCE is a current Dispatch schema first — a wrong connection string otherwise surfaced as a raw 'relation "config" does not exist' partway through migrating production. Exposed as `Dispatch.Service migrate-database --to "<connection string>"`, which reads nothing but the source and never deletes: if the result is wrong, point the connection string back and retry. SQL Server is rejected as a target, since explicit identity inserts need SET IDENTITY_INSERT handling that is neither implemented nor tested. Verified end to end against a populated PostgreSQL database: 502 rows copied, ids 1-500 preserved, foreign keys intact, and sqlite_sequence advanced to 500 so the first post-migration insert does not collide with copied history. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Provider knowledge was spread across six places - ISqlDialect, two dialect
implementations, DatabaseBootstrap, DispatchDbContextFactory and
DatabaseProviderResolver - each knowing a different subset of engines. For an open
source project, "add a database engine" should mean implementing one documented
contract, not finding six.
IDatabaseProvider is now that contract: identification, EF wiring, schema
conventions, bootstrap, and the handful of operations EF cannot express (the
counter upsert, size reporting, space reclamation). DatabaseProviders.All is the
only list of engines; the DbContext, initializer, migrator and test matrix all read
from it, so registering a provider is what makes the shared suite start running
against it.
Implementations for all four engines, with the differences declared rather than
discovered:
* SQL Server uses MERGE ... WITH (HOLDLOCK) for the counter upsert. The hint is
load-bearing, not defensive: without it two workers can both miss the row and
both insert, violating the unique constraint on the contended hot path.
* MariaDB/MySQL uses ON DUPLICATE KEY UPDATE, and is created utf8mb4_bin.
SQL Server is created Latin1_General_BIN2. Both default to case-INSENSITIVE
collations, which would silently widen every Message Log search relative to
PostgreSQL. Every engine is normalised to case-sensitive.
* MariaDB/MySQL is the one engine with no filtered indexes, so IX_relays_default
cannot exist there and the single-default invariant stays in SqlRelayRepository.
That is declared in ProviderCapabilities rather than left as a surprise.
ProviderCapabilities makes varying behaviour explicit, and ProviderConformanceTests
(29 tests, no database required) stops declarations drifting from reality - a
provider claiming filtered-index support while returning no filter would otherwise
pass every functional test while dropping the invariant the index enforces. The
suite also caught a real design flaw: Pomelo's ServerVersion.AutoDetect opens a
connection during Configure, so the provider could not be constructed offline,
breaking design-time scaffolding and any startup preceding a reachable database.
It now falls back to a pinned version.
Licensing, since this is Apache-2.0 and the README promises commercial
redistribution: every data-layer dependency is permissive - Pomelo and
MySqlConnector MIT, Npgsql under the PostgreSQL licence, the Microsoft providers
MIT, Dapper and SQLitePCLRaw Apache-2.0. Oracle's MySql.EntityFrameworkCore, the
alternative to Pomelo, is GPL-2.0 with a FOSS exception and was already rejected on
MariaDB compatibility grounds.
docs/database.md documents choosing a backend, moving between engines, exactly what
differs and why, and how to add one - including the four mistakes that have already
cost us time here.
ISqlDialect survives, marked superseded: the Dapper repositories still use it, which
is why they still work on PostgreSQL and SQLite only. Porting them deletes it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dashes
DatabaseMigratorTests proves the copy preserves rows and keys. This proves the
thing an operator actually does: run the service on PostgreSQL, put real mail
through it via the ingestion API, stop it, run migrate-database, repoint the
connection string, start it again, and find everything still working.
The assertions are chosen for failures that would otherwise be SILENT:
* logging in with the SAME password proves the config table crossed, bcrypt hash
and all - a fresh install would demand first-run setup instead
* the relay keeps its id, so historical mail stays attributed to it
* the TLS state read back from /api/settings is only resolvable by DECRYPTING the
stored certificate password, so it proves ciphertext survived untouched and the
key directory still opens it
* sending a NEW message after cutover proves identity sequences were advanced to
the end of the copied data rather than left at 1, where the first insert would
collide with migrated history
Wired into build.yml so it cannot rot. Also fixes a duplicate retention-days key
that had crept into the artifact upload step.
Separately: this repo has a CI job rejecting em dashes, because a mis-decoded
em-dash byte becomes a smart-quote string delimiter and breaks Windows PowerShell
parsing. I had introduced 109 of them across this branch, which would have failed
the build. All replaced with ASCII hyphens; the gate now passes repo-wide.
Two API details worth recording, both found by this test failing first: dashboard
mutations require the X-Dispatch-Request CSRF header (WebAuthMiddleware) and return
a bodyless 403 without it, and the settings route is /api/settings rather than
/api/settings/tls, which otherwise silently serves the SPA fallback HTML.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All 12 SQL-touching repositories now run through DispatchDbContext, reaching the
engine only via IDatabaseProvider for the handful of things EF cannot express.
ISqlDialect, both dialect implementations, SqliteTypeHandlers, SqlConnectionFactory
and the Dapper package are gone.
The same 64 repository tests now pass against ALL FOUR engines - SQLite,
PostgreSQL, MariaDB and SQL Server. MariaDB and SQL Server previously scored 9/33
because the data layer only spoke PostgreSQL and SQLite.
Most of the port is a simplification: LINQ predicates replace hand-built WHERE
strings (so values are parameters by construction, not by discipline), .Contains()
replaces LIKE with manual wildcard escaping (EF escapes per provider), and
retention cutoffs computed in C# replace engine-specific interval arithmetic. Two
things needed real thought:
* The counter upsert stays raw SQL, via IDatabaseProvider.CounterUpsertSql. It is
the one write that cannot be load-modify-save: every worker bumps the same
(date, relay_id) row, so a read-then-write loses increments under load.
* The ROW_NUMBER() OVER (PARTITION BY ...) lifecycle dedup becomes MAX(id) per
group. Ids are monotonic, so within a spool id the highest id IS the latest
event - and it translates on all four engines, where LINQ's group-then-take-
first does not.
Two engine-specific bugs the four-way run exposed, both silent:
* SQLite's CURRENT_TIMESTAMP has whole-second precision, but EF writes DateTime
with sub-second digits. Timestamps are text, so within the same second
'...:16' sorts BEFORE '...:16.847' - a row the database stamped appeared OLDER
than one written moments earlier, and the Message Log showed them out of order.
logged_at is now stamped in code so every row has identical precision.
* MySQL/MariaDB tables did not inherit the case-sensitive collation set at
database creation, so subject search silently matched case-insensitively.
Collation is now declared on the provider and applied to the MODEL, which is
what makes EF emit it per table.
SqlRelayRepository keeps enforcing at-most-one-default-relay in code, inside a
transaction, on every engine - MySQL/MariaDB has no filtered index to catch it.
CI now runs the suite against all four engines rather than two.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…vers
Completes the two-mode story: bundled SQLite by default, or point Dispatch at a
database server the site already runs. Dispatch no longer installs, configures or
manages a database server on any platform.
* Windows: deletes installer/windows/postgres entirely and the Burn chain step
that ran it. That step downloaded and silently installed PostgreSQL, took
15-20 minutes, was the most failure-prone part of a Windows install, and left
a database service behind after uninstall. SQLCONN now defaults to a SQLite
file under %ProgramData%\Dispatch.
* Linux: install.sh drops --install-postgres and its distro-specific bootstrap.
Omitting --sql-connection uses /var/lib/dispatch/dispatch.db, which the
existing DATA_DIR chown already covers.
* Appliance: deletes firstboot.sh and dispatch-firstboot.service outright. That
unit existed only to generate a per-VM database password, start PostgreSQL and
create the role - all of which the bundled file makes unnecessary. Also drops
the offline PostgreSQL .deb staging, so the image is smaller and first boot no
longer waits on a database bootstrap.
* Docker: one container, one volume. The PostgreSQL service moves to
docker-compose.postgres.yml as an opt-in override.
* Azure: cloud-init no longer generates a database password.
Both installers gain --db-provider / DBPROVIDER, needed only when a connection
string is ambiguous - "Server=...;Database=...;User Id=..." is valid for both SQL
Server and MySQL/MariaDB, and Dispatch refuses to guess rather than failing later
at an opaque syntax error.
Verified: a default install boots healthy with no database setup at all, creating
its database and WAL on first start. All 357 tests pass and the PostgreSQL to
SQLite upgrade smoke still passes. CI's Linux tarball smoke now installs the way a
user will - no flags beyond the admin password.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nce another property
Two errors from the first Windows CI run of the SQLite default:
* WIX0006 - <Property Id="DBPROVIDER" Value="" /> is illegal; a property with no
value must simply omit the attribute.
* WIX1077 - Property/@value cannot reference another property, so the SQLCONN
default of "Data Source=[CommonAppDataFolder]Dispatch\dispatch.db" would have
been written out with the reference UNEXPANDED. It built as a warning, which
means an install would have produced a literally broken connection string.
SQLCONN is now left unset and WriteAppSettings derives the SQLite default from the
data directory it is already handed - the one place where that path is resolved.
Found only by running the Windows job; neither is reproducible off Windows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Windows functional smoke asserted relay_log tableBytes > 0. That is only true on an engine that can attribute size per table - SQLite needs the dbstat module, which is absent from most builds, so Dispatch reports 0 there rather than inventing a number (ProviderCapabilities.PerTableSizeReporting, docs/database.md footnote 1). Now asserts the whole-database size, which every engine reports and which is what the storage page actually needs. The per-table figure is only checked for non-negativity, since requiring it to be positive is really asserting "the backend is PostgreSQL". The other 16 sections of the smoke - API and SMTP delivery, STARTTLS, key revocation, SMTP AUTH, attachments, routing, reports, config round-trip, audit log and all three purge behaviours - passed on the SQLite default install. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The storage page showed "relay_log: 0 KB" on the bundled default. That is not
graceful degradation - it tells an operator something false about their own system,
on the page whose entire job is showing them what is consuming their disk.
dbstat would give an exact page-level answer but is genuinely absent from the
shipped SQLitePCLRaw builds (verified against both 2.1.12 and 3.0.4). So the size
is measured rather than guessed: real database size from page_count * page_size,
real per-table content from the octet lengths of actual rows, and the former split
across tables in proportion to the latter. Every input is measured; only the split
of shared overhead (indexes, page slack) is inferred, so a heavily-indexed table
reads slightly low. Sampled from both ends of each table, since relay_log rows
change width over time and reading only the oldest 1,000 would misjudge a table
whose recent traffic differs from its history.
Printing the actual numbers exposed a worse bug that had been shipping since the
first SQLite commit and passed every test:
total=181,446KB relay_log=183,000KB (3,000 rows)
60 KB per row, and a table larger than the database containing it.
GetDatabaseSizeBytesAsync summed the files on disk, and in WAL mode the -wal
sidecar holds not-yet-checkpointed pages - 3,000 inserts produced a 180 MB WAL
against ~3 MB of data. Because it kept growing between calls, one table could
exceed the whole database. Now uses the logical size, which already accounts for
pages living in the WAL and is what the file settles at after a checkpoint:
total=3,464KB relay_log=3,428KB (3,000 rows) audit_log=35KB (100 rows)
Three cross-engine tests guard it: a populated table must report non-zero, a table
cannot exceed its database, and content must move the number. Deliberately no
assertion that an empty table reports zero - on PostgreSQL, SQL Server and MySQL an
empty table owns real allocated pages, and InnoDB's minimum extent means it reports
~48 KB regardless. Asserting otherwise would be asserting a storage engine's
allocation policy rather than Dispatch's behaviour; both of those wrong assumptions
were caught by running the matrix.
The Windows smoke assertion is restored to tableBytes > 0, plus a check that a
table cannot exceed its database. All 67 repository tests pass on all four engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… fixed 1 MB The Windows smoke forced size-pressure with a hardcoded sizeTriggerGb of 0.001, commented "~1 MB trigger -> always over". That only held on a backend whose reported database size was already large - PostgreSQL, or SQLite back when the size wrongly included the WAL. Now that SQLite reports its real logical size, a fresh install is about 144 KB, the trigger never fired, and nothing was archived. The step's own comment says it sets the trigger "below current usage", so it now does exactly that: read the reported size and set the trigger to half of it. Correct on every backend, and it stops the test from silently depending on a database being big for reasons unrelated to what it is testing. Formatted fixed-point with InvariantCulture - these are small fractions of a GB, and PowerShell's default double formatting would put exponent notation and a culture-dependent decimal separator into the JSON body. The storage assertion this run was chasing now passes on Windows against the bundled SQLite: "relay_log=61KB of 144KB". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
IDatabaseProvider.OnConnectionOpenedAsync was dead code after the EF port. Nothing called it for real connections - it ran once, on a throwaway connection, during bootstrap. It looked like a guarantee and was decorative, which is the same failure mode as the legacy schema adopter removed in 60518e2. The consequence was not cosmetic. SQLite's `synchronous` pragma, unlike `journal_mode`, is NOT stored in the database file: it is per-connection state. So every connection the application used ran with the default synchronous=FULL, an fsync on every commit. Concurrent write throughput had quietly fallen from 5,970/sec before the port to 998/sec, with every test still green - a missing pragma changes durability settings and speed, not correctness, so nothing failed. A DbConnectionInterceptor now runs the hook on every connection EF opens, and synchronous=NORMAL moved into it (journal_mode stays at bootstrap, since that one does persist). Measured on the same 16-writer test: 998/sec -> 3,509/sec Still short of Dapper's 5,970 - EF has real per-insert cost - but roughly 1,170 messages/sec, far beyond what a relay needs. Postgres and SQL Server measure 1,079/sec and 931/sec on the same test, so SQLite remains the fastest of the three for this workload. Context pooling was tried first and moved the number by under 2%, which is what pointed at fsync rather than object construction. Also reports both logical and on-disk size in the volume test - they differ by more than the WAL is worth ignoring, and conflating them is what hid the earlier bug. All 67 repository tests pass on all four engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…opy guard Four of the six Hyper-V import tests have failed on every run since they were written - main, v0.5.0 and every dependabot PR since 2026-07-09. Both the test and the guard it trips over landed in the same commit (91f4cc2), so this suite has never been green. The script verifies the VHDX copy actually landed before handing the path to New-VM: Copy-Item -LiteralPath $VhdxPath -Destination $destVhdx -Force if (-not (Test-Path -LiteralPath $destVhdx -PathType Leaf)) { throw "VHDX copy failed..." } That guard is worth keeping - a silently failed copy would otherwise produce a VM pointed at a disk that is not there. The test mocked Copy-Item to a no-op, so the destination never appeared and the guard threw on every test that reached it. The two that passed were the two that fail during validation, before the copy. The mock now creates the destination file, which is what a real copy does. The existing assertion on Copy-Item's Destination parameter still holds. These tests need Windows identity APIs (WindowsIdentity::GetCurrent) and cannot run off Windows, so this is verified by the Scripts workflow rather than locally. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rated at all
The customer upgrade was broken and the smoke test could not see it, because the
test built its source database with CURRENT code. That source was therefore already
an EF schema - the one shape a real 0.6 install is not.
A genuine pre-0.7 database has a schema_version table and no __EFMigrationsHistory.
The migrator reads the source THROUGH the EF model, so it refused outright:
Migration failed: The source database is not a current Dispatch schema
(1 migration(s) not applied).
This restores the pre-EF schema adoption removed in 60518e2. That removal was made
on the basis that every install is a new install, which was corrected immediately
afterwards - and I then wrongly reasoned the removal still stood because "that was
schema adoption, this is data migration". It is not a separate concern: the data
migration depends on the source being a schema EF recognises.
* DatabaseInitializer adopts a pre-0.7 schema again, recording the baseline
migration as applied rather than re-creating tables that hold live mail.
* DatabaseMigrator detects a pre-0.7 source and says what to do about it - start
the 0.7 service against it once - instead of reporting "not a current schema" to
someone midway through upgrading production.
* The shipped 0.6 PostgreSQL scripts are kept as a test fixture, so the upgrade is
exercised against the real old schema rather than a reconstruction of it.
The smoke test now starts where the customer starts: build the pre-0.7 schema, seed
300 rows of 0.6-era mail, start 0.7 (adopting in place), add new mail, migrate to
SQLite, restart, and verify BOTH eras of history survive along with the relay ids
and encrypted config.
That rewrite immediately exposed a second bad assertion of my own: the final check
compared row counts to prove new mail arrived, which only worked while the log fitted
on one page. With 300 rows of history the page is full and the count never moves. It
now asserts the message is present, which is what it meant.
Verified end to end: 300 rows of 0.6 history adopted in place and migrated, relay id
preserved, config and encrypted settings intact, new mail delivering. 69 repository
tests pass on SQLite and PostgreSQL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eleased build Making SQLite the default introduced a silent data-loss-shaped failure. install.sh rewrites appsettings.json wholesale, and appsettings holds exactly the connection string and TLS cert. Before this branch, omitting --sql-connection was a hard error; now it means "bundled SQLite". So an operator upgrading a PostgreSQL install in place - the normal way, with no flags - would have had their config silently repointed at a brand-new empty database. The service starts, reports healthy, shows no mail history and asks for first-run setup, while every message they have relayed sits untouched in the database they were using. Nothing reports an error. install.sh now leaves an existing appsettings.json completely alone when no --sql-connection is given. Deliberately preserved byte for byte rather than parsed and rebuilt: reading a connection string back out of JSON with shell tools means getting quote escaping right inside a password, and getting that subtly wrong is the same silent failure by another route. --generate-cert is skipped with a note in that case, pointing at the dashboard where certs are managed anyway. Adds tests/smoke/upgrade-from-released-version.sh, which is the first test here that exercises a real upgrade rather than a simulation of one: it downloads the published release from GitHub, installs it with its own installer (PostgreSQL and all), sends mail through it, then upgrades to this build in place and migrates onto SQLite. It asserts the connection string is unchanged by the upgrade, the pre-EF schema is adopted, the same dashboard password still works, and mail sent on the OLD version is still visible after the binaries, the schema and the engine have all changed. Runs last in the linux-tarball job, after the fresh-install smoke, because it rewrites the machine's install - and starts by removing any existing one, so it cannot accidentally test an upgrade from a build made moments earlier. Two prior upgrade defects were invisible without this class of test: the migrator refusing a pre-0.7 database, and now this. Both passed every test that built its source from current code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The upgrade test passed --prebuilt "$dir/bin" to the released installer, which failed with "--prebuilt dir not found". The 0.5.0 tarball is universal - it ships bin-x64 and bin-arm64, not bin - and its installer selects by architecture when the flag is omitted. Its README gives the command with no --prebuilt at all. Both installs now run exactly the documented command, which is the point of this test: it is worth less the moment it stops being what an operator would type. The new install already had the same arch fallback, so it needs no flag either. Also bounds the journal dump on failure. An unbounded one buried the single line that said what actually broke. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
installers.yml executes tests/smoke/*.sh but only triggered on installer/** and its own file. So the fix to the release-upgrade smoke never ran - the workflow simply did not fire, and the last result on the branch stayed the failing one from before the fix. A smoke test whose corrections do not trigger the workflow that runs it is worse than no smoke test: it reports stale verdicts. Also corrects the header, which still described the Linux smoke as installing PostgreSQL. It no longer does; only the upgrade smoke does, via the OLD released installer, which is the point of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…o key
The release-upgrade smoke got all the way through - old release installed on
PostgreSQL, mail sent, upgraded in place with the connection string preserved,
schema adopted, migrated to SQLite, all 20 rows of old mail present and visible -
then died on its last step with:
KeyError: 'key'
/api/keys answers either 200 with the key or 400 with {"error": ...}, and both are
valid JSON, so pulling the field out blind turned an API error into an opaque
Python traceback with nothing to diagnose from. It now prints the response.
Verified locally that this is not a product fault: replaying the same sequence - a
0.5.0-shaped PostgreSQL database with an existing api_keys row, adopted by 0.7,
migrated to SQLite - the endpoint returns 200 with a key and no primary-key
collision. DatabaseMigratorTests now asserts that directly too: a key created after
a migration must not collide with the copied rows and must verify.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t never write
The release-upgrade smoke found a real operational fault, and the symptom is the
reason it was worth building: after migrating, the service starts, reports healthy,
serves the dashboard and shows every migrated message - and silently fails on every
write, which for a relay means every message that arrives.
migrate-database has to run as root, since it reads the install's appsettings.json.
The SQLite file it creates is therefore owned by root, while the service runs as the
dispatch account. SQLite fails writes on a file it cannot write while continuing to
serve reads, so nothing looks wrong until you look for something new.
Reproduced exactly by making the file read-only locally: reads return rows, writes
return {"error":"An unexpected error occurred."} and the log says "attempt to write
a readonly database".
The CLI now sets the new file (and its -wal/-shm siblings) to the same owner as the
directory holding it, which on a real install is the service account, and says so.
chown --reference rather than P/Invoke: marshalling struct stat across libc versions
to read the directory's uid is fragile, and this is a one-shot interactive admin
command where shelling out is honest and visible.
The smoke asserts the ownership and then proves a WRITE persists, rather than
chowning the file itself - fixing it in the test would hide the bug the test exists
to catch. Every assertion that passed on the failing run was a read.
Also fixes the smoke's fail() writing diagnostics to stdout. These helpers run inside
command substitutions, so the journal dump was being captured into the variable being
assigned instead of reaching the log - which is why the first failure reported only
"KeyError: 'key'" with no context.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restoring the pre-0.7 schema detection broke SQL Server outright:
SqlException: Invalid object name 'information_schema.tables'
Two of my own decisions collided. Dispatch creates SQL Server databases with
Latin1_General_BIN2 so LIKE matches the other engines - and a binary collation makes
IDENTIFIER resolution case-sensitive too. The view is really INFORMATION_SCHEMA.TABLES,
so the lowercase spelling that is correct on PostgreSQL and MySQL is not a name that
exists there.
The detection runs whenever no migrations have been applied - which is exactly a
fresh install - so every new SQL Server deployment would have failed at startup.
Now uses sys.tables, which is canonically lowercase and unaffected by collation.
Caught only because the four-engine matrix runs the full suite on SQL Server. The
detection was written when the matrix did not exist, deleted before it did, and
restored after - so this combination had never actually run until now.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…oss backends
Migration only worked one way: onto bundled SQLite. Moving the other way - off the
default onto a server you already run, once you outgrow it - was unsupported, and
SQL Server was refused outright.
SQL Server now works as a target. It rejects an explicit value for an identity
column unless IDENTITY_INSERT is on for that table, which is per-session and covers
one table at a time, so the connection is held open across the toggle and the write.
Afterwards the identity seed is reseeded with DBCC CHECKIDENT, or the first row
written after the migration would collide with a copied key - the same problem
PostgreSQL sequences already had. DBCC takes a literal, not a subquery, so the
maximum is read first.
Writing the test for it immediately found a worse bug: migrating INTO PostgreSQL
failed on the first row.
Cannot write DateTime with Kind=Unspecified to PostgreSQL type
'timestamp with time zone', only UTC is supported.
Dispatch stores UTC everywhere, but only PostgreSQL round-trips that fact.
SQLite (ISO text), SQL Server (datetime2) and MySQL (datetime) return
Kind=Unspecified because their column types carry no zone - so three of the four
engines could not be migrated onto the fourth.
The quieter half of the same bug was already visible and I had not chased it: an API
response serialises Kind=Unspecified with no trailing Z, so the identical timestamp
was emitted as "2026-07-20T08:27:14" on three backends and "...Z" on PostgreSQL. A
client parsing those gets two different instants depending on which database the
operator runs.
Fixed in the model with a value converter applied on every provider - specifying UTC
on a value that is already UTC is a no-op, and a conversion present everywhere cannot
be forgotten when an engine is added.
MigrationIntoEachEngineTests covers the direction, and because the fixture chooses
the engine, one test exercises all four as targets. 70 repository tests pass on
SQLite, PostgreSQL, MariaDB and SQL Server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pre-0.7 upgrade smoke failed in CI on its last assertion - the message sent after cutover was not in the log - while passing locally every time. It was not a logic fault. Delivery is asynchronous: the API accepts, the spool worker picks the file up, the provider responds, and only then is the row written. How long that takes depends on how busy the machine is, and a fixed `sleep 5` is a guess about that rather than a check. It held on a quiet laptop and failed on a runner that had just finished four engine test suites. Every send is now followed by polling for the message to appear, up to a minute. The same fixed sleeps in the released-version upgrade smoke are replaced too - they had not failed yet, which only means they had not been unlucky yet. This is the same mistake as the earlier run-watcher that could not tell "not started" from "finished": waiting for a duration when the thing being waited on is a condition. A test that is timing-dependent reports on the machine's load, not on the software. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three places still described an installer that no longer exists. SPEC.md gets a new entry in section 1.1, which the document itself designates as authoritative over the older passages: Dispatch does not install a database server, defaults to bundled SQLite, and supports four engines chosen by connection string. The body passages that assume PostgreSQL are left alone - section 1.1 states that the deltas win and the older text is kept for context, so rewriting it would fight the document's own convention. The Windows delta is corrected too: the bundle no longer chains an InstallPostgres.exe that downloaded and installed a server. installer/README.md said the MSI defaults SQLCONN to a local bundled PostgreSQL. It defaults to the bundled SQLite file, and DBPROVIDER exists for the ambiguous case. Also documents the SQL Server object schema rather than pinning it. Tables land in the connection's default schema, dbo unless the server or login says otherwise. I had flagged the implicitness as a decision to make, and the decision is to leave it: pinning would bake an assumption into the migrations that a site with a different schema convention could not override, and it prevents no bug - it only restates EF's existing default. Saying so in the docs is the part that was actually missing. One stale code comment: to_addresses has not been "mapped by Dapper" since the port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 20, 2026
chrismuench
added a commit
that referenced
this pull request
Jul 20, 2026
…t 2-3x (#16) Following the pooling fix (#15) to its evidence exposed that my own commit overstated it. The fix corrected DI to register the pooled context factory and added a test that the resolved factory is the pooled TYPE - but a type assertion is not a throughput number, and the comments claimed pooling was "the difference between about 1,000 and several thousand concurrent writes per second." ProductionPoolingBenchmarkTests runs the ingest write pattern through the real AddDispatchData registration - the exact ILogRepository/ICounterRepository the service resolves - and against a hand-built non-pooled equivalent. Measured on SQLite, 16 writers: pooled 9,000-11,900/sec, non-pooled 7,600-8,800/sec. Roughly 1.1x-1.5x. Real, but not the headline the comments claimed. The ~3.5x jump earlier this branch (998 -> 3,509/sec) was the WAL synchronous=NORMAL pragma (ProviderConnectionInterceptor), which is I/O-bound and dominates because SQLite serialises writers on one lock regardless of context allocation. I conflated the two: pooling is the small CPU-bound saving on top, not the big one. Production before the pooling fix already had the synchronous fix (it shipped in #14 and applies to every path), so it was never at ~1,000/sec - it was around 8,000. The pooling fix took it to ~9,500. Corrected both comments to the measured figure and to distinguish the two fixes. Pooling is still worth registering - it is free once the factory exists and it is what makes the test and production paths measure the same thing - but the reason is correctness of measurement, not the speed-up. Co-authored-by: Chris Muench <chris@MacBook-Air.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answers "can we switch Postgres to SQLite, and can it perform?" — and grows into a documented four-engine framework.
What ships
installer/windows/postgres/and the appliance's first-boot PostgreSQL bootstrap are deleted. A default install creates a file beside the service and starts.Database:Providerdisambiguates the SQL-Server-vs-MySQL case sniffing cannot).DispatchDbContext) rendered per engine by four migrations assemblies. The Dapper data layer,ISqlDialect, andSqlConnectionFactoryare gone.migrate-databasecopies a full database between any two engines, in either direction, preserving primary keys and encrypted config.schema_versionPostgreSQL install → 0.7 → bundled SQLite) is adopted in place and migrated, verified against the real shipped 0.5.0 artifact.Performance (measured, same 16-writer test)
Message Log stays ~17ms out to 2M rows; keyset pagination is flat.
Verification
All five workflows green on GitHub runners:
A
ProviderConnectionInterceptor, per-providerIDatabaseProvider, and a conformance suite keep the engines honest.docs/database.mddocuments choosing a backend, migrating, what differs per engine and why, and how to add one.🤖 Generated with Claude Code