fix: reduce cold-start D1 queries from 159 to ~25#532
Conversation
- Short-circuit migrations when all are already applied (1 query vs ~116) - Fix FTS triggers to exclude soft-deleted content from search index - Use rebuildIndex in enableSearch to prevent duplicate FTS rows - Prevents false-positive FTS index rebuilds on every Worker cold start
🦋 Changeset detectedLatest commit: 3372f11 The changes in this PR will be included in the next version bump. This PR includes changesets to release 9 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | 3372f11 | Apr 13 2026, 03:18 PM |
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/blocks
@emdash-cms/cloudflare
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
There was a problem hiding this comment.
Pull request overview
Reduces Worker cold-start D1 query volume by avoiding expensive Kysely migrator schema introspection when migrations are already applied, and by fixing/avoiding unnecessary FTS index rebuilds caused by soft-deletes and duplicate index population.
Changes:
- Add a migration “fast-path” that skips the Kysely Migrator when
_emdash_migrationsindicates all migrations are applied. - Update FTS triggers to exclude soft-deleted content from the index and keep counts consistent.
- Change
enableSearch()to rebuild the index from scratch to prevent duplicate rows (especially during seeding).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/core/src/search/fts-manager.ts | Adjusts FTS triggers to respect deleted_at, and switches enableSearch() to rebuildIndex() to avoid duplicated index rows and startup rebuild loops. |
| packages/core/src/database/migrations/runner.ts | Adds a cheap query-based short-circuit to skip Migrator initialization when migrations are already applied. |
| .changeset/wide-flies-think.md | Patch changeset documenting the cold-start query reduction and FTS/migration optimizations. |
Comments suppressed due to low confidence (1)
packages/core/src/search/fts-manager.ts:334
- The new trigger logic (excluding soft-deleted rows) and the new
enableSearch()behavior (rebuild-from-scratch to avoid duplicates) are key to preventing cold-start rebuild loops. There are existing integration tests for FTSManager, but none appear to cover soft-delete behavior or the duplicate-index scenario; adding a test that soft-deletes content and asserts the FTS row count stays in sync would help prevent regressions.
* Enable search for a collection.
*
* Uses rebuildIndex to ensure a clean state -- drop any existing FTS
* table/triggers, recreate them, and populate from content. This avoids
* duplicate rows when triggers have already populated the index (e.g.
* during seeding where content is inserted before search is enabled).
*/
async enableSearch(
collectionSlug: string,
options?: { weights?: Record<string, number> },
): Promise<void> {
if (!isSqlite(this.db)) {
throw new Error("Full-text search is only available with SQLite databases");
}
// Get searchable fields
const searchableFields = await this.getSearchableFields(collectionSlug);
if (searchableFields.length === 0) {
throw new Error(
`No searchable fields defined for collection "${collectionSlug}". ` +
`Mark at least one field as searchable before enabling search.`,
);
}
// Rebuild from scratch to ensure clean state (no duplicate rows)
await this.rebuildIndex(collectionSlug, searchableFields, options?.weights);
// Update search config
await this.setSearchConfig(collectionSlug, {
enabled: true,
weights: options?.weights,
});
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const result = await sql<{ count: number }>` | ||
| SELECT COUNT(*) as count FROM ${sql.ref(MIGRATION_TABLE)} | ||
| `.execute(db); | ||
| if (result.rows[0]?.count === MIGRATION_COUNT) { |
There was a problem hiding this comment.
The fast-path compares result.rows[0]?.count directly to MIGRATION_COUNT. Depending on the dialect/driver, COUNT(*) can come back as a string or bigint, which would make this comparison always fail and negate the optimization. Consider normalizing the value (e.g., Number(...)) before comparing.
| if (result.rows[0]?.count === MIGRATION_COUNT) { | |
| const appliedMigrationCount = Number(result.rows[0]?.count); | |
| if (appliedMigrationCount === MIGRATION_COUNT) { |
| try { | ||
| const result = await sql<{ count: number }>` | ||
| SELECT COUNT(*) as count FROM ${sql.ref(MIGRATION_TABLE)} | ||
| `.execute(db); | ||
| if (result.rows[0]?.count === MIGRATION_COUNT) { | ||
| return { applied: [] }; | ||
| } | ||
| } catch { | ||
| // Table doesn't exist yet (first run). Fall through to the Migrator | ||
| // which will create it. | ||
| } |
There was a problem hiding this comment.
The catch {} around the fast-path will swallow any error (including unexpected DB issues) and silently fall back to the migrator, which can make real problems harder to detect. Consider only ignoring the specific "table does not exist" error and rethrowing others (or at least logging unexpected failures).
| // Insert trigger - only index non-deleted content | ||
| await sql | ||
| .raw(` | ||
| CREATE TRIGGER IF NOT EXISTS "${ftsTable}_insert" | ||
| AFTER INSERT ON "${contentTable}" | ||
| WHEN NEW.deleted_at IS NULL | ||
| BEGIN | ||
| INSERT INTO "${ftsTable}"(rowid, id, locale, ${fieldList}) | ||
| VALUES (NEW.rowid, NEW.id, NEW.locale, ${newFieldList}); | ||
| END | ||
| `) | ||
| .execute(this.db); |
There was a problem hiding this comment.
These triggers are created with IF NOT EXISTS. Since this PR changes trigger behavior, existing installations that already have the old triggers will keep the old definitions until something explicitly drops/recreates them (e.g., a rebuild). If you want the fix to take effect immediately, consider dropping the triggers first (or otherwise forcing recreation/versioning) before creating them.
- Short-circuit migrations when all are already applied (1 query vs ~116) - Fix FTS triggers to exclude soft-deleted content from search index - Use rebuildIndex in enableSearch to prevent duplicate FTS rows - Prevents false-positive FTS index rebuilds on every Worker cold start
What does this PR do?
Fixes a cold-start query explosion where every Worker cold start on D1 executes 159+ queries and takes 3s+ to load. Reduces to ~25 queries.
A user reported 150+ D1 queries per pageload. Root cause analysis of the query log revealed three compounding issues:
Kysely Migrator introspects the entire schema twice (~116 queries on D1). The Migrator calls
pragma_table_info()for every table in the database -- twice -- just to check if two migration tables exist. On D1 (which can't use cross-joins with pragma functions), each check is a separate query per table.FTS triggers keep soft-deleted content in the search index, causing a count mismatch between the FTS and content tables.
verifyAndRepairAll()at startup detects the mismatch and triggers a full DROP/CREATE/INSERT rebuild for every search-enabled collection -- on every cold start.enableSearch()double-populates the FTS index during seeding. Triggers insert rows during content creation, thenenableSearch()does a second bulk INSERT, doubling the row count and guaranteeing a mismatch on the next startup.Changes
Migration short-circuit (
runner.ts): Before creating the Kysely Migrator, do a singleSELECT COUNT(*) FROM _emdash_migrations. If the count matches the known migration count, all migrations are applied -- skip the Migrator entirely. Saves ~116 queries on D1.FTS trigger fix (
fts-manager.ts): The INSERT trigger now hasWHEN NEW.deleted_at IS NULL. The UPDATE trigger uses a conditional INSERT (INSERT ... SELECT ... WHERE NEW.deleted_at IS NULL) so soft-deleted content is removed from the FTS index instead of being re-inserted. This keeps FTS and content counts in sync, preventing false-positive rebuilds.enableSearch() uses rebuildIndex (
fts-manager.ts): Instead ofcreateFtsTable()+populateFromContent()(which appends to existing trigger-populated rows), usesrebuildIndex()which drops and recreates cleanly.Before vs After (D1, 3 search-enabled collections, ~57 tables)
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change)pnpm formathas been runpnpm locale:extracthas been run (if applicable)AI-generated code disclosure