Skip to content

Add IndexedDB sync queue with FIFO drain and OPFS blob store - #17

Merged
unforced merged 1 commit into
mainfrom
sync-queue
Apr 19, 2026
Merged

Add IndexedDB sync queue with FIFO drain and OPFS blob store#17
unforced merged 1 commit into
mainfrom
sync-queue

Conversation

@unforced

Copy link
Copy Markdown
Contributor

Summary

Foundation for offline-capable CRUD in v0.2. Mutations issued offline are persisted in IndexedDB and drained in FIFO order when the vault is reachable.

  • src/lib/sync/db (stores: pending autoIncrement seq, id_map, blob_path_map, blobs, meta), queue (enqueue + drain), engine (online/interval tick), blob-store (OPFS → IDB fallback), id-map (local-id + blob-ref resolution), storage-quota.
  • src/providers/SyncProvider.tsx — opens the DB, builds the blob store, listens for online/offline, runs the engine, invalidates TanStack caches on drain.
  • src/lib/vault/queries.ts — existing hooks now fall through to enqueue(...) when navigator.onLine === false. Same signatures; create returns an optimistic Note with a local-<uuid> id.

Design decisions worth calling out

  • Local → remote ID mapping, not mutation-row routing. When create-note drains, we store {localId → serverId} in id_map. Subsequent update/delete rows in the queue call resolveNoteId(localId) before dispatching. Same pattern for blob uploads (blob:<blobId> → storage path). This keeps the mutation rows themselves small and independent, and means UI code never has to be told "your note id changed."
  • FIFO via autoincrement seq. Queue depth is small (one user, thinking-pace mutations), so the cursor filters vault + status in-app rather than maintaining a compound index.
  • ArrayBuffer, not Blob, across the blob-store interface. Both jsdom and fake-indexeddb mangle Blob on structured clone. Callers convert via the exported blobToArrayBuffer() helper.
  • OPFS mimeType in sidecar files. OPFS has no metadata channel of its own — we write <blobId>.meta alongside each payload.
  • Error classification: VaultAuthError → halt drain + meta["auth-halted"] marker; VaultConflictError → stash as needs-human, continue past it; VaultNotFoundError → drop the row; anything else → exponential backoff (2^attempts seconds, capped at 10 min).
  • Vault isolation. A drain only touches rows for the active vault. Other vaults' queues are left alone.

Six mutation kinds

create-note, update-note, delete-note, upload-attachment, link-attachment, delete-attachment. All resolve local ids + blob refs before hitting the network.

Out of scope (per PR #2 spec)

UI for queue status, dedicated conflict-resolution UI, voice capture wiring, scribe. This is plumbing only — the existing UI simply stops failing when offline.

Gates

  • bun run lint
  • bun run typecheck
  • bun run test --run — 231/231 passing (39 new sync tests)
  • bun run build

Test plan

  • DevTools → Network → Offline, create a note → optimistic note appears with local-* id; reload → note list empty (expected, since the queue lives in IDB and drains later).
  • Back online, wait for next tick (or fire an online event) → note appears with server id.
  • Offline → update / delete of an existing server note → queued; drains when online.
  • Auth halt: invalidate token, mutate offline → back online → drain halts + meta["auth-halted"] set.
  • Restart resilience: enqueue offline, hard reload tab, confirm pending count persists.

🤖 Generated with Claude Code

Mutations issued while offline are persisted to an IndexedDB `pending`
store and drained in seq order when the vault is reachable. Six
mutation kinds: create/update/delete-note, upload/link/delete-
attachment. Local-UUID note IDs and blob refs are mapped to server IDs
via dedicated stores so subsequent rows in the queue can resolve their
target ids before issuing the real call.

Blob payloads live in OPFS when available (with sidecar mimeType
files — OPFS has no metadata channel) and fall back to an IndexedDB
object store. The interface operates on ArrayBuffer + mimeType so the
test harness (jsdom + fake-indexeddb) doesn't mangle Blob objects via
structured clone.

Drain classifies errors: auth halts the whole drain and stamps a
meta marker; conflicts stash the row as `needs-human` and continue
past it; 404s drop the row; anything transient hits an exponential
backoff capped at 10m. The engine ticks on `online` events plus a
30s interval and no-ops when navigator.onLine is false or no vault
is active.

The existing mutation hooks (useCreate/Update/Delete-Note,
useDeleteAttachment) keep the same signature. When offline, the
create hook returns an optimistic Note with a `local-<uuid>` id;
update/delete enqueue and resolve to a no-op Note / void. TanStack
query caches are invalidated in onDrain when rows flush.

Storage persistence is requested on SyncProvider mount. iOS' ~50MB
PWA cap and eviction caveats remain documented in the install
section.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@unforced
unforced merged commit 2ddac48 into main Apr 19, 2026
@unforced
unforced deleted the sync-queue branch April 19, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant