Skip to content

Opt-in decryption for o11y tooling (CLI + web)#1256

Merged
TooTallNate merged 34 commits into
mainfrom
nate/opt-in-decrypt-rebased
Mar 4, 2026
Merged

Opt-in decryption for o11y tooling (CLI + web)#1256
TooTallNate merged 34 commits into
mainfrom
nate/opt-in-decrypt-rebased

Conversation

@TooTallNate
Copy link
Copy Markdown
Member

Summary

Makes decryption an explicit opt-in for observability tooling (CLI inspect and web dashboard), building on the wire encryption from #1251.

Key changes

  • CLI: --decrypt flag for workflow inspect commands. Encrypted data displays as [Encrypted] by default. When --decrypt is passed, fetches the encryption key (via getEncryptionKeyForRun) and hydrates data in-place. Key is cached per runId.
  • Web dashboard: "Decrypt" button in the run detail view title bar. Click triggers audit-logged key retrieval via server action. Once the key is available, all encrypted fields are automatically decrypted on subsequent fetches. Encryption key is lifted to run-level state.
  • Encrypted data display: EncryptedDataRef sentinel class with util.inspect.custom for CLI, isEncryptedMarker() checks for web UI components. Lock icon shown for encrypted fields.
  • Event data: Lazy-loaded event data is re-fetched and decrypted when the encryption key becomes available.
  • world-vercel: teamId parameter passed to the get-key endpoint.

Packages changed

Package Changes
@workflow/cli --decrypt flag, EncryptedDataRef display, key caching per runId
@workflow/core serialization-format.ts encrypted marker utilities
@workflow/web-shared isEncryptedMarker, hydrateResourceIOWithKey, encrypted field UI (Lock icon, EncryptedFieldBlock)
@workflow/web Decrypt button, encryption key state, auto-decrypt on fetch
@workflow/world-vercel teamId in get-key endpoint

Rebased from nate/opt-in-decrypt onto the latest nate/wire-encryption.

@TooTallNate TooTallNate requested a review from a team as a code owner March 4, 2026 08:56
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Mar 4, 2026

🦋 Changeset detected

Latest commit: f3e3ded

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages
Name Type
@workflow/cli Patch
@workflow/core Patch
@workflow/web Patch
@workflow/web-shared Patch
workflow Patch
@workflow/world-testing Patch
@workflow/builders Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copilot AI review requested due to automatic review settings March 4, 2026 08:56
@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Mar 4, 2026

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 4, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 538 0 67 605
✅ 💻 Local Development 576 0 84 660
✅ 📦 Local Production 576 0 84 660
✅ 🐘 Local Postgres 576 0 84 660
✅ 🪟 Windows 52 0 3 55
❌ 🌍 Community Worlds 117 48 15 180
✅ 📋 Other 138 0 27 165
Total 2573 48 364 2985

❌ Failed Tests

🌍 Community Worlds (48 failed)

turso (48 failed):

  • addTenWorkflow
  • addTenWorkflow
  • wellKnownAgentWorkflow (.well-known/agent)
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • importedStepOnlyWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • parallelSleepWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling retry behavior workflow completes despite transient 5xx on step_completed
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • instanceMethodStepWorkflow - instance methods with "use step" directive
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument
  • cancelRun - cancelling a running workflow
  • cancelRun via CLI - cancelling a running workflow
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 48 0 7
✅ example 48 0 7
✅ express 48 0 7
✅ fastify 48 0 7
✅ hono 48 0 7
✅ nextjs-turbopack 53 0 2
✅ nextjs-webpack 53 0 2
✅ nitro 48 0 7
✅ nuxt 48 0 7
✅ sveltekit 48 0 7
✅ vite 48 0 7
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 46 0 9
✅ express-stable 46 0 9
✅ fastify-stable 46 0 9
✅ hono-stable 46 0 9
✅ nextjs-turbopack-canary 52 0 3
✅ nextjs-turbopack-stable 52 0 3
✅ nextjs-webpack-canary 52 0 3
✅ nextjs-webpack-stable 52 0 3
✅ nitro-stable 46 0 9
✅ nuxt-stable 46 0 9
✅ sveltekit-stable 46 0 9
✅ vite-stable 46 0 9
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 46 0 9
✅ express-stable 46 0 9
✅ fastify-stable 46 0 9
✅ hono-stable 46 0 9
✅ nextjs-turbopack-canary 52 0 3
✅ nextjs-turbopack-stable 52 0 3
✅ nextjs-webpack-canary 52 0 3
✅ nextjs-webpack-stable 52 0 3
✅ nitro-stable 46 0 9
✅ nuxt-stable 46 0 9
✅ sveltekit-stable 46 0 9
✅ vite-stable 46 0 9
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 46 0 9
✅ express-stable 46 0 9
✅ fastify-stable 46 0 9
✅ hono-stable 46 0 9
✅ nextjs-turbopack-canary 52 0 3
✅ nextjs-turbopack-stable 52 0 3
✅ nextjs-webpack-canary 52 0 3
✅ nextjs-webpack-stable 52 0 3
✅ nitro-stable 46 0 9
✅ nuxt-stable 46 0 9
✅ sveltekit-stable 46 0 9
✅ vite-stable 46 0 9
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 52 0 3
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 3 0 2
✅ mongodb 52 0 3
✅ redis-dev 3 0 2
✅ redis 52 0 3
✅ turso-dev 3 0 2
❌ turso 4 48 3
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 46 0 9
✅ e2e-local-postgres-nest-stable 46 0 9
✅ e2e-local-prod-nest-stable 46 0 9

📋 View full workflow run

Comment thread packages/web-shared/src/components/sidebar/events-list.tsx Outdated
Comment thread packages/web-shared/src/components/sidebar/entity-detail-panel.tsx Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes observability decryption explicitly opt-in across the CLI and web dashboard, so encrypted run/step/event payloads render as “Encrypted” placeholders unless the user requests decryption (via --decrypt in CLI or a Decrypt button in the UI). It builds on the existing wire encryption work by adding encrypted-marker detection/display and on-demand key retrieval for hydration.

Changes:

  • CLI: adds --decrypt support with per-run key resolution/caching and encrypted placeholders in inspect output.
  • Web: adds a Decrypt button that retrieves the per-run key via an RPC server action and threads the key into event/resource hydration.
  • Core/web-shared/world-vercel: adds encrypted data detection utilities and marker rendering; updates Vercel key lookup flow and passes teamId to the run-key endpoint.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
packages/world-vercel/src/index.ts Refactors Vercel World to inline getEncryptionKeyForRun logic and changes exports.
packages/world-vercel/src/encryption.ts Updates run-key fetch contract/parsing and request headers.
packages/web/app/server/workflow-server-actions.server.ts Adds server action to retrieve run encryption key for client-side decryption; switches to createVercelWorld.
packages/web/app/routes/api.rpc.tsx Exposes getEncryptionKeyForRun via CBOR RPC handler.
packages/web/app/lib/rpc-client.ts Adds RPC client wrapper for getEncryptionKeyForRun.
packages/web/app/components/run-detail-view.tsx Adds Decrypt button and threads key through event hydration + trace viewer props.
packages/web-shared/src/lib/hydration.ts Adds encrypted markers, encrypted detection helpers, and async hydration with key.
packages/web-shared/src/index.ts Adjusts barrel exports for new encryption-related helpers.
packages/web-shared/src/components/workflow-trace-view.tsx Threads encryptionKey prop to entity detail panel/events list.
packages/web-shared/src/components/ui/data-inspector.tsx Adds encrypted marker rendering and minor deep-equality change.
packages/web-shared/src/components/sidebar/events-list.tsx Adds key-aware reload behavior for expanded events.
packages/web-shared/src/components/sidebar/entity-detail-panel.tsx Adds encryptionKey prop wiring (but currently introduces duplicate rendering).
packages/web-shared/src/components/sidebar/attribute-panel.tsx Adds encrypted-field UI block for run/step/hook/event attributes.
packages/web-shared/src/components/event-list-view.tsx Preserves custom constructors in parsing and reloads expanded events on key availability.
packages/core/src/serialization-format.ts Adds ENCRYPTED format + isEncryptedData, and async hydrateDataWithKey.
packages/core/src/serialization-format.test.ts Adds tests for encrypted prefix detection and pass-through behavior.
packages/cli/src/lib/inspect/output.ts Wires --decrypt into CLI inspect commands and shows encrypted placeholders.
packages/cli/src/lib/inspect/hydration.ts Adds encrypted placeholder ref + optional decryption during hydration.
packages/cli/src/lib/config/types.ts Adds decrypt?: boolean inspect option.
packages/cli/src/commands/inspect.ts Adds --decrypt flag to CLI command definition.
.changeset/opt-in-decrypt.md Adds changeset for CLI/core/web/web-shared but currently omits world-vercel.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread packages/web-shared/src/components/sidebar/events-list.tsx Outdated
Comment thread packages/cli/src/lib/inspect/hydration.ts Outdated
Comment thread packages/web-shared/src/index.ts
Comment thread .changeset/opt-in-decrypt.md
Comment thread packages/world-vercel/src/index.ts Outdated
Comment thread packages/web-shared/src/lib/hydration.ts Outdated
Comment thread packages/core/src/serialization-format.ts
Comment thread packages/world-vercel/src/encryption.ts
Comment thread packages/world-vercel/src/index.ts
Comment thread packages/web-shared/src/components/sidebar/entity-detail-panel.tsx Outdated
The rebase incorrectly picked up older versions of these files from
early encryption branch commits. The main versions are correct and
up-to-date.
Cherry-pick conflict resolution incorrectly took the older opt-in-decrypt
versions of these files, reverting improvements from main (dispatcher,
createGetEncryptionKeyForRun extraction, nullable key response).
- Remove duplicate AttributePanel/EventsList rendering in entity-detail-panel.tsx.
  Thread encryptionKey into the existing EventsList render instead.
- Restore missing re-exports (isClassInstanceRef, isStreamId, isStreamRef)
  in web-shared/src/index.ts to maintain backwards compatibility.
- Add 'error' to replaceEncryptedWithMarkers field list in web-shared
  hydration.ts to match the decrypt path.
- Extend CLI hydration eventData decrypt/placeholder to cover all known
  serialized fields (output, metadata, payload) not just result/input.
- Add 'error' to CLI replaceEncryptedWithRef field list.
- Remove invalid encryptionKey option from useWorkflowResourceData call
  (hook doesn't support it yet), add TODO.
- Add 4 unit tests for hydrateDataWithKey in serialization-format.test.ts:
  encrypted+key decrypts, encrypted+noKey returns raw, non-encrypted
  hydrates normally, non-Uint8Array legacy data passes through.
Instead of leaving a TODO, implement the encryptionKey support directly:
- Add optional encryptionKey to useWorkflowResourceData options
- When key is available, use hydrateResourceIOWithKey (async decrypt)
  instead of hydrateResourceIO for all resource types
- Remove redundant hydrateResourceIO from fetchResourceWithCorrelationId
High priority:
- Gate showStream key fetch on --decrypt flag, warn when --decrypt
  used without --run
- Fix workflow-server-actions.server.ts missing cryptoKey params
  (undefined for both getExternalRevivers and getDeserializeStream)
- Add hydration + decryption to listEvents (was completely missing)
- Fix error/eventData display: check isEncryptedMarker before
  hasDisplayContent so encrypted markers don't silently disappear

Medium priority:
- handleDecrypt: use toast.error() instead of console.error for
  user-visible feedback on key fetch failures
- CLI maybeDecryptFields: add try/catch with graceful fallback to
  encrypted placeholders + warning, also decrypt error field
- use-resource-data: wrap hook/sleep hydrate() in try/catch to
  prevent stuck loading state on decryption errors
- Decrypt button: also check run.error and step input/output for
  encrypted markers, not just run.input/output

Low priority:
- event-list-view: add .catch() to re-load useEffect promise
- Export ENCRYPTED_DISPLAY_NAME from hydration.ts and import in
  data-inspector.tsx instead of raw 'Encrypted' string
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 4, 2026

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 0.032s (-1.8%) 1.006s (~) 0.974s 10 1.00x
💻 Local Nitro 0.033s (~) 1.005s (~) 0.973s 10 1.02x
💻 Local Next.js (Turbopack) 0.038s 1.005s 0.967s 10 1.18x
🌐 Redis Next.js (Turbopack) 0.044s 1.005s 0.961s 10 1.36x
🐘 Postgres Next.js (Turbopack) 0.049s 1.012s 0.963s 10 1.52x
🐘 Postgres Express 0.053s (-11.1% 🟢) 1.012s (~) 0.959s 10 1.64x
🐘 Postgres Nitro 0.057s (-7.0% 🟢) 1.013s (~) 0.956s 10 1.77x
🌐 MongoDB Next.js (Turbopack) 0.092s 1.009s 0.917s 10 2.86x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 0.450s (+19.4% 🔺) 2.022s (+13.4% 🔺) 1.572s 10 1.00x
▲ Vercel Nitro 0.457s (+2.8%) 2.172s (+8.7% 🔺) 1.715s 10 1.01x
▲ Vercel Next.js (Turbopack) 0.480s (+12.6% 🔺) 2.201s (+12.5% 🔺) 1.721s 10 1.07x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.100s (~) 2.006s (~) 0.906s 10 1.00x
💻 Local Nitro 1.103s (~) 2.006s (~) 0.903s 10 1.00x
💻 Local Next.js (Turbopack) 1.103s 2.006s 0.902s 10 1.00x
🌐 Redis Next.js (Turbopack) 1.107s 2.006s 0.900s 10 1.01x
🐘 Postgres Next.js (Turbopack) 1.123s 2.012s 0.889s 10 1.02x
🐘 Postgres Nitro 1.126s (-1.2%) 2.013s (~) 0.887s 10 1.02x
🐘 Postgres Express 1.127s (-0.6%) 2.012s (~) 0.885s 10 1.02x
🌐 MongoDB Next.js (Turbopack) 1.317s 2.009s 0.691s 10 1.20x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 1.977s (-10.3% 🟢) 3.231s (-6.9% 🟢) 1.253s 10 1.00x
▲ Vercel Next.js (Turbopack) 2.057s (-41.3% 🟢) 3.321s (-33.6% 🟢) 1.264s 10 1.04x
▲ Vercel Express 2.075s (+2.0%) 3.200s (+7.5% 🔺) 1.125s 10 1.05x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 10.679s 11.024s 0.345s 3 1.00x
💻 Local Next.js (Turbopack) 10.687s 11.024s 0.337s 3 1.00x
💻 Local Express 10.764s (~) 11.022s (~) 0.259s 3 1.01x
💻 Local Nitro 10.777s (~) 11.024s (~) 0.247s 3 1.01x
🐘 Postgres Next.js (Turbopack) 10.786s 11.041s 0.255s 3 1.01x
🐘 Postgres Express 10.877s (~) 11.045s (~) 0.168s 3 1.02x
🐘 Postgres Nitro 10.888s (~) 11.044s (~) 0.155s 3 1.02x
🌐 MongoDB Next.js (Turbopack) 12.335s 13.026s 0.691s 3 1.16x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 16.964s (+2.7%) 18.108s (+5.6% 🔺) 1.144s 2 1.00x
▲ Vercel Nitro 17.216s (-0.6%) 18.552s (-0.8%) 1.336s 2 1.01x
▲ Vercel Next.js (Turbopack) 18.090s (+5.3% 🔺) 19.891s (+9.0% 🔺) 1.801s 2 1.07x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 26.609s 27.051s 0.442s 3 1.00x
🐘 Postgres Next.js (Turbopack) 26.882s 27.068s 0.186s 3 1.01x
💻 Local Next.js (Turbopack) 26.889s 27.052s 0.162s 3 1.01x
🐘 Postgres Express 26.960s (~) 27.059s (~) 0.099s 3 1.01x
🐘 Postgres Nitro 27.162s (~) 28.074s (+2.5%) 0.912s 3 1.02x
💻 Local Express 27.184s (~) 28.054s (~) 0.870s 3 1.02x
💻 Local Nitro 27.203s (~) 28.054s (~) 0.851s 3 1.02x
🌐 MongoDB Next.js (Turbopack) 30.488s 31.043s 0.556s 2 1.15x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 43.822s (-1.1%) 45.564s (~) 1.741s 2 1.00x
▲ Vercel Express 44.158s (+1.0%) 45.510s (+1.5%) 1.352s 2 1.01x
▲ Vercel Nitro 45.532s (-4.8%) 47.606s (-2.7%) 2.074s 2 1.04x

🔍 Observability: Next.js (Turbopack) | Express | Nitro

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 53.109s 53.598s 0.488s 2 1.00x
🐘 Postgres Next.js (Turbopack) 53.545s 54.107s 0.561s 2 1.01x
🐘 Postgres Express 53.752s (~) 54.101s (~) 0.349s 2 1.01x
🐘 Postgres Nitro 54.115s (~) 54.614s (+0.9%) 0.499s 2 1.02x
💻 Local Next.js (Turbopack) 55.447s 56.100s 0.653s 2 1.04x
💻 Local Express 56.117s (~) 56.601s (-0.9%) 0.484s 2 1.06x
💻 Local Nitro 56.156s (~) 57.106s (+1.8%) 0.950s 2 1.06x
🌐 MongoDB Next.js (Turbopack) 60.952s 61.085s 0.134s 2 1.15x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 92.992s (~) 93.665s (~) 0.673s 1 1.00x
▲ Vercel Nitro 93.284s (-1.8%) 94.892s (-1.3%) 1.608s 1 1.00x
▲ Vercel Next.js (Turbopack) 100.695s (+2.8%) 102.418s (+3.3%) 1.723s 1 1.08x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 1.250s 2.007s 0.756s 15 1.00x
🐘 Postgres Express 1.354s (-0.6%) 2.011s (~) 0.657s 15 1.08x
🐘 Postgres Next.js (Turbopack) 1.375s 2.011s 0.636s 15 1.10x
🐘 Postgres Nitro 1.383s (~) 2.012s (~) 0.629s 15 1.11x
💻 Local Express 1.414s (+0.6%) 2.006s (~) 0.593s 15 1.13x
💻 Local Nitro 1.431s (+0.6%) 2.006s (~) 0.575s 15 1.14x
💻 Local Next.js (Turbopack) 1.433s 2.006s 0.573s 15 1.15x
🌐 MongoDB Next.js (Turbopack) 2.143s 3.010s 0.867s 10 1.71x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.326s (-9.4% 🟢) 3.788s (-6.1% 🟢) 1.462s 9 1.00x
▲ Vercel Next.js (Turbopack) 2.401s (+0.7%) 3.617s (+6.4% 🔺) 1.215s 9 1.03x
▲ Vercel Express 2.498s (+5.2% 🔺) 3.592s (+2.9%) 1.093s 9 1.07x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 2.031s (-4.1%) 2.599s (-3.0%) 0.568s 12 1.00x
🐘 Postgres Nitro 2.046s (-8.1% 🟢) 2.743s (~) 0.697s 11 1.01x
🐘 Postgres Next.js (Turbopack) 2.082s 2.739s 0.658s 11 1.02x
🌐 Redis Next.js (Turbopack) 2.525s 3.008s 0.483s 10 1.24x
💻 Local Next.js (Turbopack) 2.641s 3.008s 0.367s 10 1.30x
💻 Local Express 2.642s (-0.7%) 3.007s (~) 0.365s 10 1.30x
💻 Local Nitro 2.691s (+2.9%) 3.008s (~) 0.317s 10 1.33x
🌐 MongoDB Next.js (Turbopack) 4.810s 5.346s 0.536s 6 2.37x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.649s (-5.9% 🟢) 3.700s (-7.4% 🟢) 1.051s 9 1.00x
▲ Vercel Express 2.814s (+15.7% 🔺) 3.898s (+19.4% 🔺) 1.084s 8 1.06x
▲ Vercel Next.js (Turbopack) 3.161s (+25.4% 🔺) 4.069s (+15.6% 🔺) 0.908s 8 1.19x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 3.564s 4.590s 1.027s 7 1.00x
🐘 Postgres Express 4.119s (+18.2% 🔺) 4.736s (+14.3% 🔺) 0.617s 7 1.16x
🐘 Postgres Nitro 4.142s (+10.5% 🔺) 4.744s (+6.3% 🔺) 0.602s 7 1.16x
🌐 Redis Next.js (Turbopack) 4.339s 5.011s 0.673s 6 1.22x
💻 Local Next.js (Turbopack) 7.509s 8.022s 0.513s 4 2.11x
💻 Local Express 7.673s (+0.6%) 8.020s (~) 0.347s 4 2.15x
💻 Local Nitro 7.823s (+2.0%) 8.018s (~) 0.195s 4 2.20x
🌐 MongoDB Next.js (Turbopack) 10.200s 10.686s 0.487s 3 2.86x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.995s (-7.0% 🟢) 4.312s (-13.5% 🟢) 1.317s 7 1.00x
▲ Vercel Next.js (Turbopack) 3.246s (-24.7% 🟢) 4.481s (-20.9% 🟢) 1.235s 7 1.08x
▲ Vercel Express 4.103s (+32.3% 🔺) 5.297s (+18.5% 🔺) 1.194s 7 1.37x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 1.239s 2.007s 0.767s 15 1.00x
🐘 Postgres Next.js (Turbopack) 1.381s 2.011s 0.631s 15 1.11x
🐘 Postgres Express 1.384s (-1.0%) 2.011s (~) 0.628s 15 1.12x
🐘 Postgres Nitro 1.391s (+1.3%) 2.012s (~) 0.621s 15 1.12x
💻 Local Next.js (Turbopack) 1.427s 2.006s 0.579s 15 1.15x
💻 Local Express 1.436s (+1.0%) 2.005s (~) 0.569s 15 1.16x
💻 Local Nitro 1.436s (+1.1%) 2.005s (~) 0.569s 15 1.16x
🌐 MongoDB Next.js (Turbopack) 2.153s 3.008s 0.854s 10 1.74x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.029s (-17.8% 🟢) 3.291s (-16.3% 🟢) 1.263s 10 1.00x
▲ Vercel Express 2.462s (-0.6%) 3.533s (+4.6%) 1.071s 9 1.21x
▲ Vercel Next.js (Turbopack) 2.561s (+19.9% 🔺) 4.150s (+26.3% 🔺) 1.589s 8 1.26x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.025s (-1.6%) 2.681s (~) 0.656s 12 1.00x
🐘 Postgres Next.js (Turbopack) 2.141s 2.597s 0.456s 12 1.06x
🐘 Postgres Express 2.164s (+2.5%) 2.601s (-3.0%) 0.437s 12 1.07x
🌐 Redis Next.js (Turbopack) 2.522s 3.008s 0.487s 10 1.25x
💻 Local Next.js (Turbopack) 2.639s 3.009s 0.370s 10 1.30x
💻 Local Express 2.731s (-0.5%) 3.009s (~) 0.278s 10 1.35x
💻 Local Nitro 2.766s (+1.8%) 3.109s (+3.4%) 0.343s 10 1.37x
🌐 MongoDB Next.js (Turbopack) 4.666s 5.177s 0.511s 6 2.30x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.339s (-10.5% 🟢) 3.429s (-4.8%) 1.090s 9 1.00x
▲ Vercel Nitro 2.582s (-0.7%) 3.573s (-8.2% 🟢) 0.991s 9 1.10x
▲ Vercel Express 3.632s (+43.9% 🔺) 4.614s (+33.3% 🔺) 0.981s 7 1.55x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 4.008s 4.594s 0.587s 7 1.00x
🐘 Postgres Nitro 4.139s (+11.4% 🔺) 4.893s (+9.9% 🔺) 0.754s 7 1.03x
🌐 Redis Next.js (Turbopack) 4.357s 5.013s 0.655s 6 1.09x
🐘 Postgres Express 4.830s (+27.0% 🔺) 5.529s (+24.1% 🔺) 0.699s 6 1.21x
💻 Local Next.js (Turbopack) 7.745s 8.020s 0.275s 4 1.93x
💻 Local Express 8.370s (-3.3%) 9.022s (-2.7%) 0.652s 4 2.09x
💻 Local Nitro 8.454s (-1.4%) 9.022s (-2.7%) 0.568s 4 2.11x
🌐 MongoDB Next.js (Turbopack) 10.099s 10.685s 0.586s 3 2.52x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 3.413s (+16.2% 🔺) 4.817s (+3.4%) 1.404s 7 1.00x
▲ Vercel Nitro 3.716s (+17.5% 🔺) 5.138s (+19.5% 🔺) 1.422s 6 1.09x
▲ Vercel Express 4.059s (+23.6% 🔺) 5.224s (+18.7% 🔺) 1.165s 6 1.19x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 0.145s 1.000s 0.002s 1.008s 0.863s 10 1.00x
💻 Local Next.js (Turbopack) 0.149s 1.001s 0.012s 1.018s 0.868s 10 1.03x
💻 Local Express 0.170s (-2.3%) 1.002s (~) 0.012s (+1.8%) 1.017s (~) 0.847s 10 1.17x
💻 Local Nitro 0.175s (+2.1%) 1.003s (~) 0.012s (+6.2% 🔺) 1.018s (~) 0.843s 10 1.21x
🐘 Postgres Next.js (Turbopack) 0.187s 1.002s 0.002s 1.013s 0.826s 10 1.29x
🐘 Postgres Express 0.193s (+2.2%) 0.995s (~) 0.002s (-6.3% 🟢) 1.012s (~) 0.820s 10 1.33x
🐘 Postgres Nitro 0.214s (+6.1% 🔺) 0.996s (~) 0.002s (~) 1.014s (~) 0.800s 10 1.48x
🌐 MongoDB Next.js (Turbopack) 0.498s 0.948s 0.002s 1.010s 0.511s 10 3.44x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 1.711s (-22.7% 🟢) 2.873s (-17.4% 🟢) 0.287s (+130.5% 🔺) 3.603s (-9.7% 🟢) 1.892s 10 1.00x
▲ Vercel Nitro 1.723s (+9.9% 🔺) 2.441s (+2.6%) 0.151s (-48.7% 🟢) 3.069s (-2.6%) 1.346s 10 1.01x
▲ Vercel Express 1.986s (+22.6% 🔺) 2.711s (+14.9% 🔺) 0.704s (+404.8% 🔺) 63.810s (+2111.4% 🔺) 61.824s 10 1.16x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Next.js (Turbopack) 9/12
🐘 Postgres Next.js (Turbopack) 9/12
▲ Vercel Nitro 5/12
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 6/12
Next.js (Turbopack) 🌐 Redis 6/12
Nitro 🐘 Postgres 6/12
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants