Skip to content

[core] Handle RunNotSupportedError in flow/step handlers#1661

Closed
VaguelySerious wants to merge 1 commit into
mainfrom
peter/handle-run-not-supported-error
Closed

[core] Handle RunNotSupportedError in flow/step handlers#1661
VaguelySerious wants to merge 1 commit into
mainfrom
peter/handle-run-not-supported-error

Conversation

@VaguelySerious
Copy link
Copy Markdown
Member

Summary

  • When a World rejects a run via RunNotSupportedError (e.g., community world built against an older @workflow/world sees a specVersion it doesn't support), the workflow and step handlers now catch it, log a clear error, and consume the queue message.
  • Previously the error bubbled up as a 500, the queue retried with exponential backoff, and the run stayed in pending forever — causing tests to time out at the 30-minute job limit instead of failing promptly at the per-test timeout.

Context

This is a defense-in-depth companion to #1658 (which fixes the root cause by not sending specVersion 3 runs to community worlds). Even with #1658 merged, this handler ensures that any future specVersion mismatch fails fast instead of silently timing out.

Test plan

  • All 581 core unit tests pass
  • Typecheck passes
  • CI green

🤖 Generated with Claude Code

When a World rejects a run via RunNotSupportedError (specVersion too
high), the flow and step handlers now catch it, log a clear error, and
consume the queue message. Previously the error bubbled up as a 500,
the queue retried with backoff, and tests would time out waiting for
a run that could never reach a terminal state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Apr 8, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Apr 8, 2026 8:08pm
example-nextjs-workflow-webpack Ready Ready Preview, Comment Apr 8, 2026 8:08pm
example-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-astro-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-express-workflow Building Building Preview, Comment Apr 8, 2026 8:08pm
workbench-fastify-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-hono-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-nitro-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-nuxt-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-sveltekit-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workbench-vite-workflow Ready Ready Preview, Comment Apr 8, 2026 8:08pm
workflow-docs Ready Ready Preview, Comment, Open in v0 Apr 8, 2026 8:08pm
workflow-swc-playground Ready Ready Preview, Comment Apr 8, 2026 8:08pm

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 8, 2026

🦋 Changeset detected

Latest commit: 3b641e7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
Name Type
@workflow/core Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
workflow Patch
@workflow/world-testing Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch
@workflow/ai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 8, 2026

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 0.040s (-4.0%) 1.005s (~) 0.965s 10 1.00x
💻 Local Nitro 0.044s (-1.1%) 1.006s (~) 0.962s 10 1.09x
🐘 Postgres Express 0.057s (+3.4%) 1.012s (~) 0.954s 10 1.42x
🐘 Postgres Nitro 0.058s (-5.5% 🟢) 1.010s (~) 0.952s 10 1.43x
🐘 Postgres Next.js (Turbopack) 0.058s 1.011s 0.953s 10 1.44x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 0.204s (-25.0% 🟢) 1.920s (-22.7% 🟢) 1.717s 10 1.00x
▲ Vercel Express 0.235s (-6.9% 🟢) 2.143s (-17.7% 🟢) 1.908s 10 1.16x
▲ Vercel Next.js (Turbopack) 0.293s (-31.0% 🟢) 2.444s (-2.0%) 2.150s 10 1.44x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.133s (+0.9%) 2.006s (~) 0.872s 10 1.00x
💻 Local Nitro 1.137s (~) 2.007s (~) 0.870s 10 1.00x
🐘 Postgres Next.js (Turbopack) 1.141s 2.011s 0.870s 10 1.01x
🐘 Postgres Nitro 1.141s (-0.7%) 2.011s (~) 0.870s 10 1.01x
🐘 Postgres Express 1.158s (+1.0%) 2.010s (~) 0.852s 10 1.02x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 1.943s (-7.2% 🟢) 3.807s (+1.4%) 1.864s 10 1.00x
▲ Vercel Nitro 2.031s (+5.9% 🔺) 3.703s (-4.8%) 1.672s 10 1.05x
▲ Vercel Express 2.220s (+15.8% 🔺) 3.965s (+5.9% 🔺) 1.745s 10 1.14x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 10.836s 11.023s 0.187s 3 1.00x
🐘 Postgres Nitro 10.870s (~) 11.019s (~) 0.149s 3 1.00x
💻 Local Express 10.926s (~) 11.024s (~) 0.098s 3 1.01x
🐘 Postgres Express 10.935s (~) 11.026s (~) 0.091s 3 1.01x
💻 Local Nitro 10.975s (~) 11.024s (-2.9%) 0.049s 3 1.01x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 17.146s (+0.9%) 19.205s (+1.2%) 2.059s 2 1.00x
▲ Vercel Nitro 17.381s (+2.7%) 19.251s (+1.5%) 1.870s 2 1.01x
▲ Vercel Next.js (Turbopack) 17.602s (+2.9%) 19.211s (+1.9%) 1.609s 2 1.03x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 14.421s 15.023s 0.602s 4 1.00x
🐘 Postgres Nitro 14.547s (~) 15.026s (~) 0.479s 4 1.01x
🐘 Postgres Express 14.603s (~) 15.021s (~) 0.418s 4 1.01x
💻 Local Express 14.983s (~) 15.279s (+1.7%) 0.296s 4 1.04x
💻 Local Nitro 15.043s (~) 15.780s (~) 0.737s 4 1.04x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 32.035s (+4.4%) 34.327s (+7.2% 🔺) 2.291s 2 1.00x
▲ Vercel Express 32.590s (+5.3% 🔺) 34.340s (+4.6%) 1.750s 2 1.02x
▲ Vercel Next.js (Turbopack) 32.950s (+2.7%) 34.853s (+3.3%) 1.902s 2 1.03x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 13.937s 14.164s 0.227s 7 1.00x
🐘 Postgres Nitro 14.025s (-0.6%) 14.594s (-1.0%) 0.570s 7 1.01x
🐘 Postgres Express 14.153s (+0.5%) 15.025s (+1.0%) 0.872s 6 1.02x
💻 Local Nitro 16.742s (-1.4%) 17.032s (-1.9%) 0.291s 6 1.20x
💻 Local Express 16.886s (+2.7%) 17.366s (+2.0%) 0.480s 6 1.21x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 56.373s (+2.3%) 58.146s (+1.5%) 1.772s 2 1.00x
▲ Vercel Express 56.708s (+4.4%) 58.244s (+4.1%) 1.537s 2 1.01x
▲ Vercel Next.js (Turbopack) 60.032s (+3.4%) 62.030s (+2.7%) 1.998s 2 1.06x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 1.238s 2.009s 0.772s 15 1.00x
🐘 Postgres Express 1.252s (-0.9%) 2.009s (~) 0.757s 15 1.01x
🐘 Postgres Nitro 1.269s (-0.7%) 2.011s (~) 0.742s 15 1.03x
💻 Local Express 1.535s (-1.5%) 2.005s (-3.3%) 0.471s 15 1.24x
💻 Local Nitro 1.535s (-1.2%) 2.005s (~) 0.471s 15 1.24x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.254s (-9.2% 🟢) 3.984s (-9.9% 🟢) 1.730s 8 1.00x
▲ Vercel Express 2.503s (+6.0% 🔺) 3.913s (-10.9% 🟢) 1.410s 8 1.11x
▲ Vercel Next.js (Turbopack) 2.567s (+6.4% 🔺) 4.131s (-1.0%) 1.564s 8 1.14x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 2.326s (~) 3.009s (~) 0.683s 10 1.00x
🐘 Postgres Nitro 2.350s (~) 3.008s (~) 0.657s 10 1.01x
🐘 Postgres Next.js (Turbopack) 2.385s 3.009s 0.624s 10 1.03x
💻 Local Express 3.050s (+7.6% 🔺) 3.760s (+21.0% 🔺) 0.710s 8 1.31x
💻 Local Nitro 3.082s (-2.4%) 3.760s (-6.3% 🟢) 0.678s 8 1.32x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.659s (-3.9%) 4.028s (-18.1% 🟢) 1.369s 8 1.00x
▲ Vercel Nitro 3.693s (+44.2% 🔺) 5.842s (+36.2% 🔺) 2.149s 6 1.39x
▲ Vercel Express 3.716s (+36.8% 🔺) 5.431s (+17.4% 🔺) 1.715s 6 1.40x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 3.479s (~) 4.008s (~) 0.530s 8 1.00x
🐘 Postgres Nitro 3.485s (+0.5%) 4.013s (~) 0.527s 8 1.00x
🐘 Postgres Next.js (Turbopack) 3.629s 4.012s 0.383s 8 1.04x
💻 Local Express 8.539s (+5.5% 🔺) 9.023s (+5.8% 🔺) 0.484s 4 2.45x
💻 Local Nitro 8.942s (~) 9.522s (+2.7%) 0.580s 4 2.57x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.829s (-4.4%) 4.556s (-10.0% 🟢) 1.727s 7 1.00x
▲ Vercel Express 2.954s (-9.4% 🟢) 4.930s (-3.8%) 1.976s 7 1.04x
▲ Vercel Next.js (Turbopack) 153.702s (+3822.0% 🔺) 156.472s (+2568.4% 🔺) 2.770s 1 54.33x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 1.212s 2.009s 0.797s 15 1.00x
🐘 Postgres Nitro 1.246s (~) 2.008s (~) 0.762s 15 1.03x
🐘 Postgres Express 1.260s (+0.8%) 2.007s (~) 0.748s 15 1.04x
💻 Local Express 1.530s (+1.5%) 2.006s (~) 0.477s 15 1.26x
💻 Local Nitro 1.561s (~) 2.006s (~) 0.444s 15 1.29x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.150s (+7.4% 🔺) 3.908s (+6.5% 🔺) 1.758s 8 1.00x
▲ Vercel Nitro 2.382s (+14.1% 🔺) 3.758s (+1.3%) 1.376s 8 1.11x
▲ Vercel Express 53.210s (+2404.6% 🔺) 54.782s (+1306.9% 🔺) 1.572s 6 24.75x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.328s (-0.5%) 3.010s (~) 0.682s 10 1.00x
🐘 Postgres Express 2.340s (+1.0%) 3.010s (~) 0.670s 10 1.00x
🐘 Postgres Next.js (Turbopack) 2.364s 3.008s 0.644s 10 1.02x
💻 Local Express 2.917s (-17.4% 🟢) 3.309s (-23.0% 🟢) 0.392s 10 1.25x
💻 Local Nitro 3.100s (-1.4%) 3.886s (-3.1%) 0.785s 8 1.33x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.686s (-7.7% 🟢) 4.420s (-14.7% 🟢) 1.734s 7 1.00x
▲ Vercel Express 2.812s (+11.2% 🔺) 4.404s (-4.6%) 1.591s 7 1.05x
▲ Vercel Nitro 3.311s (+31.4% 🔺) 5.010s (+16.5% 🔺) 1.699s 6 1.23x

🔍 Observability: Next.js (Turbopack) | Express | Nitro

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 3.457s (~) 4.009s (~) 0.552s 8 1.00x
🐘 Postgres Nitro 3.477s (+0.6%) 4.009s (~) 0.533s 8 1.01x
🐘 Postgres Next.js (Turbopack) 3.664s 4.014s 0.350s 8 1.06x
💻 Local Nitro 8.689s (-6.6% 🟢) 9.023s (-10.0% 🟢) 0.334s 4 2.51x
💻 Local Express 9.050s (+2.4%) 9.776s (+5.4% 🔺) 0.726s 4 2.62x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.039s (-1.3%) 4.904s (+6.4% 🔺) 1.865s 7 1.00x
▲ Vercel Express 3.571s (+24.0% 🔺) 5.365s (+15.8% 🔺) 1.794s 6 1.17x
▲ Vercel Next.js (Turbopack) 3.961s (+9.4% 🔺) 5.828s (+5.0%) 1.868s 6 1.30x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 10 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.755s 1.006s 0.252s 60 1.00x
🐘 Postgres Nitro 0.829s (-1.7%) 1.006s (~) 0.177s 60 1.10x
🐘 Postgres Express 0.844s (-5.3% 🟢) 1.022s (-3.4%) 0.178s 60 1.12x
💻 Local Nitro 0.997s (-1.8%) 1.469s (-18.4% 🟢) 0.472s 41 1.32x
💻 Local Express 1.049s (+7.7% 🔺) 1.881s (+68.7% 🔺) 0.832s 32 1.39x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 9.250s (-1.9%) 10.875s (-4.7%) 1.625s 6 1.00x
▲ Vercel Express 9.301s (+2.2%) 10.899s (-1.8%) 1.598s 6 1.01x
▲ Vercel Next.js (Turbopack) 9.928s (-8.3% 🟢) 11.820s (-6.4% 🟢) 1.892s 6 1.07x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 25 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 1.840s 2.029s 0.188s 45 1.00x
🐘 Postgres Nitro 2.033s (~) 2.685s (-4.8%) 0.652s 34 1.10x
🐘 Postgres Express 2.046s (+3.6%) 2.767s (+19.5% 🔺) 0.721s 33 1.11x
💻 Local Nitro 3.067s (-0.6%) 3.923s (-2.2%) 0.856s 23 1.67x
💻 Local Express 3.104s (+4.4%) 3.801s (+16.7% 🔺) 0.697s 24 1.69x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 27.345s (-9.0% 🟢) 29.125s (-9.3% 🟢) 1.780s 4 1.00x
▲ Vercel Next.js (Turbopack) 30.209s (+1.4%) 32.021s (+1.5%) 1.812s 3 1.10x
▲ Vercel Express 31.047s (+7.9% 🔺) 33.175s (+8.7% 🔺) 2.129s 3 1.14x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

workflow with 50 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 3.812s 4.109s 0.297s 30 1.00x
🐘 Postgres Nitro 4.097s (-1.9%) 4.850s (-3.2%) 0.752s 25 1.07x
🐘 Postgres Express 4.152s (+1.9%) 4.892s (+4.9%) 0.740s 25 1.09x
💻 Local Nitro 9.058s (-3.9%) 9.633s (-3.9%) 0.575s 13 2.38x
💻 Local Express 9.122s (+0.7%) 9.710s (+1.6%) 0.588s 13 2.39x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 73.784s (~) 75.489s (-1.1%) 1.706s 2 1.00x
▲ Vercel Express 76.685s (+2.3%) 78.261s (+2.1%) 1.576s 2 1.04x
▲ Vercel Next.js (Turbopack) 92.298s (+12.7% 🔺) 93.855s (+11.4% 🔺) 1.557s 2 1.25x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 10 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.252s 1.007s 0.755s 60 1.00x
🐘 Postgres Nitro 0.282s (-1.2%) 1.007s (~) 0.725s 60 1.12x
🐘 Postgres Express 0.284s (+2.4%) 1.007s (~) 0.723s 60 1.13x
💻 Local Express 0.589s (-1.2%) 1.021s (+1.6%) 0.432s 60 2.33x
💻 Local Nitro 0.595s (-4.4%) 1.004s (-1.7%) 0.409s 60 2.36x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 1.651s (-15.7% 🟢) 3.163s (-27.2% 🟢) 1.513s 19 1.00x
▲ Vercel Next.js (Turbopack) 1.836s (-6.6% 🟢) 3.480s (-6.4% 🟢) 1.644s 18 1.11x
▲ Vercel Nitro 3.405s (+132.7% 🔺) 5.031s (+55.9% 🔺) 1.626s 13 2.06x

🔍 Observability: Express | Next.js (Turbopack) | Nitro

workflow with 25 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.475s 1.006s 0.531s 90 1.00x
🐘 Postgres Express 0.480s (-1.2%) 1.006s (~) 0.526s 90 1.01x
🐘 Postgres Nitro 0.515s (+2.1%) 1.007s (~) 0.493s 90 1.08x
💻 Local Express 2.459s (-1.5%) 3.008s (~) 0.549s 30 5.18x
💻 Local Nitro 2.593s (~) 3.010s (~) 0.417s 30 5.46x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.843s (+2.0%) 4.358s (-8.0% 🟢) 1.515s 21 1.00x
▲ Vercel Nitro 2.977s (+9.0% 🔺) 4.772s (+5.2% 🔺) 1.795s 19 1.05x
▲ Vercel Next.js (Turbopack) 3.421s (-4.4%) 5.039s (-4.4%) 1.618s 18 1.20x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 50 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.743s 1.006s 0.263s 120 1.00x
🐘 Postgres Express 0.794s (-1.1%) 1.008s (~) 0.214s 120 1.07x
🐘 Postgres Nitro 0.801s (~) 1.008s (-0.9%) 0.207s 120 1.08x
💻 Local Express 10.753s (-2.6%) 11.302s (-1.6%) 0.549s 11 14.46x
💻 Local Nitro 11.322s (-2.0%) 11.845s (-1.5%) 0.523s 11 15.23x
💻 Local Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 6.627s (-3.2%) 8.476s (+0.8%) 1.849s 15 1.00x
▲ Vercel Express 6.856s (+13.7% 🔺) 8.803s (+13.7% 🔺) 1.948s 15 1.03x
▲ Vercel Nitro 6.987s (+6.5% 🔺) 8.698s (+4.0%) 1.710s 14 1.05x

🔍 Observability: Next.js (Turbopack) | Express | Nitro

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.197s (-2.0%) 0.998s (~) 0.002s (-11.8% 🟢) 1.010s (~) 0.813s 10 1.00x
💻 Local Nitro 0.205s (-4.3%) 1.005s (~) 0.012s (-9.3% 🟢) 1.018s (~) 0.813s 10 1.04x
🐘 Postgres Express 0.206s (+1.6%) 0.996s (~) 0.001s (+8.3% 🔺) 1.009s (~) 0.803s 10 1.05x
💻 Local Express 0.219s (+9.4% 🔺) 1.004s (~) 0.011s (-3.4%) 1.018s (~) 0.799s 10 1.11x
🐘 Postgres Next.js (Turbopack) 0.326s 1.101s 0.002s 1.111s 0.785s 10 1.65x
💻 Local Next.js (Turbopack) ⚠️ missing - - - - -

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 1.545s (-10.9% 🟢) 2.893s (+7.2% 🔺) 0.666s (+5.0% 🔺) 3.947s (-1.4%) 2.402s 10 1.00x
▲ Vercel Express 1.702s (-33.3% 🟢) 3.198s (-21.6% 🟢) 0.389s (+52.0% 🔺) 4.044s (-15.4% 🟢) 2.341s 10 1.10x
▲ Vercel Next.js (Turbopack) 1.770s (-45.3% 🟢) 3.249s (-33.3% 🟢) 0.343s (-9.6% 🟢) 4.091s (-27.8% 🟢) 2.320s 10 1.15x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

stream pipeline with 5 transform steps (1MB)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.585s 1.008s 0.002s 1.018s 0.433s 59 1.00x
🐘 Postgres Express 0.617s (-5.6% 🟢) 1.002s (~) 0.004s (-29.1% 🟢) 1.021s (~) 0.404s 59 1.05x
🐘 Postgres Nitro 0.631s (+2.4%) 1.005s (~) 0.004s (+4.2%) 1.022s (~) 0.391s 59 1.08x
💻 Local Express 0.719s (-10.7% 🟢) 1.012s (~) 0.010s (-1.7%) 1.023s (-8.3% 🟢) 0.304s 59 1.23x
💻 Local Nitro 0.731s (-4.4%) 1.012s (~) 0.010s (-5.2% 🟢) 1.024s (~) 0.292s 59 1.25x
💻 Local Next.js (Turbopack) ⚠️ missing - - - - -

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 4.180s (+11.3% 🔺) 5.654s (+5.6% 🔺) 0.200s (-28.0% 🟢) 6.373s (+5.3% 🔺) 2.193s 10 1.00x
▲ Vercel Next.js (Turbopack) 5.810s (+40.5% 🔺) 7.290s (+24.4% 🔺) 0.371s (-1.0%) 8.125s (+20.7% 🔺) 2.314s 8 1.39x
▲ Vercel Express 309.315s (+7049.6% 🔺) 310.905s (+5095.4% 🔺) 0.280s (-3.2%) 311.565s (+4546.8% 🔺) 2.250s 1 74.01x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

10 parallel streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.933s 1.133s 0.000s 1.139s 0.206s 53 1.00x
🐘 Postgres Nitro 0.942s (-0.8%) 1.168s (~) 0.000s (~) 1.184s (~) 0.242s 52 1.01x
🐘 Postgres Express 0.946s (~) 1.149s (-5.8% 🟢) 0.000s (-100.0% 🟢) 1.161s (-5.8% 🟢) 0.216s 52 1.01x
💻 Local Express 1.243s (-11.4% 🟢) 2.021s (~) 0.000s (+40.0% 🔺) 2.023s (-8.0% 🟢) 0.780s 30 1.33x
💻 Local Nitro 1.256s (-3.9%) 2.021s (~) 0.000s (-21.4% 🟢) 2.023s (~) 0.767s 30 1.35x
💻 Local Next.js (Turbopack) ⚠️ missing - - - - -

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.734s (+7.8% 🔺) 4.310s (+11.1% 🔺) 0.000s (-100.0% 🟢) 4.709s (+4.7%) 1.975s 13 1.00x
▲ Vercel Nitro 2.921s (+9.6% 🔺) 4.279s (+5.4% 🔺) 0.000s (-100.0% 🟢) 4.661s (+1.0%) 1.740s 13 1.07x
▲ Vercel Next.js (Turbopack) 3.220s (-3.9%) 4.652s (-5.3% 🟢) 0.000s (-97.8% 🟢) 5.097s (-5.0% 🟢) 1.876s 12 1.18x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

fan-out fan-in 10 streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.753s (+0.6%) 2.101s (~) 0.000s (+50.0% 🔺) 2.113s (~) 0.360s 29 1.00x
🐘 Postgres Next.js (Turbopack) 1.802s 2.106s 0.000s 2.121s 0.319s 29 1.03x
🐘 Postgres Nitro 1.814s (+3.6%) 2.176s (+5.2% 🔺) 0.000s (+3.6%) 2.191s (+3.9%) 0.377s 28 1.03x
💻 Local Nitro 3.568s (-4.3%) 4.034s (-1.7%) 0.000s (~) 4.036s (-1.7%) 0.468s 15 2.03x
💻 Local Express 3.602s (+5.1% 🔺) 4.100s (+3.3%) 0.001s (+6.7% 🔺) 4.103s (+3.3%) 0.500s 15 2.05x
💻 Local Next.js (Turbopack) ⚠️ missing - - - - -

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 4.252s (-1.0%) 5.610s (-2.9%) 0.000s (-50.0% 🟢) 6.056s (-3.8%) 1.804s 10 1.00x
▲ Vercel Nitro 4.471s (+19.8% 🔺) 5.624s (+14.2% 🔺) 0.000s (-86.4% 🟢) 6.031s (+13.2% 🔺) 1.561s 11 1.05x
▲ Vercel Express 4.510s (+8.0% 🔺) 6.005s (+7.5% 🔺) 0.000s (+Infinity% 🔺) 6.463s (+6.8% 🔺) 1.953s 10 1.06x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Express 14/21
🐘 Postgres Next.js (Turbopack) 14/21
▲ Vercel Nitro 11/21
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 17/21
Next.js (Turbopack) 🐘 Postgres 21/21
Nitro 🐘 Postgres 17/21
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 8, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 890 0 67 957
✅ 💻 Local Development 866 0 178 1044
✅ 📦 Local Production 866 0 178 1044
✅ 🐘 Local Postgres 866 0 178 1044
✅ 🪟 Windows 79 0 8 87
❌ 🌍 Community Worlds 16 61 8 85
✅ 📋 Other 219 0 42 261
Total 3802 61 659 4522

❌ Failed Tests

🌍 Community Worlds (61 failed)

mongodb-dev (1 failed):

  • dev e2e should rebuild on imported step dependency change

redis-dev (1 failed):

  • dev e2e should rebuild on imported step dependency change

turso-dev (1 failed):

  • dev e2e should rebuild on imported step dependency change

turso (58 failed):

  • addTenWorkflow | wrun_01KNQBAJCCZB4XPFSZJ18H5JRQ
  • addTenWorkflow | wrun_01KNQBAJCCZB4XPFSZJ18H5JRQ
  • wellKnownAgentWorkflow (.well-known/agent) | wrun_01KNQBC482Z2KQSE2HRV10472H
  • should work with react rendering in step
  • promiseAllWorkflow | wrun_01KNQBASE63ZD0HK9ANQ5THYDB
  • promiseRaceWorkflow | wrun_01KNQBAXKS8PSCD2QRJKYW6YVY
  • promiseAnyWorkflow | wrun_01KNQBAZJNPG0H45VW06G7PQ4F
  • importedStepOnlyWorkflow | wrun_01KNQBCFGY1MDN5MPVGPSCTVJF
  • hookWorkflow | wrun_01KNQBBBADMJ5M8X3FARAY0TD5
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KNQBBMT7ETB85TEB09DHVKK3
  • webhookWorkflow | wrun_01KNQBBXAZHKQJCBE8P1NM54TX
  • sleepingWorkflow | wrun_01KNQBC485BXS9PC3763D6WSF6
  • parallelSleepWorkflow | wrun_01KNQBCGCZY693NQXAC639JKH6
  • nullByteWorkflow | wrun_01KNQBCMC5PZJWWD2Y3WDHN18F
  • workflowAndStepMetadataWorkflow | wrun_01KNQBCP42WP6YDDDFJ7KM0H9X
  • fetchWorkflow | wrun_01KNQBF9KTA6105362A4T5AJQ4
  • promiseRaceStressTestWorkflow | wrun_01KNQBFCK635R7VS0ER910XHPK
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • error handling not registered WorkflowNotRegisteredError fails the run when workflow does not exist
  • error handling not registered StepNotRegisteredError fails the step but workflow can catch it
  • error handling not registered StepNotRegisteredError fails the run when not caught in workflow
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01KNQBJHHXCW1TA9T28Y0E4QAT
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KNQBK46HV4BSB2Z96S5FKPVR
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running | wrun_01KNQBKPT97QCP0NX8DE7KP53V
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars) | wrun_01KNQBM9AHQT8NGJGT4P3WRZBX
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument | wrun_01KNQBMGYZ8M70EDPRFXCJWVDH
  • closureVariableWorkflow - nested step functions with closure variables | wrun_01KNQBMNTPKHF73WSCGK3VB4R7
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KNQBMR2B6ECG0GR95QZD6GQZ
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly | wrun_01KNQBN55GVXZH77QS1RZ0DDGG
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01KNQBN9VS9ZB9T98QGN2ZPMND
  • AllInOneService.processNumber - static workflow method using sibling static step methods | wrun_01KNQBNGCT2X7VDQTQV2JN7B2C
  • ChainableService.processWithThis - static step methods using this to reference the class | wrun_01KNQBNP6THV585M4Q6E16TS3G
  • thisSerializationWorkflow - step function invoked with .call() and .apply() | wrun_01KNQBNW7VRE07K23P2NFNSKHV
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KNQBP32FS0XK0DDGACESKAP8
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KNQBP9Y2YGD1179XBN7R445P
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KNQBPKK89E8CCC8GDTSG9PYF
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01KNQBPTGJQBK4X79KHRX46GNG
  • cancelRun - cancelling a running workflow | wrun_01KNQBQ2QW68G7A4Q0XW29CBQP
  • cancelRun via CLI - cancelling a running workflow | wrun_01KNQBQB5WY5ES1QQPJ5HT1PNC
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router
  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01KNQBQP1P2PB33BYN5WXREHBQ
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KNQBR8XDSZPEMPC49PJMCD32
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01KNQBRKWNZVP40FFFG8EWSG6E
  • importMetaUrlWorkflow - import.meta.url is available in step bundles | wrun_01KNQBRSNJH7QSA9AMP3FS5ANR
  • metadataFromHelperWorkflow - getWorkflowMetadata/getStepMetadata work from module-level helper (#1577) | wrun_01KNQBRVDJ1TECKD9QB5TQJYD7
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KNQBRX6CAH5MFSJGDE9PWP9J
  • getterStepWorkflow - getter functions with "use step" directive | wrun_01KNQBS2K7WTHVQ5GM8M4J8HYC

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 80 0 7
✅ example 80 0 7
✅ express 80 0 7
✅ fastify 80 0 7
✅ hono 80 0 7
✅ nextjs-turbopack 85 0 2
✅ nextjs-webpack 85 0 2
✅ nitro 80 0 7
✅ nuxt 80 0 7
✅ sveltekit 80 0 7
✅ vite 80 0 7
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 73 0 14
✅ express-stable 73 0 14
✅ fastify-stable 73 0 14
✅ hono-stable 73 0 14
✅ nextjs-turbopack-canary 62 0 25
✅ nextjs-turbopack-stable 79 0 8
✅ nextjs-webpack-canary 62 0 25
✅ nextjs-webpack-stable 79 0 8
✅ nitro-stable 73 0 14
✅ nuxt-stable 73 0 14
✅ sveltekit-stable 73 0 14
✅ vite-stable 73 0 14
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 73 0 14
✅ express-stable 73 0 14
✅ fastify-stable 73 0 14
✅ hono-stable 73 0 14
✅ nextjs-turbopack-canary 62 0 25
✅ nextjs-turbopack-stable 79 0 8
✅ nextjs-webpack-canary 62 0 25
✅ nextjs-webpack-stable 79 0 8
✅ nitro-stable 73 0 14
✅ nuxt-stable 73 0 14
✅ sveltekit-stable 73 0 14
✅ vite-stable 73 0 14
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 73 0 14
✅ express-stable 73 0 14
✅ fastify-stable 73 0 14
✅ hono-stable 73 0 14
✅ nextjs-turbopack-canary 62 0 25
✅ nextjs-turbopack-stable 79 0 8
✅ nextjs-webpack-canary 62 0 25
✅ nextjs-webpack-stable 79 0 8
✅ nitro-stable 73 0 14
✅ nuxt-stable 73 0 14
✅ sveltekit-stable 73 0 14
✅ vite-stable 73 0 14
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 79 0 8
❌ 🌍 Community Worlds
App Passed Failed Skipped
❌ mongodb-dev 4 1 0
❌ redis-dev 4 1 0
❌ turso-dev 4 1 0
❌ turso 4 58 8
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 73 0 14
✅ e2e-local-postgres-nest-stable 73 0 14
✅ e2e-local-prod-nest-stable 73 0 14

📋 View full workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant