Skip to content

[ai] Convert tool call errors to error-text results#1212

Merged
VaguelySerious merged 2 commits into
mainfrom
peter/agent-tool-call
Feb 27, 2026
Merged

[ai] Convert tool call errors to error-text results#1212
VaguelySerious merged 2 commits into
mainfrom
peter/agent-tool-call

Conversation

@VaguelySerious
Copy link
Copy Markdown
Member

@VaguelySerious VaguelySerious commented Feb 27, 2026 •
edited
Loading

Fixes #1180. Also see #376

@VaguelySerious
[ai] Convert tool call errors to error-text results
a183b54 
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Feb 27, 2026 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
example-nextjs-workflow-webpack Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
example-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-astro-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-express-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-fastify-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-hono-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-nitro-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-nuxt-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-sveltekit-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workbench-vite-workflow Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workflow-docs Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workflow-nest Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm
workflow-swc-playground Ready Ready Preview, Comment, Open in v0 Feb 27, 2026 6:43pm

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Feb 27, 2026 •
edited
Loading

🦋 Changeset detected

Latest commit: cfb7374

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@workflow/ai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 27, 2026 •
edited
Loading

🧪 E2E Test Results

❌ Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 523 0 49 572
✅ 💻 Local Development 556 0 68 624
✅ 📦 Local Production 556 0 68 624
❌ 🐘 Local Postgres 555 1 68 624
✅ 🪟 Windows 49 0 3 52
❌ 🌍 Community Worlds 111 45 9 165
✅ 📋 Other 135 0 21 156
Total 2485 46 286 2817

❌ Failed Tests

🐘 Local Postgres (1 failed)

express-stable (1 failed):

  • webhookWorkflow
🌍 Community Worlds (45 failed)

turso (45 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • parallelSleepWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling retry behavior workflow completes despite transient 5xx on step_completed
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • instanceMethodStepWorkflow - instance methods with "use step" directive
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument
  • cancelRun - cancelling a running workflow
  • cancelRun via CLI - cancelling a running workflow
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 47 0 5
✅ example 47 0 5
✅ express 47 0 5
✅ fastify 47 0 5
✅ hono 47 0 5
✅ nextjs-turbopack 50 0 2
✅ nextjs-webpack 50 0 2
✅ nitro 47 0 5
✅ nuxt 47 0 5
✅ sveltekit 47 0 5
✅ vite 47 0 5
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 45 0 7
✅ express-stable 45 0 7
✅ fastify-stable 45 0 7
✅ hono-stable 45 0 7
✅ nextjs-turbopack-canary 49 0 3
✅ nextjs-turbopack-stable 49 0 3
✅ nextjs-webpack-canary 49 0 3
✅ nextjs-webpack-stable 49 0 3
✅ nitro-stable 45 0 7
✅ nuxt-stable 45 0 7
✅ sveltekit-stable 45 0 7
✅ vite-stable 45 0 7
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 45 0 7
✅ express-stable 45 0 7
✅ fastify-stable 45 0 7
✅ hono-stable 45 0 7
✅ nextjs-turbopack-canary 49 0 3
✅ nextjs-turbopack-stable 49 0 3
✅ nextjs-webpack-canary 49 0 3
✅ nextjs-webpack-stable 49 0 3
✅ nitro-stable 45 0 7
✅ nuxt-stable 45 0 7
✅ sveltekit-stable 45 0 7
✅ vite-stable 45 0 7
❌ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 45 0 7
❌ express-stable 44 1 7
✅ fastify-stable 45 0 7
✅ hono-stable 45 0 7
✅ nextjs-turbopack-canary 49 0 3
✅ nextjs-turbopack-stable 49 0 3
✅ nextjs-webpack-canary 49 0 3
✅ nextjs-webpack-stable 49 0 3
✅ nitro-stable 45 0 7
✅ nuxt-stable 45 0 7
✅ sveltekit-stable 45 0 7
✅ vite-stable 45 0 7
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 49 0 3
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 3 0 0
✅ mongodb 49 0 3
✅ redis-dev 3 0 0
✅ redis 49 0 3
✅ turso-dev 3 0 0
❌ turso 4 45 3
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 45 0 7
✅ e2e-local-postgres-nest-stable 45 0 7
✅ e2e-local-prod-nest-stable 45 0 7

📋 View full workflow run

❌ Some E2E test jobs failed:

  • Vercel Prod: success
  • Local Dev: success
  • Local Prod: success
  • Local Postgres: failure
  • Windows: success

Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 27, 2026 •
edited
Loading

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Nitro 0.032s (+21.4% 🔺) 1.005s (~) 0.973s 10 1.00x
💻 Local Express 0.032s (+1.9%) 1.005s (~) 0.973s 10 1.01x
💻 Local Next.js (Turbopack) 0.034s 1.005s 0.971s 10 1.07x
🌐 Redis Next.js (Turbopack) 0.050s 1.005s 0.955s 10 1.56x
🐘 Postgres Nitro 0.056s (+5.3% 🔺) 1.012s (~) 0.956s 10 1.76x
🐘 Postgres Express 0.060s (-4.6%) 1.010s (~) 0.950s 10 1.88x
🌐 MongoDB Next.js (Turbopack) 0.097s 1.008s 0.910s 10 3.06x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 0.402s (-38.9% 🟢) 1.720s (-21.5% 🟢) 1.318s 10 1.00x
▲ Vercel Nitro 0.506s (-2.2%) 2.116s (+11.6% 🔺) 1.610s 10 1.26x
▲ Vercel Next.js (Turbopack) 0.596s (-14.0% 🟢) 1.881s (-9.9% 🟢) 1.285s 10 1.49x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 1.072s 2.006s 0.935s 10 1.00x
💻 Local Nitro 1.105s (+2.7%) 2.006s (~) 0.901s 10 1.03x
💻 Local Express 1.110s (+0.6%) 2.006s (~) 0.896s 10 1.04x
🌐 Redis Next.js (Turbopack) 1.117s 2.006s 0.889s 10 1.04x
🐘 Postgres Express 1.120s (-1.9%) 2.012s (~) 0.892s 10 1.05x
🐘 Postgres Nitro 1.128s (+0.5%) 2.011s (~) 0.883s 10 1.05x
🌐 MongoDB Next.js (Turbopack) 1.318s 2.010s 0.691s 10 1.23x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.004s (-6.9% 🟢) 2.900s (-12.7% 🟢) 0.896s 10 1.00x
▲ Vercel Nitro 2.116s (-6.8% 🟢) 3.350s (-5.8% 🟢) 1.234s 10 1.06x
▲ Vercel Next.js (Turbopack) 2.127s (-0.6%) 3.327s (-2.0%) 1.201s 10 1.06x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 10.590s 11.020s 0.431s 3 1.00x
🌐 Redis Next.js (Turbopack) 10.754s 11.023s 0.269s 3 1.02x
💻 Local Nitro 10.817s (+2.2%) 11.022s (~) 0.205s 3 1.02x
💻 Local Express 10.857s (~) 11.022s (~) 0.166s 3 1.03x
🐘 Postgres Nitro 10.888s (+0.6%) 11.046s (~) 0.158s 3 1.03x
🐘 Postgres Express 10.897s (~) 11.040s (~) 0.144s 3 1.03x
🌐 MongoDB Next.js (Turbopack) 12.318s 13.024s 0.706s 3 1.16x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 17.068s (-3.3%) 18.952s (~) 1.884s 2 1.00x
▲ Vercel Nitro 17.939s (+8.3% 🔺) 19.325s (+5.3% 🔺) 1.386s 2 1.05x
▲ Vercel Express 18.625s (+9.8% 🔺) 19.393s (+7.4% 🔺) 0.768s 2 1.09x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 26.852s 27.048s 0.196s 3 1.00x
🌐 Redis Next.js (Turbopack) 27.081s 27.720s 0.639s 3 1.01x
🐘 Postgres Nitro 27.231s (~) 28.066s (~) 0.835s 3 1.01x
🐘 Postgres Express 27.283s (~) 28.066s (~) 0.782s 3 1.02x
💻 Local Nitro 27.487s (+2.5%) 28.050s (+3.7%) 0.563s 3 1.02x
💻 Local Express 27.509s (~) 28.052s (~) 0.543s 3 1.02x
🌐 MongoDB Next.js (Turbopack) 30.422s 31.032s 0.610s 2 1.13x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 43.949s (-3.3%) 47.634s (+2.5%) 3.685s 2 1.00x
▲ Vercel Nitro 44.884s (-0.6%) 45.638s (-2.3%) 0.754s 2 1.02x
▲ Vercel Express 46.038s (+4.0%) 46.948s (+2.0%) 0.911s 2 1.05x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 54.382s 55.094s 0.712s 2 1.00x
🐘 Postgres Express 55.037s (-0.6%) 55.600s (-0.9%) 0.563s 2 1.01x
🐘 Postgres Nitro 55.084s (~) 55.600s (+0.9%) 0.515s 2 1.01x
💻 Local Next.js (Turbopack) 55.987s 56.094s 0.107s 2 1.03x
💻 Local Nitro 57.425s (+2.9%) 58.103s (+3.6%) 0.679s 2 1.06x
💻 Local Express 57.468s (~) 58.102s (~) 0.634s 2 1.06x
🌐 MongoDB Next.js (Turbopack) 60.828s 61.050s 0.222s 2 1.12x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 93.948s (-5.2% 🟢) 95.173s (-5.2% 🟢) 1.225s 1 1.00x
▲ Vercel Nitro 95.420s (-0.7%) 97.132s (-0.7%) 1.712s 1 1.02x
▲ Vercel Express 96.516s (-1.0%) 97.233s (-1.5%) 0.717s 1 1.03x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 1.252s 2.006s 0.754s 15 1.00x
💻 Local Next.js (Turbopack) 1.357s 2.005s 0.648s 15 1.08x
🐘 Postgres Express 1.366s (-2.5%) 2.010s (~) 0.644s 15 1.09x
🐘 Postgres Nitro 1.386s (+2.3%) 2.010s (~) 0.625s 15 1.11x
💻 Local Nitro 1.402s (+1.6%) 2.005s (~) 0.603s 15 1.12x
💻 Local Express 1.426s (+0.7%) 2.005s (~) 0.579s 15 1.14x
🌐 MongoDB Next.js (Turbopack) 2.140s 3.007s 0.868s 10 1.71x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.197s (-14.0% 🟢) 3.342s (-9.8% 🟢) 1.145s 10 1.00x
▲ Vercel Express 2.219s (-7.8% 🟢) 3.065s (-16.6% 🟢) 0.847s 10 1.01x
▲ Vercel Nitro 2.421s (+2.7%) 3.658s (+1.7%) 1.237s 9 1.10x

🔍 Observability: Next.js (Turbopack) | Express | Nitro

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.932s (-12.8% 🟢) 2.476s (-9.8% 🟢) 0.544s 13 1.00x
🐘 Postgres Nitro 1.992s (-0.6%) 2.396s (-4.7%) 0.404s 13 1.03x
💻 Local Next.js (Turbopack) 2.415s 3.007s 0.592s 10 1.25x
🌐 Redis Next.js (Turbopack) 2.505s 3.008s 0.503s 10 1.30x
💻 Local Express 2.564s (~) 3.008s (~) 0.444s 10 1.33x
💻 Local Nitro 2.631s (+14.3% 🔺) 3.007s (~) 0.377s 10 1.36x
🌐 MongoDB Next.js (Turbopack) 4.666s 5.176s 0.510s 6 2.41x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.850s (+4.9%) 3.791s (-1.1%) 0.941s 8 1.00x
▲ Vercel Nitro 3.317s (-2.2%) 4.491s (-0.6%) 1.175s 7 1.16x
▲ Vercel Next.js (Turbopack) 3.449s (+8.9% 🔺) 4.340s (-4.8%) 0.891s 7 1.21x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 3.449s (+2.6%) 4.147s (-3.8%) 0.698s 8 1.00x
🐘 Postgres Nitro 3.634s (~) 4.598s (+7.8% 🔺) 0.964s 7 1.05x
🌐 Redis Next.js (Turbopack) 4.041s 4.439s 0.398s 7 1.17x
💻 Local Next.js (Turbopack) 5.648s 6.412s 0.764s 5 1.64x
💻 Local Express 7.287s (+0.7%) 8.017s (~) 0.730s 4 2.11x
💻 Local Nitro 7.611s (+19.9% 🔺) 8.018s (+14.3% 🔺) 0.407s 4 2.21x
🌐 MongoDB Next.js (Turbopack) 9.931s 10.346s 0.415s 3 2.88x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 3.069s (~) 3.814s (-1.1%) 0.745s 8 1.00x
▲ Vercel Nitro 3.400s (-1.0%) 4.603s (~) 1.203s 7 1.11x
▲ Vercel Next.js (Turbopack) 3.589s (+17.9% 🔺) 4.766s (+17.7% 🔺) 1.177s 7 1.17x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Redis 🥇 Next.js (Turbopack) 1.251s 2.006s 0.754s 15 1.00x
🐘 Postgres Express 1.373s (-1.9%) 2.011s (~) 0.638s 15 1.10x
💻 Local Next.js (Turbopack) 1.376s 2.004s 0.629s 15 1.10x
🐘 Postgres Nitro 1.387s (+1.9%) 2.010s (~) 0.623s 15 1.11x
💻 Local Nitro 1.420s (+3.3%) 2.005s (~) 0.585s 15 1.13x
💻 Local Express 1.423s (~) 2.005s (~) 0.582s 15 1.14x
🌐 MongoDB Next.js (Turbopack) 2.180s 3.007s 0.827s 10 1.74x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.139s (+1.6%) 2.976s (-9.8% 🟢) 0.838s 11 1.00x
▲ Vercel Nitro 2.250s (+3.2%) 3.446s (+0.5%) 1.196s 9 1.05x
▲ Vercel Next.js (Turbopack) 2.272s (-8.5% 🟢) 3.424s (-2.9%) 1.152s 9 1.06x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.022s (-1.2%) 2.515s (-3.3%) 0.493s 12 1.00x
🐘 Postgres Express 2.037s (~) 2.597s (+3.3%) 0.559s 12 1.01x
💻 Local Next.js (Turbopack) 2.446s 3.009s 0.563s 10 1.21x
🌐 Redis Next.js (Turbopack) 2.500s 3.007s 0.508s 10 1.24x
💻 Local Express 2.693s (~) 3.007s (~) 0.314s 10 1.33x
💻 Local Nitro 2.770s (+12.7% 🔺) 3.108s (+3.3%) 0.338s 10 1.37x
🌐 MongoDB Next.js (Turbopack) 4.808s 5.175s 0.367s 6 2.38x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.435s (+7.1% 🔺) 3.265s (-3.0%) 0.830s 10 1.00x
▲ Vercel Nitro 2.701s (-11.4% 🟢) 3.751s (-9.5% 🟢) 1.051s 8 1.11x
▲ Vercel Next.js (Turbopack) 3.167s (+13.5% 🔺) 4.043s (+1.0%) 0.876s 8 1.30x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 3.534s (-8.8% 🟢) 4.452s (-6.1% 🟢) 0.918s 7 1.00x
🐘 Postgres Nitro 3.541s (+4.1%) 4.310s (+4.0%) 0.769s 7 1.00x
🌐 Redis Next.js (Turbopack) 4.041s 4.438s 0.397s 7 1.14x
💻 Local Next.js (Turbopack) 5.919s 6.412s 0.493s 5 1.68x
💻 Local Express 7.894s (+2.3%) 8.270s (+3.1%) 0.375s 4 2.23x
💻 Local Nitro 8.329s (+16.5% 🔺) 9.020s (+16.2% 🔺) 0.691s 4 2.36x
🌐 MongoDB Next.js (Turbopack) 9.768s 10.347s 0.579s 3 2.76x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.859s (-9.3% 🟢) 3.750s (-17.1% 🟢) 0.891s 8 1.00x
▲ Vercel Nitro 3.584s (+8.9% 🔺) 4.797s (+8.6% 🔺) 1.213s 7 1.25x
▲ Vercel Next.js (Turbopack) 4.075s (+4.6%) 5.146s (+2.4%) 1.071s 6 1.43x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 0.119s 1.000s 0.011s 1.016s 0.897s 10 1.00x
🌐 Redis Next.js (Turbopack) 0.150s 1.000s 0.001s 1.007s 0.857s 10 1.27x
💻 Local Nitro 0.170s (+44.1% 🔺) 1.003s (~) 0.011s (+20.2% 🔺) 1.017s (~) 0.847s 10 1.43x
💻 Local Express 0.173s (+2.2%) 1.002s (~) 0.011s (+5.6% 🔺) 1.017s (~) 0.843s 10 1.46x
🐘 Postgres Nitro 0.195s (+3.5%) 0.995s (~) 0.002s (+13.3% 🔺) 1.012s (~) 0.817s 10 1.64x
🐘 Postgres Express 0.198s (+2.5%) 0.992s (~) 0.002s (+7.1% 🔺) 1.011s (~) 0.813s 10 1.67x
🌐 MongoDB Next.js (Turbopack) 0.519s 0.924s 0.002s 1.008s 0.489s 10 4.37x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - - -

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 1.587s (-1.2%) 2.427s (+19.0% 🔺) 0.130s (-8.1% 🟢) 2.935s (+10.6% 🔺) 1.348s 10 1.00x
▲ Vercel Nitro 1.636s (-4.5%) 2.146s (+12.0% 🔺) 0.117s (-58.4% 🟢) 2.788s (+3.4%) 1.152s 10 1.03x
▲ Vercel Next.js (Turbopack) 1.638s (-5.1% 🟢) 2.148s (-9.1% 🟢) 0.138s (-6.0% 🟢) 2.775s (-8.5% 🟢) 1.137s 10 1.03x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Next.js (Turbopack) 11/12
🐘 Postgres Express 7/12
▲ Vercel Express 8/12
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 6/12
Next.js (Turbopack) 💻 Local 7/12
Nitro 🐘 Postgres 7/12
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run

@VaguelySerious VaguelySerious marked this pull request as ready for review February 27, 2026 02:48
@VaguelySerious VaguelySerious requested a review from a team as a code owner February 27, 2026 02:48
iNishant
iNishant reviewed Feb 27, 2026
View reviewed changes
Comment thread packages/ai/src/agent/durable-agent.ts Outdated
toolName: toolCall.toolName,
output: {
type: 'error-text',
value: error instanceof Error ? error.message : String(error),
Copy link
Copy Markdown
Contributor

@iNishant iNishant Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can use this util for similar behavior as core? https://github.com/vercel/ai/blob/main/packages/provider-utils/src/get-error-message.ts

TooTallNate
TooTallNate approved these changes Feb 27, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@TooTallNate TooTallNate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me. @iNishant's comment would be good to do as well.

@VaguelySerious
Use code similar to AI SDK
cfb7374 
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
@VaguelySerious VaguelySerious merged commit eea4a02 into main Feb 27, 2026
97 of 98 checks passed
@VaguelySerious VaguelySerious deleted the peter/agent-tool-call branch February 27, 2026 18:50
@vercel-release-bot vercel-release-bot mentioned this pull request Feb 27, 2026
Merged
This was referenced Feb 27, 2026
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TooTallNate TooTallNate TooTallNate approved these changes

+1 more reviewer

@iNishant iNishant iNishant left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

DurableAgent: single tool exception breaks entire agent stream

3 participants

@VaguelySerious @TooTallNate @iNishant