Sentry slowing down DB queries

[raymondhechen]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/node - express

SDK Version

10.29.0

Framework Version

No response

Link to Sentry event

No response

Reproduction Example/SDK Setup

No response

Steps to Reproduce

@sentry/node >= 10.29.0 when using postgres and drizzle-orm results in DB query latency increases. Increase at ~1pm reflects this. Below charts show duration of spans that wrap drizzle queries.

I've tested with every version >= 10.29.0, none of which resolve this issue. 10.28.0 is fine. Also it seems that the versions with errors seem to break some postgres instrumentation? (below images are directly from db spans)

Any ideas what the issue might be? For some context: we're computing the first image's charts spans like this:

export const withServerTrace = async <T>(
  { name, op = 'service', attributes }: ServerSpanArgs,
  fn: (span: Sentry.Span) => Promise<T> | T
): Promise<T> => {
  return Sentry.startSpan({ name, op, attributes }, async (span) => {
    try {
      return await fn(span)
    } catch (err) {
      span.setStatus({ code: 2, message: 'internal_error' })
      Sentry.captureException(err)
      throw err
    }
  })
}

And here's an example usage with drizzle-orm:

export const getChat = async ({ chatId }: { chatId: string }) => {
  return await withServerTrace(
    { name: 'getChat', op: 'api.query' },
    async () => {
      return await db.query.chats.findFirst({
        where: (chats, { eq }) => eq(chats.id, chatId),
      })
    }
  )
}

Expected Result

No DB query latency increases.

Actual Result

DB query latency increasing.

Additional Context

No response

Priority

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it.}

[linear]

JS-1327

[s1gr1d]

Hi, is the latency increase at 1pm due to a version bump to 10.29.0 or what was done differently from that moment onwards?

Which drizzle version do you use and how do you set up the SDK?

I just checked the changelog and at first glance, there is nothing specific inside that would explain this increased latency: https://github.com/getsentry/sentry-javascript/blob/develop/CHANGELOG.md#10290

[raymondhechen]

Hi @s1gr1d! Yes, the latency increase happened directly after a version bump to 10.29.0. Reverting back to 10.28.0 immediately resolves the latency increase. We're using "drizzle-orm": "^0.45.1".

Below is how we setup Sentry:

import * as Sentry from '@sentry/node'
import 'dotenv/config'

const DISABLED_DEFAULT_INTEGRATIONS = [
  'Amqplib',
  'Kafka',
  'OpenAI',
  'Tedious',
  'Prisma',
  'Mysql2',
  'Mysql',
  'Mongoose',
  'Mongo',
  'Fastify',
  'Hapi',
  'Koa',
  'Firebase',
  'Anthropic_AI',
]

Sentry.init({
  // debug: true,
  environment: process.env.NODE_ENV,
  dsn: process.env.SENTRY_SERVER_DSN,
  // filter out default integrations that are not needed
  integrations: (integrations) => {
    return integrations.filter(function (integration) {
      return !DISABLED_DEFAULT_INTEGRATIONS.includes(integration.name)
    })
  },
  ignoreErrors: ['AbortError'],
  tracesSampleRate: 0.1,
  sendDefaultPii: true,
})

And start script:
"start": "NODE_OPTIONS='--max-old-space-size=24576' tsx --import ./src/sentry/instrument.ts src/index.ts",

[s1gr1d]

I analyzed the 17 commits between 10.28.0 and 10.29.0 and the performance degradation is puzzling because:

• No direct changes to postgres instrumentation
• No changes to OpenTelemetry core libraries
• No changes to span creation or context management
• The OpenAI refactor is isolated and shouldn't affect other instrumentations (and you are also filtering OpenAI)

In general, your setup seems alright, nothing out of the ordinary. Can you provide a small reproduction so we can have a closer look?