Improve paramaterization of transaction names

[smeubank]

Describe the idea

We need to revisit how the JS SDK captures transactions (URL routes) and sends them to sentry.

Reasons for doing this and things we need to consider:

• parameterization: We should parameterize URLs whenever possible

Because

• grouping: transaction name influences transaction grouping; raw URL names lead to ungroupted transactions
• indexing and high cardinality: Raw URL transaction names lead to a high cardinality of transactions (because they're not grouped)
• PII: Raw URL transaction names can contain sensitive data/PII (e.g. IDs, auth tokens, etc.)
• Dynamic Sampling: Propagating raw URLs in DSC vs. showing parameterized routes in the Sentry UI (and the DS settings) creates a lot of user confusion

Best effort has been decent so far, but the fallback to unparameterized or whole URL maybe sub-optimal.

Examples:

• Low-cardinality transaction name:
  • {"transaction": "/users/{username}", "transaction_source": "route"}
• Presumably a high-cardinality transaction name:
  • {"transaction": "/users/123235", "transaction_source": "uri"}
• User-defined transaction name, cardinality unknown:
  • {"transaction": "my_transaction_name", "transaction_source": "custom"}

Requirement:

what should be sent and where?

baggage header

• sentry-transaction
trace Envelope Header
• transaction (string)
• transaction envelope payload

• error envelope payload

• in the envelope header

Possible implementation

There's a couple of things we can do or at least check:

Existing Routing Instrumentations with parameterization

As listed in #5345, we have a lot of popular routers covered with routing instrumentations. However, we might be able to improve paramenterizations in some of them. Hence, for each instrumentation

• check if there is a way to parameterize earlier
• try to match routes better (if it turns out that there are cases where our current matching fails)
• (send source information; tracked in Add support for transaction source #5345)

Existing Routing Instrumentations without parameterization

TODO: Check to which routers this applies

There are some routing instrumentations that don't parameterize currently.

• add parameterization whenever possible

TBD: Approximative Parameterization

This has been discussed quite a bit in the past but given that we have to make our best effort for parameterization, let's revisit this topic. The idea is seemingly simple: We could try to add a mechanism that takes a raw URL and tries to guess parts of that URL that might be parameters (e.g. IDs, tokens, etc). The mechanism would then replace these parts with a generic param placeholder.

Example:

/users/1235/credentials ==> users/:id/credentials

There are a lot of possible issues with this because obviously, there are going to be loads of edge cases, where this approximation might be off or miss parameters completely.

Why is this challenging?

• In the product, we have to explain why DSC might have another (or no) transaction name than what's visible in the UI
• We have to explain why unparameterized routes are sent and even full URLs.
• Some frameworks and routers provide unique challenges
• What about custom routing instrumentations we don't control?

Places to improve parameterization

• nextjs (@lobsterkatie ) Improve Nextjs tracing #5505
• express integration (ref(node): Improve Express URL Parameterization #5450, @Lms24)
• angular (feat(Angular): Add URL Parameterization of Transaction Names #5416, @Lms24)
• vue (I think Vue is pretty much optimal, @lforst)
• react router (pretty much optimal)
  • react router v6 fix(react): Fix React Router v6 paramaterization #5515
• without framework (feat(tracing): Try parameterizing URLs in default routing instrumentation #5411, @lforst)
• remix fix(remix): Paramaterize server side transactions #5491
• Ember
• Gatsby

[smeubank]

@Lms24 let me know when you have time, this needs refinement. We can loop in others of course

• identify ways we could quantify this problem
• identify problem areas we want and importantly could address

[Lms24]

Always happy to refine/discuss this. Since this became a very important issue, we should sync with everyone here who has context on how transaction names/parameterization/routing instrumentation currently works

[Lms24]

@smeubank fleshed out the issue description. Feel free to add/change stuff