ops(observability): audit_log_insert_failed metric (closes #3574)#3769
Merged
Conversation
…ssification_log insert failure (closes #3574) The audit-log insert helper intentionally swallows DB errors so observability never blocks a profile save / crawler promote / backfill row. Until now the only signal of a failed insert was a warn log line — too fragile to alert on. The catch block now also fires a PostHog metric: captureEvent("server-metrics", "audit_log_insert_failed", { source, error_class }) error_class is the 2-char SQLSTATE class (e.g. '23' = integrity violation, '08' = connection failure) or 'unknown' when the thrown error has no pg code. The class — not the full 5-char code — is what ops alerts on. Pairs with #3567 (the table itself) and the future audit-log query UI, both of which assume audit-log volume is reliable.
…ice (review nits) Two follow-ups from review on PR #3769: 1. SQLSTATE length check is now strict (=== 5 instead of >= 2). A 5-char well-formed SQLSTATE yields its 2-char class; anything else is 'unknown'. The previous '>= 2' guard was inconsistent: a malformed 1-char code fell back to 'unknown' but a malformed 3-char code yielded a truncated 2-char "class" that wouldn't map to any real PostgreSQL class. Refuse ambiguous input — pick one shape and stick to it. 2. Added a comment next to the captureEvent call explaining why agent_url and member_id are not labels: unbounded cardinality would blow up event volume. The {source × error_class} cross product is the alertable shape; the warn log already carries per-row detail for forensics. New test pins the malformed-code boundary: code: 'XYZ' → error_class: 'unknown' (not 'XY'). 10/10 unit tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
server/src/db/type-reclassification-log-db.ts) intentionally swallows DB errors so a failed audit write never blocks the caller's primary intent (member-profile save / crawler promote / backfill row). Until now, the only signal of a swallowed insert was awarnlog line — fragile to alert on.captureEvent("server-metrics", "audit_log_insert_failed", { source, error_class }).error_classis the 2-char SQLSTATE class (e.g.'23'for integrity violations,'08'for connection failures) — the class, not the full 5-char code, is what ops alerts on. Falls back to'unknown'when the thrown error has no pg.code.audit_log_insert_failed > 0 over 5m.Why this metrics surface
The triage on #3574 noted there's no
metrics.increment()helper in this repo — the canonical metrics layer is PostHog viacaptureEvent()(seeserver/src/middleware/request-metrics.ts:43). This PR uses that same pattern.Test plan
npx vitest run tests/unit/type-reclassification-log-db.test.ts— 9/9 pass (6 existing + 3 new)error_class: '23'from acode: '23505'errorerror_class: 'unknown'for plainErrornpm run typecheck— cleannpm run build— cleanprecommithook (full unit suite + dynamic-import check + callapi state-change + typecheck) — cleanOut of scope