Skip to content

Grafana-to-GitHub alert bridge with metric classification#3224

Open
x3c41a wants to merge 1 commit intomasterfrom
grafana-github-alert-bridge
Open

Grafana-to-GitHub alert bridge with metric classification#3224
x3c41a wants to merge 1 commit intomasterfrom
grafana-github-alert-bridge

Conversation

@x3c41a
Copy link
Contributor

@x3c41a x3c41a commented Feb 18, 2026

design doc

Summary

Cloudflare Worker that converts Grafana Alertmanager webhooks into categorized GitHub issues, triggering a Claude Code agent to diagnose and act on bridge alerts.

flowchart TD
    P["Prometheus :9615"] --> G["Grafana Alert Rules"]
    G --> AM["Alertmanager"]
    AM -->|webhook| W["Cloudflare Worker"]
    AM -->|webhook| M["Matrix"]

    W -->|"warning + known"| T["Issue: label claude"]
    W -->|"critical / unknown"| E["Issue: label claude-escalate"]

    T -->|Haiku| H["Fast Triage"]
    E -->|Sonnet| S["Deep Investigation"]

    H --> V["Engineer Reviews"]
    S --> V
Loading
  • Classifies 28 bridge alerts into 8 categories based on metric patterns
  • Detects environment and bridge pair from alert names
  • Tiered model: warning/known → Haiku, critical/unknown → Sonnet. Haiku handles ~90% of alerts at 1/10th the cost.
  • Deployed at https://grafana-github-bridge.parity-bridges.workers.dev
  • Located in deployments/local-scripts/grafana-github-bridge/

Grafana notification policy

- receiver: GitHub parity-bridges-common
  matchers:
    - alertname =~ ".*Bridge.*|.*bridge.*|.*headers mismatch"
  continue: true

Test plan

  • All 8 alert categories correctly classified via local curl tests
  • Resolved alerts are skipped (only firing creates issues)
  • Environment detection works for both domain label and chain name fallback
  • Bridge pair extraction from alert title patterns
  • Deployed and responding at workers.dev URL
  • Set GITHUB_TOKEN secret and verify issue creation end-to-end
  • Verified no false positives: regex does not match any non-bridge alerts
  • Configure Grafana notification policy with updated matcher
  • Test e2e with real Grafana alert

🤖 Generated with Claude Code

@x3c41a x3c41a requested a review from a team as a code owner February 18, 2026 14:35
@x3c41a x3c41a removed the request for review from a team February 18, 2026 14:39
@x3c41a x3c41a marked this pull request as draft February 18, 2026 14:40
steps:
- uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be a service account

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be hosted on Parity's public infra if any...

@cla-bot-2021
Copy link

cla-bot-2021 bot commented Feb 19, 2026

User @claude, please sign the CLA here.

@x3c41a x3c41a changed the title Add Grafana-to-GitHub alert bridge Grafana-to-GitHub alert bridge with metric classification Feb 19, 2026
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if that's (deployments/local-scripts/grafana-github-bridge/) the right place for this bridge but I could not find a better one

@x3c41a x3c41a force-pushed the grafana-github-alert-bridge branch 2 times, most recently from 0663195 to c10b545 Compare February 26, 2026 13:33
issues: write
env:
# Grafana API access for live metric queries.
# Claude can use: curl -H "Authorization: Bearer $GRAFANA_TOKEN" "$GRAFANA_URL/api/..."
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are set in Github's Project settings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracted to a separate PR - #3236 - but may be merged as part of this PR

@x3c41a x3c41a marked this pull request as ready for review February 26, 2026 13:52
@x3c41a x3c41a force-pushed the grafana-github-alert-bridge branch 2 times, most recently from fc90b20 to ef89177 Compare February 26, 2026 15:00
Cloudflare Worker that converts Grafana Alertmanager webhooks into
categorized GitHub issues, triggering a Claude Code agent (Haiku)
to diagnose and act on bridge alerts.

- Classifies 28 bridge alerts into 8 categories (finality-lag,
  delivery-lag, confirmation-lag, reward-lag, relay-down,
  version-guard, headers-mismatch, low-balance)
- Detects environment and bridge pair from alert names
- GitHub Action triggers on issue label:claude, uses Grafana API
- Located in deployments/local-scripts/grafana-github-bridge/
@x3c41a x3c41a force-pushed the grafana-github-alert-bridge branch from ef89177 to 8b606c0 Compare February 26, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant