IncidentRelay

Self-hosted on-call scheduling, alert routing, alert delivery, reminders, and escalations for teams that want control over their incident workflow.

IncidentRelay helps SRE, DevOps, platform, infrastructure, and operations teams route alerts to the right people through the right channels without depending on a hosted incident-management platform.

It provides the core building blocks of an on-call system:

• access groups and RBAC-style group roles;
• teams and on-call rotations;
• alert intake routes with per-route tokens;
• Alertmanager, Zabbix, and generic webhook integrations;
• Mattermost, Slack, Telegram, Discord, Microsoft Teams, email, webhook, and voice-call notifications;
• acknowledge and resolve workflows;
• reminders and escalation to the next on-call user;
• rotation overrides;
• alert silences;
• calendar view for on-call schedules;
• personal API tokens;
• Swagger/OpenAPI documentation.

IncidentRelay is designed for self-hosted environments where teams need predictable behavior, clear ownership, easy integrations, and full control over alert routing.

---

Why IncidentRelay?

Many teams need on-call routing, but do not always need a large SaaS incident platform.

IncidentRelay focuses on the practical workflow:

Monitoring system -> Route -> Team -> Rotation -> Notification channels -> ACK / Resolve

A route owns its own intake token, so external systems send alerts to an exact alert path:

ROUTE_INTAKE_TOKEN -> Route -> Team -> Rotation -> Channels

Channels only describe where notifications are delivered. Routes decide which team receives the alert and which channels are used.

This keeps alert delivery easier to reason about, easier to audit, and safer for self-hosted deployments.

---

Highlights

Self-hosted by design

Run IncidentRelay in your own environment, with your own database, your own network rules, and your own operational policies.

Route-based alert intake

Alert intake tokens belong to routes, not channels. This makes it clear which incoming integration is allowed to submit alerts to which team and rotation.

Multi-team and multi-group support

Use groups as access boundaries. Teams, rotations, routes, channels, alerts, and silences are scoped through groups and memberships.

Escalation and reminders

Unacknowledged alerts can trigger repeated reminders and then escalate to the next on-call user according to the team configuration.

Mattermost with real actions

Mattermost Bot API mode supports interactive Acknowledge and Resolve buttons, message updates, and severity-based attachment colors.

Telegram actions and worker

Telegram notifications can include action buttons and alert links. Telegram callback/polling processing is handled by the optional incidentrelay-telegram-worker service.

Pluggable voice calls

IncidentRelay can be extended with custom voice providers for self-hosted installations. Providers can implement text-to-speech calls, call status callbacks, DTMF button callbacks, ACK / Resolve actions from phone keypad, and optional call status polling.

API-first

IncidentRelay includes Swagger/OpenAPI documentation and personal API tokens with scopes for alerts, resources, and profile access.

---

Supported integrations

Incoming alert sources

Source	Endpoint
Alertmanager	POST /api/integrations/alertmanager
Zabbix	POST /api/integrations/zabbix
Generic webhook	POST /api/integrations/webhook

Notification channels

Channel	Notes
Mattermost	Incoming webhook mode or Bot API mode with buttons and updates
Slack	Webhook notifications
Telegram	Bot notifications and optional action buttons
Discord	Webhook notifications
Microsoft Teams	Webhook notifications
Email	Email recipients
Webhook	Generic outbound webhook
Voice call	Pluggable provider API for self-hosted voice integrations

---

Installation

Choose the installation method that matches your environment.

Docker Compose

Recommended for quick start, testing, and simple self-hosted deployments.

docker compose up -d --build

With PostgreSQL:

docker compose \
  -f docker-compose.yml \
  -f docker-compose.postgres.yml \
  up -d

Read more: Docker installation

RedHat-like distributions from RPM repository

Recommended for RHEL, Rocky Linux, AlmaLinux, and CentOS Stream.

sudo dnf install -y curl
sudo curl -fsSL \
  https://repo.incidentrelay.io/incidentrelay.repo \
  -o /etc/yum.repos.d/incidentrelay.repo
sudo dnf makecache
sudo dnf install -y incidentrelay

For older yum-based systems:

sudo yum install -y curl
sudo curl -fsSL \
  https://repo.incidentrelay.io/incidentrelay.repo \
  -o /etc/yum.repos.d/incidentrelay.repo
sudo yum makecache
sudo yum install -y incidentrelay

Read more: RedHat RPM installation

Manual systemd installation

Recommended when you want to run IncidentRelay directly from source code or manage the Python environment manually.

Read more: Systemd installation

---

Runtime layout

Common paths used by the RPM and systemd installation:

/var/www/incidentrelay                  # application code
/var/www/incidentrelay/venv             # Python environment or venv-compatible wrapper
/etc/incidentrelay/incidentrelay.conf   # main configuration file
/var/lib/incidentrelay                  # runtime data
/var/log/incidentrelay                  # logs
/usr/local/lib/incidentrelay/voice_providers  # custom voice providers

Systemd services:

incidentrelay-web.service               # HTTP API, UI, webhooks
incidentrelay-scheduler.service         # reminders, escalations, periodic jobs
incidentrelay-telegram-worker.service   # optional Telegram callbacks/polling

System user:

incidentrelay

---

Configuration

IncidentRelay reads the configuration file path from:

INCEDENTRELAY_CONFIG_FILE

Example:

export INCEDENTRELAY_CONFIG_FILE=/etc/incidentrelay/incidentrelay.conf

For production, set the public URL used for generated links and callback URLs:

[server]
public_base_url = https://incidentrelay.example.com

SQLite is suitable for small single-node installations:

[database]
type = sqlite
path = /var/lib/incidentrelay/incidentrelay.db

PostgreSQL is recommended for larger or long-running production installations:

[database]
type = postgresql
host = 127.0.0.1
port = 5432
name = incidentrelay
user = incidentrelay
password = change-me

Read more: Configuration

---

First setup

After installation, initialize the database and create the first administrator.

Run database migrations

For RPM/systemd installations:

sudo -u incidentrelay \
  INCEDENTRELAY_CONFIG_FILE=/etc/incidentrelay/incidentrelay.conf \
  /var/www/incidentrelay/venv/bin/python \
  /var/www/incidentrelay/manage.py migrate

For Docker installations:

docker compose exec incidentrelay-web \
  python manage.py migrate

Create the first admin user

For RPM/systemd installations:

sudo -u incidentrelay \
  INCEDENTRELAY_CONFIG_FILE=/etc/incidentrelay/incidentrelay.conf \
  /var/www/incidentrelay/venv/bin/python \
  /var/www/incidentrelay/manage.py create-admin \
  --username admin \
  --password 'change-me-123' \
  --email admin@example.com

For Docker installations:

docker compose exec incidentrelay-web \
  python manage.py create-admin \
  --username admin \
  --password 'change-me-123' \
  --email admin@example.com

Change the password and email before using these commands in production.

---

Basic UI setup flow

After the first login:

1. Create a group
2. Create users
3. Add users to the group
4. Create a team
5. Add users to the team
6. Create a rotation
7. Add rotation members
8. Create notification channels
9. Create a route
10. Copy the route intake token
11. Configure Alertmanager, Zabbix, or webhook sender
12. Send a test alert
13. Acknowledge or resolve the alert

Detailed guide: First login and initial setup

---

Example Alertmanager request

curl -X POST http://127.0.0.1:8080/api/integrations/alertmanager \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer ALERTMANAGER_ROUTE_TOKEN' \
  -d '{
    "status": "firing",
    "alerts": [
      {
        "status": "firing",
        "labels": {
          "alertname": "DiskFull",
          "severity": "critical",
          "team": "infra",
          "instance": "host1"
        },
        "annotations": {
          "summary": "Disk is full",
          "description": "/var is 95% full"
        },
        "fingerprint": "disk-full-host1-var"
      }
    ]
  }'

More examples:

• Alertmanager integration
• Zabbix integration
• Generic webhook integration

---

Mattermost buttons and message updates

Mattermost has two modes.

Incoming webhook mode sends plain messages only.

Bot API mode is recommended when you want:

• Acknowledge button;
• Resolve button;
• message updates after ACK / Resolve;
• severity-based colors.

More details: Mattermost integration

---

Custom voice providers

IncidentRelay supports custom voice providers for self-hosted installations.

A provider is a Python module that can be placed into:

/usr/local/lib/incidentrelay/voice_providers

Custom providers can implement:

• text-to-speech call creation;
• provider call ID tracking;
• call status callbacks;
• DTMF button callbacks;
• ACK / Resolve actions from phone keypad;
• optional call status polling.

Start here:

• Custom Voice Providers
• Provider API
• Configuration
• Callbacks and DTMF
• Security
• Troubleshooting
• Example providers

---

API documentation

Swagger UI is available at:

/docs

OpenAPI JSON is available at:

/api/openapi.json

---

Documentation

Topic	Link
Getting started	docs/getting-started/
Docker installation	docs/getting-started/docker.md
RedHat RPM installation	docs/getting-started/redhat-rpm-installation.md
Systemd installation	docs/getting-started/systemd.md
Configuration	docs/getting-started/configuration.md
First login	docs/getting-started/first-login.md
Groups and RBAC	docs/concepts/groups-and-rbac.md
Teams, rotations, routes	docs/concepts/teams-rotations-routes.md
Route intake tokens	docs/concepts/route-intake-tokens.md
Channels	docs/concepts/channels.md
Alertmanager	docs/integrations/alertmanager.md
Zabbix	docs/integrations/zabbix.md
Generic webhook	docs/integrations/generic-webhook.md
Mattermost	docs/integrations/mattermost.md
Alerts	docs/usage/alerts.md
Calendar	docs/usage/calendar.md
Silences	docs/usage/silences.md
Rotation overrides	docs/usage/rotation-overrides.md
Profile and API tokens	docs/usage/profile-and-tokens.md
Logging	docs/administration/logging.md
Troubleshooting	docs/administration/troubleshooting.md
Demo data	docs/administration/demo-data.md
Schema check	docs/administration/schema-check.md
Custom voice providers	docs/voice-providers/index.md

---

Demo data

Create demo data:

python manage.py demo-data

The command creates demo groups, users, teams, rotations, channels, routes, and route intake tokens.

Static demo-data check:

python app/check_demo_data.py

More details: Demo data

---

Schema check

After running migrations, verify that all Peewee model tables and columns exist in the configured database:

python app/check_schema.py

Expected output:

Schema check OK: all model tables and columns exist.

More details: Schema check

---

Troubleshooting

If an alert is not visible or not delivered:

1. Check that the correct route intake token was used.
2. Check that the endpoint matches the route source.
3. Check that route matchers match alert labels.
4. Check that the group is active.
5. Check that the team is active.
6. Check that the UI active group is correct.
7. Select "All my groups" and reload the Alerts page.
8. Check routing_error in the integration response.
9. Check JSON logs by error_id if the server returned one.

More details: Troubleshooting

---

License

See LICENSE.