Incidents
Auto-created from monitor downtime, manually created from the dashboard or API, with severity, ack, escalation, and timeline updates.
Incidents
An incident is a record of an outage or service disruption. Each incident has a status (investigating, identified, monitoring, resolved), a severity (minor, major, critical), affected monitors, a timeline of updates, and audit fields for acknowledgement and resolution.
How they get created
When a monitor's confirmed status flips to down and there is no active incident for that monitor, the auto-create path runs:
- Insert
incidentsrow withauto_created = 1andsource_monitor_idset. - Link the monitor via
incident_monitors. - Add an "investigating" update with details.
- Severity inferred from how many regions failed:
- All regions failed →
critical - 2+ regions failed →
major - 1 region failed →
minor
- All regions failed →
- If a deploy correlation exists (GitHub repo linked), annotate with the suspect commit.
- If the org has a schedule with an escalation policy, start the escalation timer.
POST /incidents with title, optional severity, and affected_monitors. The first update is created automatically.
Acknowledgement & escalation
When an incident exists with an active escalation policy, a per-minute cron checks: has anyone acked? If not and the next level's delay_minutes has elapsed, the dispatcher fires a notification to that level's target (user / schedule / channel) and bumps current_level.
Acking happens via:
- Dashboard "Acknowledge" button
- Slack interactive button (
Acknowledgeon the alert message) /happyuptime ack <id-or-title>slash commandPOST /incidents/:id/ack
Acking sets acknowledged_at and acknowledged_by, and stops escalation.
Auto-resolution
When a monitor recovers (confirmed up), Happy Uptime:
- Adds a "resolved" update with downtime duration.
- If all linked monitors are up, sets
incidents.status = 'resolved'andresolved_at = now. - Stops any pending escalation.
- Purges the status page edge cache so subscribers see green again.
Updates timeline
Each incident has a stream of updates (incident_updates table) — same model PagerDuty / status pages use. Updates have a status (matches the incident status enum), a message, optional is_internal flag (visible only to team), and an author.
Add updates via:
- Dashboard "Add Update" button
POST /incidents/:id/updates- AI:
POST /ai/draft-updateto generate update copy from the incident's history