Live
#17 medium manual
BlockedTest
Opened 2026-07-26 16:48 UTC · Resolved 2026-07-26 16:50 UTC
Handoff Notes ← Dashboard
Timeline
OPENED
2026-07-26 16:48 UTC
Incident created manually. Severity: MEDIUM
CONTEXT AGGREGATED
2026-07-26 16:49 UTC
Sources available: 3/3 — Runbook: ✓ | Past incidents: ✓ | Infra health: ✓
Response Plan
2026-07-26 16:49 UTC

Severity

MEDIUM-HIGH: CPU at 99% with sharp upward trend; memory at 95%; immediate action needed to prevent cascade failure.

Root Cause

  • Runaway process or uncontrolled workload spike consuming CPU/memory
  • Resource leak or inefficient query generating sustained load

Actions

  1. SSH to affected host; run top -b -n 1 | head -20 to identify CPU/memory hog process.
  2. Kill or throttle offending process; if test/batch job, pause execution.
  3. Check recent deployments (last 2h); rollback if correlation exists.
  4. Drain traffic to other instances if load-balanced; restart affected service if isolated.
  5. Increase memory/disk allocation if sustainable load, or tune thresholds if false alarm.

Watch

  • CPU trend direction (target: <70% within 5min; confirm downward vector)
  • Memory pressure and disk I/O (watch for secondary spikes indicating queue buildup)

Escalate If

CPU does not drop below 80% within 10 minutes, or memory continues climbing.

STATUS CHANGE
2026-07-26 16:50 UTC
OPEN -> RESOLVED (closed after validation; public manual form disabled and host pressure was transient)
Update Status
Details
ID #17
Severity MEDIUM
Source manual
Status RESOLVED
Opened 2026-07-26 16:48