HIVE80lab — Ops notes

Incident Response Playbook Matrix

Purpose: Quick-reference matrix matching incidents to playbooks. Use this during triage to select the right playbook immediately.

Incident Matrix

Incident TypeTriggerSeverityCategoryPlaybookTierFirst Response
Service DownHTTP 500 / 503 / timeoutP1Availabilityfirst-30-minutes-incident-responseTier 1Verify, assess
Database DownConnection refusedP1Availabilityfirst-30-minutes-incident-responseTier 1Check DB status
Load Balancer DownNo routingP1Availabilityload-balancer-failoverTier 1Check LB config
Connection Pool FullError 503P1Availabilitydatabase-connection-pool-exhaustionTier 1Restart pool
Data BreachUnusual log activityP1Securitydata-breach-first-24-hoursTier 1Contain, preserve
Credential LeakSuspicious auth logsP1Securitycredential-compromise-responseTier 1Revoke, rotate
Phishing AttackUser reportsP2Securityphishing-response-checklist-small-teamsTier 1Investigate, report
Slow Responsep95/p99 > thresholdP2Performanceperformance-degradation-first-15-minTier 1Check metrics
Slow CheckoutRevenue impactP2Performanceperformance-optimizationTier 1Review code
Feature BrokenUser ticketP2Functionalfeature-broken-detectedTier 1Check code
Wrong AddressCustomer errorP4Functionalintegration-failureTier 1Fix integration
Data LossNo rows in tableP1Datadata-loss-restore-from-backupTier 1Restore from backup
Data CorruptionData integrity issuesP2Datadata-corruption-detectedTier 1Stop writes
Migration FailedMigration incompleteP2Datadata-migration-failedTier 1Rollback, retry
Vendor DownVendor status pageP1Third-Partyvendor-incident-coordinationTier 1Escalate vendor
API DownHTTP 502 / 504P1Third-Partythird-party-api-failureTier 1Check API status
Customer Payment DownPayment failedP1Third-Partythird-party-api-failureTier 1Notify customer

Quick Triage Matrix

Severity → Tier Mapping

SeveritySLAUpdate Frequency
P115 minEvery 15 min
P230 minEvery 30 min
P31 hourEvery 1 hour
P44 hoursNext business day

Category → Playbook Family

CategoryCore PlaybookTier-Specific Playbooks
Availabilityfirst-30-minutes-incident-responseTier 1 → Tier 2 → Tier 3 → Tier 4
Securityfirst-30-minutes-incident-responseTier 1 → Tier 2 → Tier 3 → Tier 4
Performancefirst-30-minutes-incident-responseTier 1 → Tier 2 → Tier 3 → Tier 4
Functionalfirst-30-minutes-incident-responseTier 1 → Tier 2 → Tier 3 → Tier 4
Datafirst-30-minutes-incident-responseTier 1 → Tier 2 → Tier 3 → Tier 4
Third-Partyfirst-30-minutes-incident-responseTier 1 → Tier 2 → Tier 3 → Tier 4

Tier-Specific Playbooks

Tier 1 (Investigation + Communication)

IncidentPlaybookFocus
Service Downfirst-30-minutes-incident-responseVerify, assess, communicate
Data Breachdata-breach-first-24-hoursContain, preserve, escalate
Slow Responseperformance-degradation-first-15-minCheck metrics, investigate
Feature Brokenfeature-broken-detectedCheck code, investigate
Data Lossdata-loss-restore-from-backupCheck backup, restore
Vendor Downvendor-incident-coordinationEscalate vendor, find workaround

Tier 2 (Resolution + Customer Comms)

IncidentPlaybookFocus
Service DownTier 2 Availability PlaybookResolve, customer comms
Data BreachTier 2 Security PlaybookInvestigate, notify customers
Slow ResponseTier 2 Performance PlaybookOptimize, customer comms
Feature BrokenTier 2 Functional PlaybookFix, customer comms
Data LossTier 2 Data PlaybookRestore, customer comms
Vendor DownTier 2 Third-Party PlaybookCustomer comms, SLA tracking

Tier 3 (Major Incident + Executive Involvement)

IncidentPlaybookFocus
Service DownTier 3 Availability PlaybookExecutive, major comms
Data BreachTier 3 Security PlaybookLegal, compliance, press comms
Slow ResponseTier 3 Performance PlaybookEngineering changes, optimization
Feature BrokenTier 3 Functional PlaybookEngineering changes, stakeholder comms
Data LossTier 3 Data PlaybookComplex restoration, compliance comms
Vendor DownTier 3 Third-Party PlaybookVendor penalties, legal escalation

Tier 4 (RCA + Prevention)

IncidentPlaybookFocus
Service DownTier 4 Availability PlaybookRCA, prevention
Data BreachTier 4 Security PlaybookPrevention, training
Slow ResponseTier 4 Performance PlaybookRCA, monitoring
Feature BrokenTier 4 Functional PlaybookRCA, feature improvements
Data LossTier 4 Data PlaybookRCA, backup improvements
Vendor DownTier 4 Third-Party PlaybookRCA, alternative providers

Common Playbook Combinations

Payment Gateway Down

IncidentPlaybookTier
Initialfirst-30-minutes-incident-responseTier 1
Recoveryavailability-recovery-from-backupTier 2
Customer Commscustomer-vendor-incident-commsTier 2
Post-Mortemincident-post-mortem-templateTier 4

Phishing Attack

IncidentPlaybookTier
Initialfirst-30-minutes-incident-responseTier 1
Investigationphishing-response-checklist-small-teamsTier 1
Trainingtraining-response-planTier 4
Preventionsimulation-after-action-reportTier 4

Slow Checkout Flow

IncidentPlaybookTier
Initialfirst-30-minutes-incident-responseTier 1
Performanceperformance-optimizationTier 2
Customer Commscustomer-comms-for-degradationTier 2
Post-Mortemincident-post-mortem-templateTier 4

Feature Broken (Release 1.0.0)

IncidentPlaybookTier
Initialfirst-30-minutes-incident-responseTier 1
Fixfeature-broken-detectedTier 1
Rollbackrollback-procedureTier 2
Post-Mortemincident-post-mortem-templateTier 4

How to Use This Matrix

Step 1: Identify Incident

Step 2: Find Playbook

Step 3: Start Playbook

Step 4: Adjust if Needed

Documentation Fields

Usage: Use during triage to quickly select the right playbook. Match incident type, severity, and category to the matrix row. Link to playbook in incident timeline.

Product links: /l/ops-starter-kit-vol-2 | /l/ops-starter-kit | /l/automation-starter-pack

From the HIVE80lab kit

Part of the five-pillar incident-response set: see the pillars overview and the blameless post-incident review template.