HIVE80lab — Ops notes

Playbook Triage Decision Flow

Purpose: Quick decision flow to select the right playbook for any incident. Match category × severity × impact → playbook.

Triage Decision Flow

Step 1: 60-Second Severity Call

Is money or data stuck?

Step 2: Business Impact Call

Is this a business/brand/compliance risk?

Step 3: Technical Category Call

What type of issue is this?

Step 4: Playbook Selection

Select playbook based on Category × Severity:

CategoryP1 ImpactP2 ImpactP3 ImpactP4 Impact
AvailabilityTier 1 Availability PlaybookTier 2 Availability PlaybookTier 3 Availability PlaybookTier 4 Availability Playbook
SecurityTier 1 Security PlaybookTier 2 Security PlaybookTier 3 Security PlaybookTier 4 Security Playbook
PerformanceTier 1 Performance PlaybookTier 2 Performance PlaybookTier 3 Performance PlaybookTier 4 Performance Playbook
FunctionalTier 1 Functional PlaybookTier 2 Functional PlaybookTier 3 Functional PlaybookTier 4 Functional Playbook
DataTier 1 Data PlaybookTier 2 Data PlaybookTier 3 Data PlaybookTier 4 Data Playbook
Third-PartyTier 1 Third-Party PlaybookTier 2 Third-Party PlaybookTier 3 Third-Party PlaybookTier 4 Third-Party Playbook

Key Incident Playbooks

Availability Playbooks

PlaybookPurposeFirst Response
first-30-minutes-incident-responseInitial response for any incidentVerify, assess, communicate
availability-recovery-from-backupService down, restore from backupCheck backup, restore, verify
load-balancer-failoverLB not routing trafficCheck LB config, restart, verify
database-connection-pool-exhaustionDB connection pool fullRestart pool, increase limit, fix leak

Security Playbooks

PlaybookPurposeFirst Response
data-breach-first-24-hoursData breach detectedIsolate, preserve, escalate, notify
credential-compromise-responseCredential leak/compromiseRevoke credentials, rotate passwords, notify
phishing-response-checklist-small-teamsPhishing attackReport, investigate, train

Performance Playbooks

PlaybookPurposeFirst Response
performance-degradation-first-15-minSlow response, latencyCheck metrics, review changes, optimize
cache-invalidation-strategyStale data responseClear cache, verify invalidation, monitor
database-optimizationDB query slownessCheck queries, add indexes, optimize

Functional Playbooks

PlaybookPurposeFirst Response
feature-broken-detectedFeature not workingCheck code, revert, fix, test
integration-failureExternal API failureCheck API status, fallback, retry
race-condition-detectedUnexpected behaviorReview concurrency, add lock, fix race

Data Playbooks

PlaybookPurposeFirst Response
data-loss-restore-from-backupData deleted/corruptedCheck backup, restore, verify
data-corruption-detectedData integrity issuesStop write access, restore from backup, validate
data-migration-failedMigration incompleteRollback migration, fix, retry

Third-Party Playbooks

PlaybookPurposeFirst Response
vendor-incident-coordinationVendor outageCheck vendor status, escalate, workaround
third-party-api-failureExternal API downCheck API status, fallback, notify vendor
customer-vendor-incident-commsCustomer impacted by vendorNotify customer, provide updates, compensate

Quick Reference: Decision Tree

`` START │ ├─ 60-second severity call: money/data stuck? │ ├─ YES → P1 → Availability/Security playbooks │ └─ NO → business impact call │ ├─ YES → P2 → High Impact playbooks │ └─ NO → technical category call │ ├─ Availability → Tier 1 Availability Playbook │ ├─ Security → Tier 1 Security Playbook │ ├─ Performance → Tier 1 Performance Playbook │ ├─ Functional → Tier 1 Functional Playbook │ ├─ Data → Tier 1 Data Playbook │ └─ Third-Party → Tier 1 Third-Party Playbook │ └─ Document decision in incident timeline ``

Triage Documentation Fields

Common Playbook Combinations

Scenario 1: Payment Gateway Down (Availability + Business)

Scenario 2: Phishing Attack (Security)

Scenario 3: Slow Checkout Flow (Performance + Business)

Scenario 4: Wrong Customer Address (Functional)

Playbook Selection Rules

Rule 1: Always Start with First 30 Minutes

Every incident should start with first-30-minutes-incident-response playbook to establish baseline: verify, assess, communicate.

Rule 2: Severity Trumps Category

A P1 Security incident (data breach) always uses Security playbooks, even if it's also an Availability issue.

Rule 3: Business Impact Defines Escalation

A P2 data corruption issue (no breach, but data lost) should be escalated to Tier 2, not Tier 1.

Rule 4: Use Tier-Specific Playbooks

Tier 1 playbooks focus on quick investigation and communication. Tier 2+ playbooks focus on resolution and customer comms.

Rule 5: Multiple Playbooks Allowed

Advanced incidents may require multiple playbooks. Example: Payment down (Availability) → Data loss (Data) → Customer comms (Third-Party).

Usage: Use immediately after incident classification. Select the right playbook based on severity × category. Link to playbook in incident timeline.

Product links: /l/ops-starter-kit-vol-2 | /l/ops-starter-kit | /l/automation-starter-pack

From the HIVE80lab kit

Part of the five-pillar incident-response set: see the pillars overview and the blameless post-incident review template.