Incident Response Template: Contain, Recover, Learn

What This Is

Every organization faces incidents. The difference between a disaster and a manageable event is preparation. This template gives you a repeatable structure for containing damage, restoring service, and learning from what happened. Use it to turn chaos into data.

Why You Need This

Faster containment: Clear roles and responsibilities reduce response time by 50-70%.
Less damage: Structured communication prevents panic and escalation.
Legal protection: A documented response shows due diligence in the event of audit or litigation.
Psychological safety: Team members who know their role handle stress better.

Phase 1: Detection & Containment (15 Minutes)

The first 15 minutes matter most. Confirm the incident, isolate it, and limit the blast radius.

Step-by-Step

Confirm the incident: Is this a real event or a false alarm? Use data, not fear.
Identify the scope: Which systems, services, users, or data are affected?
Stop the bleeding: Shut down access, disconnect compromised systems, or revert recent changes.
Document what you know: Time, source, initial impact. Write it down immediately.
Notify the incident response team: Appropriate stakeholders get notified based on severity.

Phase 2: Eradication (30-60 Minutes)

Remove the root cause and close the vulnerability. This phase can take hours to days depending on complexity.

Eradication Steps

Root cause analysis: How did this happen? Who did it? Did we miss a signal?
Remove the threat: Delete malware, patch vulnerabilities, revoke compromised credentials.
Verify the fix: Test that the issue is truly resolved. Don't guess.
Restore from backups: Roll back to a clean state if possible. Monitor closely for reinfection.
Restart systems: If you shut down services, bring them back up. Watch for regression.

Phase 3: Recovery (15-30 Minutes)

Return to normal operations, but do it carefully. Don't rush and don't skip validation.

Recovery Checklist

Partial restoration: Restore non-critical systems first. Monitor before bringing up everything.
Full restoration: Bring up all services. Verify each one works correctly.
Failover verification: If you had backup systems, confirm they were used correctly.
User notification: Tell customers/users what happened, what you're doing, and when service will be back.
Performance monitoring: Watch for unexpected behavior. Slow systems are often the next symptom.

Phase 4: Post-Incident Activity (30-60 Minutes)

The incident is over. Now learn from it so it doesn't happen again. This is the most important phase.

Post-Incident Steps

Write an incident report: What happened, why, how it was handled, what changed going forward. This is your legal evidence and your learning record.
Conduct a blameless postmortem: What should we do differently? Who should we talk to? What systems should we improve? No blame — focus on process.
Implement preventive changes: Patch vulnerabilities, update policies, improve monitoring, schedule rotation.
Notify stakeholders: Executives, board, clients, legal. Be honest about impact.
Document lessons learned: Update your incident response playbook. Archive the report for future reference.

Decision Tree: When to Escalate

Escalation Triggers

Legal/PDPA breach: Personal data exposed → engage legal immediately.
Financial loss: $10,000+ lost or at risk → notify finance and legal.
Public impact: Customer-facing outage, brand damage → executive notification.
Regulatory violation: Health data, payment data, or other regulated data → regulator notification.
Long duration: Service down > 4 hours → executive notification.

Worked Example

A SaaS company's authentication service was compromised. Here's how their incident response played out:

0:00: The monitoring system detected failed login attempts from multiple IP addresses.

0:05: Security team confirmed it was an automated attack. They disabled the affected API keys and blocked the attacker IPs.

0:15: Customer-facing sign-up was down. They isolated the affected service and restored it with a known-good backup from 15 minutes ago.

1:00: Service restored. They resumed monitoring and confirmed no data was exposed.

2:00: Root cause analysis: the old API key had a weak hash algorithm. They rotated all keys and updated the secret rotation policy to use SHA-256 going forward.

3:00: Blameless postmortem revealed: no one knew API key rotation was required. They updated the security handbook and added a 90-day rotation policy.

Result: Customer notification was sent 10 minutes after discovery. Customer churn dropped from 8% to 3%. External audit later praised their response as "professional and timely."