The drill calendar: a plan you never rehearsed is a wish
Every small team has an incident response plan. Almost none have rehearsed it. The difference shows up at 2AM: the team that drills fumbles for fifteen minutes while they remember where things live, and the team that doesn't fumbles for two hours while they discover the plan was fiction. The fix is not more plan — it is a schedule. A drill that is not on the calendar does not happen, because drills are always the first casualty of a busy week. This page is the calendar itself: four quarterly drills, a handful of fifteen-minute monthly ones, the roles, the pass criteria, and the loop that turns every drill into fixes instead of feelings.
1. Why the calendar is the drill
- Drills decay on their own schedule, so schedule against it. People rotate off, tools get renamed, the runbook that worked in March is wrong by October. A quarterly cadence means nothing drifts more than six months before someone rediscovers it. The annual security review is the deep audit; the drill calendar is the quarterly heartbeat that keeps the plan between reviews honest.
- Each quarter gets one big drill, each month gets one small one. That is the whole budget: about four hours of practice a year plus an hour a month. Teams fail at drills by attempting a full-scale simulation of a datacenter fire with four people and a shared calendar. Small, frequent, and specific beats large, annual, and theatrical.
- The schedule is public. The drill calendar lives where the rotation schedule lives, with names and dates. A private calendar is a calendar that quietly dies. When a new hire joins, the drill calendar goes on their onboarding reading list — their first scheduled drill is part of their certification.
2. The four quarterly drills
- Q1 — the tabletop: one scenario, ninety minutes, no laptops. Pick the scenario your business actually fears (for most small teams: ransomware or a payment-path outage), and walk it end to end using the tabletop exercise checklist. The output is a list of stall points — places the room went quiet — not a grade. The ransomware recovery checklist is the usual script.
- Q2 — the restore: prove the backup can pay out. Restore one real system from backup onto a clean machine, timed, with the key ceremony included if backups are encrypted. The restore test checklist is the runbook; if encryption is involved, the backup encryption checklist drill is the same afternoon. A backup that has never been restored is a rumor, not a backup.
- Q3 — the failover: rehearse the recovery path, not the disaster. Pick the rollback or failover you would need on a bad Tuesday: a deployment rollback on staging, a DNS cutover walk-through against the DNS outage runbook, or a vendor-down tabletop against the vendor outage runbook. The point is muscle memory for the path, because at 2AM nobody reads a page they have never seen.
- Q4 — the break-glass and keys walkthrough: daylight rehearsal of the emergency path. Hands on: open the sealed break-glass kit (or walk the steps that would open it), rotate one real credential against the API key rotation or secrets rotation checklist, and confirm the alerting fires on emergency-account use. If the walkthrough requires breaking the seal for real, rotate the sealed contents the same hour — a break-glass kit that cannot survive being tested is a decoration.
3. The monthly fifteen-minute mini-drills
- The severity recitation. The on-call person for the week classifies two hypothetical events aloud against the severity matrix, one sentence per level, and starts a timeline entry for the fake incident. Two people, ten minutes, zero infra. This is the drill that makes the first ten minutes of a real incident calm.
- The lost-device walk. One person narrates the MFA device lost runbook from memory while the other checks against the page. The mismatch list is the fix list. Nobody should meet this runbook for the first time while holding a found phone in a parking lot.
- The handoff under pressure. Write a complete handover for a fictional open incident in ten minutes, then have the receiver restate it. The handoff checklist exists precisely so the real 2AM version is a copy-paste job; the mini-drill keeps the copy-paste honest. The 2AM handoff test is the full-size version.
- The cleanup sandbox. Once a quarter, fill a scratch disk or a test volume and walk the disk-full runbook on it. Disk-full is the most common small-team incident and the most preventable; a team that has practiced the cleanup once stops fearing it forever.
4. Drill-day roles (four people, three of whom can be the same person)
- Facilitator. Brings the scenario, keeps the clock, refuses to answer questions the runbook should answer. The facilitator's discipline is the drill's quality: the moment they rescue the room, the room stops learning. In a two-person team, the facilitator is whoever did not own the system last.
- Incident commander and scribe. Someone drives (classifies, escalates, delegates) and someone writes the timeline as it would be written for real, per the first-30-minutes runbook. Rotating these roles is the point: your only IC should not be the person who is already on-call every quarter.
- Observer with the checklist. One person holds the written pass criteria and marks every stall point with the reason: missing doc, missing access, missing tool, or missing skill. Grading against the runbooks, not the people, is what keeps drills blameless — and what makes the findings actually get fixed. The postmortem template takes the findings from here.
5. Pass criteria and the fix loop
- Write the pass criteria before the drill, not during it. One line per drill: "restore of the billing database on a clean machine, under one hour, from sealed keys, with the old system untouched." A drill without pre-written criteria is a conversation with a timer. The same discipline as a change ticket: if you cannot state success in a sentence, you are not ready to run the drill.
- Split the findings into doc fixes and skill gaps. Every stall point is either the documentation failing the team (fix the doc, ship it this week) or a person missing a skill (schedule the pairing, not a lecture). Teams that mix these two lists end up neither training nor fixing. The doc-fix list is usually longer, and that is good news — docs are cheap to fix and the fix pays out on every future drill and incident.
- Re-test the fix, not the whole drill. If Q2's restore failed on the encryption key ceremony, the October mini-drill is one key ceremony, not another full restore. Small re-tests keep the loop tight and make the next quarterly pass on the first try. Anything unresolved rides to the next annual review as a named open item, with an owner.
- Let drills write to the plan. The drill that finds "the escalation path in the plan skips the finance system" edits the incident response plan the same day. A plan that never changes after four drills a year is not stable — it is unread.
6. The one-page template
- The calendar line (one row per drill): quarter · drill name · owner (a person, not a team) · scenario · pass criteria sentence · date · result (pass / fix list).
- The standing quarters: Q1 tabletop (business-feared scenario) · Q2 restore-from-backup (timed, with keys) · Q3 failover path (rollback, DNS, or vendor-down) · Q4 break-glass + one real rotation.
- The standing minis: weekly severity recitation on the active on-call · monthly handoff-under-pressure · quarterly lost-device walk and cleanup sandbox.
- The fix log: date · finding · type (doc / access / tool / skill) · owner · fix date · re-test date. Same columns as a postmortem action list, so the two artifacts stay compatible.
- The cadence rule: never two consecutive quarters without a completed drill. A missed drill is rescheduled within thirty days or the failure goes on the same fix log — the schedule polices itself or it does not exist.
Small-team honesty note: if the team is one founder and a part-time admin, this is one shared doc with four rows and a phone timer — and it still matters, because the founder is usually both the plan and the single point of failure inside it. The drill that matters most is the one that scares you to think about; put that one in the next quarter, not the one that is easy to pass. And when a real incident happens anyway, its postmortem action list feeds straight into this fix log — reality is just a drill you didn't schedule.
Related: tabletop exercise failure points · backup restore test · disaster recovery plan template · first 30 minutes · severity matrix · the 2AM handoff test · deployment rollback · break-glass account · annual security review · postmortem template · incident response plan · on-call rotation schedule · vendor escalation ladder