Deployment Rollback Checklist (Tested Before You Need It)

Most teams discover their rollback plan is fictional at 2am, mid-incident, when someone types the words "just roll it back" and nobody can say how. A rollback that has never been run is not a rollback — it's a hope. The fix isn't a bigger process. It's a one-page pre-deploy card that forces the rollback to be written as commands, tested, and timed. Here's the checklist.

The pre-deploy rollback card — copy this

1. Write the rollback as exact commands. Not "revert the deployment" — git revert 4f2c1a9 && ./deploy.sh staging or kubectl rollout undo deploy/billing or "restore snapshot pre-2026-09-11 from the panel". The version of you at 2am executes text, not judgment. If you can't write the commands, you don't have a rollback — you have an incident.
2. Time it, then halve your patience. Run the rollback once on staging and write the real number next to it: "measured 6m40s". Your rollback-the-moment threshold is roughly half that — if the new deploy looks bad at +3 minutes and the rollback takes 7, you're already behind. Slow rollbacks get executed too late to matter.
3. Check the blast radius of the rollback itself. Rolling back code is usually safe. Rolling back a database migration can destroy data written since. Rule: schema changes go forward-only (add nullable column, never drop in the same release), and the rollback note must say explicitly whether data written during the bad window survives.
4. Freeze the state you can revert to. Note the previous tag/commit/image digest, the config version, and (for schema) the migration number. If you deploy by "latest" tags and shared mutable config, there is no previous state to return to — pin versions before every deploy.
5. Decide rollback criteria before the deploy. Write the tripwire down: "rollback if error rate >2% for 5 min, or checkout fails twice, or p95 latency >800ms". Judging by vibes means arguing with yourself during an outage and losing.
6. Verify the rollback the way you'd verify a deploy. Hitting the rollback button is not success. Run the same smoke checks as the deploy: login works, a test order completes, error rate back under baseline. A half-rolled-back system is a new incident wearing a costume.
7. Announce it. One line to the channel: "Rolling back billing to 4.1 — cause, ETA 5m." If you use a status page, the customer line goes out before the commands do. Silence during a rollback reads as "nobody is driving".

The one-page rollback record

The rules that make rollbacks real

• Test rollbacks on slow weeks. The change record already asks "has anyone actually run the rollback?" — the answer should be yes, on staging, within the last quarter. A rollback drill costs ten minutes; an untested rollback during an outage costs the weekend.
• Feature flags beat redeploys. The fastest rollback is a toggle. Put risky behavior behind a flag, and "roll back" becomes a config flip measured in seconds, not a redeploy measured in minutes. Reserve the deploy-revert path for actual code defects.
• Forward-only schemas, always. Every classic rollback disaster is a migration that ate data on the way back. Two-phase migrations (expand, then contract in the next release) make every deploy reversible by default — that's the 2am test applied to databases.
• Rollbacks are deploys too. They get a watcher, a smoke check, and a note in the same runbook. "It's just a rollback" is how a recovery becomes a second outage.
• Keep the last-known-good deployable. If your previous good build only exists as a commit hash that no longer builds (deps moved, images pruned), pin and archive it. Last-known-good that can't be built is a memory, not a rollback.

---

The Ops Starter Kit Vol. 2 ($27) includes the change-control pack with the rollback log and freeze calendar this card plugs into — launch week: 30% off any paid kit with code HIVE-LAUNCH30 at checkout.