Incident Response Changelog Automation: Keep the 2am Log Without Relying on 2am Memory
Every serious incident ends the same way: someone asks "when did this start,
exactly, and who changed what?" The teams that can answer have a changelog —
one append-only, timestamped record of what was seen, decided, and done. The
teams that can't answer are reconstructing the night from memory, chat
scrollback, and guesses.
An incident response changelog is not documentation. It's a running log with
three rules: one place, timestamps on everything, append only. You never
edit an old entry — you add a correction as a new entry. This page covers how
small teams keep that log, and which parts of it are worth automating.
The log format (30 seconds to learn)
Each entry: HH:MM INITIALS — what was observed/decided/done.
14:05 JT — customer reports "can't open invoices" (email quote)14:07 JT — file server shows ransom note; screenshot saved to inc-0911/14:09 PM — DECISION: payments provider will be disconnected (owner: PM)14:12 PM — correction to 14:09: disconnect sync client too, not just the share
That's the whole format. The dash matters more than the tool. Entries start
with what was observed (a fact) versus what was decided (a call someone
made) versus what was done (an action). Insurance claims, police reports,
and post-mortems all consume the same three categories.
What to automate first: timestamps and the incident folder
The highest-value automation is the most boring one: make the correct
behaviour the easy behaviour. Two options teams actually keep using after
week one:
Option 1 — a timestamp helper on every desk. A one-liner that prints a
ready-made log line (current time + your initials) that you paste your note
onto. On macOS/Linux, a tiny alias does it:
# ~/.zshrc — `ilog JT did X` -> "14:07 JT — did X"
ilog() { printf '%(%H:%M)T %s — %s\n' -1 "${1:-?}" "${*:2}" | tee -a ~/Desktop/incident.log; }
On Windows, the same job is a two-line PowerShell function that echoes
Get-Date -Format "HH:mm" in front of the text. The point is not the shell —
it's that the timestamp is never typed by a human at 2am, because tired
people mistype timestamps and wrong timestamps poison the timeline.
Option 2 — the incident folder script. One command creates the structure
you'll need by minute five, so nobody invents a folder structure mid-incident:
#!/bin/sh
# newincident.sh "short name" — creates inc-YYYYMMDD/
D=$(date +%Y%m%d)-"$1"
mkdir -p "$D"/screenshots
echo "# log — append only" > "$D"/log.txt
date "+%H:%M — incident declared: $D" >> "$D"/log.txt
echo "$D"
Run it, and everything that follows — screenshots, log, comms drafts, hashes
of affected files — has somewhere to live from the first minute.
Chat-channel logging: the automation most teams already have
If your team lives in Slack/Teams/WhatsApp, your cheapest changelog is a
dedicated incident channel plus one rule: every message starts with a
timestamp. Most chat platforms can give you this semi-automatically:
- Slack/Teams: create a workflow or scheduled-reminder template that
prompts the owner for a status line every 30 minutes. The prompt is the
automation — it forces the log to stay alive without anyone remembering.
- The 30-minute heartbeat: `14:30 STATUS: still isolated, awaiting
restore verdict — PM`. If a heartbeat has no reply, that itself is a signal
the incident has lost its owner.
What you don't do is automate the writing of the log with AI summarisers
during the incident. Summarise in the post-mortem, by all means — but the
live record must be exactly what a human typed at the time. A log that a
model can quietly rewrite is not evidence, and insurers and lawyers care
about the difference.
Snapshots: the changelog that writes itself
Some of the best incident changelog entries are automatic because they're
just copies:
- Screenshot immediately, rename later. The filename is metadata you can
fix; the pixels are the evidence.
- Keep the system log. Before rebooting or wiping anything, copy the
machine's own logs (Windows Event Logs, macOS log show, syslog) into the
incident folder. The machine's timestamps are independent of human memory.
- Hash the affected files if you can:
shasum -a 256 file > hashes.txt.
Later, proving "this file is the same one we saw at 14:07" is worth five
minutes of calm-day learning.
What not to automate
Three things resist automation, and trying to automate them is how teams end
up with logs nobody trusts:
- The decision record. "PM decided to disconnect" is a human call. The
log can remind you (a template line with DECISION: in it), but the entry
itself must come from the person who made the call.
- The customer timeline. Auto-announcing "we are experiencing issues"
to every customer the moment a script fires is a good way to turn a
contained incident into a public one. Comms are deliberate, scheduled,
human.
- Severity changes. Automated alerts will happily reclassify your
incident six times an hour. In the log, severity changes only when the
incident owner writes it.
Calm-day checklist (20 minutes, once)
- [ ] Create the
iloghelper or its Windows equivalent on the two machines
most likely to be awake at 2am.
- [ ] Save
newincident.sh(or a folder template) somewhere obvious. - [ ] Create the incident chat channel now, name it, pin the log format.
- [ ] Put the 30-minute status reminder template in the channel.
- [ ] Print the one-page First 30 Minutes checklist and put it where the
backup drive is — same cupboard, same crisis.
That's the whole automation layer. It's deliberately small: the failure mode
of small teams isn't "not enough tooling", it's "the plan existed but nobody
could run it at 2am". Automate the typing; keep the thinking human.
Related pages
- [The First 30 Minutes: what to do while it's happening](first-30-minutes-incident-response.html)
- [Incident response plan template for small teams](incident-response-plan-template-small-teams.html)
- [Severity matrix: why 3 levels stick](severity-matrix-3-levels.html)
- [Tabletop exercises: the 5 failure points](tabletop-exercise-failure-points.html)
Templates
The free [First 30 Minutes checklist](https://hive80lab.gumroad.com/l/first-30-minutes)
includes the log format and declaration sentence. The
[Ops Starter Kit](https://hive80lab.gumroad.com/l/ops-starter-kit) ($14) adds
the fill-in IR plan, comms templates, and five tabletop exercises that
pressure-test your logging habit before a real incident does. Everything is
also on [GitHub](https://github.com/Hive80-lab/hive80-toolkit) with SHA256
checksums if you prefer to review before buying.
*This is general guidance, not legal or regulatory advice — adapt it to your
environment and obligations.*
Deciding what to automate? The Automation Starter Pack ($19) ships the 4-question filter as a worksheet, 12 workflows ranked by payback, and the one-page spec that keeps automations honest — launch month: 30% off with code HIVE-LAUNCH30.