From read-only to 2AM-ready: onboarding a new hire onto the pager
The fastest way to turn a good hire into a liability at 2AM is to hand them the pager with a link to the wiki and the words "you'll figure it out." The fastest way to burn out your senior engineer is the opposite: keeping them as the only person who can take a page forever. On-call onboarding is the bridge between those two failure modes — a short, written path from read-only observer to certified responder that usually takes two to three rotations, not six months of osmosis. It rides on the structures you already have: the rotation schedule, the handover template, the severity matrix, and the incident response plan the new hire is joining.
1. Before the first rotation: the reading list and the read-only access
- Give four documents, not forty. The reading list that actually gets read is short: the incident response plan, the severity matrix, the first-30-minutes runbook, and the current rotation schedule with names and dates. Everything else is reference material the new hire will meet when a page forces them to. A reading list that takes a week is a reading list that gets skimmed and forgotten.
- Grant read-only everything on day one. Dashboards, alert history, past incident timelines, the status-page archive: a new hire who can see last month's pages learns what real pages look like before they own one. What they do not get yet: production change rights, the break-glass credentials, or the power to acknowledge-and-mute. Read access is safe and cheap; it is also the difference between shadowing and spectating.
- Set the expectation in writing. One line in the onboarding doc: "You are not expected to resolve anything on your first two rotations. You are expected to watch, take the timeline, and ask why-questions." New hires silence alerts they do not understand out of fear of looking slow; the written permission to not know is what keeps the noise visible.
2. The shadow rotation: watch pages happen, write the timeline
- Pair every page, no exceptions. For the shadow rotation, alerts go to both phones, and the on-call engineer talks through their triage out loud: which severity, which runbook, who gets pinged, what gets written where. The new hire's job is the timeline — timestamps, who did what, what the alert actually said. Taking the timeline forces attention without handing over the wheel.
- Debrief every page within 24 hours. Ten minutes, three questions: what did you see first, what surprised you, what would you have looked up that we didn't? The surprises are the syllabus. If the new hire was surprised that a disk-full page resolves with a log rotate and not a server rebuild, that gap gets a runbook note before the next rotation — the onboarding is now improving the docs for everyone, which is exactly what a good shadow week produces.
- Include one handoff in each direction. The new hire should see a shift handoff performed and then perform one themselves, watched. The handover document is where tribal knowledge actually lives; writing one under supervision, while nothing is on fire, is the cheapest possible rehearsal of the real thing.
3. The drill: a first-30-minutes tabletop they run, not watch
- Flip the roles once before they go solo. Book 45 quiet minutes, hand the new hire a scenario (the same style as the first-30 tabletop: "it's 2AM, the checkout endpoint is throwing 500s"), and let them drive while a senior observer says nothing unless they would have reached for something harmful. The drill passes when the new hire, unprompted: classifies the severity, opens the timeline, posts the first customer communication draft, and states the escalation threshold — not when they "know the docs."
- Drill the awkward parts on purpose. The parts new responders freeze on are social, not technical: waking up the boss, paging a person (not a channel), saying "I don't know yet" in an incident channel, and deciding to stop investigating and start escalating. The drill should force at least one deliberate escalation so the first real one at 2AM is muscle memory, not a moral crisis. The 2AM handoff test is the same idea applied to the whole team.
- Grade the drill against the runbooks, not the person. Every stall point in the drill is a documentation defect first: if the new hire did not know where the rollback lives, the rollback checklist is not linked from the alert, and that is a fix in the alerting, not a lecture. Onboarding that ends with a list of doc fixes has paid for itself.
4. Keys, access, and the break-glass line
- Escalate access in step with responsibility. The certification rotation is when production change rights arrive — granted through the same review your onboarding checklist uses for any production access, not as pager-passing ceremony. The rule that matters: by the time someone can be paged alone, someone else must be able to see everything they can do. Shared visibility first, shared break-glass only if the team is small enough that it cannot be avoided.
- Walk the break-glass path once, hands on. If your emergency access is a sealed kit or a break-glass account, the new hire walks it once during onboarding: where it lives, when it may be used, what gets logged and rolled immediately after. The classic break-glass failure is a credential nobody under 30 has ever typed; a five-minute supervised walkthrough in daylight is the fix. The same hands-on rule applies to the password manager vault and any API keys the rotation relies on.
- Do the leak-and-rotate speech now, not after an incident. New on-call people will eventually touch credentials in an incident — which is exactly when mistakes happen. Five minutes: what counts as a leaked key, where the leak runbook lives, and the rule that no incident ever gets worse by pausing to rotate a credential properly.
5. Certification: the gate that decides when they take the pager alone
- Write the gate down before the first rotation, not when the pressure arrives. A certification page is a dozen checkboxes with a sign-off line: completed reading list, one full shadow rotation with timeline, one handoff written, one first-30 drill passed, break-glass walkthrough done, and the severity matrix recited from memory in one sentence each level. When the boxes are checked, the next rotation is theirs. No vibes, no "whenever you feel ready" — feelings are not a rotation policy, dates are.
- Cap the shadow period at two rotations. An open-ended "shadow until ready" quietly becomes never, which is how teams end up with a single point of failure named Dave. Two rotations, then certify or name the specific gap and a date. The same deliberate-review rhythm as the annual security review, applied to people instead of systems.
- The first solo rotation is still staffed — by a phone, not a body. Certification means they take the page; it does not mean they are alone. The senior on-call stays reachable for the first solo shift with one agreement: calling is free, breaking things alone is expensive. After that rotation, the new hire joins the normal rotation and the compensation policy applies like it does for everyone else.
6. The one-page template
- Reading card: the four documents with links, the read-only access list with who granted it, and the written expectation line.
- Shadow card: rotation dates, pages paired (with debrief dates), timeline samples written, handoff performed both directions.
- Drill card: scenario used, date, pass/fix list (doc fixes separated from skill gaps), observer sign-off.
- Access card: rights escalated (what, when, who approved), break-glass walkthrough date, keys covered.
- Certification line: checklist complete yes/no, gaps named, certification date, first solo rotation date, post-solo debrief scheduled.
Small-team honesty note: if the team is two engineers and a founder, this whole page is one shared doc and a calendar invite — and it still matters, because the alternative is that the founder is the pager until burnout or a holiday forces the issue. On-call onboarding is how a small team buys its first real day off: the cheapest insurance is a second person who knows where the rollback lives. And when a page goes badly anyway, the postmortem feeds the next hire's reading list — which is how the whole system compounds instead of rusting.
Related: on-call rotation schedule · on-call handover template · on-call handoff checklist · the 2AM handoff test · severity matrix · first 30 minutes · incident timeline · incident communications · postmortem template · employee onboarding · incident response drill schedulenew admin first week >