The SSO Outage Runbook for Small Teams

A DNS outage makes you hard to find. A single sign-on outage makes you impossible to enter — including for you. The status page is up, the site renders, the database is fine, and every door is locked because the front door (Google Workspace, Okta, Entra ID, or whatever issues the tokens) is having a bad day. Worst of all, the people who could fix it are locked out too. The fix is not heroics at 2am. It is five checks to tell lockout from outage, one local door you built in sunlight, and a comms path that does not run through the thing that is down.

First five minutes: is it them or you?

Half of "SSO is down" is not an IdP outage — it is one misconfigured app, one expired signing certificate, or one person whose session died. Five checks before you declare anything:

1. Reproduce from a second device on a different network. Phone on mobile data, incognito window, another office. One person failing = their session; everyone failing = the IdP.
2. Test a second app behind the same IdP. The SSO login button on two different apps. Both dead = provider. One dead = that app's SAML/OIDC config, and the runbook is not this one — it is that app's admin page.
3. Check the IdP's status page and your own auth logs. No successful authentications for fifteen minutes across all apps is a signal your logs can confirm before any vendor admits anything. Errors like invalid_signature or a today-dated cert expiry point at your side, not theirs.
4. Check the auth domain itself. If login.yourcompany.com or the IdP's auth host stops resolving, this is a DNS problem wearing an SSO costume — run the DNS outage runbook first.
5. Decide the path, out loud, in one sentence: "Provider-side — activating break-glass and posting status" or "Our side — last change was X, rolling back X." An IdP outage is the one incident where the person deciding should not be the person trying to log in.

Break-glass: the local door you built in sunlight

When the IdP is down, the only way in is a path that does not go through the IdP. That path must exist before the outage — building it during one means building it without testing it. Four properties of a break-glass that actually works:

• Local admin account(s) on the critical few systems — the production host, the payment admin panel, the DNS/registrar console — kept in a password manager vault that is not SSO-gated. An emergency login stored inside the login system is a joke with a punchline at 3am.
• Printed 2FA recovery codes, sealed, in two locations. The most common break-glass failure is the second factor that lives in the same dead IdP. Paper survives a provider outage.
• A named short list of who may use it — two or three people, written down. Break-glass with unlimited users is just "no security."
• Every use is logged and closed the same day: who used it, on what system, for how long, and the ticket that rotates the password after. A break-glass door that stays open forever is a breach you have not noticed yet. Drill it quarterly — a ten-minute game where one person proves they can get into production without the IdP is the cheapest insurance you will ever buy.

What still works when login does not

The outage is smaller than it feels — if you know what survives:

• Existing sessions usually hold. Tokens were issued before the outage and are verified locally or cached; customers already logged in keep working. Panic-resetting sessions converts a sign-in problem into a full outage.
• Service accounts and CI keep running if they use API keys rather than interactive SSO — deploys, backups, and monitors usually keep running. Check before assuming the whole machine is down.
• New signups and password resets are the real victims. That is your blast-radius line for comms: "existing users are unaffected; new sign-ins are paused."
• Email may be behind the same door. If your mail lives in Google Workspace or Microsoft 365, the IdP outage took your email too. Your comms fallback — an independently hosted status page, a personal-domain group alias, a phone/WhatsApp tree — must not depend on the thing that is down. Write the fallback on a calm afternoon.

Tell people before they tell you

Nothing turns a 40-minute IdP hiccup into a trust incident faster than silence. Three updates, on a clock:

• Internal, minute 5: "IdP provider outage suspected; break-glass active for [two named people]; existing sessions unaffected; next update 15:20." One message, in the channel everyone reads.
• Status page, minute 15: "Sign-in is unavailable for some users. Data is safe. Existing sessions are unaffected. We are working with our identity provider and will update by [time]." Name what works, not just what is broken.
• Every promised update, on time, even when it is "no change yet." The update cadence is the product during an outage.

The mistakes that turn an hour into a day

• The single IdP dependency with no local door. If every system — production, DNS, the password manager itself — federates to one provider and nothing has a local login, an IdP outage is a full company outage. One break-glass account on the three systems you would cry about losing is the minimum.
• Break-glass stored inside the dead system. Recovery codes emailed to the SSO-gated inbox, the local admin password in the SSO-gated vault. Store the escape hatch outside what it escapes from.
• The password-reset loop. The reflex during a login outage is resetting everyone's passwords. It does nothing while the IdP is down, and after recovery it produces a lockout storm, a support queue, and MFA re-enrollment for people who were never affected. Nobody resets anything during an IdP outage.
• Fixing the provider. You cannot fix Okta from your laptop. Your job is the two things you own: the local door and the comms. Everything else is waiting well.
• Skipping the review because "the vendor fixed it." The post-mortem question is not "why did the IdP fail" — it is "why did we have no path in, and how long did it take us to notice." The answer list is the next quarter's hardening work.

Worked example

An eleven-person agency, Friday 9:12am: Google Workspace authentication went partially down (Google later confirmed a global incident). The old way: everyone tried their own browser in turn for an hour, two admins reset passwords "to see if it helps" (it did not), the founder could not reach production because production SSH'd through SSO, and the first customer-facing word was a support ticket at 11:30 asking if the company had disappeared. Total: five hours of degraded work, 60+ unnecessary password resets, and one churned account.

The rerun, six months later: second-network reproduction at 9:14, second-app test confirmed provider-side by 9:19, break-glass opened for the two named on-call people at 9:21 — local admin on the deploy host and the payment panel, password rotated and ticket closed by 9:40. Status page updated 9:26 with "existing sessions unaffected" (and it was true — client work never stopped). Email fallback ran through the founder's personal-domain alias. Zero password resets. Total: 34 minutes of lockout for two break-glass users, zero for everyone else, and the post-mortem produced one permanent fix: a second local admin on the staging host, which the drill the next quarter proved in eleven minutes.

Metrics (for the SSO outage plan itself)

• Break-glass drill cadence: quarterly, with a pass rate recorded — a drill nobody ran is a door that does not open.
• Time from "confirmed provider-side" to status posted: under 15 minutes. Track the median.
• Password resets during IdP outages: zero, always. Any nonzero number is the reset loop re-emerging.
• Every critical system has a documented local-login path, tested in the last 90 days. Count it as a percentage; anything under 100% is a list of future 3am surprises.
• Comms fallback verified independent of the IdP: if your status page, email, and chat all authenticate through the IdP, you have one outage with no microphone.

From the HIVE80lab kit

• The First 30 Minutes — free incident quick-start checklist
• Ops Starter Kit — incident response for small teams — $14
• Ops Starter Kit Vol. 2 — advanced incident response & communications — $27
• Ops Mega Bundle — all 5 kits in one download — $49

Related: the MFA device lost runbook is the single-person version of this incident; the DNS outage runbook is its sibling when the auth domain itself stops resolving; and the vendor outage runbook is the general playbook for any provider having the bad day the IdP is having. and the configuration drift audit is the pass that reconciles every break-glass edit this runbook produces back into the repo