API key rotation checklist and scheduled-job monitoring: why cron silences are worse than cron errors, the heartbeat pattern that catches dead schedules, runtime alerting that catches hangs, and the 15-minute setup.">
Cron Job Monitoring Checklist for Small Teams
Cron fails in two ways and teams only notice one of them. The job that errors writes logs; the job that silently stops running — disabled during an incident and never re-enabled, renamed by a refactor, killed by a timezone change — writes nothing at all. The checklist has four parts: heartbeat every critical job, alert on runtime, alert on silence, and keep the schedule owned.
1. Heartbeat every critical job (the part that catches silence)
The only reliable way to know a scheduled job ran is for the job to say so. A heartbeat is one line at the end of a successful run:
Job class
Heartbeat
Alert condition
Backups
Touch a timestamp file, or ping a monitor URL
No heartbeat in >25h (daily job)
Sync / ingest jobs
Ping a monitor URL on success
No heartbeat within 1–2 expected intervals
Reports / digests
Ping + record row count
No heartbeat, or zero rows
Free heartbeat monitors (healthchecks-style services, or a 10-line script + your existing alerting) are enough. The rule: a job without a heartbeat is a job you're not allowed to trust.
2. Alert on runtime, not just exit codes
Exit code 0 covers less than you think:
The job that hangs. No error, no exit — the backup is stuck on an NFS mount since Tuesday. Wrap critical jobs with a timeout, and alert when runtime exceeds 2× the historical median.
The job that succeeds uselessly. Zero rows processed, zero files found — exit 0, nothing done. Log the counts and alert on the zero.
The job that succeeds at the wrong time. A schedule that drifted (DST, edited crontab) is a finding even when the run is green — heartbeats with timestamps catch it.
3. Own the schedule (the part cron doesn't give you)
One inventory of every schedule — system crontabs, systemd timers, CI cron, provider-side schedulers, and the "temporary" scripts in someone's home directory. If it fires on a clock, it's on the list.
Each entry has an owner and a "what breaks if this doesn't run" line. The inventory nobody can explain is the inventory you'll disable by accident.
Changes to schedules go through review. Most silent deaths are self-inflicted: a deploy that rewrote crontab, a rename that missed one host.
4. The 15-minute setup (do it today)
List your critical jobs (5 min): backups, syncs, alerts, digests. Everything else waits.
Add a heartbeat line to each: append a timestamp ping at the end of a successful run (curl to a monitor URL, or touch /var/lib/jobcheck/backup.last).
Add the silence check: one cron that verifies every heartbeat file is younger than its threshold — and alerts you when it isn't.
Test it once: disable one job for an hour on purpose and confirm the silence alert fires. An untested silence alert is a wish.
5. The mistakes that keep cron silent
Monitoring the server, not the job. The box is up; the job died three days ago. Uptime and schedules are different facts.
Trusting exit codes alone. A hung job has no exit code. A useless job exits 0.
Heartbeats that fire even when the job did nothing. Ping success only, and record the counts.
The untested silence alert. Simulate a dead job once a quarter; otherwise the first real test is the outage.
Takeaways
Cron silence is worse than cron errors — and only heartbeats catch it.
Alert on runtime and output, not just exit codes: hangs, zeros, and drift are all green-on-the-outside failures.
Every schedule has an owner and a consequence line — or it's one refactor away from disappearing.
---
The Automation Starter Pack ($19) includes pick-first workflows for exactly this layer — heartbeat checks, retention sweeps, and the weekly schedule review — pre-assembled so the boring checks run by themselves. The Ops Starter Kit ($14) covers what happens when a check does fire. Launch week: 20% off any paid kit with code HIVE20 at checkout.
Put this to work.The Ops Starter Kit bundles the highest-leverage templates — incident response, runbooks, onboarding, checklists — into one download ($14).