The Production Engineering Digest

The Production Engineering Digest

Home
Podcast
Notes
Archive
About
When the Alert Never Arrives: How Alerting Pipelines Fail Silently
Lucidworks got a 30-day warning that its certificate would expire. The check kept firing on schedule, but for four weeks none of those alerts reached…
 • Arun Sankar A S K
What to Ask About On-Call Before You Accept the Offer
The questions that show whether a rotation is sustainable, and how to read the answers when nobody wants to call it bad
 • Arun Sankar A S K
The Roundup #7: Three Outages That Broke at the Seams
Azure, Slack and GitHub Actions each failed where two working systems met: two unrelated changes colliding, a third-party app tripping Slack's sending…
 • Arun Sankar A S K
When Rollback Doesn't Work: The Production Changes You Can't Undo
Reverting a deploy puts the old code back. It doesn't un-write data, take back a flag's new meaning, or reach a machine that can no longer boot, and…
 • Arun Sankar A S K
You Caused a Production Outage. Here's What to Do in the Week After
Almost everyone who works near production breaks it eventually. The outage rarely decides anything about your career. The next few days usually do.
 • Arun Sankar A S K

September 2026

The Roundup #6: Three Outages Where Starting Fresh Was What Broke
AWS EC2, Azure OpenAI and Claude all failed the same way last week: what was already running stayed up, and the usual recovery moves (relaunch, restart…
 • Arun Sankar A S K
Diagnosis Is the Outage: Where the Time in a Software Incident Actually Goes
In config and software outages, the fix is usually small. Finding it is what eats the hours, and the teams that recover fastest stop waiting to be sure…
 • Arun Sankar A S K
How to Put Incident Work in Your Promotion Case
Being fast on the pager gets you thanked in Slack. It rarely gets you promoted. Here's how to turn the incidents you handled into evidence a promo…
 • Arun Sankar A S K
The Roundup #5: Three Outages Caused by the Service Nobody Was Watching
Salesforce, Microsoft 365, and Google Cloud all went down this month — and each time, the failure was in the layer underneath the product, not the…
 • Arun Sankar A S K
Three Outages in One Night: Why On-Call Teams Wrongly Assume They're Connected
Cloudflare had three separate incidents in about 24 hours last week. None of them shared a cause — but a team that doesn't check will treat them like…
 • Arun Sankar A S K
The AI Skills That Actually Matter for a DevOps or SRE Career Right Now
Job postings want agents and LLMs on the resume. Interviewers and promo committees are testing for something narrower — and it isn't prompt engineering.
 • Arun Sankar A S K
The Roundup #4: Three Outages Caused by Someone Else's Outage
GitHub Copilot, Shopify checkout, and a production database all broke over a vendor's outage — the difference was how fast each team found out.
 • Arun Sankar A S K
© 2026 Arun Sankar A S K · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture