I have written a lot of post-mortems. The first few I wrote were quietly terrible. Not because I was dishonest or lazy, but because I didn't understand what a post-mortem was supposed to accomplish. I thought it was a record of what happened. It is not. It is a document designed to prevent the same thing from happening again.
That difference sounds small. It changes everything.
What a post-mortem is
A post-mortem (some teams call it an incident review, or a retrospective, or a PIR - Post Incident Review) is a document written after something goes wrong in production. An outage, a data issue, a degraded service - anything significant enough to have caused a real problem.
A good post-mortem answers four questions:
1. What happened, and when?
2. Why did it happen?
3. Why didn't we catch it sooner?
4. What are we going to do so it doesn't happen again?
A bad post-mortem answers one question: whose fault was it?
The blameless part is harder than it sounds
"Blameless post-mortems" is a concept that has become fashionable in the last decade, and like a lot of fashionable concepts it is frequently misunderstood to mean "don't say what anyone did wrong."
That's not what it means.
Blameless means recognizing that people make the decisions that seem most reasonable given what they know and can see at the time. When something goes wrong, the question isn't "why did this person make this decision" - it's "why did the system create conditions where this decision seemed reasonable?"
I'll give you a concrete example. At one firm I worked at, an engineer accidentally took down a production service by running a deployment script on the wrong environment. The script was designed for staging. It deleted and rebuilt containers that, in staging, had no persistent data. In production, those same containers had been accumulating session data that we hadn't realized was being stored there.
The initial reaction in the room was to ask why the engineer hadn't been more careful. The blameless reaction is to ask why the script didn't have an environment check, why the production and staging environments had different behavior that wasn't documented anywhere, and why we had session data sitting in containers rather than in a dedicated session store.
The engineer made a mistake. But the system made that mistake easy to make and hard to notice. Fix the system, and the next engineer who makes the same mistake won't cause an outage.
The format I actually use
I've settled on a fairly simple template after years of iteration:
Summary - two to three sentences. What happened, when, how long it lasted, what the impact was. Written for someone who knows nothing about the incident.
Timeline - chronological list of events with timestamps. Not narrative prose. Just: "14:32 - alert fired. 14:35 - on-call engineer paged. 14:41 - incident declared." This is the factual record. It should be boring.
Root cause - this is the one section where I encourage people to be ruthless. Not about the person, but about the system. Ask "why" at least five times. The first answer is usually a symptom. The fifth answer is usually close to the real cause.
What we did to fix it - specifically, the steps taken to restore service. Different from the timeline; this is focused only on the response.
Action items - numbered list, each with an owner and a due date. No unowned action items. No action items without dates. An action item that nobody owns is a hope, not a plan.
The thing most post-mortems get wrong
They're written too late. The best post-mortem is written within 48 hours of the incident, while people's memories are fresh and the emotional urgency is still there to push action items forward.
I have written post-mortems three weeks after incidents. They are useless documents. The timeline is hazy, the root cause analysis is reconstructed from imperfect notes, and nobody is going to complete action items for an incident that happened a month ago.
Write them immediately. Even a rough draft written the day after the incident and cleaned up two days later is better than a polished document written two weeks later.
The goal of the post-mortem is not to have a nice document. The goal is to change something. And changes happen when the incident is still fresh.
