Google has a pretty good culture of doing postmortems. When something fails, someone close to the failure tries to document what happened and why. A good postmortem document should also point the way to avoid similar mistakes in the future. Mistakes happen, but you don’t want to make the same mistakes over and over again. Instead, it’s important to try to get to the root of a problem and fix it there. Failure can be a good thing if you learn valuable lessons along the way.
But it’s also a rule of thumb of software engineering that it’s 10x harder to catch a problem at each new level of deployment. That’s why solid tests (e.g. unit tests) are so helpful; it’s much easier to fix a problem when you catch it earlier. Likewise, requiring code reviews before submitting code changes can avoid lots of stupid mistakes. Once buggy or poor code is checked in, debugging a problem can get much harder. And if a problem makes it out into production, it’s typically even more difficult to fix.
When you take a solid practice like postmortems and think about going further upstream, you land on the idea of a “premortem.” An example of a premortem would be going to each member of a team before a product launches and asking them: “What’s going to break or fail? What’s the mostly likely thing to go wrong?” After all, the people who have been working hard on a project have been steeped in it and are intimately familiar with potential weaknesses and failures.
I first heard the idea of a premortem from a Freakonomics podcast, but the idea is so simple that it practically explains itself. However, there are a couple subtleties. First, don’t bring everyone into one big room and ask “Hey, what’s going to go wrong?” That’s a recipe for groupthink unless there are a small number of glaring problems.
Instead, you want to collect the initial feedback independently and privately so that people won’t be biased by hearing what others are saying. With private feedback you might end up hearing opinions that people are afraid to express in public. If you do have to settle for a big group meeting, ask people to channel their personal voice of doom on their own before opening up the public discussion.
The second subtlety is that “dogfooding,” the process of testing your own product internally before introducing them to the world, is almost like a premortem if you can get good feedback internally. The touchy issue here is authenticity: people want their feedback to be taken seriously, but internal feedback might be biased or skewed for various reasons. Even if you disagree with internal feedback, it makes sense to take a clear-eyed look at what people are saying. And if you do disagree, it helps to explain your reasons for adjusting to feedback or not.
Could a pre-mortem help your next project avoid a massive failure? Why not give it a shot to find out? Premortems can be an easy and fast exercise, and you might get some really useful insights. Just ask the people close to the launch to brainstorm “What’s most likely to go wrong?” before the project launches.
Great post Matt! I have to ask if you feel like this process has had a positive impact to minimize post-release public feedback on issues? You know, as well as anyone with your role at Google, what harsh teeth the public can have when things go south. Like you said, mistakes happen and I’m curious if there is data to support the value of these in-process steps? Thanks!
I’d suggest that “premortems” don’t just apply to code…they apply to businesses in general. A lot of marketing plans, for example, are destined to fail, and the failure points can easily enough be determined at a surface glance by experienced employees and contractors. For example, a hyperaggressive cold calling strategy targeting blue collar business owners is almost a surefire recipe for disaster, but I’ve seen it play out.
I think the better term here would be strategical thinking/development.
But. The reality is often coders and teams pushing hard to make deadlines so the owner or the manager can push the product out and hit a deadline. If you do a pre-launch premortum there is a chance you would never launch because there is something else that can always be done.
We like to call premortems as ‘anticipation’. Just recently, we launched a new strategy and used this method to look ahead at the possible problems we’d be facing. During the execution, I had to meet with my managers and see how far we’re come along. Cooperation is very important. If people are not honest or do their share of the work, it falls apart.
Properly testing stuff on a full tech copy (identical data identical hardware) helps back when i worked for dialcom uk we spent 9 months dual running our new billing system and checking against the live system.
Is it a concept similar to the “pretotyping” of Leonardo Zangrando?
A couple of additional ideas
– Could be done not only on code being checked in for a project, but on an overall project, a business area, or even at an enterprise level on a regular (annual) basis
– Could implement elements of typical risk assessments, which take each risk and assign it a score for both likelihood of it occurring and impact / fallout if it does happen to occur