Truth-seeking series · № 3
Survivorship bias
Where would you armor the bomber?
It's 1943. Your bombers keep getting shot down over Europe. Armor is heavy — you can only add a little, so it has to go where it matters most.
Below is the damage data your crews collected: every dot is a bullet hole, mapped across the bombers that returned to base. You have 4 armor plates. Tap the plane to place them.
Armor where the holes aren't
Most officers wanted armor where the holes clustered — wings, mid-fuselage, tail. The statistician Abraham Wald told them to do the opposite: armor the engines and cockpit, the places with almost no holes.
The data only included planes that made it home. A bomber riddled through the wings could still fly. The holes you never saw — engines, cockpit — were on the planes at the bottom of the Channel. The clean spots on the survivors weren't the safe spots. They were the fatal ones.
Every dataset walked in through a door. Before you trust it, ask what got filtered out on the way in.
Where else this bites
- Founder advice. "We ignored the doubters and it worked" — collected exclusively from companies that survived. The graveyard ignored the doubters too; it doesn't give keynotes.
- "They built things to last back then." The flimsy old buildings are gone. You're sampling the survivors.
- Fund performance. Indexes of funds quietly drop the ones that closed, flattering the average.
- Transformation photos. You see the routine that worked for one person, not the thousand people it silently failed.