Bayes & base rates

You tested positive. How worried should you be?

A test catches 90% of people who have a disease. It also flags 9% of healthy people by mistake. 1 in 100 people have the disease. Your test comes back positive.

Most people guess there's a 90% chance you're sick. So do most doctors. Move the sliders and count the people instead.

How common is the disease? 1%

Test catches the sick (sensitivity) 90%

Test wrongly flags the healthy 9%

—

sick and flagged (true positives) healthy but flagged (false alarms) sick but missed healthy and cleared

Why the guess is wrong

This mistake has a name: base-rate neglect. The 90% is a fact about the test, and on its own says nothing about you. There are far more healthy people than sick ones. So a small error rate flags a lot of healthy people, often more than the sick ones it catches. The question to ask is: of everyone who tested positive, how many are sick?

Gerd Gigerenzer's team asked doctors this using percentages, and most got it wrong. When they asked with counts (of 1,000 people, 10 are sick, 9 get caught, 89 healthy people get flagged), most got it right. The math was identical.

The same goes for hiring signals, fraud alerts, drug-dog alerts and gut feelings about startups. Before you trust an alarm, ask how rare the thing it's looking for is.