The same is true of the statistical concept of sample bias. Say you want to know what Americans think of the job the president is doing. That should be simple enough. Just ask some Americans. But which Americans you ask makes all the difference. If you go to a Republican rally and ask people as they leave, it’s pretty obvious that your sample will be biased (whether the president is a Republican or a Democrat) and it will produce misleading conclusions about what “Americans” think. The same would be true if you surveyed only Texans or Episcopalians or yoga instructors. The bias in each case will be different, and sometimes the way it skews the numbers may not be obvious. But by not properly sampling the population you are interested in—all Americans—you will obtain distorted and unreliable results. Pollsters typically avoid this hazard by randomly selecting telephone numbers from the entire population whose views are being sought, which creates a legitimate sample and meaningful results. (Whether the sample is biased by the increasing rate at which people refuse to answer surveys is another matter.)
In the silicone breast implant scare, what the media effectively did was present a deeply biased sample. Story after story profiled sick women who blamed their suffering on implants. Eventually, the number of women profiled was in the hundreds. Journalists also reported the views of organizations that represented thousands more women. Cumulatively, it looked very impressive. These were big numbers and the stories—I got implants and then I got sick—were frighteningly similar. How could you not think there was something to this? But the whole exercise was flawed because healthy women with implants didn’t have much reason to join lobby groups or call reporters, and reporters made little effort to find these women and profile them because “Woman Not Sick” isn’t much of a headline. And so, despite the vast volume of reporting on implants, it no more reflected the health of all women with breast implants than a poll taken at a Republican rally would reflect the views of all Americans.
Our failure to spot biased samples is a product of an even more fundamental failure: We have no intuitive feel for the concept of randomness.
Ask people to put fifty dots on a piece of paper in a way that is typical of random placement and they’re likely to evenly disperse them—not quite in lines and rows but evenly enough that the page will look balanced. Show people two sets of numbers—1, 2, 3, 4, 5, 6 and 10, 13, 19, 25, 30, 32—and they’ll say the second set is more likely to come up in a lottery. Have them flip a coin and if it comes up heads five times in a row, they will have a powerful sense that the next flip is more likely to come up tails than heads.
All these conclusions are wrong because they’re all based on intuitions that don’t grasp the nature of randomness. Every flip of a coin is random— as is every spin of the roulette wheel or pull on a slot machine’s arm—and so any given flip has an equal chance of coming up heads or tails; the belief that a long streak increases the chances of a different result on the next go is a mistake called gambler’s fallacy. As for lotteries, each number is randomly selected and so something that looks like a pattern—1, 2, 3, 4, 5, 6—is as likely to occur as any other result. And it is fantastically unlikely that fifty dots randomly distributed on paper would wind up evenly dispersed; instead, thick clusters of dots will form in some spots while other portions of the paper will be dot-free.
Misperceptions of randomness can be tenacious. Amos Tversky, Tom Gilovich, and Robert Vallone famously analyzed basketball’s “hot hand”— the belief that a player who has sunk his last two, three, or four shots has the “hot hand” and is therefore more likely to sink his next shot than if he has just missed a shot—and proved with rigorous statistical analysis that the “hot hand” is a myth. For their trouble, the psychologists were mocked by basketball coaches and fans across the United States.
Our flawed intuitions about randomness generally produce only harmless foibles like beliefs in “hot hands” and Aunt Betty’s insistence that she has to play her lottery numbers next week because the numbers she has played for seventeen years have never come up so they’re due any time now. Sometimes, though, there are serious consequences.
One reason that people often respond irrationally to flooding—why rebuild in the very spot where you were just washed out?—is their failure to grasp randomness. Most floods are, in effect, random events. A flood this year says nothing about whether a flood will happen next year. But that’s not what Gut senses. A flood this year means a flood next year is less likely. And when experts say that this year’s flood is the “flood of the century”— one so big it is expected to happen once every hundred years—Gut takes this to mean that another flood of similar magnitude won’t happen for decades. The fact that a “flood of the century” can happen three years in a row just doesn’t make intuitive sense. Head can understand that, with a little effort, but not Gut.
Murder is a decidedly nonrandom event, but in cities with millions of people the distribution of murders on the calendar is effectively random (if we set aside the modest influence that seasonal changes in the weather can have in some cities). And because it’s random, clusters will occur—periods when far more murders than average happen. Statisticians call this “Poisson clumping,” after the French mathematician Siméon-Denis Poisson, who came up with a calculation that distinguishes between the clustering one can expect purely as a result of chance and clustering caused by something else. In the book Struck by Lightning, University of Toronto mathematician Jeffrey Rosenthal recounts how five murders in Toronto that fell in a single week generated a flurry of news stories and plenty of talk about crime getting out of control. The city’s police chief even said it proved the justice system was too soft to deter criminals. But Rosenthal calculated that Toronto, with an average of 1.5 murders per week, had “a 1.4 percent chance of seeing five homicides in a given week, purely by chance. So we should expect to see five homicides in the same week once every seventy-one weeks— nearly once a year!” The same calculation showed that there is a 22 percent chance of a week being murder-free purely by chance, and Toronto often does experience murder-free weeks, Rosenthal noted. “But I have yet to see a newspaper headline that screams ‘No Murders This Week!’ ”
Cancer clusters are another frightening phenomenon that owes much to our inability to see randomness. Every year in developed countries, publichealth authorities field calls from people convinced that their town’s eight cases of leukemia or five cases of brain cancer cannot possibly be the result of mere chance. And people always know the real cause. It is pesticides on farm fields, radiation from the region’s nuclear plant, or toxins seeping out of a nearby landfill. In almost every case, they don’t have any actual evidence linking the supposed threat to the cancers. The mere fact that a suspicious cancer rate exists near something suspect is enough to link them in most people’s minds.