Mob Rule by Scientists

Neat article in the NYTimes the other day about how, in science, the "general consensus" can be self-perpetuating, even if fallacious. How well this applies to sacrosanct items is still to be determined, and the author, probably wisely, chose not to go there.

Since the NYT tends to pull articles after a week or so, I'll just violate a little copyright and post the text here:

Diet and Fat: A Severe Case of Mistaken Consensus

By JOHN TIERNEY
Published: October 9, 2007

In 1988, the surgeon general, C. Everett Koop, proclaimed ice cream to a be public-health menace right up there with cigarettes. Alluding to his office’s famous 1964 report on the perils of smoking, Dr. Koop announced that the American diet was a problem of “comparable” magnitude, chiefly because of the high-fat foods that were causing coronary heart disease and other deadly ailments.

He introduced his report with these words: “The depth of the science base underlying its findings is even more impressive than that for tobacco and health in 1964.”

That was a ludicrous statement, as Gary Taubes demonstrates in his new book meticulously debunking diet myths, “Good Calories, Bad Calories” (Knopf, 2007). The notion that fatty foods shorten your life began as a hypothesis based on dubious assumptions and data; when scientists tried to confirm it they failed repeatedly. The evidence against Häagen-Dazs was nothing like the evidence against Marlboros.

It may seem bizarre that a surgeon general could go so wrong. After all, wasn’t it his job to express the scientific consensus? But that was the problem. Dr. Koop was expressing the consensus. He, like the architects of the federal “food pyramid” telling Americans what to eat, went wrong by listening to everyone else. He was caught in what social scientists call a cascade.

We like to think that people improve their judgment by putting their minds together, and sometimes they do. The studio audience at “Who Wants to Be a Millionaire” usually votes for the right answer. But suppose, instead of the audience members voting silently in unison, they voted out loud one after another. And suppose the first person gets it wrong.

If the second person isn’t sure of the answer, he’s liable to go along with the first person’s guess. By then, even if the third person suspects another answer is right, she’s more liable to go along just because she assumes the first two together know more than she does. Thus begins an “informational cascade” as one person after another assumes that the rest can’t all be wrong.

Because of this effect, groups are surprisingly prone to reach mistaken conclusions even when most of the people started out knowing better, according to the economists Sushil Bikhchandani, David Hirshleifer and Ivo Welch. If, say, 60 percent of a group’s members have been given information pointing them to the right answer (while the rest have information pointing to the wrong answer), there is still about a one-in-three chance that the group will cascade to a mistaken consensus.

Cascades are especially common in medicine as doctors take their cues from others, leading them to overdiagnose some faddish ailments (called bandwagon diseases) and overprescribe certain treatments (like the tonsillectomies once popular for children). Unable to keep up with the volume of research, doctors look for guidance from an expert — or at least someone who sounds confident.

In the case of fatty foods, that confident voice belonged to Ancel Keys, a prominent diet researcher a half-century ago (the K-rations in World War II were said to be named after him). He became convinced in the 1950s that Americans were suffering from a new epidemic of heart disease because they were eating more fat than their ancestors.

There were two glaring problems with this theory, as Mr. Taubes, a correspondent for Science magazine, explains in his book. First, it wasn’t clear that traditional diets were especially lean. Nineteenth-century Americans consumed huge amounts of meat; the percentage of fat in the diet of ancient hunter-gatherers, according to the best estimate today, was as high or higher than the ratio in the modern Western diet.

Second, there wasn’t really a new epidemic of heart disease. Yes, more cases were being reported, but not because people were in worse health. It was mainly because they were living longer and were more likely to see a doctor who diagnosed the symptoms.

To bolster his theory, Dr. Keys in 1953 compared diets and heart disease rates in the United States, Japan and four other countries. Sure enough, more fat correlated with more disease (America topped the list). But critics at the time noted that if Dr. Keys had analyzed all 22 countries for which data were available, he would not have found a correlation. (And, as Mr. Taubes notes, no one would have puzzled over the so-called French Paradox of foie-gras connoisseurs with healthy hearts.)

The evidence that dietary fat correlates with heart disease “does not stand up to critical examination,” the American Heart Association concluded in 1957. But three years later the association changed position — not because of new data, Mr. Taubes writes, but because Dr. Keys and an ally were on the committee issuing the new report. It asserted that “the best scientific evidence of the time” warranted a lower-fat diet for people at high risk of heart disease.

The association’s report was big news and put Dr. Keys, who died in 2004, on the cover of Time magazine. The magazine devoted four pages to the topic — and just one paragraph noting that Dr. Keys’s diet advice was “still questioned by some researchers.” That set the tone for decades of news media coverage. Journalists and their audiences were looking for clear guidance, not scientific ambiguity.

After the fat-is-bad theory became popular wisdom, the cascade accelerated in the 1970s when a committee led by Senator George McGovern issued a report advising Americans to lower their risk of heart disease by eating less fat. “McGovern’s staff were virtually unaware of the existence of any scientific controversy,” Mr. Taubes writes, and the committee’s report was written by a nonscientist “relying almost exclusively on a single Harvard nutritionist, Mark Hegsted.”

That report impressed another nonscientist, Carol Tucker Foreman, an assistant agriculture secretary, who hired Dr. Hegsted to draw up a set of national dietary guidelines. The Department of Agriculture’s advice against eating too much fat was issued in 1980 and would later be incorporated in its “food pyramid.”

Meanwhile, there still wasn’t good evidence to warrant recommending a low-fat diet for all Americans, as the National Academy of Sciences noted in a report shortly after the U.S.D.A. guidelines were issued. But the report’s authors were promptly excoriated on Capitol Hill and in the news media for denying a danger that had already been proclaimed by the American Heart Association, the McGovern committee and the U.S.D.A.

The scientists, despite their impressive credentials, were accused of bias because some of them had done research financed by the food industry. And so the informational cascade morphed into what the economist Timur Kuran calls a reputational cascade, in which it becomes a career risk for dissidents to question the popular wisdom.

With skeptical scientists ostracized, the public debate and research agenda became dominated by the fat-is-bad school. Later the National Institutes of Health would hold a “consensus conference” that concluded there was “no doubt” that low-fat diets “will afford significant protection against coronary heart disease” for every American over the age of 2. The American Cancer Society and the surgeon general recommended a low-fat diet to prevent cancer.

But when the theories were tested in clinical trials, the evidence kept turning up negative. As Mr. Taubes notes, the most rigorous meta-analysis of the clinical trials of low-fat diets, published in 2001 by the Cochrane Collaboration, concluded that they had no significant effect on mortality.

Mr. Taubes argues that the low-fat recommendations, besides being unjustified, may well have harmed Americans by encouraging them to switch to carbohydrates, which he believes cause obesity and disease. He acknowledges that that hypothesis is unproved, and that the low-carb diet fad could turn out to be another mistaken cascade. The problem, he says, is that the low-carb hypothesis hasn’t been seriously studied because it couldn’t be reconciled with the low-fat dogma.

Mr. Taubes told me he especially admired the iconoclasm of Dr. Edward H. Ahrens Jr., a lipids researcher who spoke out against the McGovern committee’s report. Mr. McGovern subsequently asked him at a hearing to reconcile his skepticism with a survey showing that the low-fat recommendations were endorsed by 92 percent of “the world’s leading doctors.”

“Senator McGovern, I recognize the disadvantage of being in the minority,” Dr. Ahrens replied. Then he pointed out that most of the doctors in the survey were relying on secondhand knowledge because they didn’t work in this field themselves.

“This is a matter,” he continued, “of such enormous social, economic and medical importance that it must be evaluated with our eyes completely open. Thus I would hate to see this issue settled by anything that smacks of a Gallup poll.” Or a cascade.