At a press conference in March, 2004 Department of Health and Human Services Secretary Tommy Thompson stood flanked by Centers for Disease Control and Prevention (CDC) director Dr. Julie Gerberding, the Surgeon General, and other top health officials and said: “Americans need to understand that overweight and obesity are literally killing us.” Dr. Gerberding added that obesity is “going to overtake tobacco” as the number one cause of preventable death. She was referring to the CDC’s then-forthcoming study attributing 400,000 deaths a year to poor eating habits and physical inactivity. The next morning’s front page headline in USA Today typified the study’s coverage: “Obesity on Track as No. 1 Killer.“
Since March, the study’s key finding on obesity has come unraveled. The Wall Street Journal reported on November 23 that it suffered from a crippling mathematical error. Then, after the CDC authors submitted a correction to JAMA, where the study was originally published, an independent review committee determined that the study’s underlying methodology was limited.
The CDC study used the risk associated with excess weight that had been calculated in a 1999 study in JAMA by professor D.B. Allison. His study blamed obesity for 300,000 deaths in 1990. The CDC’s now-discredited study upped that number to 400,000, noting: “We used the same procedure reported by Allison et al. to estimate annual overweight-attributable deaths.” In addition to the CDC’s admitted mathematical error, the 400,000 study has several other substantial flaws that trace back to Allison’s method. The following serious flaws serve to inflate the number of deaths attributable to obesity:
1. The Study relied on outdated data
Both the CDC’s and Allison’s studies rely on data dating as far back as 1948. The 300,000 paper relied on six population studies to calculate the mortality risk from excess weight. The average start date of these studies was 1963 (more than 40 years ago) and the average end date was 1983. Consequently, the 400,000 figure assumes that our ability to treat high blood pressure, diabetes, heart disease, and other illnesses linked to obesity has not improved in more than a generation. Allison admits this is a problem. “When most of the cohort studies used were initiated,” he writes, “there were fewer intervention strategies to reduce risk factors associated with obesity and fewer medical therapies for postponing death from obesity-related diseases.“
|Cohort Study||Years Studied|
|Alameda County Health Study||1965-1975|
|Tecumseh Community Health Study||1959-1985|
|Framingham Heart Study||1948-1980|
|Americans Society Cancer Prevention Study||1960-1972|
|Nurses Health Study||1976-1992|
|NHANES 1 Epidemiologic Follow-up Study||1971-1992|
In an unrelated matter, Dr. Allison himself criticized another researcher for using one of the very same cohorts that Allsion relied on for his 300,000 study. In a letter published in JAMA, he wrote:
Finally, there are other factors that may account for apparent differences. First, Peeter et. al. looked at a cohort of individuals residing in Framingham, Mass beginning in approximately 1948… Since 1948, treatments for cardiovascular disease have improved and mortality rates have been reduced. For this and other reasons, the true effect of BMI on longevity of mortality rate may change over time.
Despite all of this, no adjustment was made.
2. Deaths due to overweight vs. deaths due to obesity
While most of the 400,000 deaths derive from the “obese” (Body Mass Index, or BMI 30 or higher), a substantial minority derive from the “overweight” (BMI 25 to 29.9). But Allison’s own data show no statistically significant relationship between being merely “overweight” and increased risk of death. If the overweight deaths had been excluded, the study would have reported 17 percent fewer deaths. And 17 of the reported relationships seem to indicate that being overweight has a protective effect. The failure to report a significant relationship between overweight and increased mortality is not surprising. Most studies find no correlation (much less causation) between the two.
According to a CDC researcher who is familiar with the topic and who wishes to remain anonymous, the range of Allison’s confidence intervals (which set the bounds for statistical significance) makes it difficult to draw a reliable conclusion about the relationship between excess weight and mortality:
Uncertainty around any estimates [of deaths from obesity] make it very difficult to use for public health purposes. Any estimate is not going to have much value, simply because it’s currently not possible to reach a level of precision that most scientists and policy makers would be comfortable with. The margin of error that people talk about — the 95 percent confidence interval — would end up showing a range of outcomes so that obesity has anything from a protective effect to a deleterious effect. For numbers of deaths, the range would go from negative to positive.
The CDC researcher also explained that Allison did not report a final confidence interval for the aggregate number of deaths in each BMI category. He contends that if Allison had reported these confidence intervals, the resulting range of deaths would be huge.
3. “Our estimates may be biased toward higher numbers due to confounding by unknown variables.” — Allison, et al.
The 400,000 figure presumes that any increased rate of death in overweight or obese people is the result of their excess weight — a very unlikely assumption. Those without a high school diploma are nearly twice as likely to be obese as those with a four-year degree. They are also much less likely to have health insurance and to receive quality health care. Other factors that could increase the risk of death among obese people include sedentary lifestyles, genetic ailments, and the negative effects of diet pills — including amphetamines, the weight loss drug of choice for much of the last half century. In 1970, eight percent of all U.S. prescriptions were for weight-loss amphetamines.
The authors of the 300,000 study admit that their calculations assume “all excess mortality in obese people is due to obesity,” and that such an assumption would have the effect of overestimating the total number of deaths due to obesity. They continue: “Our estimates may be biased toward higher numbers due to confounding by unknown variables.” Of course, access to health care and physical activity levels are known variables. The authors simply chose not to account for them. And as The New England Journal of Medicine pointed out in a 1998 editorial: “Mortality among obese people may be misleadingly high because overweight people are more likely to be sedentary and of low socio-economic status.” Michigan State Professor Jon Robinson explains the study’s flawed logic:
Perhaps the most glaring absurdity about the 400,000 deaths due to obesity study resides in the pronouncement that ‘all excess mortality in obese people is due to their obesity.’ This is as preposterous as claiming that differences in mortality rates between blacks and whites are solely a result of the color of their skin!
University of Virginia professor Glenn Gaesser reinforces the point, saying:
The authors made no attempt to determine whether other factors, such as physical inactivity, low fitness levels, poor diet, risk weight loss practices, weight fluctuation, use of weight loss drugs, less than adequate access to health care, etc. could have explained some of all of the excess mortality in large people.
Obesity-associated diseases are not necessarily caused by increased body weight. There is evidence to support a number of alternative explanations: (1) A disease or its treatment may itself promote obesity. (2) Lifestyle factors which cause disease may independently promote obesity. (3) The obese may be unhealthy as a group because they are more likely to be older, to have a low socioeconomic status, and to belong to an ethnic minority. (4) Hazardous weight loss methods and the dangers of repeated loss and regaining of weight may be major contributors to obesity-related diseases.
4. The study failed to control for the influence of age
In a recent study published in the American Journal of Public Health, a group of researchers from the CDC found that Allison overestimated the number of deaths attributable to obesity because the risk of obesity substantially decreases with age. They wrote: “[Allison’s] approach to calculating deaths attributable to obesity did not fully allow for age as a confounder (associated with both mortality and with obesity) or as an effect modifier (the relative risk varies with age), and thus it is unlikely to adequately account for the differential effects of age on the mortality relative risk for obesity.” Specifically commentating on Allison’s study, they wrote: “Even within these narrow ranges of relative risk estimates, we observed over a 10-fold difference in the magnitude of the estimates, from 23,313 to 297,835 deaths, depending on age specific mortality relative risks.”
The article ultimately concludes: “Given the present knowledge about the epidemiology of obesity, and especially the impact of age on mortality risks associated with obesity, it may be difficult to develop accurate and precise estimates. We urge caution in the use of current estimates of the number of deaths attributable to obesity.”
A second study by an overlapping team of CDC researchers found that Allison’s original study overestimated the total number of deaths by at least 17 percent — and probably much more — by not controlling for the influence of either age or sex. Writing in the American Journal of Epidemiology, they found: “The method used by Allison et al. did not allow for effect modification and only partially adjusted for confounding. Use of this approach is contradicted by several published statistical papers, which show that such an approach can lead to bias.” They went on to say: “When the mortality relative risk is estimated with a single age-adjusted relative risk, overestimation of deaths attributable to overweight and obesity in the US population is more likely than underestimation.” The authors also point out that, “If we restrict attention to deaths occurring in people under 75 years of age, the estimated number of deaths due to obesity would be considerably smaller (ranging from about 110,000 to about 149,000 in our examples), regardless of what method is used.”
5. Data not nationally representative
Only one of Allison’s six population studies used to calculate the likelihood of death from excess weight was nationally representative. The others over-represented whites and upper socioeconomic classes. One earlier journal article points out:
The databases on which height-weight tables are based have sampling and design problems … The largest databases, those of the Build studies and the American Cancer Society study, are not particularly representative of the United States as a whole. These studies disproportionately include men, whites, members of the upper and middle classes, and persons under age 60 … The Framingham Study and other studies based on community or occupational samples likewise do not represent the population.
If the authors had included only nationally representative data from the CDC’s NHANES 1 Follow-up Study, they would have reported 47,171 — or 13 percent — fewer deaths. It may seem counterintuitive that over-representing whites and upper socioeconomic classes would increase the number of deaths. But there’s a simple explanation: Many studies show that African Americans have little, if any, increased mortality risk from obesity.
An article in the Harvard Health Policy Review sums all this up well: “The major problem with this ‘obesity kills’ statistic is the lack of compelling evidence to substantiate it.“