Thanks to a Freedom of Information Act request by the Center for Consumer Freedom, the Centers for Disease Control and Prevention (CDC) has posted internal documents on its website that add to the growing body of evidence showing the agency’s scientists knew of substantial scientific shortcomings well before they published their study blaming overweight and obesity for 400,000 deaths a year. The documents were part of a report by an internal CDC committee commissioned to examine the headline-grabbing finding. They were finally released this week, three months after the agency initially refused to turn them over to the Center for Consumer Freedom. What follows are some highlights from the individual committee members’ analyses of the 400,000-deaths study.
“Methods used to calculate number of deaths due to obesity were incorrect and possibly miscalculated … The use of the improper formula is a rather serious mistake to make. At the time this study was being conducted, the scientific literature had several papers describing potential bias. Following Allison et al. [which attributed 300,000 deaths to overweight and obesity per year] in using an incorrect method was not justified. From the cross-clearance, it seems as if this bias from the wrong formula was pointed out to the authors.” (emphasis added)
“The knowledge about inappropriate use of adjusted relative risks in certain attributable-fraction formulas was in the literature prior to the preparation of this manuscript and was apparently shared with the authors prior to publication.” (emphasis added)
“This review has clarified that we should no longer be using the relatively simple methodology of the model used by several of those previous papers [which arrived at the estimates of 300,000 and 400,000 deaths annually].”
“My general conclusion is that the Mokdad et al. paper [the 400,000-deaths study] makes some bold statements, bolder than the original McGinnis and Foege paper [attributing 300,000 deaths to poor diet and physical inactivity in 1990], which might have been better off being presented as a policy exercise rather than a scientific study; the estimates seemed a combination of scientific calculation and expert opinion. Furthermore, the scientific reviewers of the paper who mentioned the problems were not taken as seriously as they could have been.” (emphasis added)
“Double counting [of deaths] is mentioned clearly in the Discussion section but not in the Introduction and not clearly in the Methods … A sensitivity analysis would have helped the authors direct critique of the paper to the impact of some of the assumptions and methodological holes on the inferences. ” (emphasis added)
“The [CDC’s] clearance process appeared to have been begun in good faith, though given dissent, it veered off the path. This likely happened since the Director [click here to see our open letter to CDC Director Julie Gerberding] is a co-author and presumably approved the paper. I think if the scientists had believed that their concerns were being considered, the issue of clearance may not have arisen. Estimation based on best educated guesses should be acknowledged as such up front: for this paper the experience of the authors supports their ability to make some decisions with weak data, though some decisions may ultimately be incorrect.”
“Written details of exact response to each reviewer’s comments were not provided so I cannot comment on whether CDC reviewers’ comments were addressed.”
Regarding the last point, one wonders: why were these responses not provided to the committee? They are the crux of the matter. What did the CDC know and when did they know it?
Many newspaper headlines screamed that obesity was set to overtake smoking as the number one cause of preventable death when the CDC released its original 400,000-deaths estimate. That’s not surprising, considering that CDC’s Gerberding proclaimed the day the study was released: “Obesity is catching up to tobacco as the leading cause of death in America. If this trend continues, it will soon overtake tobacco.” But according to more than one committee member, that analysis was “inappropriate.” One wrote:
“Direct comparisons of deaths attributable to different causes is inappropriate due to variety of methods used (e.g. all cause mortality for obesity vs. cause specific for smoking; effect of adjusted vs. unadjusted estimates). This is the most legitimate concern to me. I don’t believe such comparisons were the primary aim of the authors but are inevitable when precise estimates are given in a table, and this was certainly highlighted in media reports. Given the uncertainties in the data, it is difficult to conclude that obesity is now equal to smoking or will shortly overtake it.”
A number of committee members suggested that the CDC submit an official correction to JAMA regarding the methodological problems. One wrote: “If possible, the proper formula should be used to calculate the deaths attributed to obesity. If using the published assumptions but the more suitable equation leads to different estimates, an erratum should be submitted.” Yet the CDC’s subsequent erratum to JAMA, which lowered the estimate to 365,000 deaths, only addressed mathematical errors – and not the substantial methodological problems raised in the review committee’s report.