In an October 2003 column and a November 2005 commentary, Steven Milloy, founder and publisher of junkscience.com and adjunct scholar at the Cato Institute, attempts to debunk the scientific claim from the Helena and Pueblo studies that smoking bans resulted in a dramatic, immediate reduction of heart attack admissions by 40% and 27%, respectively.
About the Helena study, Milloy accepts that fewer heart attacks occurred during the six-month period that the smoking ban was in effect, but he argues that there is no basis upon which to attribute that decline to the smoking ban. He points out, for example, that a very similar decline in heart attacks occurred in 1998. He suggests that there is no credible explanation for "why the 1998 dip in heart attack rates was just an anomaly but the 2002 dip was definitely due to the smoking ban."
The 2002 decline in heart attacks, Milloy argues, appears to be simply part of a cyclical pattern. And there is not enough of a historical context (the study did not go back far enough in time), he argues, to determine that the observed decline was anything other than a reflection of the random variation in this statistic over time.
Milloy also criticizes the study for failing to study any of the pre-ban and post-ban patients (no information is available, for example, on the smoking status and reported secondhand smoke exposure of these patients). This, he argues, further weakens the credibility of drawing a causal conclusion from the study.
A similar criticism of the Pueblo study was offered by an associate professor of political science at the University of Colorado who teaches statistics and commented that: "I'd like to see data from the last 10 years. They just studied this for a year and a half, and the conclusions could be coincidental or caused by other factors."
The Rest of the Story
I have to admit that I find Milloy's comments on Helena and Pueblo to be largely on the mark. In fact, anyone who is familiar with my own commentaries of these studies will note a striking similarity between the main arguments that I have made and precisely what Milloy is arguing above.
While I don't similarly question the link between secondhand smoke and heart disease, on the particular question of whether the Helena and Pueblo studies provide solid evidence that smoking bans dramatically reduce heart attacks, I agree with Milloy, and for the precise reasoning that he provides in his own criticism of these studies.
If one looks at historical trends in heart attack rates in these relatively small localities like Helena and Pueblo, one will note that there are often large changes from year to year in heart attacks, and some of these changes are of a similar magnitude as the changes associated with the implementation of their smoking bans.
But there is simply no credible evidence that I am aware of that demonstrates that the observed changes are anything more than simply random variation in the underlying data. At very least, there is nothing to demonstrate that the magnitude of any effect on heart attack rates is 40% or even 27%, even if a small effect of these smoking bans on heart attacks did occur. In fact, as I have argued, it is not plausible or even mathematically possible that these declines were due to the smoking ban.
And I agree with the University of Colorado professor that one really needs to look at data further back in time, such as from the past 10 years, to adequately understand what the baseline level of variation in these data is, and that is essential before when could conclude that the observed changes in these particular years is attributable to the smoking ban rather than to random variation.
I also should add that Milloy makes one other point that is worthy of highlighting. He discusses the fact that the Helena results were widely disseminated to the media and the public before the study was ever published or available for public scrutiny and he criticizes this "science-by-press conference" approach to releasing results by anti-smoking advocates "because they know their immediate audience likely will not be able to ask probing questions -- a tough thing to do when only sketchy details are hurriedly presented to people with no familiarity of the research conducted."
I completely agree. I would add that the release and dissemination of the Pueblo study followed the same pattern (science by press release - there is no actual study available for the public to review and so there is no way that anyone can scrutinizes the study methodology and results adequately). And I would add that it is not just the immediate audience that will not be able to ask probing questions, but the remote audience (the thousands of people who read news articles about the study) as well.
This is inappropriate, because it seems contrary to the integrity of science to present and widely disseminate research findings without making the details of the research available for public scrutiny. There are limited exceptions to this, I think, such as studies with dramatic clinical relevance for which a delay in sharing the results could affect the medical treatment of patients. But for the most part, if you're not willing or able to share your research, you probably shouldn't be releasing the results of that research via press release.
For many reasons, I am seriously concerned about the implications of this story for the credibility of the tobacco control movement. The public's interest and that of the movement itself, will certainly not be best served if the integrity of our science degrades into the realm of "junk science."
No comments:
Post a Comment