In a rapid response to the original BMJ study which concluded that a 40% decline in heart attack hospital admissions in Helena, Montana was attributable to the city's smoking ban, Drs. Brad Rodu and Philip Cole of the University of Alabama at Birmingham presented historical data on trends in the heart attack mortality rate for Lewis and Clark County (which includes Helena) for the years 1979-2001.
These data demonstrate that there is significant random variation in heart attack mortality trends (which suggests that there is also substantial random variation in heart attack incidence trends) in Helena, since 85% of the county population lives in Helena. This random variation is so marked that the observed heart attack mortality in several years is dramatically lower than, or higher than, the trend in preceding years.
As the authors point out: "The variability is most clearly illustrated by two periods: 1989-1993 and 1997-2001 (the bars for these periods are yellow). The latter period is notable because of the year 2000, in which the mortality rate was 110, about 50% higher than the mean for the other years in the period. The former period includes 1992, in which the mortality rate was 40% lower than the mean for the other years in the period. In addition, 1992 follows three years of increasing AMI mortality. In isolation, this period presents a picture very similar to that described by Sargent et al from 1998 to 2003."
The authors conclude that random variation cannot be ruled out as an explanation for the observed 40% decline in heart attack admissions in Helena coincident with the city's smoking ban.
In response, the authors of the original study wrote: "Rodu and Cole are following a well-established tobacco industry strategy of trying to shift the focus away from our actual observations."
The Rest of the Story
Rodu and Cole have, I think, convincingly illustrated the dangers in drawing a causal conclusion from an association between one specific factor and a single observed change in a complex phenomenon such as heart attack admissions, especially when there is tremendous baseline variation and substantial secular changes in that phenomenon and when one looks only at a small slice of the overall secular trend.
Using the same methodology as in the original Helena study, the authors could have concluded that there was a significant decline in heart attacks in Helena in 1992 (it seems quite obvious from the data that there was a drop in heart attacks in 1992 and if one was looking only at the period 1989-1992, one could easily conclude that something happened in 1992 to bring down the heart attack rate).
Does this mean that the 40% decline in heart attacks in 1992 was due to the absence of a smoking ban? Or to the election of Bill Clinton as president of the United States? Or to the Toronto Blue Jays having won the World Series? Or to the Pittsburgh Penguins winning the Stanley Cup?
Of course not. The point is that in order to truly have a picture of the underlying secular trend and random variation in a phenomenon as variable as heart attack admissions in a small city, one has to go back many years. Examining an isolated five-year period runs a huge risk of interpreting what is really random variation as being a significant annual change.
And this is precisely what I think is the fundamental flaw of the Sargent et al. analysis.
Given this serious flaw, I do not suggest that the paper does not provide any evidence that heart attack admissions may have fallen in response to the smoking ban. But I think the flaw is serious enough so that one cannot credibly conclude that the smoking ban was the cause of the observed reduction in heart attacks.
Perhaps most disturbing in this story, however, is the original authors' response to the very appropriate and enlightening analysis by Rodu and Cole. Instead of addressing the criticism on its scientific merits, the authors accuse Rodu and Cole of using some sort of tobacco industry strategy. And they accuse them of trying to shift the focus away from the observations, when in fact they are doing an excellent job of trying to spotlight the observations and show how they are within the overall range of random variation observed in heart attacks in Helena over time.
Sargent et al. seem to be following what is a common tobacco control strategy of failing to address the scientific merits of an argument by instead issuing an ad hominem attack on the messenger - in this case, accusing them of using tobacco industry tactics.
The rest of the story is that the underlying random variation in heart attacks in Helena seems to be high enough to fully explain the 40% drop in heart attacks observed in association with the implementation of a smoking ban. This doesn't mean that the observed decline could not be due to the smoking ban, but it does suggest, to me at least, that it is premature to conclude that the heart attack decline was caused by the smoking ban, as opposed to simply representing random variation.
More importantly, the rest of the story reveals the use of what I have recently come to learn is a common tobacco control strategy in responding to scientific criticism - deflecting the merits of that criticism by attacking its messenger.