The study uses "an interrupted time series design with routinely collected hospital episode statistics data" to analyze "admissions from July 2002 to September 2008 (providing five years’ data from before the legislation and 15 months’ data from after) using segmented Poisson regression."
It concludes that the legislation resulted in a 2.4% decline in heart attacks, equating to 1200 fewer emergency admissions than would have occurred in the absence of the smoking ban.
The paper concludes as follows: "We therefore conclude that the implementation of smoke-free public places is associated with significant reductions in hospital admissions for myocardial infarction even in jurisdictions with pre-existing smoking restrictions."
The conclusions of the study have been disseminated widely in the media, such as in a Daily Mail article which states that "The number of heart attacks has fallen dramatically since the smoking ban came in."
The Rest of the Story
The paper reports the annual number of heart attack admissions throughout all of England for the years beginning in July 2002 through 2008. The smoking ban went into effect in July 2007. The data are shown in Table 1 below:
Table 1. Annual Heart Attack Admissions -- England, 2002-2008
(Years begin in July of the corresponding year)
As you can easily see from the graph, there was no observable change in the trend in heart attacks associated with the smoking ban during its first year. The rate of heart attacks had been falling prior to the ban and continued to fall at exactly the same rate after the ban. If anything, the rate of decline was slightly lower than it had been during the period 2004-2005.
Here are the exact figures for the annual declines in heart attacks observed in England according to these data:
2003-2004: 1.3%
2004-2005: 3.1%
2005-2006: 5.2%
2006-2007: 3.2%
2007-2008: 4.3%
Thus, in the two year period preceding the smoking ban, heart attacks fell by an annual average of 4.1%.
In the year following the smoking ban, heart attacks fell by 4.3%.
Thus, one can see that these data simply do not support the conclusion that the smoking ban led to a decline in heart attacks.
When I read the headlines in the newspapers, it then was hard to believe it when I saw the actual data from the study, which simply do not show any change in the trend in heart attacks in England in association with the smoking ban. It's just a straight line that continues down at the same rate as before!
What makes the conclusion even less valid, in addition to the fact that it is unsupported by the data presented in the study, is that there is no comparison group in this study!
Even if there were a very small increase in the rate of decline in heart attacks in England in 2007-2008, one cannot conclude that it was attributable to the smoking ban unless one can rule out the possibility that this decline was simply a reflection of secular trends in heart attacks occurring elsewhere as well. Without any comparison group, there is no indication of what was going on with heart attack rates elsewhere, so one has no idea whether the rates in England were unique and therefore can be attributed to the smoking ban as opposed to a general trend due to other factors.
In fact, with an effect as small as 2.4%, I find it striking that the paper would conclude that such a small effect is due to the smoking ban, rather than to even consider that the reported effect could be due to other changes, such as improved treatment for coronary artery disease and hypertension. While the paper acknowledges that one limitation is the absence of a control population, it completely dismisses this limitation without any reason or justification.
If you are going to acknowledge a limitation, you need to explain why that limitation does not threaten the validity of your study's conclusions. Every study has limitations, but the paper must demonstrate why those limitations do not negate the study's ability to draw its conclusion. In this case, the study points out the limitation but then glosses over it, providing the reader with not a single reason to overlook the limitation and accept the study conclusion. We are supposed to, in other words, take it on faith that the observed 2.4% decline in heart attacks observed in England was a unique phenomenon and that the rate of decline in heart attacks elsewhere did not increase at all during the same period.
I find that to be too big of a pill to swallow. A conclusion like this needs to be justified based on examining the data, not simply on speculation. Actually, in this case, the paper doesn't even speculate. It just ignores the whole problem.
The rest of the story is that once again, we have a study on smoking bans and heart attacks whose conclusion is inconsistent with the very data presented in the paper. For some reason, researchers working on this genre of study are so anxious to find an effect of smoking bans that they appear to have lost their ability to critically analyze actual data and make causal judgments with any kind of rigor.
By the way, the reasoning being used in this study is exactly the same as that used by the tobacco companies when they used to argue that smoking bans result in huge declines in restaurant sales. Anti-smoking advocates criticized those studies because they failed to include a control group. Now, the absence of a control group is somehow no longer a problem. When restaurant sales fell in association with a smoking ban, there were lots of alternative explanations that needed to be taken into account. Now, with heart attacks falling in association with a smoking ban, alternative explanations apparently no longer need to be considered.
Studies of the effects of interventions that do not include a comparison group of any kind generally represent an extremely weak study design. With an huge effect size, you can sometimes get away without a comparison group because it is implausible that an effect that large represents a common secular trend. However, in this case, the effect size is miniscule (2.4%) and there is simply no way one can credibly attribute that small an effect to the smoking ban without ruling out the possibility that it is, instead, simply a reflection of a decline in heart attacks that is occurring anyway due to secular trends.
Finally, readers should always be skeptical about conclusions that are not consistent with the actual data presented in a paper. When you have a graph which clearly shows no demonstrable effect of a smoking ban on heart attacks (a.k.a., a straight line), then be wary of a complex statistical analysis that shoots out a specific number. If you can't see the effect in the graph, then it is going to be difficult to argue that any number which comes out of a computer is more believable than your own eyes. Statistical analysis is important as an adjunct to visual inspection of data, especially to help confirm visual impressions, but it is not a substitute for it.
(Thanks to Christopher Snowdon for the tip. Read Christopher's excellent analysis of this article over at Velvet Glove Iron Fist).
No comments:
Post a Comment