Large Meta-Analysis Purports to Find Huge Effect of Smokefree Laws on Heart Attacks, But Instead Fails to Demonstrate an Effect

In the largest meta-analysis to date of studies examining the effect of smoking bans on heart attack admissions, a paper published this week in the journal Circulation concludes that smoking bans lead to an immediate 15% decline in heart attack admissions or deaths. The typical follow-up period in the reviewed studies was approximately one year. Thus, the paper is concluding that smoking bans produce a 15% decline in heart attacks in the first year following implementation of these bans. Furthermore, the paper argues that the observed decline is due to a reduction in secondhand smoke exposure. Most of the policies were bans on smoking in restaurants and/or bars.

The basic method of the study was as follows: "We included studies examining the association between smokefree laws and hospitalizations or deaths due to cardiovascular or respiratory disease with sufficient data to calculate the relative risk and confidence interval before and after... ."

The article concludes: "Consistent with 3 prior meta-analyses that concluded that smoke-free laws are associated with significant decreases in AMI and other cardiac hospital admissions, we found that comprehensive smoke-free laws (covering workplaces, restaurants, and bars) were associated with a
15% decrease in AMI hospitalizations."

The Rest of the Story

Unfortunately, this article doesn't do what it purports to do.

What the Study Does: The study examines changes in the rates of heart attack admissions or deaths from before to after the implementation of smoking bans in a large number of localities. It provides convincing data to conclude that in these localities, there was an overall 15% decline in heart attack admissions or deaths during the time period when smoking bans were implemented in these localities. There is no question that the implementation of smoking bans has been associated with a decline in heart attacks, and the best estimate for the magnitude of that decline is about 15%.

What the Study Doesn't Do: The study doesn't examine the observed changes in heart attacks in localities with smoking bans in light of secular changes in heart attack rates that were occurring during the same time period in localities that did not enact smoking bans. In other words, the meta-analysis did not include any comparison group. In fact, the study did not restrict inclusion to articles that used a comparison group. Generally, the estimates in the paper refer to the relative risk for heart attack admissions or deaths after the smoking ban compared to before the ban. All we can conclude from the study is that there was a decline in heart attacks during the study period. However, there is no way to discern whether the observed decline was attributable to the smoking ban. To do that, we would need to compare the rate of decline observed in these localities to the rate of decline in heart attacks during the same time period in localities without smoking bans.

This might not be a problem if heart attack rates had generally been stable during the study period. But due to medical interventions and pharmaceutical advancements, heart attack rates have generally been falling during the study period, even in the absence of smoking bans. To conclude that the observed changes were due to the smoking bans, one must compare the rates of decline in heart attacks in the localities with smoking bans to the rates of decline in localities without smoking bans. The meta-analysis fails to make this comparison. Therefore, while I believe that it demonstrates a clear reduction in heart attacks in localities with smoking bans, its conclusion that these reductions are attributable to the smoking bans is invalid.

We can, however, derive estimates of the overall decline in heart attack admissions in the United States during the approximate study period. The Health Care Utilization Project (HCUP) provides data on heart attack admissions from a large national inpatient sample. Based on these data, the national trend in heart attack admissions during the period 2002 to 2006 shows a decline of 17.2%. From 2003 to 2005 alone, the decline in heart attack admissions nationally was 15.7%.

So if the expected decline in heart attacks during the period 2003-2005 is 15.7% and in localities with smoking bans, the observed decline is 15%, how can one possibly conclude that the smoking bans led to a decline in heart attacks? My point here is simply to demonstrate that declines in heart attack admissions on the order of about 15% would not be unexpected in the U.S. in the absence of smoking bans, based on an examination of overall trends in the country occurring anyway.

One final note. As with the study I have been discussing over the past few days in which I revealed investigator bias, this study too shows evidence of the same type of bias. Specifically, the authors excluded from the analysis a study which found no effect of smoking bans on heart disease deaths in six states.

In that study, the authors examined age-adjusted rates of heart attack mortality during the 3 years before implementation of the smoking ban and during the first year after the smoking ban was implemented in the eight states that implemented smoking bans between 1995 and 2003. These trends were also compared with those in the 44 other states without smoking bans.

The results were that in four of the six states (California, Utah, Delaware, and South Dakota), the smoking bans were not associated with any significant short-term decline in heart attack mortality. In one of these states - South Dakota - there was an 8.9% increase in heart attack mortality during the first year of the smoking ban which was significantly different from the expected decline of 7.2%.

The meta-analysis excludes this study because it uses "nonstandard methodology." But the "nonstandard" methodology it uses is the fact that it: (1) examines heart attack death trends in an entire set of localities, rather than just in one city or state; and (2) it, unlike nearly all other studies, includes a large number of comparison localities so that the trends in places with smoking bans can be adequately compared to trends in places without bans.

Thus, the very study which uses the most appropriate methods for studying this issue is excluded.

And that makes sense, because in a study that includes the appropriate comparison groups, one is not going to find a dramatic decline in heart attacks.

In other words, when one actually compares the decline in heart attacks in the smoking ban studies with the declines occurring elsewhere, the "dramatic" declines no longer appear to be so "dramatic.":

If anything, this meta-analysis demonstrates that the observed short-term decline in heart attacks from smoking bans is not all that different from the magnitude of decline that is occurring anyway because of secular trends.

The rest of the story is that unfortunately, I believe we have resorted to shoddy science in order to try to produce evidence favorable to our cause (which I believe is a noble one: the protection of nonsmokers from the demonstrated hazards of secondhand smoke). This science, which has now abandoned the concept that a comparison group is necessary to draw valid conclusions, has deteriorated to a level that we previously would attack when we saw it being relied upon in tobacco industry studies that purported to show declines in sales associated with restaurant smoking bans.

The methodology that was once below us is now our mainstay. The only thing that has changed is that this methodology is now yielding favorable results, whereas when used by the tobacco industry to show the economic harms from smoking bans, it was yielding unfavorable results.