Pueblo Study Concludes that Smoking Ban Reduced Heart Attacks by 27%; Conclusion is Premature and Based on Shoddy Science

A study published online this week ahead of print in the journal Circulation concludes that the smoking ban in Pueblo, Colorado resulted in a 27% decline in hospital admissions for acute myocardial infarction (heart attacks).

The study ascertained the number of hospital admissions for acute MI in Pueblo for 18 months before and 18 months after the implementation of a smoke-free bar and restaurant ordinance. Poisson regression was used to analyze monthly counts of heart attack admissions and to compare the counts prior to the ordinance with those after the ordinance took effect. A neighboring county without a smoking ordinance (El Paso County) served as a comparison group.

The number of heart attack admissions in Pueblo declined by 27%, from 399 to 291, while there was no significant decline in heart attacks in El Paso County (from 984 to 955) during the same period.

The regression analyses indicated that there was a significant difference in monthly heart attack counts in Pueblo from before to after the smoking ban, and this difference remained significant after accounting for seasonal differences in the occurrence of heart attacks. The study concludes that: "A public ordinance reducing exposure to secondhand smoke was associated with a decrease in AMI [acute myocardial infarction] hospitalizations in Pueblo, Colorado, which supports previous data from a smaller study."

The Rest of the Story

While it would be nice to think that a smoking ban could produce an almost instantaneous dramatic effect on heart attacks as this article leads us to believe, this study unfortunately fails to provide adequate evidence to reach such a conclusion.

There are two major problems with the study which are so severe that in my view they invalidate the study conclusions and make the assertion that the smoking is responsible for the observed changes in heart attack rates premature.

First, the study is unable to control for the very real possibility that the observed changes in heart attacks simply reflect random variation in this phenomenon. If one looks at the year-to-year variation in heart attack admissions in a relatively small population like Pueblo, one will see that there is substantial variation, or instability in the data. There are relatively large increases and declines from year to year that occur simply by chance. Because this study only looked at heart attack admissions for a 1 1/2 year period before and after the smoking ban, it is unable to assess the level of random variation in the underlying data and therefore unable to determine whether the observed changes in heart attack rates are due to the smoking ban or simply due to random variation in the data that could have been observed in the absence of the smoking ban.

Essentially, this is a study with 2 data points: one before the ordinance (399) and one after the ordinance (291). Is the drop from 399 to 291 significantly more than one would observe from year to year (or 18-month period to 18-month period) due simply to random variation? It is impossible to know because the study does not go back or ahead any further than 18 months.

The second major problem is that because the study does not go back more than 18 months prior to the ordinance, it is also impossible to assess whether the observed decrease in heart attacks simply reflects secular changes that may have been occurring in heart attack admissions in Pueblo anyway, irregardless of the smoking ordinance. One cannot even evaluate this possibility because you cannot assess a trend when you only have one data point prior to the ordinance. The 18-month baseline period does not allow enough time to assess pre-existing secular trends, since these trends take place over a period of several years.

The combination of the study's failure to be able to assess whether the observed decline in heart attack admissions is attributable to random variation and whether the decline is attributable to secular changes in heart attacks in Pueblo render its conclusions invalid. A reasonable alternative explanation to the study conclusions exists: that the observed changes in heart attacks are due to a combination of random variation and to a secular trend of declining heart attack admissions.

The dangers of drawing the kind of conclusions being drawn from the Pueblo study with this kind of data can be demonstrated by examining trends in heart attack admissions for the state of Colorado as a whole (available from the HCUPnet database).

Based on changes observed from 1997 to 1998 for the entire state of Colorado, which has a substantially higher number of total heart attacks than Pueblo (about 17 times higher), one would have concluded that something happened in 1998 which resulted in a 16.4% increase in heart attack admissions.

My guess is that it was the Denver Broncos winning the Super Bowl in January 1998. That must have shocked so many long-time Broncos fans that it caused innumerable heart attacks.

But this is exactly the type of reasoning that is being used to support the Pueblo study conclusions.

In fact, for the entire state of Colorado, there was a 12% decline in heart attack admissions from 2002 to 2004, indicating that there was indeed a secular trend of declining heart attacks during the study period. Does the fact that heart attacks apparently did not decline in El Paso County indicate that the absence of a smoking ban in that county protected its residents from heart attacks?

Obviously not, but again, this is the type of reasoning that is being relied upon to draw the conclusions in the Pueblo study.

Another significant problem with the Pueblo study conclusion is that the research was not able to ascertain whether the heart attack patients were smokers or not. Thus, the paper provides no idea of the extent to which the observed decline in heart attacks was attributable to a decline among smokers or among nonsmokers. This makes it impossible to evaluate the plausibility of the study's conclusions, because the study is unable to determine whether a decline in smoking by active smokers precipitated by the smoking ban or a decline in secondhand smoke exposure is the mechanism by which the purported effect on heart attacks occurred.

Yet another problem is that the study failed to examine whether the ordinance actually resulted in substantial changes in secondhand smoke exposure and smoking behavior. How much did secondhand smoke exposure decrease? How many people experienced such a significant decline in their exposure? Did the smoking prevalence decrease, and by how much? Without answers to any of these questions, it is far too premature for the study to be drawing conclusions as sweeping as this one does.

In addition, and perhaps most importantly, the Pueblo study conclusions are implausible on their face. Even if the ordinance completely eliminated secondhand smoke exposure for everyone in Pueblo, one would not expect to see a 27% in heart attacks within 2 months. And even if the ordinance caused every smoker in Pueblo to quit smoking, one would only expect to see about a 27% long-term decline in heart attacks.

Finally, by chance alone, heart attack admissions would be expected to fall in some cities following smoking bans. Showing that heart attack admissions did fall in one particular city does not demonstrate that it had anything to do with the smoking ban. It could simply be a chance finding. To draw that conclusion, one would need evidence from a large number of different cities.

It's kind of like concluding that a smoking ban caused an increase in restaurant sales because in one particular establishment, the sales went up significantly after the smoking ban. Even assuming that the increase in sales was real, it does not indicate that the increase was attributable to the smoking ban. By chance alone, sales in some establishments are going to increase. To conclude that the smoking ban had an effect on sales, one would have to sample a decent number of these establishments.

Yet concluding that the smoking ban reduced heart attacks in Pueblo based on these study results is essentially the equivalent of concluding that a smoking ban resulted in increased restaurant sales because in one restaurant, sales went up.

In my view, this is an example of really shoddy science. Not the study itself. But its conclusions.

Equally shoddy is the propaganda based on the study that is already being spewed by anti-smoking groups. The Campaign for Tobacco-Free Kids, for example, stated in its press release about the new study that: "These studies add to the overwhelming evidence that secondhand smoke poses serious, even life-threatening risks to health... ."

However, this study says nothing about secondhand smoke. Even if one fully accepts the conclusion, it does not add any evidence about the harms of secondhand smoke. The observed effects could just as easily be due to reduced active smoking. In fact, that is the more likely explanation for any actual observed effects of a smoking ban on heart attacks.

I really fear that this shoddy science on the part of the anti-smoking movement is going to harm our credibility. Junk science is supposed to be the hallmark of the tobacco companies. Why are we now embracing it as our own?