New Study Concludes that State Smoking Bans Significantly Reduce Heart Attacks

A new study, published recently in the online journal Public Health, concludes that statewide smoking bans have led to a significant reduction in the prevalence of self-reported heart attacks.

See: Lippert WC, Gustat J. Clean Indoor Air Acts reduce the burden of adverse cardiovascular outcomes. Public Health 2012; 126:279-285.

In the study, the authors used state-specific data from the Behavioral Risk Factor Surveillance System (BRFSS) surveys to determine the prevalence of self-reported heart attacks one year prior to the implementation of a statewide smoking ban and in 2009, after the smoking ban had been in effect. There were 17 states included in the study. Each had enacted a statewide smoking ban in either 2006, 2007, or 2008. For a smoking ban enacted in 2006, data from the 2005 BRFSS would be compared to data from the 2009 BRFSS. For a smoking ban enacted in 2008, data from the 2007 BRFSS would be compared to data from the 2009 BRFSS.

In this way, the study was able to compare changes in the prevalence of self-reported heart attacks among adults from before to after the state smoking bans in these 17 states.

The results were reported as follows: "Ten of the 17 states/territories (58.8%) were found to have a significant decrease in the prevalence of CHD/angina (Arizona, District of Columbia, Hawaii, New Hampshire, New Jersey, New Mexico, Pennsylvania) or AMI (District of Columbia, Hawaii, Iowa, Minnesota, New Hampshire, New Jersey, Puerto Rico) between baseline and 2009. Two states/territories (11.8%) had a significant increase in the prevalence of CHD/angina (Colorado and Louisiana) between baseline and 2009. Four states (23.5%) had an increase in the prevalence of AMI (Colorado, Louisiana, Nevada, Pennsylvania) between baseline and 2009, but these increases were not significant."

Based on these results, the paper concludes: "The data suggest that CIAAs reduce the prevalence of current smokers and adverse cardiovascular outcomes between 1 and 4 years following implementation (average time between baseline and 2009 was 3.06 years). State/territory-wide reductions in the prevalence of CHD/angina or AMI were observed 1 year (Iowa and Pennsylvania), 2 years (Arizona, District of Columbia, Minnesota, New Hampshire, New Mexico, Puerto Rico) and 3 years (Hawaii, New Jersey, Ohio) after CIAA implementation. Overall, 10 of the 17 states/territories (58.8%) had a decrease in the prevalence of CHD/angina or AMI. ... In conclusion, state/territory-wide CIAAs appear to significantly reduce the prevalence of CHD/angina, AMI, and current and former smokers in the immediate period following CIAA implementation (1–4 years)."

The paper's abstract concludes: "State/territory-wide CIAAs are beneficial in reducing adverse cardiovascular health outcomes in the short term. The prevalence of AMI, CHD/angina, and former and current smokers decreased significantly following CIAA implementation."

The Rest of the Story

There are two major flaws of this study which render its conclusion invalid.

1. There is no control group.

First, there is no control group. The study simply compares changes in self-reported prevalence of heart attacks in states with smoking bans from approximately 2006 to 2009. The study finds that in some states, there was a significant decline during this three-year period. However, without knowing what happened in states without a smoking ban, it is impossible to attribute this change in heart attack prevalence to the smoking ban. One needs to know what was the change in heart attack prevalence from 2006 to 2009 in states that did not enact smoking bans.

The study does not report this information. However, from the Health Care Utilization Project (HCUP) data, we can obtain the changes in hospital discharges with a primary diagnosis of heart attack (i.e., incident heart attacks) in states without smoking bans between the years 2006 and 2009. Here are the data for all states without smoking bans in the HCUP database for which there are data for these years (the last column shows the percentage change from 2006 to 2009):

State	2006	2009
SC	9825	8890	-9.5
OK	8687	8030	-7.6
AR	7192	6837	-4.9
KY	12839	12444	-3.1
MO	15198	14310	-5.8
TN	17229	16493	-4.3
WV	6645	5891	-11.3
TOTAL	77615	72895	-6.1

From this table, one can see that in every state without a smoking ban for which HCUP data are available during the study period, there was a substantial decline in heart attacks, ranging from a decline of 3.1% in Kentucky to a decline of 11.3% in West Virginia. Overall, the decline in heart attacks in these 7 states without smoking bans was 6.1% from 2006 to 2009.

Therefore, how can this study conclude that the decline in self-reported heart attacks in the 17 smoking ban states from 2006 to 2009 was different than what would have been observed in the absence of these bans. Clearly, there is a secular trend of declining incident heart attacks in the United States that is independent of statewide smoking bans.

Given this baseline secular trend, the study cannot conclude that the observed declines in self-reported heart attacks observed in the 17 study states were attributable to the smoking bans in those states, as opposed to simply reflecting underlying secular trends, which are readily observable in states without such smoking bans.

2. The study conducts the wrong statistical analysis.

The study's conclusion that the smoking bans led to a significant reduction in heart attacks is based on the observation that in 10 of the 17 states, the prevalence of heart attacks declined. Of course, another way to look at this is to say that in 7 of the 17 states, the prevalence of heart attacks increased. The real question is this: if there were no true change in heart attacks, what percentage of the time would 10 out of 17 states show a decrease in heart attacks by chance alone?

Think of it this way. Suppose you flip a coin 17 times and come up with 10 heads. Can you conclude that this is not a fair coin, and that it must be weighted more heavily towards heads?

Well one can calculate the probability of obtaining 10 or more heads out of 17 coin tosses with a fair coin. Using the binomial distribution, one can determine that if one flips a fair coin 17 times, the chances of getting at least 10 heads is 31.5%.

Thus, by chance alone, if one were to examine changes in heart attack prevalence in 17 states, one would find that heart attacks decreased in 10 of those 17 states 31.5% of the time (if there were actually no true change in heart attacks). This is far beyond any reasonable level of statistical significance (which is usually set at about 5%).

Thus, the reasoning used by the study to conclude that there was a significant effect of the smoking bans on heart attacks is flawed. The truth is that the observed results (10 out of 17 states showing a decline in heart attacks) would occur by chance about 31% of the time, in the absence of any effect of smoking bans on heart attacks.

A more powerful statistical analysis in this situation would be to calculate the change in heart attack prevalence for each of the 17 states and then determine whether the average change in heart attacks across the states differs significantly from zero, based on standard errors that take into account the total number of observations (i.e., states).

If one conducts this analysis, one will find that the average change in heart attack prevalence among the 17 states is -0.34 percentage points. However, the standard deviation is 0.42 percentage points. Using these data, we can construct a 95% confidence interval for the change in heart attack prevalence, which is [-0.13 to -0.56]. Thus, it turns out that the change in heart attack prevalence is statistically significant.

From this study, then, one can conclude that there was a small, but significant decline in heart attacks in the 17 intervention states. However, in the absence of data from the control states, one cannot conclude that this small decline in heart attack prevalence was attributable to the smoking bans in these states.