More Science by Press Release on Smoking Bans and Heart Attacks; Mississippi Study Conclusions are Based on Shoddy Science with No Comparison Group

Failure to Analyze Pre-Existing Trends in Heart Attacks Negates Conclusions of Study

A press release issued yesterday by Mississippi State University claims that a new study has demonstrated that a local smoking ban in Starkville resulted in a 27% decline in heart attacks.

According to the press release: "A Mississippi State study released Monday [Nov. 9] shows a 27 percent decrease in heart attacks among Starkville residents since the city passed a smoking ban in 2006. Researchers associated with the university report also are recommending a statewide public ban on smoking. The study by Robert McMillen and Dr. Robert Collins shows fewer heart attacks being treated at the Oktibbeha County Hospital. It focused on Starkville residents in the three-year span after the ban became law, compared to three years prior."

The Starkville smoking ban went into effect in May 2006. The researchers found that the rate of heart attack admissions among Starkville residents at the Oktibbeha County Hospital decreased by 27% from the three -year period proceeding the ban (2003 through 2005) to the three-year period following the ban (2006 through 2008). They attribute this decline to the smoking ban.

The study contains no comparison group, nor was there any assessment of baseline trends in heart attacks prior to the ban to determine whether the observed decline was occurring anyway, even without the smoking ban. Nevertheless, the study boldly concludes that this research "clearly demonstrates" that the smoking ban was what caused the 27% decline in heart attacks in Starkville.

The study was apparently not peer reviewed nor published, nor does it appear to be available to the public for scrutiny. Nevertheless, the conclusions have been widely disseminated through the media (example 1; example 2; example 3; example 4). Headlines boasted that: "Study Links Decline in Heart Attacks to Smoking Bans."

A summary of the study appears on the internet, but the actual study itself does not (at least I could not find it). According to the study summary: "The scientists compared the standardized number of heart attacks for three years prior to the smoke-free law and the three years following the enactment of the smoke-free law, and found that the number of heart attacks decreased substantially. Although these results are preliminary and the study will not be completed until several other Mississippi communities are examined, this finding highlights the immediate impact on health that smoke-free laws in Mississippi can have."

One of the study authors is quoted in the Starkville Daily News as quantifying the precise effect of the smoking ban on reduced heart attacks: "We have saved $750,000." This calculation assumes that all of the 27% decline in heart attacks during the period 2006-2008 was attributable to the smoking ban.

The Rest of the Story

The problem with the study, and the reason why I call it shoddy science, is that it fails to investigate whether or not the observed decline in heart attacks in Starkville was attributable to the smoking ban, or whether the decline was part of an already existing pattern of secular decline in heart attacks occurring throughout Mississippi that was happening anyway, and would have continued even in the absence of the smoking ban.

Just because you observe a decline in a particular phenomenon from before to after a given year does not mean that something which happened during that year is responsible for that decline. For example, there was an 11.3% decline in divorces in Mississippi from 2003 to 2008. Can I conclude that it was the Rebels upset victory over the Florida Gators in 2003 that led to this substantial decline in divorces?

Of course not. What is first necessary is to examine whether the observed decline differs from what was happening anyway. In other words, you have to go back in time and determine the baseline or pre-existing trends in the number of divorces. In Mississippi, from 2000 to 2003, the number of divorces declined by 6.0%. If you plot the number of divorces out over a long period of time, say from 2000 through 2008, you'll see that there has been a relatively steady decline over the entire study period, and there is no reason to believe that any particular event in 2003 had any effect on this trend.

Now, back to the study. The study compared the change in heart attacks in Starkville from 2003-2005 to 2006-2008 and found a 27% decline. What we need to know is: what was the decline in heart attacks during the previous period? In other words, what was the decline in heart attacks in Starkville from 2000-2002 to 2003-2005? Without that information, we cannot possibly determine whether the 27% decline was due to the smoking ban, or whether it was merely a reflection of a long-term, secular decline in heart attacks that was occurring anyway.

Importantly, the study cannot answer this question, at least with the data it currently analyzes. However, we can investigate this question by examining data on heart attack deaths for the state of Mississippi, which are available from the state Department of Health. Heart attack deaths are likely correlated highly with the number of heart attack admissions. So looking at the pre-existing trend in heart attack deaths in Mississippi gives us some indication of what would likely have been observed in Starkville in the absence of the smoking ban.

As it turns out, from 2000-2002 to 2003-2005, there was a 19.3% decline in heart attack deaths in Mississippi. And from the period 2000-2002 to 2006-2008, there was a 29.2% decline in heart attack deaths in Mississippi.

From 2003 to 2008, heart attack deaths in Mississippi fell by 19.1%. From 2002 to 2008, they fell by 29.4%. And from 2000 to 2008, they fell by more than one-third: by 33.9%.

This demonstrates that heart attacks were declining substantially in the state of Mississippi anyway, even in the absence of a smoking ban. It suggests that if Starkville had not enacted a smoking ban, the study still would have found a very large decline in heart attacks from 2003-2005 to 2006-2008.

Just looking at the change in heart attack deaths in the state from 2005 to 2006 (the first year of the Starkville smoking ban) alone, there was a large 8.7% decline. And for the entire period 2000-2006, there was a 29.7% decline. Thus, a decline in heart attacks in Starkville on the order of about 27% appears to be exactly what one would have expected in the absence of a smoking ban.

Given these facts, I do not understand how the study can conclude that the smoking ban in Starkville resulted in a 27% decline in heart attacks. I won't quibble and argue that the smoking ban had zero impact, but it certainly appears that a decline on the order of about 27% would have occurred anyway. I simply don't see how the study can attribute all 27% of the observed decline to the smoking ban.

The decline in heart attack deaths in Mississippi during the study period is a reflection of broader underlying trends in cardiovascular disease in the state. For example, cerebrovascular disease deaths in Mississippi declined by 21.3% from 2000 to 2008. From 2000-2002 to 2006-2008, there was a 20% reduction in cerebrovascular disease deaths in the state.

It's interesting that the study summary acknowledges that the study conclusions are preliminary and that the study will not be complete until other communities are examined (which is a good idea, because it will show that heart attacks have been declining throughout the state during the study period), but that the investigators nevertheless have no problem disseminating their unequivocal conclusions throughout the country through the media.

This is once again an example of science by press release. Although I have demonstrated that the study conclusion is flawed because at least a large part of the observed decline in heart attacks was attributable to the pre-existing secular trend of declining heart attacks in Mississippi, it is too late. The conclusions have already been disseminated, and I don't believe that these media outlets are now going to publish a retraction or a new article that says: "Remember that article last week about how the smoking ban in Starkville reduced heart attacks by 27% and saved $750,000. Well, never mind."