Tuesday, September 15, 2009

Another Day, Another Smoking Ban/Heart Attack Study Disseminated by the Tactic of Science by Press Release; Study Appears to Be Invalid on Its Face

Not a day went by after reporting the use of science by press release to disseminate results of a study of the effect of England's smoking ban on heart attacks before the same tactic of science by press release was used to disseminate the preliminary findings of another smoking ban/heart attack study.

Heartwire reported yesterday the results of a new study in Iceland, presented at the 2009 meeting of the European Society of Cardiology, which reportedly showed that the occurrence of acute coronary syndrome (heart attacks and unstable angina) declined in Iceland during the first five months following implementation of the country's ban on smoking in restaurants and bars, which went into effect in June 2007. According to the article, the incidence of acute coronary syndrome declined by 21% among men, but did not decline among women. The study authors conclude that the effect was due to the smoking ban, and that this finding is consistent with the other literature on this topic.

Because this is science by press release, the full study is not available, so there is limited opportunity to scrutinize the methods, results, and conclusions to assess the validity of the study. Neverthless, the findings will be widely disseminated by the media, spreading the conclusion that the smoking ban in Iceland resulted in an immediate 21% decline in the incidence of acute coronary syndrome.

The Rest of the Story

I find it very curious that the study abstract does not mention the date (month and year) in which the smoking ban was implemented in Iceland. Given that this study examined only five months prior to and five months after the smoking ban (which itself is quite odd - why not wait until you have a full year's data), it seems quite important for the abstract to reveal the specific months that were included in the study.

Why is this important? For a very simple reason: there is a well-recognized seasonal variation in acute coronary syndrome, with peak incidence during the winter months, high incidence during the spring and the lowest incidence during the summer. This is recognized in the literature as a universal phenomenon (see Cheng TO. Seasonal variation in acute myocardial infarction. International Journal of Cardiology 2009; 135:277-279).

Cheng explains the reasons for this observation: "Cold temperature in the winter can cause increased cardiac workload, higher coronary and vascular resistance, higher blood pressure and higher fibrinogen levels, all of which are conducive to acute myocardial infarction. That it is the colder environmental temperature in the winter in most parts of the world rather than the winter season per se that is the principal reason for increased hospital admissions for acute myocardial infarction is evidenced by the observation that the peak month of acute myocardial infarction admitted to hospitals in Melbourne, Australia was July, which is the coldest month of the year."

Given this seasonal variation in heart attacks, I was surprised to find out that the smoking ban in Iceland went into effect on June 1, 2007. This means that the pre-ban period included the months of January, February, March, April, and May. And the post-ban period included the months of June, July, August, September, and October.

The average temperatures (degrees Centigrade) during the pre-ban months in Reykjavik are: 1.9, 2.8, 3.2, 5.7, and 9.4. The average temperatures during the post-ban months are: 11.7, 13.3, 13.0, 10.1, and 6.8.

Thus, based on seasonal variation alone, one would expect to observe a much lower incidence of acute coronary syndrome in Iceland during the period of June through October compared with the period of January through May. One would expect acute coronary syndrome incidence to peak in the winter (the pre-ban period), remain high in the spring (also pre-ban period), and to wane in the summer (the post-ban period).

Thus, the study methodology is basically a set-up to detect a reduction in the incidence of acute coronary syndrome. The fact that the abstract reports finding an overall 20% reduction is not at all surprising given the months during which these data were collected. One would expect a 20% reduction based on the seasonal variation alone.

In fact, one of the most comprehensive studies of seasonal variation in acute myocardial infarctions found that there is a 40% reduction in the incidence of heart attacks in the summer compared to the winter and spring (see: Rumana et al. Seasonal pattern of incidence and case fatality of acute myocardial infarction in a Japanese population [from the Takashima AMI Registry, 1988 to 2003]. American Journal of Cardiology 2008; 102: 1307-1311).

Therefore, the study results, as taken from the information provided in the study abstract, do not support a conclusion that the smoking ban had any effect on reducing acute coronary syndrome incidence in Iceland.

By the way, this is precisely why peer review is essential and why science by press release is so problematic. The apparent failure to consider seasonal variation in heart attack incidence is likely something that would be picked up in a vigorous peer review. It would become apparent that the study results are invalid. But now, it is too late. Even if this is picked up in peer review, it is almost moot because the results and conclusion have already been widely disseminated.

Are the study authors going to pull an Emily Litella and put out a subsequent press release that says: "Never mind." I doubt it. And even if they did, it's too late anyway because the conclusion has already been so widely disseminated.

I should also point out that even ignoring the seasonality issue, the abstract itself does not report a significant effect of the smoking ban on heart attacks. In the overall study population, the level of significance associated with the observed decline in heart attacks was 0.08. In other words, it did not meet the 0.05 level of significance and the study cannot conclude that there was a significant effect of the smoking ban.

The study authors do two things to try to create the appearance of a significant effect.

First, they stratify the results by gender (there is no a priori reason to do this, since the study hypothesis was not that the smoking ban would only affect heart attacks among men). While they report a significant effect for men, there is no effect for women. Such a finding argues strongly against the hypothesis that the smoking ban was the cause for the decline in acute coronary syndrome, since if the effect were due to secondhand smoke exposure reduction, one would expect to observe the effect among both men and women.

Second, rather than honestly reporting the results of the study - that they found no significant effect of the smoking ban on heart attacks overall, they twist the words that are used to describe this finding, and instead, they state that: "In the total population a trend was seen towards a 20% reduction in ACS (p=0.08)."

No trend was seen toward a 20% reduction. What the study found was that there was no significant reduction in acute coronary syndrome. By twisting the way they report this finding, the authors appear to be trying to make it sound like there was a significant effect when there was not one. This type of language is not appropriate, in my opinion.

When you read a study that reports a result "trended toward significance," what the authors are really saying is: "We did not actually find a significant effect, but we wanted to, and we are afraid to actually state they we didn't find a significant effect, so instead we're going to hide our failure to find significance by saying that the results 'trended' toward significance.

Today, I don't what is worse: the fact that the results of the study were disseminated by press release rather than by peer review and publication, the fact that the study apparently failed to account for seasonal variation in the incidence of acute coronary syndrome, or the fact that the paper actually found no significant effect of the smoking ban on acute coronary syndrome but tried to disguise that finding - which, after all, is the critical finding of the entire study.

No comments: