Monday, December 01, 2008

Data Released by Scottish National Health Service Show that Conclusions of Earlier Study about Effect of Smoking Ban on Heart Attacks Were Wrong

Data from the Scottish National Health Service on hospital admissions for heart attacks and unstable angina (acute coronary syndrome) obtained and released by Christopher Snowdon over at Velvet Glove Iron Fist reveal that admissions for acute coronary syndrome rose sharply during the second year of the smoking ban, such that there was little overall change over the first two years during which the smoking ban has been in effect.

This demonstrates that the decline in acute coronary syndrome admissions observed during the first year after the smoking ban was not due to the ban, but was simply a reflection of underlying variation in the data -- it is apparent now that the decline was just a blip down and the admissions have now climbed back up to where they were prior to the smoking ban.

As Snowdon writes: "Much was made of an apparent reduction in the number of patients being diagnosed with the life-threatening heart condition after the smoking ban came into effect in 2006, including a study published in the New England Journal of Medicine which claimed that the ban had caused emergency ACS [acute coronary sydrome] admissions to fall by 17%."

"However, official statistics show that the decline in hospitals admissions for acute coronary syndrome has been greatly exaggerated. The real decline in the first year of the smoking ban was just 7.2% - not 17% - and the rate then rose by 7.8% in the second year, cancelling out the earlier drop."

"In the last 12 months before Scotland enacted its smoking ban (April 05 to March 06) there were 16,199 admissions for acute coronary syndrome*. In the second year of the smoking ban (April 07 to March 08) there were 16,212 admissions, slightly more than there had been before the legislation was enacted." ...

"Hospital admissions for acute coronary syndrome have been declining in Scotland for many years. The new data show that the well-publicised fall in admissions following Scotland's smoke-free legislation was in line with the existing downward trend and was significantly less steep than has previously been claimed."

"The belief that the number of cases of acute coronary syndrome fell by 17% after the smoking ban stems from a study of volunteers in a selection of Scottish hospitals between 2005 and 2007. The study - 'Smoke-free Legislation and Hospitalizations for Acute Coronary Syndrome' - has since been criticised for its limited scope and for excluding data from several key months before and after the ban."

"If the 2006-07 decline had really been the result of the smoke-free legislation, it would be expected for rates to remain low in subsequent years. The fact that Scottish hospitals have seen an unusually sharp rise - despite the smoking ban being rigorously enforced - suggests that whatever lay behind the 2006-07 dip, it was not the smoking ban."

The Whole Story

In September 2007, the University of Glasgow issued a press release announcing a new study which concluded that the smoking ban in Scotland had resulted in a 17% decline in heart attack admissions during the first year in which the smoking ban was in effect. The University of Glasgow is the home of the lead researcher on the study: Dr. Jill Pell. The study itself was not made available. However, its conclusions were blasted throughout the world via the media.

For example, in an uncritical recounting of the data and conclusions in the press release, Spiegel wrote as follows:

"it seems a miracle has happened: The number of heart attacks in Scotland has suddenly dropped by no less than 17 percent in a single year. What has happened? Have the Scots stopped eating red meat? Has the whole country started knocking back cholesterol medication? Are they all training for the marathon? No. The reason is much simpler: Scots are having fewer heart attacks because they are no longer inhaling other people's cigarette smoke when they sit in the pub, the train or the office."

According to the press release: "A University of Glasgow study has found a 17 per cent fall in admissions for heart attacks in the first year after the smoking ban came into force. The evaluation, led by Professor Jill Pell from the University’s BHF Cardiovascular Research Centre, found that after the legislation came into force there was a 17 per cent reduction in heart attack admissions to the hospitals. This compares with an annual reduction in Scottish admissions for heart attack of 3 per cent per year in the decade before the ban."

At the time, I criticized the study for two reasons. First, I argued that the release of the study via a press release represented a "science by press release" approach that was not appropriate, because study conclusions should not be disseminated through the media until after they have undergone peer review and the study has been published, or at least until researchers are willing to make the entire study and its methodology available so that it can be reviewed by others.

I also criticized the study conclusions because it was not clear that the reported decline in heart attacks was not simply random variation and/or an existing secular decline in heart attacks that was occurring anyway, even without the smoking ban.

I noted: "To see why a thorough review of the actual study is necessary, just look at the hospital discharge data for heart attacks in Scotland. We find evidence of a large single-year decline in heart attacks of about 10% which occurred in the absence of any smoking ban. This indicates that there can be a large year-to-year variation in heart attacks in Scotland and makes it especially difficult to determine whether a 17% annual decline in heart attacks is due in part to random variation, and if so, to what extent. From the information that was provided to us, it is impossible to conclude that the smoking ban was responsible for the observed decline in heart attack admissions.

It is also important to consider the fact that the more rapid decline in heart attack admissions in Scotland appears to have begun prior to the smoking ban. A drop in heart attack admissions of 17% (equal to what was observed after the smoking ban) actually occurred during the three-month period immediately prior to the smoking ban. It would be very questionable to attribute the observed heart attack decline to the smoking ban in the face of this evidence that heart attacks were declining substantially prior to the ban. It is entirely possible that some other factor is responsible for the observed decline."

Importantly, about two months after the initial press release, I discussed data, reported in an article by Michael Blastland published by the BBC News, that were released by the National Health Service's Informational Services Division. These data included all hospital admissions for heart attacks nationwide in Scotland, reported as annual standardized rates. The data showed a decline of about 8% in the year following the smoking ban, which is only slightly higher than the approximate 6% decline the year preceding the smoking ban. If one looked at the trend in heart attack rates from 1998 through 2007, one would find that based on the pre-existing trends in heart attack rates, there had been no apparent change in the rate of heart attacks following the smoking ban, which was implemented in March 2006.

In addition, these data confirmed an earlier analysis, in which I pointed out that a huge decline in heart attacks in Scotland from 1999 to 2000 occurred prior to the smoking ban and revealed the large degree of baseline variability in the data, meaning that only a junk science study would conclude that the observed decline in heart attacks during the 10 months following the smoking ban was attributable to the smoking ban. In fact, the data revealed that the largest drop in heart attack admission rates occurred not following the smoking ban, but from 1999 to 2000.

Despite the shoddy nature of the study's conclusions and the unsupporting data coming from a national, population-based database, the authors proceeded to publish their study in the New England Journal of Medicine in July, 2008. The study concluded that the smoking ban in Scotland resulted in a 17% decline in hospital admissions for acute coronary syndrome (including myocardial infarctions [heart attacks] and unstable angina) (see : Pell JP et al. Smoke-free legislation and hospitalizations for acute coronary syndrome. N Engl J Med 2008; 359:482-491).

The study compared the number of admissions for acute coronary syndrome in nine hospitals in Scotland (representing 63% of admissions for acute coronary syndrome in the country) during the 10-month period prior to the smoking ban and the corresponding 10-month period the following year. The number of admissions declined from 3235 to 2684, a drop of 17%.

This 17% drop was compared to the trend in overall hospital admissions in all of Scotland during the preceding 10 years. According to the study, "the trend during the 10 years before legislation was a 3% mean annual reduction, with a maximum reduction of 9% in 2000."

Because the observed 17% reduction in admissions for acute coronary syndrome was much higher than the annual reduction during the 10 previous years and exceeded the highest annual decline between any two years, the study concludes that the observed reduction is attributable to the smoking ban.

I immediately argued that: "The problem with this article is that its conclusion is based on a comparison of apples to oranges. In order to compare the change in heart attacks in Scotland from 2006-2007 to the trend in heart attacks during the preceding ten-year period, one needs to use the same data source to compare these trends. In this article, the researchers use one source of data to estimate the change in heart attacks from 2006-2007 (observed changes in admissions for nine hospitals representing a portion of the country) and a different source of data to estimate the trend in heart attacks from 1996-2006 (national data from the Scottish National Health Service)." I argued that this is comparing apples to oranges, and it is not valid science.

In September 2008, Christopher Snowdon over at Velvet Glove Iron Fist reported that the results and conclusions of the Pell et al. study were wrong. Snowdon's article was based on a complete data set of acute coronary syndrome admissions in all of Scotland for the entire period from 1999 through 2007. He included data for the full two years following the smoking ban and for at least seven years preceding the ban. Snowdon found that contrary to the results reported by Pell et al., acute coronary syndrome admissions fell by just 9.3% in the year following the ban (compared to the year preceding the ban). But more importantly, this decline was no greater than that observed several times during the baseline period. For example: "AMI [acute myocardial infarction] admissions fell by 10.2% in 1999-2000 and angina admissions fell by 10.5% in 2005-2006 (there were also falls of 11.6%, 11% and 12% in previous years)."

Moreover, when one examined the full data, one saw that while there was a decline in acute coronary syndrome admissions in Scotland following the smoking ban, this decline was not as steep as it had been prior to the smoking ban. In other words, the rate of decline in acute coronary syndrome admissions actually decreased after the smoking ban.

Now, the data obtained by Christopher Snowdon reveal that what happened in Scotland was the opposite of what the researchers reported. Rather than to create an acceleration in the existing rate of decline in acute coronary syndrome admissions, the implementation of the smoking ban was associated with a decrease in the rate of decline. In fact, in the two years that the smoking ban has been in effect, the trend of declining acute coronary syndrome admissions in Scotland has ground to a halt.

This does not, of course, mean that the smoking ban resulted in an increase in heart attacks. It simply demonstrates that the observed decline in acute coronary syndrome admissions was a statistical blip. Data tends to zig and zag just from normal random variation. If you catch the data when it is zagging, you might inadvertently conclude that heart attacks are dropping, unless you stick around long enough to see that the next year, the heart attack rate zigs.

The zigging and zagging of statistical data is normal. What science must do is to account for this zigging and zagging and refrain from drawing conclusions about changes in data until the underlying zigging and zagging has been accounted for. This is what the authors of the Scottish smoking ban study failed to do.

And now, the most complete data yet assembled show that the conclusion of their New England Journal of Medicine study was wrong: the smoking ban in Scotland did not cause a substantial, immediate drop in heart attacks or acute coronary syndrome.

The Rest of the Story

There are three important parts to the rest of the story.

First, this story demonstrates the dangers of science by press release. The conclusions from the Scotland smoking ban study have already been disseminated widely to the public and it is far too late to retract them. Even if the researchers were to issue a media advisory in which they noted that their conclusion has been disproved, it is unlikely that it would pick up enough media attention to undo the publicity created by the initial media coverage.

Second, this story demonstrates the significant bias inherent in research by the anti-smoking movement these days. This bias is so strong that researchers can apparently not wait until they have sufficient data to draw a reasonable conclusion. The desired conclusion has to be disseminated widely when it is still preliminary in order to create a media splash that cannot be reversed if the conclusions subsequently prove to be wrong.

Third, this story demonstrates that the anti-smoking movement does not really care about the truth: it cares mostly about producing favorable evidence to support its conclusions. When the original studies came out (both the unpublished study and the published article), anti-smoking groups issued press releases and did everything they could to generate media attention about the study conclusions. Now that these conclusions have proven to be wrong, anti-smoking groups are not publicizing the error of the study. They are not issuing press releases to report these new data and to update the public with the accurate conclusion of the study. Nor are they going to do so.

I will go through the usual motions of disseminating my commentary throughout the movement, to all the key groups which have issued what turns out to be false, preliminary conclusions. But these groups are not going to do anything to correct the damage that has been done through the dissemination of these inaccurate conclusions.

You see - it doesn't really matter anyway. This is the tobacco control movement and since it is all for a good cause, there really isn't damage done by disseminating false scientific conclusions. Thus, it doesn't need to be corrected. After all, this is no longer about scientific accuracy or scientific integrity. This isn't about the truth. It's about producing favorable evidence to support the policy agenda of the anti-smoking movement.

No comments: