On Monday, Cancer Research UK issued a press release in which it boasts about the results of a new study which purportedly shows that the national smoking ban in England caused 400,000 people to quit smoking, which will result in 40,000 fewer deaths over the next 10 years.
According to the press release: "Smokefree law in England has helped more smokers to quit than ever before and will help prevent an estimated 40,000 deaths over the next 10 years - according to new research being presented in Birmingham tomorrow (Tuesday). The Smoking Toolkit Study - funded by Cancer Research UK, McNeil, Pfizer and GlaxoSmithKline and presented at the UK National Smoking Cessation Conference - interviewed more than 32,000 people in England over the nine months before and nine months after last year's smokefree law took effect on July 1. The decline in smoking prevalence for the nine months pre-July was 1.6 per cent compared to an impressive 5.5 per cent in the nine months post July. Based on the findings researchers estimate that at least 400,000 people quit smoking as a result of the ban."
These results and conclusions were widely disseminated by the media (article 1; article 2).
The Rest of the Story
The rest of the story is that although Cancer Research UK released its conclusions from the study, it refused to release the actual study itself, including details about the study methodology and results. Thus, there is no possibility for anyone (including myself) to review the methodology and determine whether the conclusions are valid or not.
It is problematic that the study authors will not release the full results because without the full methods and results, it is not possible for others in the field to adequately review the work and assess its validity. I personally feel that researchers should not publicize study findings prior to publication unless they are willing to make the full findings and methods available. Releasing results via press release to the media and the public should not be done until publication, or if it needs to be done before publication, then it should only be done with concomitant release of the entire study.
I have a lot of interesting data about changes in smoking prevalence and quitting myself, but I am not going to release them publicly until after they are published. The problem is that if I release these data without the full study, then no one has an opportunity to review the study. It is very possible that upon peer review, my conclusions might be found to be invalid, or at very least, there might be some changes I need to make in my methods to correct the analysis (this is quite common). But here's the rub: if my results and conclusions have already been disseminated in newspapers throughout the country, it's too late to correct them when the corrected - or valid - results are actually published.
To me, the story here is not so much about quitting rates in England, which we have no ability to evaluate, but instead is about the appropriateness of what I like to call "science by press release," by which I mean trying to communicate important scientific findings to the public via the media, but without making the full methods/results available for scrutiny and review. The problem with this approach is that should the results subsequently be found to be invalid upon peer review, it is too late: the information has already been disseminated.
Interestingly, I have observed this problem most readily with studies of the effects of smoking bans (particularly studies on the changes in heart attacks as a result of these bans). Interested readers may want to see some of the other posts I have written about the hasty conclusions about the effect of smoking bans on heart attacks, as well as the general approach of "science by press release." (post 1; post 2; post 3; post 4; post 5)