New Study Purports to Estimate Efficacy of Electronic Cigarettes By Studying Quit Rates for Select Group of Smokers who Failed to Quit Using E-Cigarettes

Imagine that you are a tobacco company and you are concerned about the potential threat posed by electronic cigarettes. You want to deflate the enthusiasm about these products by producing a study which shows that quit rates among smokers who try to quit using electronic cigarettes are very low and are in fact lower than quit rates for smokers who use other cessation methods.

Clearly, the proper way to conduct such a study would be to take a sample of smokers who made quit attempts using electronic cigarettes and compare their success with a sample of smokers who made quit attempts using other methods.

However, conducting such a study presents a risk: you might find that smokers who try to quit using e-cigarettes actually do better than those who try to quit using traditional methods, which you know have dismal success rates.

What, then, is a sure way of finding low quit rates that you can then report in a paper?

A great way to "manufacture" such a result would be to selectively choose a sample of smokers who tried to quit using electronic cigarettes and failed, and then to wait until they call a quitline and then see what proportion are able to quit at that point. Even though these smokers may not be actually trying to quit using electronic cigarettes when they call the quitline, you would count them as electronic cigarette users and attribute their cessation rates to the electronic cigarette group. You could then compare this group with smokers who did not fail a quit attempt using electronic cigarettes.

Or course, if a tobacco company conducted such a study in this way, it would be considered an example of scientific fraud. This is not objective science.

The Rest of the Story

Yesterday, a new study appeared online ahead of print in the journal Nicotine & Tobacco Research. To my shock, the study did exactly what the hypothetical tobacco industry study above did!

The study was purported designed to examine the efficacy of electronic cigarettes in smoking cessation. However, instead of estimating cessation rates among a cohort of smokers who made quit attempts using these products, the study analyzed cessation rates of a large number of smokers who had previously tried to quit using e-cigarettes but failed, and then called a quitline because they had failed and wanted to try again.

Then, they compared the quit rate among these smokers to that among smokers without such a history of a failed quit attempt using electronic cigarettes.

In other words, this study did not estimate quit rates among smokers trying to quit using e-cigarettes. Instead, it estimated quit rates among many smokers who were not using e-cigarettes in their quit attempt at all!

Here is how the study describes its methods and findings:

"This study ... describes differences among state quitline callers who used e-cigarettes for 1 month or more, used e-cigarettes for less than 1 month, or never tried e-cigarettes. ... Both e-cigarette user groups were significantly less likely to be tobacco abstinent at the 7-month survey compared with participants who had never tried e-cigarettes (30-day point prevalence quit rates: 21.7% and 16.6% vs. 31.3%, p < .001)."

The authors conclude that: "This study indicates that e-cigarette users were less likely than those who had never used to have quit tobacco at the time of the survey...".

But the truth is that many of the electronic cigarette users in the study did not use electronic cigarettes in their quit attempts!

According to data provided in the paper, a full 28% of the sample of electronic cigarettes did not use these products in their quit attempts.

It should be clear to readers that this study was poorly designed to investigate the efficacy of electronic cigarettes. The study systematically sampled a group of quitline callers who were unsuccessful using electronic cigarettes. These people tried and failed using electronic cigarettes. How do we know they failed? Because they wouldn’t have had to call the quitline if they weren’t still smoking. This is clearly a harder core group of smokers and it is no surprise that their cessation rates were lower after 6-months than the comparison group. The study tells us nothing about the effectiveness of electronic cigarettes, other than that they do not work for everyone. This is exactly the kind of biased research, designed specifically NOT to find an effect of electronic cigarettes, that is characterizing the anti-smoking movement today, guided by an ideology that is apparently opposed to anything that merely looks like smoking, no matter how much safer it may be. Apparently, the researchers had already reached a pre-determined conclusion and the study was apparently designed to find no effect. Any researcher sincerely interested in testing the efficacy of electronic cigarettes would not test the research question in this way.

That the organization which conducted this study had already drawn a pre-determined conclusion about electronic cigarettes is evidenced by its rant against these products, published back in September of last year, in which it warned that electronic cigarettes might kill "millions" of smokers. 

At that time, a director of the organization wrote: "E-cigs continue to gain market share with customers relying on assurances by the vendors that the product is safe. Those of us who work to help smokers quit remain skeptical and unconvinced. We remember all too well the light and ultra-light promotions by the tobacco industry back in the 70’s and 80’s. Like these, e-cigs are also being promoted as an alternative to quitting smoking. Let’s hope they don’t kill millions of Americans the way light and ultra-light cigarettes did 30 years ago."

If the tobacco industry conducted precisely this same study in order to conclude that electronic cigarettes are ineffective as a smoking cessation tool, we would call it scientific fraud. What should we call the same study, but conducted by a company that makes its money by convincing the public that traditional methods for smoking cessation should remain the mainstay of treatment?