One of the widest scientific misconceptions about evaluating the effectiveness of electronic cigarettes for smoking cessation - and one promulgated by anti-vaping groups and health agencies like the CDC and FDA - is that anecdotal evidence is insufficient to demonstrate that electronic cigarettes can be effective in helping smokers to quit smoking. In fact, this is a widespread fallacy. The truth is that the abundant anecdotal evidence of e-cigarettes helping many smokers quit is actually sufficient evidence to conclude that e-cigarettes are helping many smokers quit.
How can this be the case? Haven't we all been taught that anecdotal evidence is not sufficient? Aren't more rigorous research designs necessary to draw a conclusion that e-cigarettes can help some smokers quit? Since anecdotal evidence that a drug helps improve a medical condition among some patients cannot be used to conclude that the drug is an effective treatment, how can anecdotal evidence that many smokers have quit using e-cigarettes be used to conclude that e-cigarettes are effective for smoking cessation for many smokers?
The Rest of the Story
To understand this, we need to consider exactly why it is that anecdotal evidence is not sufficient to conclude that a drug is effective. Suppose a person with high blood pressure takes a medication for a week and her diastolic blood pressure is lower by 5 after one week (say it drops from 130/95 to 130/90). There are basically three possibilities that could explain the drop in blood pressure:
1. It would have dropped anyway, for reasons not related to her taking the drug.
2. The drop was caused by a placebo effect. It would have dropped had she taken a pill that just had sugar in it and not the medication.
3. The drop was caused by the medication.
So to conclude that the explanation for the drop in blood pressure is #3 above, we need to eliminate #1 and #2 as alternative explanations.
How do we eliminate #1? The way we eliminate #1 is to employ a control group: that is, a group of people with high blood pressure who are followed for one week without taking the medication to see whether their blood pressure drops anyway.
How do we eliminate #2? The way we eliminate #2 is to give a control group a placebo instead of the actual medication. If this group also experiences a decline in blood pressure, then it may be a placebo effect rather than a true drug effect.
In practice, conducting a placebo-controlled trial will address both #1 and #2 in one study because if either explanation #1 or #2 is true, then the blood pressure will drop in the control group and there will be no significant difference in blood pressure at follow-up between the treatment and the control group.
Now let's consider a behavioral outcome instead of a clinical (disease) outcome. Suppose a smoker uses a nicotine patch and at six month follow-up has stopped smoking. There are again three possible explanations:
1. The smoker would have quit anyway because by virtue of taking the nicotine patch, it is clear that he was motivated to quit and trying to quit. So it is quite plausible that the smoker would have quit even without the nicotine patch. The act of trying the patch indicated a desire to quit and a certain level of motivation to do so, above and beyond the motivation of other smokers.
2. The nicotine patch had a placebo effect. Simply by putting a patch on, it helped the person to quit due to placebo effect, not due to the actual nicotine.
3. The nicotine patch was effective in helping the smoker to quit.
To eliminate possible explanations 1 and 2, we again conduct a placebo-controlled trial, comparing the quit rate among a treatment group with the quit rate among a control group that receives a placebo patch. If either explanation 1 or 2 is correct, then there will be no difference in the cessation rate between the treatment and control groups.
Now, let's apply to same reasoning to a smoker who decides to try electronic cigarettes and is able to quit smoking. There are again three possible explanations:
1. The person may have quit smoking anyway, even had she not tried e-cigarettes. The act of trying e-cigarettes for smoking cessation could indicate a heightened level of motivation to quit smoking.
But there's a major problem with this explanation. It's not consistent with the available scientific data. The available data demonstrate that as a group, smokers who try electronic cigarettes are less motivated to quit and have much less confidence in their ability to quit. In fact, the very reason that smokers turn to e-cigarettes in the first place is that they have been unable to quit using other methods and have no self-efficacy regarding their ability to quit smoking otherwise. The data also show that smokers who try e-cigarettes tend to have higher levels of nicotine dependence and are thus much less likely to quit. In other words, the evidence supports the contention that the alternative to not trying e-cigarettes for most vapers is not quitting, but continuing to smoke. This alternative explanation therefore does not appear to hold water in most cases.
2. The person may have quit smoking due to a placebo effect. The act of using the e-cigarette (which simulates smoking) may be the reason for the successful cessation.
But there's a major problem with this explanation as well. The placebo effect is precisely the "mechanism of action" of vaping. It is a huge part of the reason why vaping could potentially be effective for cessation. The whole point of vaping products is to substitute for smoking by simulating the smoking experience. So in arguing that the observed association between vaping and smoking cessation is a placebo effect, one is actually arguing that the smoking cessation was a consequence of the e-cigarette use, since it works through a placebo effect. Therefore, this alternative explanation does not refute the third explanation: that the smoking cessation was attributable to the use of the e-cigarette.
The rest of the story is that the abundant anecdotal evidence of smokers quitting successfully using e-cigarettes is strong evidence that e-cigarettes are - for these smokers - effective in helping them quit. The FDA's view on this issue is misguided because it doesn't understand the basic scientific reasoning behind this relationship. The agency is used to evaluating drug studies, which must include placebo-controlled trials rather than rely on anecdotal evidence.
Please note: I am not arguing that clinical trials of smoking cessation with e-cigarettes compared to other approaches are not needed. I've made it very clear that conducting such trials should be a research priority. Neither am I arguing that anecdotal evidence that e-cigarettes can help smokers quit provides any estimate of the magnitude of the effect: we cannot, from the anecdotal evidence, determine what proportion of smokers who attempt to quit using e-cigarettes will succeed.
However, the argument that we do not have evidence to conclude that e-cigarettes can help smokers to quit is fallacious. The abundant anecdotal evidence available provides substantial evidence that e-cigarettes can and do help many smokers to quit.