The study methods are described as follows: "A total of 7436 adult smokers (18+ years) selected via random digit dialing and interviewed as part of the International Tobacco Control Four Country Survey (ITC-4) between 2002 and 2009. Primary analyses utilized the subset of respondents who participated in 2006 or later (n = 2550)." The main outcome of interest was successful quitting at either 1 month or 6 months from the quit attempt.
The chief results of the paper were reported as follows: "Among participants who recalled making a quit attempt within 1 month of interview, those who reported using varenicline, bupropion or nicotine patch were more likely to maintain 6-month continuous abstinence from smoking compared to those who attempted to quit without medication [adjusted odds ratio (OR) 5.84, 95% confidence interval (CI) (2.12–16.12), 3.94 (0.87–17.80), 4.09 (1.72–9.74), respectively]; there were no clear effects for oral NRT use."
The study concludes as follows: "Consistent with evidence from randomized controlled trials, smokers in the United Kingdom, Canada, Australia and the United States are more likely to succeed in quit attempts if they use varenicline, bupropion or nicotine patch. Previous population studies that failed to find an effect failed to control adequately for important sources of bias."
The Rest of the Story
This is a complicated study and there are many aspects to the rest of the story, so let me take them one at a time. The study procedures are extremely complex because of its retrospective design, so I will do my best to explain these important flaws.
1. Many Medication Users Were Thrown Out of the Study
The most important aspect of the study for readers to understand is that its methods were biased so as to find a higher effectiveness of smoking cessation drugs. How was this achieved?
It was achieved by throwing out from the study a significant proportion of medication users, but not using any similar procedure to exclude non-medication users, all of whom were included in the study.
Any time that you treat the intervention group different than the control group, you introduce a bias into the study and in this case, that bias works towards finding medication use to be more effective.
Specifically, here is what happened in the study: Rather than simply including all medication users (that is, reporting the quit rates for all smokers who used medication in their most recent quit attempt), the study excluded certain smokers who used medication based on their response to a question about their reasons for using smoking cessation medication.
The study explains this exclusion as follows: "During each survey wave, respondents were asked to recall their use of medications since the last survey, and those who reported using medications were asked a series of questions regarding the medications indicated, including: ‘What was the main reason you used [the medication]?’. Only those who reported using medication in an attempt to stop smoking completely were considered to be medication users for the purpose of these analyses."
In other words, the study did not define medication users as being "medication users." Instead, it defined medication users as subjects who, after having tried medication to try to quit smoking and potentially failed, retrospectively report their use of medication was to intended to help them quit smoking.
Remember that the only indicated purpose for the use of smoking cessation drugs is to quit smoking. These drugs are not approved for any other purpose. So why exclude anyone who reported using smoking cessation medications? And if you are going to exclude smoking cessation medication users based on their intended purposes (reported after the fact), then why not also exclude unassisted quitters based on their reported intentions, also after the fact?
The way the human mind works, people are likely to alter their perceptions of why they used a particular approach after they observe the result of that approach. For example, suppose I am a two pack-per day smoker and I try to quit using NRT. I fail. Instead, I am able to cut down a little. Then, a few months later, I am asked to report the reason I used the NRT. To save face, I might (subconsciously) decide that the real purpose of my using the NRT was to cut down on the amount I smoke, rather than to quit completely.
Importantly, the same phenomenon might occur with a non-medication user. A person might initially decide that they are going to try to quit (without medication). But they might find it too difficult. So instead, they decide to just cut down. When asked a few months later why they decided to make a "quit" attempt, they might respond that they were actually trying to cut down rather than to try to stop smoking completely.
The effect of this procedure would be to systematically exclude smokers who have used smoking cessation medication unsuccessfully. In other words, this procedure would artificially bolster the observed smoking cessation rate for medication users.
To make matters worse, the question about the purpose of using smoking cessation drugs referred to any use of these drugs in the past year, not to the most recent quit attempt. Therefore, someone might have used NRT to try to cut down about six months ago and then used NRT to try to quit more recently. But they might still answer the question about the use of NRT as indicating that their intent was not necessarily to quit completely.
The bottom line is that smoking cessation medications are smoking cessation medications. Any study which excludes users of smoking cessation medications from the analysis, especially retrospectively, is going to bias the results towards finding an artificially high cessation rate for medication users.
2. Cold Turkey Quitters May Have Been Included in the Medication Group
While the study describes itself as a "prospective cohort" study, it really is a "retrospective cohort" study because the assignment of exposure status is being made after the fact.
What might be the effect of this misclassification of exposure? If it were true that cold turkey quitting is more effective than NRT use, then this misclassification would mask that effect. It would result in successful quit attempts going down as being attributed to medication when they are really due to an unaided quit attempt.
In my view, the inability of the study to determine definitively the method used for the most recent quit attempt makes it suspect. Much more useful would be true cohort studies in which the classification of exposure is meaningful. An example of such an approach is the study reported by the UK National Health Service. In that study, callers were assigned to receive or not receive medication and then followed up to observe their cessation rates. The results, as I reported last week, showed no significant advantage to the use of smoking cessation drugs.
3. The Study Results Depend on the Exclusion of Anyone Other than Those Making Quit Attempts in the Past Month or Two
The study finds that the effectiveness of medication greatly increases among subjects who report having made their most recent quit attempt in the past month or two. When all quit attempts are considered, there is no significant effect of nicotine gum, the nicotine patch, or buproprion.
The study justifies the exclusion of the bulk of its data on the premise that unsuccessful cold turkey quitters are more likely than unsuccessful medication quitters to forget that they have attempted to quit smoking. On its face, this seems implausible. Would not a person making a cold turkey attempt remember that they made such an attempt? More importantly, the data purported to demonstrate this effect fail to do so.
The premise that failed cold turkey quit attempts are not remembered as easily as failed medication quit attempts is based on the finding that failed quitters who used medication are more likely to report a greater time since their quit attempt than failed quitters who did not use medication. The assumption is that the medication users made additional attempts to quit without medication more recently but forgot about those quit attempts. However, the paper provides no evidence to back up this assertion. It only demonstrates that those failed quitters who used medication reported more remote failed quit attempts than failed quitters who did not use medication.
There is a very reasonable alternative explanation for this finding, which is that when medication users fail in their quit attempts, they are very discouraged from making additional quit attempts. In contrast, cold turkey quitters do not face the same discouragement because they can always rationalize that they can do things differently. But if medication fails, it may be more likely to lead to a sense of hopelessness, which could well deter quit attempts for quite some time.
4. Non-Cold Turkey Quitters Were Included in the Unassisted Cessation Group
The real question of interest is not simply whether medication-assisted quitting is more effective than unassisted quitting, but whether medication-assisted quitting is more effecting than unassisted cold turkey quitting. I am not aware of anyone advocating for unassisted quitting using a gradual reduction approach. The question of interest is how an abrupt attempt at smoking cessation that is unassisted compares with medication-assisted quit attempts.
Unfortunately, this study does not report the effectiveness rates for cold turkey, unassisted quitting. Instead, it lumps together all unassisted attempts to quit, even those in which the smoker cut down gradually, an approach that has been shown to be ineffective and which no one is advocating. By including these attempts and reporting them in the unassisted quitting figures, the study again biases the results towards finding a greater effectiveness of medication-assisted quitting.
The comparison which nobody seems to want to make is smoking cessation drugs versus cold turkey quitting without drugs. But that is the relevant research question.
Importantly, a large proportion of the "unaided" quitters in the ITC Four-Country Survey used a gradual reduction, rather than a cold turkey approach. In fact, approximately one-third of smokers used a gradual reduction approach. Therefore, the inclusion of these subjects creates a substantial bias which artificially lowers the reported effectiveness rate of unassisted quitting, or at least fails to provide the key data of interest to the research question.
Putting it All Together
It strikes me that those who are trying to explain away the consistent finding that drugs do not appear to be much better than cold turkey quitting in population based studies (outside the context of clinical trials) are now grasping at straws, making valiant efforts at reasoning away the clear findings of these population-based studies.
Here, the study has created a premise for the entire analysis (that there is differential recall of failed cold turkey vs. medication quit attempts), a premise which is based not on the demonstration that cold turkey quit attempts tend to forget their quit attempts, but on the observation that medication users recall more remote failed quit attempts than unassisted quit attempters. Not considered is the possibility that the findings are real (rather than based on memory defects). Is it not possible that failed smoking cessation attempts are likely to deter further quit attempts for a longer period of time than failed unassisted quit attempts?
Furthermore, the study introduces two sources of potential bias, both of which would bias the results toward finding a higher rate of effectiveness of medication. The more troubling of the two is the exclusion of medication users based on a question, asked retrospectively, about the reasons for their use of stop smoking medication. In my view, once you start excluding certain users of medication, you are no longer playing fair. Smoking cessation drugs are smoking cessation drugs and to exclude failures based on smokers retrospectively reporting that they actually didn't have the desire to completely quit smoking is not a fair and balanced analytic approach.
Perhaps the most telling fact about this study is that, as you might expect based on the various biases in the study, one of the co-authors has a significant financial conflict of interest with companies that manufacture smoking cessation drugs.
According to the study's disclosure statement, one of the study investigators: "has served as a paid consultant on smoking cessation to Pfizer and Novartis, [and] has received payment from Pfizer and GlaxoSmithKline for lectures on smoking cessation to health professionals."
I am not suggesting that there is any conscious bias but subconsciously, these kinds of financial connections are going to influence the slant with which one conducts, analyzes and reports results. In this case, there is ample evidence from an analysis of the study that there is indeed bias towards finding an effect of smoking cessation medications. There is nothing wrong, since the conflict was appropriately disclosed. However, readers should take this conflict into account when interpreting the study results.