As I pointed out: "one of the authors of the new study is a recipient of grant funding from Pfizer, the manufacturer of Chantix, and therefore has a significant financial conflict of interest." I argued that: "The bias present in the study therefore has the appearance of being a result of this conflict of interest, rather than as a result of the desire to ensure the rigor of reported scientific findings."
Today, I discuss two aspects of the study which, upon further thought, lead me to more seriously question both the apparent bias in the article and the apparent influence of the conflict of interest in shaping this bias.
The Rest of the Story
The first issue that I find particularly troubling is the study's exclusion of any adverse cardiovascular events that occurred more than 30 days after discontinuation of Chantix. This might be justifiable if the effects in question were acute effects that could occur only in the presence of the drug in the bloodstream. For example, if one were studying the effects of a drug on sedation, it would be implausible that the drug would have an effect after a period of more than about five or six half-lives, because the drug is out of the system and any sedation effects would have ceased by that point. However, the hypothesized effect of the drug is not solely due to its presence in the bloodstream. The effect may be due to long-term changes in blood vessels, which do not simply disappear immediately after withdrawal of the drug.
For example, when a smoker quits smoking, her risk of cardiovascular events does not immediately drop to the level of a nonsmoker. In fact, it takes about two years - at least - before cardiovascular risk returns to normal. This is because smoking induces long-term atherosclerotic changes and vessel disease which are not immediately reversible. If one excluded events occurring more than 30 days after smoking cessation in a study of the adverse effects of smoking, one would obtain an incorrect, and substantially lower than accurate estimate of the magnitude of smoking's cardiovascular effects.
Since the adverse effect in question is related to the atherosclerotic process, rather than to some acute phenomenon, it seems inappropriate to exclude any events that occurred more than 30 days after drug withdrawal.
The second issue that I find troubling is the study's conclusion that based on this meta-analysis, Chantix has no "clinically significant" cardiovascular effects. Due to the extremely small sample sizes in the studies and to the rare nature of the events in question, the meta-analysis has limited power to detect a statistically significant effect of Chantix on cardiovascular event risk. Importantly, the meta-analysis, even with its exclusion of some adverse events, found a 0.27% excess absolute risk of adverse events among patients treated with Chantix.
This represents 27 adverse events among every 10,000 treated patients. Approximately 1.8 million patients per year receive Chantix prescriptions. If the risk difference reported in the meta-analysis is correct, then this translates into approximately 4800 adverse cardiovascular events among treated patients due to Chantix. This number hardly seems "clinically insignificant."
A judgment of the clinical significance depends further upon an evaluation of the number of treated patients who will quit smoking, and thus gain substantial health benefits. Assuming that 10% of treated patients will quit long-term because of the drug and that 10% of these patients will be saved from a severe adverse cardiovascular effect because of quitting, the drug will prevent adverse events among 18,000 people. However, this comes at the expense of causing severe harm to 4800 people. In other words, for every four smokers who are "saved" by the drug, one will be seriously harmed by it.
These numbers are of course estimates, but my point is simply that one needs to conduct an analysis like this before one can opine about the clinical significance of an observed 0.27% absolute risk increase of serious adverse cardiovascular events associated with Chantix.
The most important point, however, is that the sample sizes in the studies were so small that one should not - at this point - draw any conclusion that Chantix has no clinically significant adverse effects based on the current meta-analysis. These studies were powered to determine whether Chantix helps people quit smoking, not to specifically determine whether the drug can increase the risk of rare outcomes such as heart attacks.
Dr. Singh, the author of the original study, points out this important fact in his rapid response to the re-analysis: "the authors provide no information on the optimal information size or the power of their meta-analysis. They conflate the lack of statistical significance in an underpowered meta-analysis as clinically insignificant. Despite the removal of cardiovascular events on varenicline, and a statistical approach that has limited powered to detect an effect there is an excess risk of cardiovascular events with varenicline in all five measures reported in their study. ... Since none of the trials evaluated cardiovascular events as a primary outcome or were powered to detect individual differences in cardiovascular outcomes between varenicline and placebo, adequately powered randomized controlled trials are needed. ... A post-marketing Study to Evaluate Cardiac Assessment following different treatments of smoking cessation medications in subjects with and without psychiatric disorder (CATS) comparing varenicline, placebo, bupropion and nicotine replacement therapy of approximately 8000 patients with major adverse cardiovascular events as a primary outcome has been requested by the FDA. After 24 weeks of treatment it will collect data on cardiovascular outcomes for an additional 28 weeks of non-treatment, for a total of 52 weeks. Unfortunately, the results will only be available in 2017, just in time for the patent expiry of varenicline. ... until the results of such outcome trials are available, clinicians need to consider the overall risks of varenicline. i. e., the serious cardiovascular risk and other serious risks such as suicide, depression and violence noted in the drug label. They need to balance these serious risks against its efficacy on abstinence, compared to placebo."
In other words, it is far too premature to draw the conclusion that Chantix has no serious cardiovascular adverse effects. Were that the case, then why would the FDA be requesting a study with 8000 patients.
The point is this: none of the existing studies has the power to detect a small, but significant increase in cardiovascular events due to Chantix. The large study, with 8000 patients, is necessary before any conclusion can be drawn. Certainly, physicians and researchers should refrain from disseminating to the public the conclusion that Chantix is perfectly safe in terms of its cardiovascular risk profile.
However, this is precisely what the authors of this study have done. Their conclusion that Chantix is safe - based on their study - is completely unwarranted and I believe, inappropriate. That this conclusion is based on a study conducted by an author with a substantial financial conflict of interest - receiving grant funding from the manufacturer of the drug in question - is, in my mind, not only unfortunate but unacceptable.
The public in general, and physicians in particular, need to be able to rely on the opinions of unconflicted researchers in order to make decisions regarding the clinical safety and effectiveness of medical treatments. While there is nothing wrong with a researcher carrying out a drug-funded study, that researcher should not place themselves in the position of making national recommendations for the use of a drug, or drawing overall conclusions regarding the safety of the drug.
The rest of the story is that this study is severely biased, as evidenced by biased methodology and by a heavily biased conclusion that Chantix is perfectly safe. The severe bias has the appearance of being related to the lead author's receipt of grant funding from the manufacturer of the drug in question. This is the type of bias that proper management of conflicts of interest is intended to prevent.