A paper published in the March issue of the American Journal of Public Health, which examined the relationship between exposure to the American Legacy Foundation's "truth" campaign and the prevalence of youth smoking, concluded that the campaign "was associated with significant declines in youth smoking prevalence" (Farrelly MC, Davis KC Haviland L, Messeri P, Healton CG. Evidence of a dose-response relationship between "truth" antismoking ads and youth smoking prevalence. Am J Public Health 2005; 95:425-431). Now, the results of that study have been called into question by Dr. Joel Moskowitz of the UC Berkeley School of Public Health, who published electronic letters in AJPH as well as the BMJ (British Medical Journal) which point out that the dose-response relationship found in the study actually does not support the hypothesis that the campaign was effective in reducing smoking prevalence.
Moskowitz writes: "The theoretical rationale for inclusion of the GRP-squared term was to test whether the campaign had diminishing returns" (p. 428). This would suggest an L-shaped relationship between campaign advertising and smoking prevalence; not the U-shaped relationship found. The results suggest that the campaign had no detectable effect on smoking prevalence among those who resided in media markets that received higher levels of exposure which included students in most major metropolitan areas. Yet, the paper obscured this finding and failed to address its policy implications. Did an overdose of truth render the campaign ineffective? Or were the models improperly specified to estimate campaign effects?
When examined by grade level, the effect of truth advertising on smoking prevalence was significant only for students in grade 8 in media markets with moderate exposure (Table 2). That the campaigns impact did not sustain through high school suggests that truth advertising was no more effective than school-based, smoking prevention programs (e.g., Wiehe et al., 2005)."
The Rest of the Story
I evaluated the study in light of the alternative interpretation of the results presented by Dr. Moskowitz. My interpretation of these results is that they fail to support the conclusion of a significant effect of the "truth" campaign on youth smoking prevalence.
First, it is important for the reader to understand that the study failed to find any significant relationship between the intensity of exposure to the "truth" campaign and youth smoking prevalence under the assumption that campaign effects would increase linearly with campaign exposure. This is probably the most sound initial assumption, since the study measures not actual, individual-level exposure but the penetration of various media outlets in a geographic region (which is used as a proxy for actual exposure).
This basic result is hidden in the second half of the final paragraph on page 428 of the paper, and the quantitative result is not provided in the paper: "We also estimated a set of regressions excluding the quadratic GRP term (GRP squared) (results available on request). In this set, the effect was marginally statistically significant for 12th-grade students (OR=0.879; P<.07) but statistically nonsignificant overall and for 8th- and 10th-grade students."
This means that youth smoking did not decrease as campaign exposure increased linearly over the entire range of exposure (from 647 to 22,389 GRPs). Gross rating points (GRPs) represent the percentage of the target audience that is reach by the campaign multiplied by the number of times they are reached. If the campaign had a significant effect on youth smoking, then one would expect smoking prevalence to fall as exposure increased. This was not the case.
The paper attempts to find a relationship between campaign exposure and youth smoking by using an alternative assumption about media effects - that there are "diminishing returns" as exposure gets very high (i.e., once you reach a certain level of exposure, additional exposure is unlikely to have much of an additional effect on smoking prevalence).
While this is not an unreasonable assumption to test, the study results failed to detect a relationship that was consistent with this pattern of diminishing returns. Instead, the paper found support for a U-shaped relationship, in which the campaign was found to be associated with lower smoking prevalence at moderate exposure levels, but to have minimal, if any, effect at both very low and very high exposure levels.
This type of relationship is really not consistent with the conclusion that the "truth" campaign was effective in reducing youth smoking prevalence. While results that showed diminishing returns would be consistent with such a conclusion, the fact that there was not an observed effect among youths living in the areas with the absolute highest exposure to the campaign is not.
I will try to demonstrate this problem with some examples:
1. Under the basic analysis of the study, in which smoking rates in 2000-2002 were compared to those in the the baseline period (1997 to 1999), the results show that at an exposure level of 22,000 GRPs (the highest level), the odds ratio for youth smoking is 0.96, indicating essentially no effect of campaign exposure.
2. In the comparison of smoking in 2002 versus the baseline period, the results show that at an exposure level of 20,000 GRPs (about average for the major metropolitan areas with highest exposure), the odds ratio for youth smoking is 0.97, again indicating essentially no effect of campaign exposure.
3. In the analysis restricted to 8th-grade students, which was the only one to find a campaign effect (there was no significant effect of the campaign on 10th-grade or 12th-grade students), the results show that at an exposure level of 19,000 GRPs, there is no effect of campaign exposure (odds ratio is 0.99), and at an exposure level of 22,000 GRPs, exposed youths are actually more likely to smoke (i.e., there is actually a reverse effect of the campaign at very high exposure levels).
The reason why these findings are not apparent in the paper is that, as Dr. Moskowitz points out, Figure 2 cuts off at an exposure level of 15,000 GRPs, which leaves out a large part of the sample. Had the figure been extended to show the results for the full range of exposure, it would have become apparent that there was no campaign effect detected for youths with very high campaign exposure. It would have become clear that the pattern was not consistent with a hypothesis of diminishing campaign effects, but rather, consistent with the absence of an effect of the campaign on youth smoking prevalence.
The important point, I think, is that the results just don't appear to support a conclusion that the "truth" campaign resulted in a significant decrease in youth smoking prevalence. They certainly do not, I believe, support a causal conclusion.
While there are plenty of reasons to support and continue funding anti-smoking media campaigns such as the "truth" campaign, I do not see the results of this particular study as being one of them.
No comments:
Post a Comment