On What Passes for Science in Tobacco Control: New Study Purports to Show Effect of College Tobacco Ban on Smoking Rates, But Fails to Consider Age Differences Between Pre- and Post-Ban Samples

A study published in the journal Preventive Medicine in 2011 purported to demonstrate that a tobacco-free campus policy led to a decrease in smoking among students at Indiana University.

(See: Seo D-C, Macy JT, Torabi MR, Middlestadt SE. The effect of a smoke-free campus policy on college students' smoking behaviors and attitudes. Preventive Medicine 2011; 53:347-352.)

The study was a quasi-experiment, using a repeated cross-section design with a comparison group. Indiana University instituted a campus-wide tobacco ban in 2008. Cross-sectional surveys were conducted in 2007 - prior to the ban - and in 2009. Similar surveys were conducted during the same years at Purdue University, which did not institute a campus-wide tobacco ban.

The methods were as follows: "In fall 2007, 84 Indiana University instructors and 67 Purdue University instructors were asked for permission to administer a survey in their classes. Among those contacted, 73 Indiana instructors and 55 Purdue instructors agreed. A total of 3492 students (2057 from Indiana and 1435 from Purdue) were invited to complete a group administered paper-and-pencil survey, and 3266 students (1930 [93.8%] from Indiana and 1336 [93.1%] from Purdue) participated. In fall 2009, 77 out of 87 Indiana instructors and 54 out of 65 Purdue instructors agreed to have the survey administered in their classes. A total of 3455 students (2215 from Indiana and 1240 from Purdue) were invited to participate, and 3207 students (2042 [92.2%] from Indiana and 1165 [94.0%] from Purdue) completed the survey."

The study found that current smoking prevalence at Indiana University dropped from 16.5% to 12.8%, but at Purdue University, it increased from 9.5% to 10.1%. Based on this finding, the study concludes that the tobacco ban led to a decrease in smoking among Indiana University students.

The Rest of the Story

What the study fails to point out to readers is that the pre- and post-ban student samples were not at all comparable. The key to the validity of this study is that the pre- and post-ban samples need to be comparable, so that accurate comparisons can be made of smoking rates. However, the sampling procedure failed to yield comparable samples. In fact, the differences between the samples were striking and furthermore, would systematically lead to an underestimation of true smoking prevalence at Indiana University in 2009.

Table 1 reveals the differences in the two samples. The differences are striking. In 2007, the proportion of males in the Indiana University sample was 42.4%. In 2009, it was 37.6%. In 2007, the percentage of freshman in the sample was 19.5%. In 2009, the percentage of freshman was 26.9%. The percentage of students living in residence halls increased from 26.8% to 33.8%.

Anyone looking at these differences can easily see that the study conclusions are invalid. The same reasoning that led to the conclusion that the smoking rate dropped would lead to the conclusion that the percentage of males dropped and that the percentage of freshman dropped and that the percentage of students living in residence halls dropped as a result of the tobacco-free policy.

But it is obviously not true that the tobacco ban led to a decline in the percentage of male students at Indiana University or that it led to fewer freshmen or fewer students living in residence halls. In order to draw such conclusions, one would need comparable samples. The study failed to achieve that.

In fact, the differences in the samples would - in each case - lead to an underestimation of smoking prevalence at Indiana University in 2009 compared to 2007. In general, the post-ban sample was a younger one, with more freshmen and fewer upperclassmen. We know that smoking prevalence increases with age. Thus, just based on the differences in the sample, one would expect smoking rates to have dropped from 2007 to 2009.

But it gets worse. At Purdue University, the opposite occurred. The sample in 2009 was an older sample. The percentage of freshman in the sample decreased from 27.3% to 17.5%. Thus, sampling alone would lead to an overestimation of the smoking prevalence in 2009 compared to 2007.

The gender differences in the sample would also lead to an underestimation of smoking prevalence in 2009 at Indiana University. We know that smoking rates are substantially lower among college-aged females than males. So the much lower percentage of males in the 2009 sample at Indiana University would lead to an artificially low estimate of the percentage of smokers.

Shockingly, the study makes no attempt to control for these age and gender differences. Its conclusion is based solely on the observed difference in overall smoking prevalence in the two samples.

This is yet another example of the kind of shoddy research that passes for science in the tobacco control movement today. My impression is that researchers are so anxious to find significant effects of tobacco control policies that they subconsciously are trying to find such effects, rather than allowing the data to drive the conclusions.

Incidentally, the authors acknowledged that "the campus-wide smoke-free air policy was not actively enforced and people can be seen smoking on a regular basis." Perhaps this is why there is a need to distort the science to find an effect. The policy must be justified and based on the impossibility of enforcing such a policy, one would be hard pressed to defend it. But if one can cook up evidence showing that it reduced smoking rates, then one can justify the policy. It appears that this study succeeded in doing so.