One of the most difficult questions in the social sciences is what causes male homosexual behavior. Is it innate? (And if it is, how did not disappear due to natural selection.) Is it conditioned by society? Freely chosen? (In contrast, the level of female homosexual behavior is clearly responsive to changes in social attitudes.)
In the New York Times Magazine, Steven D. Levitt writes in his Freakonomics column that a new study by a U. of Chicago grad student named Andy Francis found that:
Not a single man in the survey who had a relative with AIDS said he had had sex with a man in the previous five years; not a single man in that group declared himself to be attracted to men or to consider himself homosexual....
Because the sample size was so small - simple chance suggests that no more than a handful of men in a group that size would be attracted to men - it is hard to reach definitive conclusions from the survey data. (Obviously, not every single man changes his sexual behavior or identity when a relative contracts AIDS.) But taken as a whole, the numbers in Francis's study suggest that there may be a causal effect here - that having a relative with AIDS may change not just sexual behavior but also self-reported identity and desire.
In other words, sexual preference, while perhaps largely predetermined, may also be subject to the forces more typically associated with economics than biology. If this turns out to be true, it would change the way that everyone - scientists, politicians, theologians - thinks about sexuality. But it probably won't much change the way economists think. To them, it has always been clear: whether we like it or not, everything has its price.
Some rather grand claims.
Unfortunately, Levitt didn't bother to inform the NYT-reading public that the sample size of men who had a relative with AIDS was so tiny, only 60 individuals in total, that Francis's study did not attain statistical significance even at the loose 5 percent level.
The percentage of men in the no-relative-with-AIDS sample of 1451 who had engaged in homosexual acts in the last five years was 4.4%. That means that if there was no actual correlation between having a relative with AIDS and sexual orientation, you’d expect only to find 2.64 men having engaged in homosexual acts out of his sample of 60 who had a relative with AIDS. Instead of two or three such men, he found zero.
As reported here in Table 6, the p-value for this correlation was 0.086, meaning that if the relationship between the two variables was purely random, you'd still expect to find that result (0 men out of 60) 8.6% of the time by pure fluke.
Is that 0 out of 60 meaningful? Maybe. Maybe not. Who knows?
Clearly, this study should be redone with an adequate sample size. Unfortunately, putting together a large enough and unbiased enough sample to do these kind of subtle analyses of male homosexuality has historically proven very difficult.
One major concern over how much credence to put in Francis's result is data-mining, or looking at a lot of relationships and then highlighting the ones that appear statistically unlikely to happen by chance. The 1992 database Mr. Francis used allows a large number of statistical tests to be run. If you looked at 100 different correlations, you’d expect to find an average of five that are statistically significant at the five percent level.
So, one relevant question is whether Mr. Francis started his research in order to test a pre-existing hypothesis that being exposed to people with AIDS would make a man less likely to engage in male homosexuality. I’d never heard the theory before, but perhaps it was going around in some circles. (Apparently, he did not -- he only formulated it after looking at data.)
I don’t have a strong opinion on Mr. Francis’s theory (Few doubt that female homosexual behavior is responsive to changing social pressures, but most researchers into homosexuality treat male and female homosexuality has highly distinct.) It sounds modestly plausible in theory, but I can’t think of much historical evidence for it. The only bit of anecdotal evidence for it that I can recall is that singer Lou Reed converted to heterosexuality (and stopped shooting heroin) right after AIDS was discovered. In contrast, San Francisco, for example, didn’t see mass conversions to heterosexuality after 1982. Indeed, AIDS seemed to lead, if anything, to an increased public commitment to homosexuality.
In summary, Mr. Francis has made a worthy beginning, but his hypothesis desperately needs to be retested with an adequate sample size.
Dr. Levitt would have been better advised to publicize this study on this blog, which is the appropriate medium for this kind of Scientific Wild Ass Guess. (In general, a better brand name for Dr. Levitt's version of social science than Freakonomics would be Scientific Wild Ass Guessonomics. Not that there's anything wrong with that. Blogs are fine sites to try out hypotheses.) Unfortunately, Levitt put it in the New York Times and failed to inform the public that the result was statistically insignificant.
Perhaps the correlation will prove to be true. But it may well not pan out. If that's so, it may be too late to recall the Instant Conventional Wisdom that Levitt has generated. "A lie goes halfway around the world before the truth gets its boots on."
Andy Francis replies here.