Saturday, June 20, 2009

NYT discovers selection bias

A not-too-bad report from the NYT on the potential for selection bias in studies that show health benefits from alcohol consumption.

My comments center on this bit:

Meanwhile, two central questions remain unresolved: whether abstainers and moderate drinkers are fundamentally different and, if so, whether it is those differences that make them live longer, rather than their alcohol consumption.

Dr. Naimi of the C.D.C., who did a study looking at the characteristics of moderate drinkers and abstainers, says the two groups are so different that they simply cannot be compared. Moderate drinkers are healthier, wealthier and more educated, and they get better health care, even though they are more likely to smoke. They are even more likely to have all of their teeth, a marker of well-being.
What matters is not whether the mean characteristics differ but whether there is what in the technical literature is called "common support", which just means overlap. Are there some non-drinkers who have the characteristics of moderate drinkers? If so, that is enough.

More important than overlap is, of course, measuring all the relevant confounders. A different, and perhaps better, way to frame the article given the likely impossibility of doing a major random assignment study on moderate alcohol use would have centered on the importance of having a large data set that contains information on all the variables that might confound the effect of alcohol on health.

The NYT piece, unfortunately, does not really make it clear how good a job the existing literature does at including all the possible confounders. I suspect the answer is "not very well" and that there is room for improvement here even without a randomized trial, though obviously that would be helpful if one could be fielded.

Hat tip: Jessica Goldberg