Reuters and response rates

Yahoo! is presently featuring this piece on a study by something called the Physicians' Foundation. The key bits are

The Physicians' Foundation, founded in 2003 as part of a settlement in an anti-racketeering lawsuit among physicians, medical societies, and insurer Aetna, Inc., mailed surveys to 270,000 primary care doctors and 50,000 practicing specialists.

The 12,000 answers are considered representative of doctors as a whole, the group said, with a margin of error of about 1 percent. It found that 78 percent of those who answered believe there is a shortage of primary care doctors.

A brief bout of division indicates that the Physicians' Foundation achieved a glorious response rate of 12,000 / 320,000 or 3.75 percent. This is quite low even by the low standards of direct mail surveys.

Put differently, this study is completely meaningless. What seems pretty clearly to have happened is that very unhappy physicians responded, but no one else did.

Reuters reports this straight up, and even suggests that the 3.75 percent of those responding are "representative" of all physicians. Why would one possibly think that? Moreover, in this context, reporting the "margin of error" due to sampling is misleading in the extreme, given its relative unimportance compared to selective non-response.

Ugh.