Leno, Stewart and Borowitz too.
Whew.
9 years ago
Sunday, May 26, 2013
Peer review follies
Abstract
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.
The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.
With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.
Gated version of the full paper here.
Hat tip: John Bound
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.
The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.
With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.
Gated version of the full paper here.
Hat tip: John Bound
Saturday, May 18, 2013
CDC takedown
Forbes provides a most excellent take-down of the CDC's input into the campaign to privatize alcohol sales in Pennsylvania.
The only thing missing is a mention of the fact that even if privatization does increase the negative consequences associated with excessive alcohol use, it might still (and perhaps easily) pass a social cost-benefit test. Increased consumption means more utils that have to be weighed against any bad consequences that do emerge.
The public health profession needs to get better at separating science from Sunday school.
The only thing missing is a mention of the fact that even if privatization does increase the negative consequences associated with excessive alcohol use, it might still (and perhaps easily) pass a social cost-benefit test. Increased consumption means more utils that have to be weighed against any bad consequences that do emerge.
The public health profession needs to get better at separating science from Sunday school.
Tuesday, May 14, 2013
Assorted links
1. DEA sadism.
2. A time series of satellite maps of Ann Arbor via Google.
3. Electronic coaching for students at Michigan.
4. The McDonald's of the future.
5. Did Notre Dame "chicken out" of its football series with Michigan?
2. A time series of satellite maps of Ann Arbor via Google.
3. Electronic coaching for students at Michigan.
4. The McDonald's of the future.
5. Did Notre Dame "chicken out" of its football series with Michigan?
Book: The End of Eternity by Isaac Asimov
I picked The End of Eternity up at the bookstore at Boston Logan Airport last week. It turns out that if you get past all the rubbish at the front of the store, there is actually some pretty good stuff in the back. And the ladies who were running it last Sunday were eager to talk about books.
As one might expect given the author and an original publication date of 1955, this is classic science fiction. There is an intricate plot, big ideas, time travel paradoxes (paradoxii?) and all the rest. It is, as one would expect from the genre, a bit short on beautiful prose and rich, multidimensional characters. Because it is old, and you can almost feel yourself brushing the dust off as you read, you get the added fun of seeing what Asimov gets right and gets wrong in his implicit predictions about technological progress.
Overall, great fun if you are into this sort of thing. There is a reason they are reprinting this in 2013.
Recommend for science fiction sorts.
Subscribe to:
Posts (Atom)
