A long pondered but only lately realized blog about economics, politics, evaluation, econometrics, Ann Arbor, academia, college football and whatever else comes to mind.
Thursday, April 7, 2011
The multiple comparisons problem arises when doing large numbers of statistical tests. For example, one might estimate the impact of some treatment on 100 different outcomes. Ignoring correlations among the outcomes, in a world where the population treatment effect equals zero for all 100 outcomes, one would still expect five estimates to be statistically significant at the five percent level.