A research agenda on school choice

Mathematica Policy Research is doing a series of blog posts that aims to lay out a research agenda on school choice for the new administration.  This seems like a worthy endeavor and I was generally impressed with the contents of the first two installments. I was particularly impressed that the first part mentions equilibrium effects of school choice, whether charters or vouchers, on traditional government schools. That is a tricky thing to study, but may well be an important part of the overall story.

I do have two quibbles. First, I would emphasize the role of innovation in the design and operation of schools as another potential benefit of school choice, though one that is constrained by the overarching regulation that governs all schools receiving public funds (as well as home schooling to some degree in some places). As best I can tell, the way schools work has changed little since I was attending them, and that was a while ago. There is some electronics around now, and the curriculum emphasizes current enthusiasms, but it all seems familiar in a broad sense. Are there better ways? If there are, could we possibly find them in the current institutional and regulatory environment?  Are there policy changes that would yield more useful innovation?

My second quibble is that the second installment implicitly adopts a binary epistemological stance in which the research world contains two kinds of studies: experiments and non-experiments. In this binary view, experiments are wonderful and non-experiments are kinda sucky, but perhaps better than the opinion of the stranger sitting next to you at the bar, at least on average. What the binary view misses is that experiments have issues, sometimes more, and sometimes less, because randomization does not solve every problem associated with empirical research; rather, it solves just one problem, albeit an important one. For example, the charter school impact studies that rely on admissions lotteries have some issues of interpretation, e.g. with multiple lottery winners, and some issues with external validity, because not all charters run lotteries. The binary view also misses substantial variation in the quality and causal compellingness among non-experimental studies. I know that the Mathematica folks know all this, but I would have liked to see it reflected a bit more in the way that they discussed the existing evidence.

Link to the first installment.

Full disclosure: I sometimes act as a paid consultant to Mathematica about various things (as I also do with several of their competitors).