Data, evaluation and the quality of public policy

This new working paper describes an evaluation data set that researchers can use to study the effects of active labor market policies in Germany. The data set combines administrative data drawn from multiple sources with innovative survey data.

There is nothing like this available in the US or Canada. Under the guise of privacy concerns, the political process in the US and Canada avoids assembling the sorts of data that would allow very high quality non-experimental evaluations (as well as descriptive analysis useful for understanding program operation and informing program implementation and design) of active labor market programs. Part of this is, at least in the US, due to the fact that both political parties have strong, but different, prior beliefs, about the effectiveness of such programs. To many democrats they are obviously effective (how could more schooling be bad?) while for republicans they are obviously ineffective (how could bureaucrats increase anyone's employment chances?) thus the demand for quality empirical analysis is low regardless of who is in power.

What we might call the "data gap" (a play on the historical "missile gap") has the effect of leading US researchers, at the margin, to spend their time working on non-US data, as with Dale Mortensen's ongoing research project in Denmark and Sandy Black's ongoing use of the Norwegian register data. Now, to be sure, no one actually moves outside the US because the salaries are much too low, but they do change their travel plans and their research agendas. In some ways this is good, because it leads to more interactions between North American and European researchers. However, the US is a big country, and policy unguided by serious evaluation in the US affects a large number of people and one of the world's most important economies.

There are real, low-cost opportunities for dong some good here. Will anyone in DC pick up the ball and run with it?

Canadian census long form

The conservative government in Canada is seeking to eliminate the Census "long form" (or, more gently, to make it voluntary). The US did this a few years ago but replaced the long form with the (still mandatory) American Community Survey, so that the issue of having data (almost) uncontaminated by non-random response bias, which to me is the key issue in the debate over the long form, does not arise. The Canadians propose to make the long form voluntary, thus corrupting all the estimates it generates with selection bias.

Here is a fairly reasonable Globe and Mail editorial and here is a somewhat over-heated and remarkably political editorial from Nature.

Here are some tweets from a Canadian government minister who does not, as the saying goes, get it.

Shame on the conservatives in Canada for not being serious on this one. They are an embarrassment to themselves and to Canada and are sending a strong signal that they are not serious about good government. And that matters in Canada, where the analogue to "life, liberty and the pursuit of happiness" is "peace, order and good government."

My pals at Reason also do not distinguish themselves on this one. The fact that the data are not perfect is an argument for improving the data, not abandoning the attempt. More broadly, it seems to me that classical liberals should support govenrment production of public goods, like knowledge, and the evidence-based policy making it supports, particularly given, at least in my experience, that most programs and policies cannot pass a cost-benefit test when rigorous evidence is available.

Regular readers may object that I cannot complain about mandatory jury duty and then not complain about a mandatory census, but the census from mandatory jury duty in three important ways. First, the tax is small. Completing the Census for takes a few minutes. Second, the tax is uniform. Everyone pays the same small tax, rather than jurors who get stuck on long trials paying a gigantic in-kind tax and jurors who escape without being assigned to a trial paying no tax. Third, volunteer jurors are probably a good substitute (perhaps even a superior substitute) for mandatory ones, while voluntary surveys are not a good substitute for mandatory ones, particularly not at the key task of establishing solid benchmark statistics against which all the other voluntary surveys can be judged.

I do think that the Census should aim to minimize the burden of the long form. 20 percent of the population may be much more than is required to get very precise population estimates, even at a provincial or metropolitan area level. In a sense, reducing the population subject to the requirement to complete the long form is what the US did by replacing it by the ACS.

More broadly, given Facebook and Google and all the rest, it seems a bit bizarre for people to worry about the privacy implications of the innocuous questions on the long form.

Hat tip on the tweets to a friend at a government contractor in Canada.

Data security

I've spent a good bit of time the last two weeks devoted to issues of data security. In this regard, my first thought is always that one hears many stories - there are some amusing ones from the UK here and the post about Canada from a couple days ago is also relevant - about politicians leaving sensitive information about, as well as stories about data being left by bureaucrats in a Starbucks in DC or somewhere similar because Jessica Cutler walked by or some other distraction intervened, but I have never, ever heard a story about ill effects from any academic researcher using confidential data.

Methinks perhaps margins are not being equated here.