Monday, March 6, 2017

On the enthusiasm of program operators for research on the effects of the programs they operate

From the Hollister, Kemper and Maynard (1981) National Supported Work Demonstration book, pages 35-36:

     "The central experimental aspect of the research, random assignment of enrollees to experimental (participant) or control groups, was carried to completion without any technical disruption. It is important to note that operators were permitted to interview all those referred prior to random assignment and to eliminate any persons they felt would be too disruptive to their programs. Once they had screened referrals, those remaining were randomly assigned to program participation or to the control group. Although there were not technical problems with this process, the whole procedure was greatly resented by most of the operators, whose negative views were strongly reinforced by the agencies that referred potential enrollees to a site program. These hostile attitudes to random assignment were deeply embedded and yielded not at all to any sort of rational argument. For example, it did not good to point out that there were, in any case, a limited number of positions because, as with any program, there were limited funds or that in most cases of referral to training or employment programs, agencies were rarely able to place anywhere near 100 percent of those referred. My own suspicion is that even tough, in reality, there was no loss of control, since the operators and the agencies had the opportunity to screen referrals, the random assignment requirement reinforced both their feelings that they had lost control as well as their resentment of the process. It was only because of MDRC's adamant refusal to concede on this issue, and its insistence on making it a prerequisite for continued funding at research sites, that the process was carried out successfully over such a long period of time. As far as the research organization could discern, there was very little resentment of the random assignment process on the part of the enrollees themselves Perhaps it will be easier to apply this method in future studies of employment programs since it will be possible to point to the successful implementation of it, on a large scale, in the Supported Work Demonstration.
     The random assignment had to be carried out during the enrollment period for the research sample, which stretched from March 1975 to July 1977, and during those months the site operators continually inquired about how much longer the process had to continue. Their reaction to the notification of the termination of enrollment and random assignment apparently was one of joy and relief"

No comments: