You wouldn’t know it by reading NEA press releases, but a majority of Americans favor publicly-funded scholarships or vouchers that give families the option of attending private schools. Blacks and Hispanics are especially supportive.
But do such programs work? The short answer is “Yes.” My colleagues and I have had the opportunity to examine data from a number of scholarship programs, and we now know quite a bit about the kinds of families that participate in central-city scholarship programs, the reasons families apply for scholarships, parental satisfaction with their private schools, the effects of school choice on student learning, and private school retention rates. Our early findings are summarized below.
Who Participates?
Choice opponents typically deride private voucher programs for providing an income supplement that is not enough to pay for private school and therefore only of use to families rich enough to make up the difference. Opponents also criticize such programs for “creaming” the best students, making matters even worse for those who remain in the public schools. Our research shows that in fact many voucher recipients are quite poor, and many have very limited educational attainment.
The best information on the kinds of people who participate in scholarship programs comes from data on the program run by the New York School Choice Scholarship Foundation. The Foundation received applications from over 20,000 students for its scholarships. In order to be eligible, families had to attend verification sessions to document their income level (i.e., eligible for the free school lunch program) and their child’s public-school attendance. The majority of those who participated in the verification sessions came from economically disadvantaged families. The program’s income limit for a family of three was $16,874 and for a family of four was $20,280, yet applicants’ average annual family income worked out to less than $10,000. Over two-thirds of the applicants came from families that were receiving food stamps and Medicaid, with nearly 60 percent receiving welfare assistance.
Average pre-test scores of eligible applicants were quite low. Only 26 percent scored at grade level in reading. The corresponding figure for mathematics was just 18 percent. By comparison, the New York public schools report that 55 percent of New York City public-school students score at the statewide average on a similar standardized test. Forty-seven percent of the applicants’ parents were of Hispanic background, and 44 percent were African American. Fewer than 5 percent of parents were non-Hispanic whites.
Parental Objectives
School choice advocates say they want to empower parents by giving them a choice among schools. But critics say that parents, especially poor parents, do not have enough information to make intelligent choices, and, when given a choice, academic considerations are not paramount. A Twentieth Century Fund report goes so far as to claim that low-income parents are not “natural ‘consumers’ of education” and that “few parents of any social class appear willing to acquire the information necessary to make active and informed educational choices.” Similarly, the American Federation of Teachers (AFT) report on a choice program in Cleveland suggests that parents sought scholarships, not because of “‘failing’ public schools” but “for religious reasons or because they already had a sibling attending the same school.”
Not much support for such suggestions can be found if we look at responses given by the parents of scholarship recipients in a state-funded program in Cleveland. Asked why they applied for a scholarship, 85 percent of parents new to choice schools said they wanted to “improve the academic quality” of their child’s education. Second in importance was the “greater safety” to be found at a choice school, a reason given by 79 percent of the recipients. “Location” was ranked third. Contrary to AFT’s suggestion, “religion” was ranked fourth, said to be very important by just 37 percent. Finally “friends,” were said to be “very important” by less than 20 percent of the scholarship recipients.
Parental Satisfaction
Many economists think customer satisfaction is the best measure of school quality. If the economists are to be believed, then both anecdotal evidence and systematic studies compel the conclusion that school choice is a smashing success. A year into the Cleveland choice program, Pamela Ballard, parent of a new choice student exclaimed: “HOPE Academy was my last hope. I took my third-grade child, who had been in several Cleveland schools and was labeled a ‘problem child.’ I now have a successful child. Where there were D’s and C’s, there are now A’s and B’s.”
In her enthusiasm, Pamela Ballard represents the norm, not the exception. My colleagues and I compared the assessments of Cleveland parents attending choice schools with the assessments of those who remained in public schools. Two-thirds of parents new to choice schools reported being “very satisfied” with the “academic quality” of their school, as compared to less than 30 percent of public-school parents. Nearly 60 percent were “very satisfied” with school safety, as compared to just over a quarter of non-recipients in public school. With respect to discipline, 55 percent of recipients from public school, but only 23 percent of non-recipients in public school, were very satisfied. The most extreme differences in satisfaction pertained to “teaching moral values”: Seventy-one percent of the recipients from public schools were “very satisfied,” as compared with only 25 percent of the non-recipients in public schools.
The Bottom Line
None of this encouraging data would be decisive if students didn’t actually learn more in choice schools. A number of national studies have found that private school students test ahead of students in public schools, but choice critics have attacked these studies for not adequately taking into account the fact that those who go to private schools may come from select families with a higher level of educational commitment. Says former Wisconsin state school superintendent and choice critic Herbert Grover, “Do private school children outperform child in public schools? It’s hard to imagine that they wouldn’t, given the initial advantages they enjoy from their parents.”
School choice experiments are providing researchers with new opportunities to circumvent the selection problem. For one thing, they are limited to inner-city children from low-income families. More importantly, to ensure fairness, scholarship winners are sometimes chosen by lottery, giving these programs the potential of becoming a classic randomized experiment of the kind found in the best medical research. When a lottery is used to pick scholarship winners, the two groups of students are similar except for the fact that the names of one group were drawn from the hat. When one works with information from a classic randomized experiment, one can in fact reasonably assume that it was the school, not the family, that made the difference.
Unfortunately, most school-choice experiments conducted thus far have not conformed to a classic randomized experiment, preferring to admit students on a first-come, first-served basis. Still, test score results from these experiments are mainly positive. For example, the scores of students participating in the school choice program in San Antonio increased between 1991-92 and 1993-94, while those of the public-school comparison group fell.
Students also seem to be learning in Cleveland choice schools. In the summer of 1997 my colleagues and I reported significant first-year test-score gains in reading and math for 263 students who attended two newly established choice schools. These test-score findings were subsequently challenged on the grounds that the gains could be expected to fade away over the summer break. In addition, another evaluation by Indiana University’s School of Education found no programmatic effects on the test scores of 94 third-grade choice students.
Well, we tested for the summer-fade theory and we found that while test score gains did slip somewhat, students in grades one through three still ended up achieving statistically significant gains in math and reading. As for the Indiana University evaluation, it suffers from a number of limitations including inflated baseline data and a complete lack of analysis of data on students in kindergarten, first, and second grades. After Indiana University released the data from its evaluation, my colleagues and I corrected for many of these deficiencies and found the choice-school effects to be positive in all subject domains. But all of these findings may be contaminated by the selection effect. After all, those families who are quick, clever, and well-connected enough to get a first-come, first-served scholarship are likely to have other attributes that favorably affect their child’s educational attainment.
The Great Milwaukee Experiment
Only in Milwaukee are data available from a randomized experiment. In analyzing the Milwaukee data, my colleagues and I found that enrollment had limited positive effects during the first two years a student was in the program, with larger gains in years three and four—as much as one quarter of a standard deviation in reading and one third of a standard deviation in mathematics. To put that into plain English, if such gains can be continued at this rate throughout a student’s educational career, existing differences in the test performances of whites and minorities could be eliminated.
That the improved performance does not become substantial until the third and fourth years is quite consistent with a commonsense understanding of the educational process. Choice schools are not magic bullets that transform children overnight. It takes time to adjust to a new teaching and learning environment. The disruption of switching schools and adjusting to new routines and expectations may hinder improvement in test scores in the first year or two of being in a choice school. As Indianapolis choice parent Barbara Lewis explains the process: “I must admit there was a period of transition, culture shock you might call it. He had to get used to the discipline and the homework. . . . But Alphonso began to learn about learning, to respect the kids around him and be respected, to learn about citizenship, discipline, and doing your lessons. . . . My son has blossomed into an honor roll student.”
Roll Your Own Evaluation
From a public-policy perspective, the most persuasive evaluation is one that examines student test scores. Although many proponents of school choice, myself included, believe that some of the most important benefits of choice are not necessarily related to test scores, in public discourse test score results trump all other findings. For this reason, it is desirable to have test score results, preferably from a randomized experiment.
Yet one should not conduct such an evaluation unless the number of scholarships to be awarded is considerable (preferably between 500 and 1,000). Otherwise, one risks being unable to detect positive school-choice results, even if the program is a success. Nor should one conduct a large-scale evaluation without baseline data, sufficient resources to follow students over several years, and a good working relationship with a credible research team. This may mean delaying the start of the scholarship program until the research design is in place—a significant cost many philanthropists may be loathe to bear.
Simpler evaluations are also possible. If resources are not available to undertake a comprehensive test-score study, one might instead conduct a study of student mobility and parental satisfaction. Such studies are less expensive to conduct, more easily undertaken, yet may be of interest to media and legislative elites. In these studies, it is important to find an appropriate comparison group; this can be done most easily by contacting public school parents by telephone. We found such a data collection strategy useful in Cleveland, where it was not possible to conduct a randomized experiment of student test scores.
School choice is not a panacea for our educational problems. But findings that have emerged from recent evaluations do suggest it is time to rethink the way we organize our public educational system. As Urban League President Hugh Price puts it, “If urban schools . . . continue to fail in the face of all we know about how to improve them, then [parents] will be obliged to shop elsewhere for quality education. We Urban Leaguers believe passionately in public education. But make no mistake. We love our children even more.” Columnist William Raspberry has written that: “It’s time for some serious experimentation.” Let the experimentation begin.
Paul E. Peterson is Henry Lee Shattuck Professor of Government and Director of the Program on Education Policy and Governance at Harvard University.
