LIKE THE REST OF US, FOUNDATIONS DO NOT like to admit mistakes and they like even less to publicize their failures. On the surface, however, there appears to be little reason for foundation reticence about communicating and explaining failure. No matter how philanthropic decisions turn out, the huge endowments of foundations provide an impenetrable shield that allows these privileged institutions to roll along quietly and securely in virtual perpetuity. Nor would foundation’s “customers” be easily put off by an occasional gaffe. At most large foundations, the mailbag is always full of new proposals, and a long line of grant applicants is delighted to accept foundation largesse regardless of how well or poorly a foundation has performed in the past.
Though not a popular subject, failure in philanthropy is an important one because it touches both foundation staffs, whose decisions are implicated, and nonprofit managers, whose performance is scrutinized. Talking about failure also raises a number of important issues about what functions the nonprofit sector can and cannot realistically be expected to perform. An open discussion of failed grants may be a first step in bringing some reality and frankness to the increasingly strained and distorted communication between foundations and their grant recipients.
Two Kinds of Failure in Philanthropy
It is necessary to distinguish two fundamental forms of failure: one constructive and the other unconstructive. The difference between the two kinds of failure comes down to knowledge creation. While all failed grants start with ineffective programs, constructive failures create value by helping us understand what went wrong. By contrast, unconstructive failures produce no new knowledge to inform future practice. The challenge for foundations in the future is to gain a greater understanding of constructive failures and to begin appreciating them as central to the diagnostic function of foundations. A pair of recent examples will highlight the difference between the two kinds of failures in philanthropy.
From 1982 to 1988, the Rockefeller Foundation provided grants to four community-based organizations in cities around the country to operate employment training programs for minority single mothers. The goal of the Minority Female Single Parent (MFSP) demonstration was to determine whether comprehensive employment training and counseling services could reduce welfare dependency and enhance self-sufficiency. The four projects — conducted by the Atlanta Urban League in Atlanta; Opportunities Industrialization Center in Providence, Rhode Island; the Center for Employment Training in San Jose, California; and Wider Opportunities for Women in Washington, D.C. — enrolled a total of 3,965 women in the demonstration projects. Services offered at the four sites included basic skills and job-skills assessments, counseling, remedial education, job-skill training, job-placement assistance, and child care assistance. Following the kind of rigorous experimental design used in scientific studies to isolate the impact of an intervention, applicants in each of the four cities were assigned randomly to either a treatment group that was offered services or to a control group that was not eligible to receive services at the funded agencies but could seek services elsewhere.
Tracking the progress of both those who were enrolled in the programs and those who were excluded from it at each of the four sites, the evaluators measured the net impact of the interventions. After 30 months of treatment and study, the evaluation produced some notable results: (1) none of the four programs had a significant long-term impact on psychological wellbeing; (2) two of three programs offering high-school equivalency courses had no impact on GED attainment; (3) none of the programs had long-term impacts on the types of child care used; and, most important of all, (4) three of the four programs had no significant impact on earnings or welfare dependency.
These outcomes — and the care with which they were documented — make the Rockefeller’s MFSP demonstration one of philanthropy’s most constructive and important failures. Though three of the four sites failed to achieve significant results, it would be wrong to conclude that philanthropic funds were wasted on this large-scale experiment. In fact, a good argument can be made for the opposite conclusion. By fully and openly documenting the program’s failure at three sites, the project as a whole created significant public value and information. The outside evaluators of the MFSP demonstration scrupulously studied the reasons for the disappointing results and catalogued their findings. In addition to measuring and explaining failure, the evaluation also studied in depth why one of the four sites managed to achieve more encouraging outcomes.
As a consequence of the detailed evaluation of the program, other foundations or government agencies interested in welfare-to-work programs can gain a tremendous amount of knowledge from this experiment — not just from the small part of the MFSP demonstration that worked, but from the much larger part that failed. Rockefeller’s MFSP demonstration was a valuable and constructive failure because it was coupled with an independent evaluation that created value in spite of disappointing program design and implementation at the site level. The multi-volume longitudinal study, which draws on both quantitative and qualitative data, represents one of the very best evaluations of a foundation program conducted to date. Though they did not receive much attention from the press at the time of their release, these reports should now be required reading for all foundations and state agencies working to move families from welfare to work.
A constructive failure, such as the one described above, occurs when a foundation grant is used to carry out a program exactly as proposed and when a good evaluation reveals that the program had no significant impact. In this sense, constructive failures are best thought of as failed experiments, cases where funds are invested, hypotheses are tested, and results obtained. Constructive failures may not be appreciated by many in the foundation field, but they have potentially tremendous value. They can inform future practices and lay the groundwork for more successful programs. Unfortunately, constructive failures in contemporary philanthropy remain all too rare.
The second kind of failure is vastly more common today. An unconstructive failure in philanthropy produces no knowledge because the funder either chooses not to communicate openly the results of the evaluation or does not conduct any evaluation at all. Unconstructive failures are discussed very discreetly among close philanthropic friends or glossed over entirely. Since foundation grantmaking can benefit from increased knowledge about both what works and what does not, this lack of open communication poses an obstacle to more effective philanthropy in the long run. Thus, the trouble with unconstructive failures is not that they produce disappointing results, but rather that they produce no new usable knowledge when it clearly is much in need.
Perhaps the most spectacular and wholly unconstructive failure in philanthropy occurred recently in Chicago. In September of 1994, the talk show host Oprah Winfrey made the surprise announcement that she was funding a new program designed to move 100 families out of public housing, off welfare, and toward independence. Jane Addams Hull House Association, one of Chicago’s most prominent social service agencies, was promised $3 million and given total freedom in designing and implementing the program. Soon after the announcement of the program, which was named Families for a Better Life, over 30,000 persons from around the country responded and requested assistance. In the end, only 4,000 of these respondents met the program’s requirement of living in public housing and, from this group, 1,600 families formally applied to the program. Amid all the early excitement about the program, Ms. Winfrey was chosen as one of seven Chicagoans of the year for her charitable work in the city. There were even rumors and questions about whether Ms. Winfrey would seek public office after her foray into social policy.
By 1996, however, the program had spent $1.3 million and had managed to serve only seven families. Of the five families who completed the program (two families dropped out), three succeeded in moving out of public housing. The employment gains were equally disappointing: Two parents found full-time work, two worked part-time, and one parent was enrolled in a training program. Of course, these modest outcomes do not justify the program costs, which amounted to over $250,000 for each of the five families. What exactly went wrong? It is difficult to say for certain, although one news report noted that the program spent too much time screening potential participants and then lost time by retooling the comprehensive counseling and training program in mid-course.
Today, the phone lines for Families for a Better Life are disconnected, the program is officially closed, and Jane Addams Hull House Association has no plans to resuscitate it. The problem with Families for a Better Life was not that it failed, but that it was a wholly unconstructive failure that produced no systematic knowledge about the transition from welfare to work. No public narrative report on the program was ever written and no systematic outside evaluation was ever made available to other funders and interested parties. It is particularly disappointing that this recent project proved so unconstructive, given the sizable resources that were invested and the fact that information on what works and what does not in the area of welfare-to-work is now more urgently needed than ever before.
The Rockefeller and Winfrey cases starkly highlight how failure in philanthropy can mean two very different things. While there should be no stigma attached to constructive failures that build knowledge (the way the Rockefeller Foundation’s program has done), heavily funded initiatives that end in unconstructive failures like Ms. Winfrey’s deserve all the criticism that they presently receive and then some.
Recovering Philanthropy’s Diagnostic Function
There is at least one good reason for foundations to stop avoiding the subject of failure in philanthropy: If they are unwilling to learn from the past, foundations will be forced to repeat the philanthropic errors of their predecessors over and over again. As a first step, foundations need to expand their horizons beyond the never-ending search for that “perfect new pilot program” that will serve as a model for “replication” and “scaling up.” Instead of believing that no idea is a good idea unless it is “original” and can be “leveraged,” grantmakers should study what has been learned in the past, which programs succeeded and which programs failed, then seek to add to this store of knowledge by evaluating and openly communicating their experiences. This more incremental and diagnostic approach to grantmaking is not glamorous, but it will likely prove effective in the long run, as ideas are tested, evaluated, improved, and refined.
If one accepts the notion that both success and failure are part of philanthropy, then the challenge for foundations becomes one of creating an environment in which honest and productive discussions of program outcomes can take place and where knowledge can be built incrementally and cumulatively. Key to any change in attitudes among foundation staff about failure in philanthropy is replacing the lingering stigma with an appreciation and respect for constructive failure. For such a change to occur, the diagnostic function of philanthropy must be recovered. At the core of this approach are sound experimental evaluations which carefully track and measure program impact. Of course, not all programs are complex and significant enough to justify the expense of experimental research designs like the one undertaken by Rockefeller in the 1980s. However, when a program addresses a significant social problem that is likely to be the focus of ongoing public and private intervention, a sound evaluation report has the potential to have lasting value — even if the program studied falls short of its goals.
Philanthropy’s diagnostic function is far less glamorous than the triumphant — though often prematurely optimistic — announcement of yet another new break-the-mold pilot project. To be sure, recovering philanthropy’s more humble diagnostic mission will be difficult given changes within the field over the past decades. The rise of philanthropic careers and increasingly close ties between donors and recipients now make it very hard for an open discussion of failure to take place. One thing is certain, however. The costs of failing to document, communicate, and appreciate failed grants will surely prove more costly in the long run than any grantmaker now neglecting this task can possibly foresee.
Peter Frumkin is an assistant professor of public policy at Harvard University’s Kennedy School of Government.