(Michael Remaley is the director of Public Policy Communicators NYC and president of HAMILL REMALEY breakthrough communications. This post originally appeared on the Foundation Center's Transparency Talk blog.)
"If you hit the bull's eye every time,
you've set the target too close."
I thought of this, one of my favorite aphorisms, at the Communications Network's annual conference last September when the Hewlett Foundation's communications director Eric Brown talked about his organization's "failed grantmaking" contest. Hewlett's smart internal exercise forces each department to name one grant from its portfolio that did not meet expectations, think through and explain what went wrong, and help the entire organization learn from its failure.
This is a learning exercise that more foundations should consider adopting. But more than that, it is an important example of how Hewlett's leadership has set the tone for candor about the unavoidable truth of philanthropic experimentation: failure is part of the equation.
It is no coincidence that Hewlett is also one of the few foundations that has talked publicly about initiatives that didn't live up to expectations. It is also no coincidence that Hewlett's profile on Glasspockets gives a good indication of its commitment to transparency. I would assert that Hewlett's reputation for being one of the most innovative, thoughtful, and effective foundations is directly related to its transparency, willingness to publicly question its strategies, and forthrightness in discussing the limitations of its successes. And that reputation further enhances its ability to exert influence and make change.
The hard sciences learned the importance of sharing candid assessments of "failed" experiments centuries ago. In fact, scientists seem to treasure results that do not meet expected outcomes even more highly than those that confirm what is already believed to be true.
I am hardly the first person to call upon foundations to talk more openly about failure, experimentation, and unexpected outcomes. (See list below.) Hewlett's Paul Brest seems to have really kickstarted the conversation in 2007 by writing and talking about his foundation's experiences. That was followed by Robert Giloth and Susan Gewirtz's seminal 2008 piece in Foundation Review, "Philanthropy and Mistakes: An Untapped Resource." Many others, including Bob Hughes, Larry Blumenthal, Edward Pauly, Grant Oliphant, and Sean Stannard-Stockton, have added important insights about the need for foundations to be more open about their lessons learned. The conversation about failure and experimentation seemed to grow and deepen over the past three years. So you might think that foundations would be making major changes in how they communicate about failure. You would be wrong.
Foundations give a lot of lip service to supporting "experimentation" in social sciences. But you almost never hear them talking about outcomes that failed to meet expectations and, even more rarely, those that call their basic strategies into question. If foundations want to be real leaders in advancing social change, they must move past the endless happy-talk that makes every grant sound like a success. Instead, they should use their Web sites to detail how they are evaluating their work and what they've learned from unexpected outcomes.
A foundation sharing its experiences with grants gone wrong is still very much the exception. Anyone who is on the receiving end of foundation annual reports and newsletters knows this is true. But to substantiate my assertion, I decided to do a little systematic poking around.
I figured the twenty-one largest supporters of the Center for Effective Philanthropy (most of which are also supporters of Grantmakers for Effective Organizations) would be the foundations most attuned to the value of self-reflection, evaluation, and sharing results that defy expectations, and also those that would have budgets big enough to support substantial evaluation efforts. I spent many hours exploring the nooks of crannies of these foundations' Web sites. I looked at numerous publications and evaluation sections of the sites, and I searched each site on the terms failure, failed, unmet expectations, unmet objective, unmet goal, experimentation, mistake, lessons learned, and assessment.
What I found was that few foundations make it easy to learn from projects that didn't go as spectacularly as planned, let alone talk frankly about what has been learned from the shortcomings of foundation strategy or execution. Many of the twenty-one foundations I examined made no mention at all of evaluation criteria and organizational outcomes, even though their association with CEP and GEO implies that they demand that kind of forthrightness from grantees. The majority of the foundation sites I examined had a few project evaluation reports scattered among other foundation-supported research -- and many of those evaluation reports were laudatory with pablum like "real collaboration is a challenge" tacked on at the end.
Some of the best exceptions were Robert Wood Johnson Foundation, the William and Flora Hewlett Foundation, and the Wallace Foundation. Each of those foundations not only makes it easy to find many project evaluations that are balanced in presenting positive and negative outcomes along with what was learned through the process, but also present self-critical examinations of foundation strategy and progress as whole. It is also not a coincidence that each of those foundations' profiles on Glasspockets indicates a commitment to transparency demonstrated by making public an assessment of overall foundation performance.
But perhaps the best example -- the foundation that gets the Gold Star for Succeeding in Failing -- is the James Irvine Foundation. The evaluation section of its site describes their approach to evaluating grantee success and links to all of its individual evaluations of initiatives. It also links to a Foundation Assessment section that has foundation annual progress reports for the last four years. These progress reports are exceptionally detailed and well-documented, as well as frank about successes and failures. Irvine has also produced "Insights: Lessons Learned" publications with candid assessments of their experiences with collaborations and other grantmaking practices. A search of the Irvine site on "lessons learned" produces lots of useful and interesting evaluative information and insightful critical analysis.
We are all members of the social science community and contributors to the social experiment that is American philanthropy. We now have enough examples of foundations talking humbly about their shortcomings to know that such candor only accelerates social progress and enhances the reputations of those philanthropic leaders. We've seen no evidence that talking forthrightly about the real-world circumstances leading to failure damages nonprofits or the foundations involved, so I wonder why foundations seem so reluctant to take on this leadership role.
What has your organization learned from experiments that didn't meet expectations?
A Chronology of the Dialogue on Failure and Experimentation in Philanthropy
Center for Effective Philanthropy (unattributed). "Indicators of Effectiveness: A Call for Foundations: Understanding and Improving Foundation Performance." 2002.
Paul Brest. "Evaluating Our Work. Hard Lessons about Philanthropy & Community Change," Commentary on William and Flora Hewlett Foundation Web site. March 2007.
Paul Brest. "Hard Lessons about Philanthropy & Community Change: Reflections on The Neighborhood Improvement Initiative." March 2007.
Stephanie Strom. "Foundations Find Benefits in Facing Up to Failures." The New York Times. July 26, 2007.
Paul Brest and James E. Canales. "Let's Stop Reinventing Potholes." The Chronicle of Philanthropy. August 9, 2007.
Sean Stannard-Stockton. "The Poster Child for Failure in Philanthropy." Stanford Social Innovation Review blog. May 14, 2008.
Robert Giloth, Ph.D., and Susan Gewirtz, Annie E. Casey Foundation. "Philanthropy and Mistakes: An Untapped Resource." Foundation Review, September 2008.
Maisie O'Flanagan, McKinsey & Company; Jacob Harold and Paul Brest, William and Flora Hewlett Foundation. "The Nonprofit Marketplace: Bridging the Information Gap in Philanthropy." 2008.
Sean Stannard-Stockton. "Anatomy of a Failed Grant." Tactical Philanthropy Advisors blog. March 25, 2009.
Grant Oliphant with Susan Herr (online video interview). "What's the Upside of Philanthropic Failure?" The Communications Network. December 2009.
Grantmakers for Effective Organizations and Council on Foundations (unattributed). "Evaluation in Philanthropy: Perspectives from the Field." December 15, 2009.
Larry Blumenthal. "A Helpful Guide to Failure in Philanthropy. Use Carefully." Philanthropy News Digest, Commentary & Opinion. January 7, 2010.
Bob Hughes. "Can Failure Be the Key to Foundation Effectiveness?" Center for Effective Philanthropy blog. January 11, 2010.
Grantmakers in Health (unattributed). "Taking Risks at a Critical Time." Essays written specifically for the 2010 GIH annual meeting. March 2010.
Edward Pauly. "Philanthropy with Impact: A Guide to Evaluative Thinking for Foundations and Donors." A guide published by Forum for Active Philanthropy. 2010.
Robert G. Hughes. "The Role of Failure in Philanthropic Learning." Book chapter In: To Improve Health and Health Care XIII, pp.93-106. Publisher: Jossey-Bass. 2010.
Patricia A. Patrizi. "Death Is Certain, Strategy Isn't: Assessing RWJF's End-of-Life Grantmaking." Published in New Directions for Evaluation, by Wiley Online Library. Volume 2010, Issue 128, pages 47–68, Winter 2010.
Ellie Buteau. "A higher bar for transparency, accountability." Philanthropy Journal. September 14, 2010.
-- Michael Remaley