Valuing What We Can Measure or Measuring What We Value?

(Michael Edwards is a leading expert on global civil society and the author of Small Change: Why Business Won't Save the World. This is the third in a series of posts in which he looks at different aspects of the Bellagio Initiative, an effort funded by the Rockefeller Foundation to produce a new framework for philanthropic and international development collaboration in pursuit of human well-being. Click here to read the first post and here to read the second. To read/download the Bellagio paper on which these posts are based --and from which the quotation below is taken -- click here.)

"There are no data or moral arguments to prove that one of these approaches is better than the others, so when interrogating the effectiveness of foundations the key question is always effective in doing what -- by what criteria, over which timescale, and who decides?..."

It is often said that we value what we can measure, but it might be more accurate to say that we measure what we value, and different people value very different things, an insight that is central to debates about the meaning of development and the need for broader indicators such as well-being. What, how and when we measure, how we use and interpret the results, and who decides these things are crucial questions in development -- and serve to highlight not just the methods, but the politics and power relations involved.

Do we measure long-term systems change as well as the numbers of lives saved and jobs created in a two- or three-year project cycle? Is democracy calculated as a cost or a benefit when we want to fix problems quickly and "get things done"? Should education be evaluated using test scores or the skills we need to live well together? As Robert Chambers, a modern icon of development research, puts it, "Whose reality counts?" Outside the science laboratory, all measurement means negotiation between different views and voices.

In the development sector, this debate has been accepted as legitimate for many years (though it remains to be resolved) -- perhaps because the exploitation of poor countries, their experience of colonialism, and/or their greater reliance on outside assistance from donor agencies have placed power at the heart of any meaningful analysis of what to do. It's a conversation that has generated whole new philosophies of knowledge and methods of participatory research to explore alongside randomized control trials and other quantitative techniques. Some have called this a shift from "using data" to "cultivating wisdom," by which they mean the ability to use information of different kinds to foster collective action and analysis, and to "co-produce" a vision of what matters most with those who are being "developed" at the center of that vision.

In philanthropy, by contrast, those who challenge the assumptions embedded in measurement and data-driven change are often labeled as idiots or reactionaries, guilty of the "hubris that measurement is the technocratic tool of the devil," as Fay Twersky from the Hewlett Foundation puts it in a recent op-ed in the Chronicle of Philanthropy. No one I know actually holds this view, and attacks like these do little except distract us from the need to ask tough questions about the influence of measurement over the kinds of work we prioritize for support.

The lesson to be learned from development is that opening ourselves to these questions is the best way to find a reasonable path forward -- a path that shifts the conversation away from the belief that measurement will tell us which are the "best" organizations or projects to support. That's a subjective evaluation at the best of times and empirically unsound, given that most of the data we collect are not strong enough to determine either causality or attribution. First and foremost, measurement is a contribution to learning and to sharing -- one input among many into well-informed but imperfect decision-making processes that continue to rely on human judgment.

In fact, arguments over measurement are not really about measurement at all. What separates enthusiasts and skeptics is not that one group wants more impact or believes in evaluation while the other group does not -- in my experience we all value those things -- but that we have substantive differences in how we approach these questions and which aspects of social change we are prepared to fight for and defend. That's why this debate generates so much heat -- and why it would be a much healthier conversation if we worked from that premise and were honest with each other about the dilemmas it creates.

Counting and measuring are only one part of a process of continuous, collective learning and communication that should be driven by the diverse needs and beliefs of all those who are working for and funding development and social change. There is no "best" approach or model or organization to be revealed by "the data," outside of a particular set of criteria and circumstances that are themselves contested.

Therefore, what we really need, as I'll explore in my fourth and final post, is an "ecosystem" of funders and funding styles and not a monoculture that uses measurement to drive resources to the causes which it, and it alone, finds of value.

-- Michael Edwards

Weekend Link Roundup (February 4-5, 2012)

Our weekly roundup of new and noteworthy posts from and about the nonprofit sector...

African Americans

BlackGivesBack's Tracey Webb gives a shout out to The Root, which earlier this week unveiled its 2012 list of Young Futurists -- African Americans under the age of 22 who are "not only achievers but also innovators in the worlds of green innovation, science and technology, arts and culture, social activism, and business enterprise."

Civil Society

"[Is there] a textbook definition of the common good?" asks Steven Fajon, a Case Foundation intern, in a guest post on the Social Citizens blog. The question occurred to Fajon after panelist Ai-Jen Poo, director of the National Domestic Workers Alliance, in an Independent Sector webinar titled "What the Heck Is the Common Good Anyway?" explained that the purpose of the question was to initiate a dialogue between communities in need. For his part, Fajon remains unconvinced. "[I]n the end," he writes, "the idea of the common good doesn't need to be an exact science –- it simply has to strike up a debate, just like it did in my mind."

Communications/Marketing

Has any public charity had a worse week, communcations-wise, than the one Komen for the Cure had this week? For a complete rundown of the missteps made by the Dallas-based organization as it tried to explain its decision to de-fund grantee Planned Parenthood, badly fumbled its response when the Internet erupted in outrage, then reversed itself a day later, see Kivi Leroux Miller's post on the "accidental rebranding" of Komen.

Global Health

On his Humanosphere blog, Tom Paulson shares a Twitter map made by Marc Smith, founder of the Social Media Research Foundation, that seems to show the global health community -- at least the portion of it "active" on Twitter -- to be fairly insular and uncommunicative. "It's mostly just an echoing of the Gates Foundation," says Smith. "There's not a lot of response, or engagement. Basically, it looks like people preaching to the choir." For his part, Paulson suggests the problem isn't so much Twitter or social media as it is "the passive...and sometimes simplistic nature of the narrative within the global health and development community itself. This is a community," Paulson adds,

devoted to -- and advertising itself as -- doing good. Humanitarians, in my experience, are exceptionally uncomfortable when forced to talk about things going bad. It also doesn't help with fund-raising, of course. But it's reality, and reality makes for better stories.

Health

In a post on the Breast Cancer Action blog, BCA executive director Karuna Jaggar writes: "The sad truth is that Komen’s willingness to restore funding to Planned Parenthood, while a victory for women who rely on those resources, will not end the epidemic....Not while Komen overemphasizes the value of mammography -- mammography will never stop cancer before it starts. Not while pinkwashing remains the status quo, and not while Komen allows companies to put pink ribbons on their carcinogenic products. Not while metastatic disease, which is what kills women, gets only 2% of research dollars in this country...."

Higher Education

The Nonprofit Quarterly's Rick Cohen takes a closer look at the tax-exempt sector's "1 percent" -- elite universities like Harvard, Yale, Princeton, and Stanford whose endowments, on average, racked up gains of 19 percent in fiscal year 2011.

International Aid/Development

In the first two installments of a four-part series here on PhilanTopic, Demos senior fellow Michael Edwards uses the paper he wrote on the Rockefeller Foundation-sponsored Bellagio Initiative to explore the relevance of "well-being" as a lens in development work and some of the lessons philanthropy can learn from that work. Coming up next: the role of metrics in that conversation.

On the White Courtesy Telephone blog, Greater New Orleans Foundation president and CEO Albert Ruesga weighs in on a disturbing article in the New York Times that exposed the unsafe working conditions at an Apple supplier in China. "It's not a given that being poor means having to work extremely hard in unsafe conditions to make very little,” writes Ruesga. “Fixing this in your supply chain needs to take priority over an on-time delivery of iPhones to eager consumers....”

Nonprofit Blogosphere

Hosting this month's Nonprofit Blog Carnival, Getting Attention's Nancy Schwartz shares a selection of posts that highlight the dreams of our fellow nonprofit bloggers.

Philanthropy

In a video reposted on the Humanosphere blog, Melinda Gates explains the purpose of the Gates Foundation's new visitor center in Seattle.

In the text of a speech reprinted on the Philanthropy Daily site, William Schambra, the director of the Hudson Institute's Bradley Center for Philanthropy and Civic Renewal, offers a typically provocative take on the difference between liberal and conservative philanthropy. "Granted, every progressive foundation has in mind a particular social ill or injustice it seeks to remedy, whether environmental degradation or poverty or racial inequity," writes Schambra. "But what do all these add up to?

Very seldom do liberal foundations tell us explicitly what their overriding political philosophy is, or how they understand the basic character of the American political order, and what’s worth preserving and what needs changing.

Instead, if you add up the range of specific problems on the agenda of liberal philanthropy, we’re left with a pretty depressing view of America -- an America beset by a wide range of social ills and injustices that desperately require philanthropic interventions of all sorts.

By contrast, conservative philanthropy tends to see beneath the problems America may be experiencing -- many of which are of course quite serious -- [and sees] a profoundly decent and good political order.

And the vision is to understand and preserve that order, the American regime of liberal democracy, in the face of powerful intellectual forces that have pulled us away from that commitment....

Social Enterprise

Is the social enterprise bubble about to burst? In an attempt to answer that question, the folks at GOOD asked half a dozen social entrepreneurs working in Africa what they think is driving the hype, where the sector is going, and what advice they have for those just starting off.

Social Media

And in a guest post on Beth’s Blog, Levi Strauss Foundation executive director Daniel Jae-Won Lee looks at how some human rights organizations are using social media to engage underserved communities on the issues that affect them. For example, the American Civil Liberties Union of Northern California developed MiACLU.org, an online Spanish-language platform that's designed to educate and engage Latinos in the region about immigration issues.

That's it for this week. What did we miss? Drop us a line at [email protected]!

-- The Editors

Can Philanthropy Put Humpty-Dumpty Back Together Again?

(Michael Edwards is a leading expert on global civil society and the author of Small Change: Why Business Won't Save the World. This is the second in a series of posts in which he looks at different aspects of the Bellagio Initiative, an effort funded by the Rockefeller Foundation to produce a new framework for philanthropic and international development collaboration in pursuit of human well-being. Click here to read the first post in Edwards' series, "Well-Being and Philanthropy," and here to read/download the Bellagio paper from which the quotation below is taken.)

"The more one disaggregates the components of well-being into smaller and more manageable pieces...the more each piece can be measured and controlled in order to improve returns....[B]ut the same pieces can't simply be re-arranged to the same effect in different contexts...."

One of my most important career lessons was taught to me by Sithembiso Nyoni, an activist in Zimbabwe. "No country in the world has developed itself through projects," she said, reflecting on the tendency of NGOs to fund their own small bits and pieces of development and hope that they add up to something more substantial over time.

Unfortunately, because the larger structures of society evolve organically rather than in assembly line fashion, they rarely do. The long and messy processes that drive our politics, culture change, and institution-building can neither be predicted nor controlled, especially if the outcome is something as complex as well-being. In that sense, development is poetry and foreign aid is prose.

Of course, a clear sense of purpose and direction is important to success. In contrast to Zimbabwe, that was one of the things that distinguished South Korea, Taiwan, and other societies that developed quickly after World War II. But as the experience of those countries also shows, clear goals were balanced by the flexibility to pursue them in lots of different ways as circumstances changed. "Evolution is always surprising," wrote Whole Earth Catalog creator Stewart Brand, "so make room for it. If you let things flourish you get a wild ride, but you also get sustainability." That's been true of all game-changing experiences in development right up to the Arab Spring.

Tension between "local ownership" and "outside intervention" has been woven through the history of development efforts for half a century or more, and it's unlikely to disappear as long as foreign aid is a tool in the foreign policy toolkit. Accountability to taxpayers, concerns about corruption, and a desire to show more "value for money" have all reinforced a project-by-project mentality that a decade ago seemed to be fading in favor of unrestricted support. Projects that are carefully planned and monitored do offer the prospect of more control, even if their influence over the deeper drivers of development is weak. On the plus side, there are many ways to leverage the impact of projects -- including through policy advocacy, capacity building, networking, and knowledge-creation -- so that they become more than small pieces in a jigsaw that can never be completed.

Nevertheless -- and here's the link to current trends in philanthropy -- the idea that successful projects can be "replicated" or "scaled-up" has, for reasons I don't entirely understand, become an article of faith. There are striking similarities between the Millennium Villages Project in sub-Saharan Africa, for example, and the Harlem Children's Zone in New York City. Both have received significant injections of resources in an effort to demonstrate that good results are possible without broader changes in the surrounding environment. The same goes for school-reform efforts and the Obama administration's Social Innovation Fund. Other schools and communities will learn from these experiences and follow a similar path, or so the theory goes.

Except that they don't, because the resources aren't there, or because the same innovation doesn't work or isn't valued when transported to another setting, or because those broader forces have a nasty habit of kicking the ladder away just when you least expect it.

If that's the case, why do pilot projects (or "policy experiments," as they are often referred to in the U.S.) continue to exercise such a powerful hold on the imagination of philanthropists? Perhaps it's because they accomplish other things that are important, like building support for their favored approaches, strengthening networks of people prepared to back them, and keeping alive the comforting thought that social progress can be removed from the influence of politics, economic restructuring, and social struggle.

Even if it could, it would be something of a pyrrhic victory, threatening to sacrifice long-term improvements in the infrastructure of problem-solving for short-term advances in services and other material indicators of success. As fifty years of trial and error in development make clear, however, investing in people's capacities to innovate is much more important than replicating any particular innovation.

But how we do measure success in that kind of scenario? And is there a way to knit all these different approaches together in a positive and constructive fashion? That's the subject of next week's posts.

-- Michael Edwards

Most Popular PhilanTopic Posts (January 2012)

These were the five most popular posts on PhilanTopic in January:

• Five Qs for...Mario Morino, Co-Founder/Chair, Venture Philanthropy Partners
• Foundation Leadership for a New Era
• 2011 Year in Review: What to Expect in 2012
• 2011 Year in Review: Foundations Bet Big on New Initiatives
• 2011 Year in Review: Microfinance Vows to Do Better

What have you been reading/watching/listening to lately that made you think? Feel free to share in the comments section....