« Weekend Link Roundup (March 27 - 28, 2010) | Main | Announcement: Foundation Center Seeks Host Orgs for Cooperating Collections Network »

P/PV President Nadya Shmavonian on Evaluative Learning

April 01, 2010

Evaluation in a nonprofit context -- what it is and isn't; who should bear the costs; and whether it's even possible -- remains a much-debated topic. Earlier today, for example, Charity Navigator president Ken Berger responded to skeptics of his organization's plans to revamp its oft-criticized rating system. You might not agree with all of the arguments Berger puts forward in his post, but most people would find it hard to disagree with his contention that a charity ought to be able to answer the following questions:

  1. Is it using outcomes in the design, management, and measurement of its efforts?
  2. Are the targets it sets "reasonable" outcomes? In other words, are they, at a minimum, meaningful, sustainable, and verifiable?
  3. Is the organization achieving those outcomes?

Evaluation also is the subject of the latest Mott Conversation between the Mott Foundation's Duane Elling and Nadya K. Shmavonian, president of Public/Private Ventures, a nonprofit social research and policy organization based in Philadelphia, about the critical role of program evaluation in designing, testing, and sharing high-quality solutions that help low-income communities create meaningful and lasting change. Here's an excerpt:

Mott: How can the nonprofit sector best approach evaluation activities to ensure that they lead to genuine learning and innovation?

Nadya Shmavonian: Evaluation is not a thing so much as it's a way of thinking. Frankly, I believe if you are not asking what you want to learn from a given intervention or program at the beginning, then it isn't necessarily structured as strategically or thoughtfully as it could be. And if it’s not a shared value with the grantees who understand -- who are really trying to advance their own knowledge and best practices -- then I think the attention, use and consumption of evaluative information can be somewhat limited, and there is not the imagination for the next time.

In a place where you have continuity of programs, where you know that you're going to be in it long term, where you’re field-building and advancing, there's a sense of history and of how you can work evaluation in at the front end, but also at the back end. I think there are too many situations where foundations and their grantees have not reaped the collective benefit at the back end to reinforce that.

But, certainly, if you don't do evaluation, then there are enormous wasted resources. You don't ever systematically build an evidence base. What I also think is quite important is that in a good evaluation you can pull apart the component elements -- and the data will surprise you in many instances -- and then you can ask the next generation questions about, "Okay, so why did that work?" and think about the next intervention to advance that knowledge.

Particularly in these times, which are so resource-constrained, we should be focusing on these issues and getting smarter about what it means to evaluate, grow and potentially even replicate programs. But I worry that in the press for resources, we are actually limiting our targeting of thoughtful evaluative inquiry....

To view a video of the conversation, click here.

Or click here to read an edited transcript of the conversation.

Where do you stand? Is Ken Berger right in asserting that every charity should use meaningful, verifiable outcomes in the design, management, and measurement of its efforts? Do you agree with Shmavonian that a failure to do evaluation invariably results in wasted resources? And which nonprofits are using evaluation effectively to advance their mission and/or build a field of practice/knowledge? Use the comments section below to share your thoughts....

-- Mitch Nauffts

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Shmavonian is dead on: If you don't know where you're going, how will you know you got there? Much less have the ability to share your lessons with others. Measuring expectations is just as critical as measuring experiences.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Contributors

Quote of the Week

  • "[L]et me assert my firm belief that the only thing we have to fear is...fear itself — nameless, unreasoning, unjustified terror which paralyzes needed efforts to convert retreat into advance...."


    — Franklin D. Roosevelt, 32nd president of the United States

Subscribe to Philantopic

Contributors

Guest Contributors

  • Laura Cronin
  • Derrick Feldmann
  • Thaler Pekar
  • Kathryn Pyle
  • Nick Scott
  • Allison Shirk

Tweets from @PNDBLOG

Follow us »

Archives

Tags