August 28, 2014
Last month, The New York Times "reviewed" the still-in-development Participant Media Index, which is designed to measure the impact and engagement of social issue documentaries. Anyone in the nonprofit world knows that impact and engagement are the buzzwords du jour. More than a passing fad, however, impact evaluation is serious business – one that many of us in the social change realm grapple with every day.
This has not always been the case in the eighteen years I've worked in the sector. Funders have increasingly driven the trend, asking grantees to not just monitor our progress, but also to develop innovative ways to quantify that progress and share our learnings more broadly. In this way, the nonprofit world is catching up with the fields of medicine, psychology and education – all of which have embraced "evidence-based practice" over the past two decades.
This is mostly a positive development. By laying out concrete objectives and outcomes at the start of a grant (in the proposal), organizations are forced to think more strategically and are held accountable for delivering on their promises. The most forward-thinking funders understand the risk inherent in our work – that social investments, like those in business, are not guaranteed to succeed, and that organizations can learn as much from their failures as their achievements. Yet careful planning (yes, even the ubiquitous logic framework) can help increase the odds that we uphold our end of the bargain: To ensure that precious resources are used to successfully mobilize positive social change.
WITNESS has always been considered an innovator in impact evaluation, starting in the mid-2000s with our groundbreaking Performance Evaluation Dashboard, and including a massive effort we launched recently to overhaul our program. Indeed, we are constantly looking for new ways to ensure we maximize our performance and learnings. But this approach is not without its challenges. Human rights advocacy is notoriously difficult to measure, change is often incremental, and ultimate "wins" can take years to achieve. Video advocacy is even more complex, since video is a complementary tool, intended to corroborate other, more traditional forms of documentation.
(A point system for tracking Outputs, Outcomes and Impacts from WITNESS’ first Performance Dashboard covering our fiscal year 2006.)