No Pain, No Gain
February 28, 2012
(Larry McGill is vice president for research at the the Foundation Center. A version of this post appeared on the center's Transparency Talk blog.)
The Foundation Center is the primary data collection, analysis, and reporting agency for the field of U.S. institutional philanthropy. Each year, we analyze more than 150,000 grants awarded by about fifteen hundred of the country's largest and most influential foundations and load them into our master database, which now comprises more than three million grants awarded over the past twenty years.
Every year, our database is accessed by thousands of grant seekers looking for funding to do their work. It also underlies all the research reports written by the Foundation Center, tracking trends in the field over time.
But here's the thing -- our data aren't perfect. And we want you to know that.
Moreover, despite the limitations of our data, we fully intend to keep publishing reports documenting and explaining the work of U.S. foundations. Even if what we produce sometimes comes back to bite us.
Case in point: We have published a number of reports in recent years on issues related to diversity in philanthropy. Not everyone is satisfied with the findings we report, regardless of the caveats we issue about the limitations of the available information. But we issue the reports anyway, because there is growing demand for this type of information.
Why would we issue reports based on imperfect data? Because it is the only way the data will get better.
To build our grants database, we have relied for most of our 55-year existence on Forms 990 and 990-PF, which are publicly filed with the IRS by foundations and grantmaking public charities. We transcribe verbatim the information provided by foundations on these forms describing the purpose of each grant awarded during a given year. Sometimes this information is richly descriptive, sometimes it's sketchy, often it's nonexistent.
In recent years, we have developed a platform that allows foundations to send their grants information directly to the center through an electronic reporting system. With more than seven hundred foundations participating, this has significantly improved both the range and depth of information available for analysis. But the quality of information we receive still varies a great deal from foundation to foundation.
As we confront the limitations of the information available to us, we have to make a choice about how best to spend our resources as we continue to build a database that describes the work of U.S. foundations. We can accept the limitations of the existing information and try to collect data on the work of as many foundations as possible each year. Or we could drastically limit the total number of foundations and grants we analyze and focus instead on trying to obtain as much additional information as we can about each grant awarded by those foundations (e.g., about beneficiary populations, geographic area served, etc.). The former strategy, the one we've chosen, allows us to add more than 150,000 grants to our database each year. The latter would allow us to add only about one-tenth that number. We believe we owe it to the hundreds of thousands of individuals who use our grants database to make it as comprehensive as possible.
Adopting that strategy means we have to live with some data limitations when doing research based on the information we have. But what is not generally understood or appreciated is that this is simply a fact of life regarding all research, at all times, and in all places. Any research study that does not come with caveats or explicitly stated limitations is not an honest piece of research.
In our reports, we use colors, italics, boldface, boxes, sidebars, methodology sections, and other strategies to make people aware of both the findings and the limitations of our research. Of course, it's never enough. Findings still have a way of disconnecting themselves from the methodologies used to generate them.
But that's fine -- as long as it leads to good faith conversations about what we think we know, what we don't know, and what we need to know. As the primary data collection agency for the field, the Foundation Center is committed to doing the best it can to answer the questions that people are asking about institutional philanthropy. The need to know will not go away, and we -- all of us who care about philanthropy -- must do whatever we can to ensure we have the kinds of data that enable us to meet this need.
Do you have thoughts about how we can collectively improve the quality of data available to the field? Let us know. Like all fields of endeavor in the twenty-first century, philanthropy, to be effective, must operate from a solid base of knowledge that can only be built from reliable data on the issues and problems it seeks to address and the approaches it takes to make a difference.
-- Larry McGill