Advancing Social Media Measurement for Foundations: A Re-Cap (Part Two)

(Beth Kanter is the author of Beth's Blog, one of the longest running and most popular blogs in the social sector, and co-author of the acclaimed books The Networked Nonprofit [J. Wiley & Sons, 2010] and Measuring the Networked Nonprofit: Using Data to Change the World, [J. Wiley & Sons, 2013]. A version of this post originally appeared on our Transparency Talk blog.)

In my post yesterday, I discussed what working transparently means as well as the benefits of transparency, and I left you hanging with a promise to discuss how to measure your foundation's transparency return on investment.

Transparency is like any other measurement challenge; you first need to be clear about what you are measuring. For our purposes, the first is assessing the impact of a change in transparency on your organization by measuring the change in the benefits created by being more transparent -- in this case greater organizational efficiency and/or constituent/stakeholder trust. And the second is evaluating how transparent your organization actually is.

How to Measure Improvements in Organizational Efficiency

To measure changes in organizational efficiency you first need to have a chat with your accounting and operations departments to figure out what the organization is tracking in terms of efficiency metrics. If no one in the organization is tracking efficiency metrics, chances are someone in one of those departments knows how to do it and can help you. Typical efficiency metrics include:

• percent reduction in response time from inquiry to satisfied resolution;
• percent reduction in staff hours responding to queries;
• percent increase in satisfaction and knowledge of employees.

The benefits of increased transparency can also be quantified by conducting a relationship survey.

How to Measure Improvements in Trust

Several studies have shown that the more transparent people perceive an organization to be, the more likely they are to trust the organization. Indeed, the willingness to be open and transparent was found to be more important than competence. In other words, people care more about an organization's willingness to be open and transparent than whether it is competent enough to do what it says it is going to do!

How to Measure Your Own Transparency

While the field of transparency measurement is relatively new, thanks to the work of Brad Rawlins and others there are established techniques to quantify the transparency of any organization.

Measurements of transparency typically look at four separate but equal components:

Participation. The organization asks for feedback, involves others, takes the time to listen, and is prompt in responding to requests for information.

Substance. The organization provides information that is truthful, complete, easy to understand, and reliable.

Accountability. The organization is forthcoming with bad news, admits mistakes, and provides both sides of a controversy.

Absence of secrecy. The organization doesn't leave out important but potentially damaging details, doesn't obfuscate its data with jargon or confusion, and isn't slow to provide data (or only provides it when required).

In Measuring the Networked Nonprofit, we recommended using questions based on the work of Dr. Rawlins or the Who Has Glass Pockets? assessment, which your foundation can use to audit its own transparency. Typically a survey like this is administered either as a group discussion or as a written survey followed by a group discussion of the results. Some areas of discussion that should emerge as a result of such an exercise are outlined below:

Is our organization participative?

1. Do we involve stakeholders to help identify the information we need?
2. Do we ask the opinions of stakeholders before making decisions?
3. Do we take the time with stakeholders to understand who we are and what we need?

Do we provide substantial information?

1. Do we provide detailed information to stakeholders?
2. Do we make it easy to find the information stakeholders need?
3. Are we prompt when responding to requests for information from stakeholders?
4. Are we forthcoming with information that might be damaging to the organization?
5. Do we provide information that can be compared to industry standards?
6. Do we present more than one side of controversial issues?

Are we accountable?

1. Do we provide information in a timely fashion to stakeholders?
2. Do we provide information that is relevant to stakeholders?
3. Do we provide information that could be verified by an outside source?
4. Do we provide information that can be compared to previous performance?
5. Do we provide information that is complete?
6. Do we provide information that is easy for stakeholders to understand?
7. Do we provide accurate information to stakeholders?
8. Do we provide information that is reliable?
9. Do we present information in language that is clear?
10. Are we open to criticism?
11. Do we freely admit when we make mistakes?

Are we secretive?

1. Do we provide only part of the story to stakeholders?
2. Do we leave out important details in the information we provide to stakeholders?
3. Do we provide information that is full of jargon and technical language that is confusing to people?
4. Do we blame outside factors that may have contributed to the outcome when reporting bad news?
5. Do we provide information that is intentionally written in a way to make it difficult to understand?
6. Are we slow to provide information to stakeholders?
7. Do we only disclose info when it is required?
8. Do we only disclose “good” news?

Measuring Stakeholder Perceptions

The second part of transparency measurement is assessing whether your stakeholders perceive your organization as being transparent. To measure that, you need to ask your stakeholders whether they agree or disagree with the following statements. (These questions also can be added to grantee perception reports or other surveys that are routinely conducted by foundations.)

1. The organization wants to understand how its decisions affect people like me.
2. The organization provides information that is useful to people like me for making informed decisions.
3. I think it is important to watch this organization closely so that it does not take advantage of people like me.
4. The organization wants to be accountable to people like me for its actions.
5. The organization wants people like me to know what it is doing and why it is doing it.
6. This organization asks for feedback from people like me about the quality of its information.
7. This organization involves people like me to help identify the information I need.
8. Provides detailed information to people like me.
9. Makes it easy to find the information people like me need.
10. Asks the opinions of people like me before making decisions.

The transition to transparency may be discomfiting, but the benefits of inviting people in and sharing in your strategy development far outweigh the potential downsides. Imagine how much stronger responses, feedback, and suggestions from your network will make your organization -- and how exciting it will feel to share your work with even more people. And with a measurement strategy in place, you'll know for sure that your organization is walking the transparency talk and changing for the better.

-- Beth Kanter