Hi there! We’re Anjie Rosga, Director, and Natalie Blackmur, Communications Coordinator, at Informing Change, a strategic consulting firm dedicated to increasing effectiveness and impact in the nonprofit and philanthropic sectors. In working with clients large and small, we’ve found that organizations are in a better position to learn if they take the time to prepare and build their capacity to evaluate. To facilitate this process, Informing Change developed the Evaluation Capacity Diagnostic Tool to measure an organization’s readiness to take on evaluation.
Rad Resource: The extent to which evaluation translates into continuous learning is in large part dependent on the organizational culture and level of evaluation experience among staff. These are the two primary categories—themselves divided into six smaller areas of capacity—in the Evaluation Capacity Diagnostic Tool. The tool is a self-assessment survey that organizations can use on their own, in preparation for working with an external evaluator or alongside an external evaluator. A lower score indicates that an organization should, for example, focus on developing outcomes and indicators, track a few key measures or develop simple data collection forms to use over time. The higher the score, the higher the evaluation capacity; staff may then be able to collect more types and a greater volume of data, design more sophisticated assessments, as well as integrate and commit to making changes based on lessons learned.
However, there’s more to the Evaluation Capacity Diagnostic Tool than the summary score. It is a powerful way to catalyze a collective discussion and raise awareness about evaluation. Taking stock and sharing individuals’ perceptions of their organization’s capacity can jumpstart the process of building a culture that’s ready to evaluate and implement learnings.
Hot Tip: Make sure everyone is on the same page. Especially if an organization is inexperienced in evaluation, it’s important to discuss the vocabulary in the Tool and how it compares with individuals’ own definitions.
Hot Tip: Assessing evaluation capacity can be a tough sell. Organizations come to us because they’ve made the decision to begin evaluation, but gauging their capacity to do so can feel like a setback. To get organizations on board, we frame evaluation capacity as an investment in building a learning culture and the infrastructure that can make the most of even relatively limited data collection efforts.
We love to hear from folks who have implemented or reviewed the tool! Feel free to reach out to us at news@informingchange.com.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Developing evaluation capacity within an organization is instrumental to measuring organization preparedness. Informing Change has a prepackaged survey to determine the level of readiness for evaluation by each organization they consult. If the organization has a lower score, then it should focus on developing outcome indicators. If it scores high, then the organization could move on to more sophisticated assessments. A great tip to help sell evaluations to organizations is cast it as an investment in the organizations cultural development.
On a personal note, our university is still in its infancy as far its research and evaluation center. This diagnostic tool may be useful for not only a current project, but for future ones as well. I am saving a digital copy to present to the professor in charge of the research and evaluation projects to see if this is something that can be of use. Knowing how engaged or open to change an organization is an important factor prior to beginning an evaluation. This component is extremely useful to know, especially if (as in our case) you know that there may be those within an organization that are wary of change.
Brady, thank you for you comment! We’re always curious to hear about the many ways organizations use and apply the evaluation capacity tool. Please feel free to drop us a line to tell us about how it went for your university.