Hi, my name is Elena Pinzon O’Quinn and I am the National Learning and Evaluation Director at LIFT. I have been a nonprofit internal evaluator for most of my career. The nonprofits I have worked with engage in ongoing performance measurement for the goal of using data for continuous program improvement. Stakeholders need to see and use data often to achieve this goal, ideally. But it doesn’t always work out this way.
As evaluators, we spend much of our time developing research questions, determining data sources, working through data collection processes, and performing analyses. Sometimes, the most important parts of the process – sharing the data, communicating results, and determining next steps based on those results, do not receive adequate focus and time. We must look for simple and efficient ways to share and use data.
I implore internal and external evaluators alike to avoid the trap of perfectionism when determining how and when to share data with its intended audience. There is a time and a place for a perfectly polished evaluation report. Most of the time, however, our programs need more rapid access to information. Focus on user-friendliness, jargon free writing, effective data visualization, and putting together a product that won’t take too long to turn around. Getting data into stakeholder’s hands is key to building a culture of data for decision-making. Get into the habit of sharing data often.
Hot Tips
- Share your demographic and process or output data with stakeholders. No need to wait until that interim report is due or even until you have your baseline outcome survey analysis complete. Demographic data and process data serve an important purpose and begin to tell the story of your population and program.
- Doing a multiple time point survey and only have the baseline collected so far? Share the results anyway, even without the change over time. Similarly, baseline data tells a story.
- Share results even if you, as the evaluator, think the data is “dirty” or incomplete. Does this sound familiar? The team was supposed to survey 30 clients and only received 10 responses or 70% of respondents left one of the questions unanswered. You can include these caveats when sharing out. This also helps to start conversations about data quality and completeness, and can help with buy-in for data collection amongst program staff.
- Focus on effective data visualization that summarizes key takeaways to facilitate discussion and feedback. For example, make your chart titles descriptive with the key takeaway message (I learned this from Ann Emery).
- Keep your writing simple. Remove excess text and overly technical terms.
- Use creative and varied formats for presenting data, especially formats that reduce the time burden on you, the evaluator.
- Do away with the Word report altogether. Instead, facilitate a data walk.
- Consider real time dashboards if you are using data management software, and schedule reports to go out to stakeholders via email periodically, especially if there are regular data check-in meetings on your calendar.
- Keep the scope small and manageable. Bring the results of one question on a survey to your regularly scheduled program meeting and build in 10 minutes for discussion and next steps.
Don’t hesitate to get data into people’s hands. It will promote the culture you want to create and the more you do it, within reason, the better.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Hi Elena,
Thank you for your thoughtful post, I truly enjoyed reading it and found that it spoke quite loudly to me. I have only recently begun to venture into the field of program evaluation, but coming from a background in health care, I immediately understood the need for accurate data collection as well as decisions/recommendations based on a foundation of solid information.
Two things you spoke about really stuck out to me. You mentioned the idea of efficient communication between evaluators and program stakeholders. I am currently developing a rudimentary program evaluation for a training course I am a part of, and it has become clear to me that the individuals who make decisions and policy aren’t looking for perfection. As you stated, they want access to information in a timely manner and presented in a way that easily conveys a message.
You also touched on the idea of reporting results as they emerge, which can spur conversations of completeness and encourage buy-in. I think, in addition to this, the regular sharing of data as well as when staff see the impact of their reporting, it can lead to a sense of ownership about the program evaluation and its completeness. With active and regular involvement from program facilitators, they can begin to see how the evaluation is providing value to their program, and increase participation to ensure the most accurate results are reported.
Regards
Jordan Robertson
Hi Elena,
My name is Nicola James, and I am currently enrolled in Program Inquiry at Queen’s University. I enjoyed reading your article on ‘Building a Culture of Data for Decision Making’ and found it interesting as I’m completing a Program Evaluation Design and data collection is a key element.
It is just recently I read and discussed the topic of evaluation use, specifically to stakeholders using evaluation information and so it was easy to make connections with your article, emphasizing the importance of helping stakeholders to see and use data for continuous improvement.
Prior to starting Program inquiry, I was of the view that stakeholders would receive feedback regarding program evaluation at the end of evaluation activity. However, this is not the case and now being more knowledgeable I realized that feedback can be provided to stakeholders at any stay of a program evaluation process. You clearly stated that some programs’ needs demand feedback in real-time and for that reason, evaluators need not wait until they have completed or perfected the report to communicate the results. Stakeholders and other program staff need evaluation information along the way to modify, make changes or improve aspects of the program in real time.
I appreciate how you pointed out that sometimes the most important parts of the evaluation process- sharing data, communicating results and next steps based on those results are not given sufficient attention and time. This is usually overshadowed by spending too much time on developing research questions, determining data sources, working through data collection processes and performance analysis. Even though I think the later activities are crucial in accumulating reliable and credible data and can be time consuming, care must be given for quality data. The way in which data is communicated plays a crucial role in getting stakeholders to understand what is being presented. I agree that methods used should be user-friendly, written in simple and clear language, and with appropriate visualization methods.
I found all the Hot Tips you suggested to be informative and can be effective strategies in getting program evaluation data in the hands of stakeholders in order to foster a culture of using data in decision-making. Thanks for sharing.
…. Nicola