Hello! We’re Bryan Higham, Innovation Specialist and former Combat Stress Recovery Specialist, and Amanda Peterson, Researcher, with Wounded Warrior Project® (WWP). WWP is a nonprofit veterans service organization that serves veterans and service members injured as a result of their service following Sept. 11, 2001. Our organization is headquartered in Jacksonville, Florida and we have offices and field staff across the country and in Germany.
We utilized a data party to explore ways to improve outcomes for Project Odyssey®, a 12-week mental health program that teaches warriors techniques to manage post-traumatic stress disorder (PTSD). The data party brought together select Combat Stress Recovery teammates who deliver the program, program leadership, and evaluators, and was facilitated by a member of our Innovation team. We successfully analyzed past outcomes and identified short- and long-term opportunities for improvement that would allow us to better serve wounded warriors. But how do we properly communicate the findings and proposed changes to WWP’s Project Odyssey team so they get onboard?
When developing a communication plan for the Project Odyssey team, we decided to do things a little differently to help with change management. Instead of presenting the changes as fact and then providing the data behind them, the data party participants traveled to regional WWP offices, presented the findings, and shared how they needed to drive change for program outcomes to improve. All program teammates also had an opportunity to ask questions and provide feedback, allowing them to read the situation for themselves and provide input.
While this approach took more time on the front end to produce actionable solutions, it saved time on the back end for implementation. Program teammates understood the reasons for the proposed changes and felt like they had a voice, and as a result supported the changes in their workflow processes. By slowing down the process, and placing the data at the forefront, we helped mitigate any change management issues and arrived at a more concrete solution.
Lessons Learned: When implementing process changes, allow program delivery staff to see and explore the data for themselves. Give them an opportunity to ask questions and provide input on how the data is informing change.
The American Evaluation Association is celebrating SEA Affiliate Week with our colleagues in the Southeast Evaluation Association. All of the blog contributions this week come from our SEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Bryan and Amanda,
I enjoyed reading your article and your perspective on program evaluation findings. Your idea of hosting a “data party” to explore ways to improve the outcomes of your mental health program, Project Odyssey, is an excellent idea! I am intrigued about the process of a “data party” and the opportunity they have to build community and encourage stakeholders to be “on board” with evaluation findings and proposed changes.
I am completing my Professional Master of Education at Queen’s University and am currently enrolled in a course involving Program Inquiry and Evaluation. The course explores the skills that are necessary for program evaluation to ensure that we can positively impact our own program context through meaningful, systemic inquiry. Your idea of a “data party” connects to several aspects that we have been exploring in our course, specifically the collaborative process of evaluation and involving stakeholders in the stages of the evaluation process.
Data parties allow stakeholders to come together and interpret what the data means and the implications for action. Data parties allow there to be a “continuous information dialogue between evaluators and program stakeholders who share the responsibility for generating, transmitting, and consuming evaluation information” (Shulha & Cousins, p.197, 1997). By doing so, it can improve the quality of the interpretation by bringing additional information that can be used to interpret evaluation data, supporting dialogue across diverse perspectives about the credibility of the data, and building support for using the findings. Ultimately, data parties are an interactive way to build relationships amongst stakeholders and allow them to be deeply involved in the evaluation process, which makes it more likely that the data will be effectively applied and used by a program.
As a high school teacher, I grapple with how we can actively engage stakeholders, whether it is within our school community or the larger community, to understand data, especially when the language in evaluations can be confusing and convoluted to those who are not familiar with it. Your example of a “data party” provides me with a tangible way to make data accessible and less intimidating for a diverse group of people. Some of the processes that can be used at a “data party” to communicate evaluation findings and allow the stakeholders to see and explore the data for themselves is gallery walks, World Cafes, and data dashboards. Each of these activities allows the stakeholders to digest the evaluation findings in an accessible format, ask questions, and provide input or feedback. By doing so, it allows the stakeholders to understand the reasons for the proposed changes and feel like they have a voice, which in turn results in increased support for the changes.