Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hello. From 2016 to 2023, Sharon Twitty, Natalie Lenhart, and I (Paul St Roseman) have implemented an evaluation design that incorporates empowerment evaluation, development evaluation, and evaluation capacity building. This blog series, which we call Walking Our Talk, presents the process in four parts:
- Laying the Foundation – The Rise of The Logic Model
- Using the Data – Emerging a Data Informed Evaluation Design through Peer Editing
- New Insights – Emerging A Collaborative Diagnostic Tool and Data Visualization
- Charting the Course of Data Use – Collaborative Data Analysis
Today’s post is Part 3 in the series.
New Insights Toward the Emergence of A Collaborative Diagnostic Tool
In spring of 2018, Sharon, Natalie, and I (ARCHES Evaluation Team) began to examine the insights gained from juxtaposing the Self-Assessment Survey with the Outcome Indicators of the ARCHES Logic Model. The Self-Assessment Survey was initially created as a reflective instrument for clients to independently gauge their progress. However, the Outcome Indicators, identified through the comparative analysis, were diagnostic by nature. These indicators were intended to inform how ARCHES could tailor its support for a collaborative’s development via its Just in Time Service Model.
In response to this finding, the Evaluation Team utilized these indicators to create the ARCHES Collaborative Diagnostic Tool (ACDT). By the fall of 2018, a preliminary version of the ACDT was ready, complete with an interactive data visualization feature that utilized mock data for demonstration purposes. This draft was presented to the Executive Leadership Team. During the presentation, the Evaluation Team highlighted the distinctions between the ACDT and original Self-Assessment Survey. We indicated how this new instrument was aligned with the existing logic model and the overarching evaluation strategy employed by ARCHES.
Lesson Learned: Movement Towards Real Time Interactive Visualizations Sets the Stage for Collaborative Data Analysis
The introduction of the ACDT also signaled a shift to interactive visualizations. A prototype dashboard had been created and efforts were underway to connect the ACDT Tool (compiled in Survey Monkey) to a Google sheet that could be updated in real time. With this function, the Evaluation Team began to augment efforts of collaborative evaluation capacity building with data science. The use of an interactive visualization would establish a foundation for a newly emerging evaluation capacity in ARCHES: Data Use through Collaborative Data Analysis.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.