My name is Mary Lou D’Allegro and I am the Vice-President for Academic Affairs at Columbia State Community College. I am responsible for the quality of our programs and the faculty who teach in those programs.
There is not much that is NOT evaluated in higher education. The double negative was intended. Yet, for all the resources expended to conduct these assessments and energy spent on creating and disseminating reports to stakeholders, what is the real value to those stakeholders and to higher education? For academic officers this is the trifecta of a balancing act: a.) practicality, b.) cost and c.) added value as a result of the evaluation.
Stakeholder input to determine evaluation priorities and that value. I am continually searching for evaluation designs that consider stakeholder identification and the impact of their input on determining those priorities.
Lessons Learned:
- Know your Audience. The dialogue with faculty about evaluation, the ultimate co-conspirators in evaluation, can be profoundly different than the discussion with students, typically the participants. Additionally, understand the roles that each type of stakeholder may assume before, during, and after an evaluation.
- Keep it Simple. For the sake of brevity, pragmatism, and sanity, design and implement the most straightforward and unpretentious evaluation possible. This also goes for the analysis; there is no need to burden stakeholders with unwieldy statistical analyses.
- Evaluation is only as good as the worst storyteller. Sharing results is important. Having those understood and interpreted accurately by relevant stakeholders is imperative. My go-to for designing easy to comprehend reports is Canva.
Hot Tips:
- Vote early, vote often and make those opportunities readily available. There is a plethora of free polling software available to survey anytime, anywhere. I have used do.com, Mentimeter. Colleagues have used Kahoot! and Zoho Survey that yielded great response.
- Use electronic Post-its
- AEA365 Tip a Day is my primary source for innovative evaluation designs that incorporate stakeholders into the evaluation. The AEA website can be navigated effortlessly. The website provides search capabilities to find what you missed or to locate additional information.
- Keep a list of Rad Resources and Hot Tips. I use Microsoft OneNote because I can quickly add pages to categorize and reorganize my RR and HTs. My OneNote pages can be retrieved on my cell phone or available in a cloud environment when I am traveling. When called about the evaluation’s design or a statistic, I can easily find and cite what other evaluation experts are doing.
Rad Resources:
- Developmental Evaluation addresses the complexities of higher education and the often conflicting formative goals of its stakeholders.
- Better Evaluation is an amazing source for planning and designing evaluations in seven color-coded task cluster. I am partial to this resource because there is mention of stakeholder involvement in every task cluster.
- Six Thinking Hats. Contributed by Sara Vaca in her AEA365 Tip A Day post, this framework identifies roles that different stakeholders may assume.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Mrs. D’Allegro,
Thank you for your wonderful insights. I thoroughly enjoyed your approach to stakeholder involvement.
I specifically resonated with your “keep it simple” tip. Design evaluation can be overwhelming to people who are not familiar with it. To be honest, I’m studying program evaluation and it’s even overwhelming for me! That’s why I like your idea of keeping it simple. Stakeholders may not have the time or patience to review unnecessarily long or complicated reports. Perhaps those reports are of use to the evaluator, but to a stakeholder who is not as knowledgeable or committed, it will seem like a snooze. I think your comparison to storytelling is interesting. If we could even make information sharing fun that would be a great benefit to increased involvement. I’ve always been an advocate for clear and simple communication; in my practice and personal life. So I agree that straightforward reporting would definitely yield the highest amount of stakeholder involvement.
I also appreciate the idea of using digital polling and voting often. In fact, I like that you mentioned the use of technology as a tool for several purposes. Since practically everyone has at least a smartphone, it allows stakeholders to contribute their inputs conveniently and for evaluators to make changes quickly. I also see stakeholder voting as enhancing buy-in. The more involvement that stakeholders have in the process, the more invested they should feel. This will hopefully lead to a positive feedback loop of increased participation and more buy-in.
I have a question about stakeholder involvement: despite giving multiple opportunities to participate to stakeholders, do you think there could be detrimental effects from quick/convenient responses methods that may exclude detailed information useful for the evaluation process and perhaps data collection later?
Once again, thank you for your post. It was very appreciated!