Jason Ravitz (Google) and David Fetterman (Fetterman & Associates, past-president of AEA, and founder of empowerment evaluation) have been using empowerment evaluation in various educational settings, including a graduate school program and with Kathy Haynie (Haynie Research and Evaluation) and Tom McKlin (The Findings Group), in our work with two computer science education evaluation learning communities.
Empowerment evaluation is the use of evaluation concepts, techniques, and findings to foster improvement and self-determination. This approach aims to increase the likelihood that programs will achieve results by increasing the capacity of program stakeholders to plan, implement, and evaluate their own programs.
3-Step Approach. One empowerment evaluation approach involves helping a group: 1) establish their mission; 2) take stock of their current status; and 3) plan for the future. Additional tools include an evaluation dashboard to help communities monitor their own progress.
CS/STEM Learning Communities. The “Evaluation Wrecking Crew” includes over 60 CS education evaluators across the country. A second group (with some overlap) is the NSF-funded Computer Science Outcomes Networked Improvement Community (CSONIC).
We have joined forces to: 1) build a CS/STEM repository of evaluation instruments and approaches; 2) build a common hub for the community, with the assistance of Oak Ridge Associated Universities; and 3) educate the CS community about the value and role of evaluation to improve the quality of CS and STEM education. We meet biweekly using Zoom video-conferencing software.
Kathy Haynie (Haynie Research and Evaluation) Remotely Facilitates Bi-monthly Meetings
Online Spreadsheet. Jason designed a 3-step online spreadsheet, using Google Sheets, to facilitate the empowerment evaluation process used in both the Evaluation Wrecking Crew and CSONIC workshops.
Mission. Our collaborative process allowed workshop members to remotely record their views about the mission or purpose of the group. Later comments were transformed into a mission statement (using Google Docs).
Taking Stock. A second sheet in the spreadsheet was devoted to “brainstorming” a list of the group’s most important activities. Members prioritized the list by “voting” for the most important activities to evaluate as a group.
A third sheet was populated with the list of the prioritized activities. The online workshop participants used a 1 (low) to 10 (high) scale to rate their performance on the “taking stock” sheet. We used the results to facilitate a dialogue about the ratings using videoconferencing software and referencing participants’ ratings.
Planning for the Future. We used a fourth sheet to help the group record plans for the future, specifying goals, strategies, and evidence.
Evaluation Dashboard. A final sheet was devoted to the dashboard to help us monitor our own performance. It included: goals, strategies, and evidence.
Computer Science Education Evaluators Conducting an Empowerment Evaluation Online
Rad Resources:
Free Template. This spreadsheet is available (free) to use to facilitate your own empowerment evaluation exercise remotely: tinyurl.com/eeblank.
Other free tools we have used include Google Forms to help graduate students evaluate their own as well as their peers’ work. We used these data to assess students’ performance, and in the process, make mid-course corrections concerning our instruction. Finally, we used Google Evaluation Worksheets to help them refine their proposals: /tinyurl.com/evalworksheet-google Additional resources can be found here (https://tinyurl.com/empowermentevaluationresources).
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Thank you for such an informative article with useful resources. As a high school teacher who is attempting to teach students to work both productively and collaboratively, this is a huge help. Additionally, I am trying to figure out how to collaborate with other teachers around the world on a cross-curricular teaching unit, and am looking forward to following the model you have set up here to ensure it is a successful endeavor.
Do you find that the students gain a lot from this model? There is mention of making mid-course corrections, which I think is very important. Are you able to recommend any further reading on empowerment evaluations to help me on my journey to improve my teaching?
Empowerment Evaluation Resources
For more information about empowerment evaluation see the following:
Web Posting
Wikipedia (look up empowerment evaluation)
Blog Postings
Stanford Social Innovations Review – This is a brief article about another empowerment evaluation book: Fetterman, D.M. (2103). Empowerment Evaluation in the Digital Villages: Hewlett-Packard’s $15 Million Race Toward Social Justice. Stanford: Stanford University Press.
See:
https://ssir.org/articles/entry/empowerment_evaluation_in_the_digital_villages_hewlett_packards_15_million
Empowerment Evaluation Blog. See: eevaluation.blogspot.com
Videos (on the web)
Brief video describing empowerment evaluation. See our Ignite Lecture (20 slides in 5 minutes). See: https://www.youtube.com/watch?v=fjUvV4HHH38
Video of an empowerment evaluation exercised used to facilitate an NSF funded computer science education evaluation initiative. It was used specifically to help them assess their efforts and move their efforts forward, including building a repository of STEM and CS evaluation tools and instruments:
See: https://www.youtube.com/watch?v=ploz0GlYPS0
Articles (on web)
Fetterman, D.M. (2011). Empowerment Evaluation and Accreditation Case Examples: California Institute of Integral Studies and Stanford University. In Secolsky, C. and Denison, D.B. (eds.) Handbook on Measurement, Assessment, and Evaluation in Higher Education. New York: Routledge.
Academic Medicine published an article about the use of empowerment evaluation to help Stanford University’s School of Medicine prepare for and pass an accreditation review. The citation is:
Fetterman, David M. PhD; Deitz, Jennifer MA; Gesundheit, Neil MD, MPH (2010). Empowerment Evaluation: A Collaborative Approach to Evaluating and Transforming a Medical School Curriculum. Academic Medicine: Volume 85 – Issue 5 – p 813-820
Fetterman, D.M. (2009). Empowerment evaluation at the Stanford University School of Medicine: Using a Critical Friend to Improve the Clerkship Experience. Ensaio: Avaliação e Políticas Públicas em Educação. Rio je Janeiro, 17(63):197-204.
Fetterman, D.M. (2017). Transformative Empowerment Evaluation and Freireian Pedagogy: Alignment with an Emancipatory Tradition. In Patton, M.Q. (ed.) Pedagogy of Evaluation: The Contributions of Paulo Freire to Global Evaluation Thinking and Practice. New Directions for Evaluation. N0. 155. San Francisco: Jossey-Bass.
Fetterman, D. and Wandersman, A. (2007). Empowerment Evaluation: Yesterday, Today, and Tomorrow. American Journal of Evaluation, 28(2):179-198.
Journal: Special Topic Edition
Empowerment Evaluation’s 21st Anniversary: A Celebration, Comment & Critique
[This is a special topic edition of Evaluation and Program Planning. It includes prominent critical friends’ comments about the 21st anniversary of the empowerment evaluation. The link to the special topic edition of EPP is presented below.]
See: http://info.davidfetterman.com/21stanniversaryempowermentevaluation.htm
A few highlights of the special topic edition are presented below:
Introduction
David Fetterman introduced empowerment evaluation to the field of evaluation during his presidential address 21 years ago. Since that time it has been used in over 16 countries, ranging from corporate offices of Google and Hewlett-Packard to squatter settlements and townships in South Africa. Empowerment evaluation has been used by the W.K. Kellogg Foundation, Centers for Disease Control and Prevention, US. Department of Education, Stanford University School of Medicine, and Native American tribes in reservations stretching from Michigan to San Diego.
David Fetterman, Shakeh Kaftarian, Abraham Wandersman, and many other empowerment evaluators, have published 5 books on empowerment evaluation. The 21st anniversary of this approach was celebrated with a panel of luminaries who have helped shape empowerment evaluation with their critiques, concerns, and congratulations. They included Drs. Steward Donaldson, Michael Scriven, Michael Patton, and Marvin Alkin. Their comments are illuminating, engaging, and presented in this special topic edition of E&PP.
Titles of Articles in Special Topic Edition:
David Fetterman: Celebrating the 21st anniversary of empowerment evaluation with our critical friends
Stewart Donaldson: Empowerment evaluation: An approach that has literally altered the landscape of evaluation
Michael Patton: Empowerment evaluation: Exemplary is its openness to dialogue, reflective practice, and process use
Michael Scriven: Empowerment evaluation 21 years later: There is much to admire about empowerment evaluation
Marvin Alkin: When is a theory a theory? A case example
David Fetterman & Abraham Wandersman: Celebratory reflections, appreciations, clarifications, and comments
Chapters
Fetterman, D.M. (2015). Empowerment Evaluation and Action Research: A Convergence of Values, Principles, and Purpose. In Bradbury, H. (ed.) The Handbook of Action Research. Thousand Oaks, CA: Sage.
Fetterman, D.M. (2013). Empowerment Evaluation: Learning to Think Like an Evaluator. In Alkin, M.C. (ed.) Evaluation Roots: a wider perspective of theorists’ views and influences (second edition). Thousand Oaks, CA: Sage.
Books
Fetterman, D.M., Kaftarian, S., and Wandersman, A. (2015). Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability (2nd edition). Thousand Oaks, CA: Sage. [This is the most recent empowerment evaluation book. It is an updated and consolidated presentation about the approach]
Fetterman, D.M. (2013). Empowerment Evaluation in the Digital Villages: Hewlett-Packard’s $15 Million Race Toward Social Justice. Stanford: Stanford University Press.
Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice. New York: Guilford Publications. (Preview.)
Fetterman, D.M. (2001). Foundations of Empowerment Evaluation. Thousand Oaks, CA: Sage.
We also have a new book out titled:
Fetterman, D.M., Rodriguez-Campos, L., and Zukoski, A. (2018). Collaborative, Participatory, and Empowerment Evaluation: Stakeholder Involvement Approaches. New York: Guilford Press
For additional information including videos, see David Fetterman’s web page at: https://www.drdavidfetterman.com/