My name is Wayne Miller and I am a senior lecturer in the Faculty of Education at Avondale College, Lake Macquarie, New South Wales, Australia. In December I received my doctorate from the University of Wollongong following acceptance of my thesis titled Practical methods to evaluate school breakfast programs – A case study. The study reports the use of empowerment evaluation with a national school breakfast program in Australia known as the Good Start Breakfast Club (GSBC).
During the project some eighty GSBC program personnel took part in ten empowerment evaluation workshops to identify key program activities for investigation; gather baseline data about the strengths and weaknesses of the activities; suggest goals and strategies to monitor and improve the activities identified; and to develop evaluation tools designed to provide evidence of success. Following workshops I asked participants …from your experiences in these initial workshops how valuable do you think the empowerment evaluation method is for collaboratively evaluating the GSBC program? 42/80 indicated ‘very’ to ‘extremely’ valuable with a further 36/80 responding ‘reasonably’ to ‘quite’ valuable. A regional coordinator commented, the model is definitely in line with the principles of our program and empowering the community. One beautiful response from an outlier, a total waste of time and all about Miller getting his doctorate!
Toward the end of the project I interviewed 29 program personnel who had been directly involved in the evaluation and I asked them to reflect on the empowerment evaluation process and particularly whether it had adhered to the ten principles of empowerment evaluation. Respondents made up of volunteers and teaching staff at the ‘coal face’, school principals, GSBC coordinators and executive staff from Red Cross the program managers, and the Sanitarium Health Food Company, the major sponsor, reported both alignment and misalignment with the principles. Two examples: On the principle of democratic participation defined as active participation by everyone in shared decision-making is valued…respondents acknowledged that the ‘taking stock’ step of the empowerment evaluation had been particularly democratic but that the democratic nature of the evaluation process had been compromised when those who came together to implement Step 3 – ‘Planning for the future’ were handed evaluands from workshop groups who had completed Steps 1 – Develop a mission, vision or unifying purpose for the program and Step 2 -Taking stock. On the principle of capacity building defined as program staff and participants learn how to conduct their own evaluations…significant gains in evaluation capacity was reported by personnel at the breakfast club level. Volunteer staff at one site designed and trialled an instrument which provided average nutrient uptake data which were subsequently used to modify food served to improve fibre intake. A negative aspect reported was that staff turnover at management level mitigated against evaluation capacity building in one region.
Hot Tip: Trustworthy relationships must be established for empowerment to occur. Community ‘champions’ committed to their communities, who use empowering processes and have good networks and communications skills, are vital partners. Ongoing commitment by senior management to nurture and support empowered staff by providing them with the resources necessary to remain so, is a ‘must have’ ingredient to avoid empowerment fade.
This week’s posts are sponsored by AEA’s Collaborative, Participatory, and Empowerment Evaluation Topical Interest Group (http://comm.eval.org/EVAL/cpetig/Home/Default.aspx) as part of the CPE TIG Focus Week. Check out AEA’s Headlines and Resources entries (http://eval.org/aeaweb.asp) this week for other highlights from and for those conducting Collaborative, Participatory, and Empowerment Evaluations.
You are most welcome Wayne – always good to see good folks doing good work. Happy to help whenever I can. Best wishes and keep up the good work.
– David
Hello David
Talk about a community champion! You embody that comment made by Donaldson and Scriven (2003) that ‘evaluator characteristics may inspire or constrain one’s ability to practice any particular approach’. Thanks for your responses to my empowerment evaluation story and for introducing June to the exchange. Empowerment evaluation resonated with me in my early days as a doctoral student and your willingness to engage from the start and June’s expertise and encouragement is greatly appreciated.
Hello Nicole. Thanks for your helpful comment. You’ve given me ideas about ways to induct and nurture newcomers to the evaluation so they can capture at least some of the enthusiasm generated in those initial empowerment evaluation workshops. There’s something however about those red dot stickers that’s hard to replicate!
All the best
Wayne
On behalf of Liliana and me, thanks for participating in this exchange and we look forward to seeing you at the annual meetings to continue this discussion in a similarly participatory but face-to-face format. Best wishes.
Hi Wayne
I really encourage folks to read your article with June Lennie about “empowerment evaluation with a national school breakfast program in Australia known as the Good Start Breakfast Club (GSBC).” They should also read your dissertation for a lot more of the details and nuances at: http://ro.uow.edu.au/theses/3059/
Empowerment evaluation: A practical method for evaluating a national school
breakfast program, Evaluation Journal of Australasia, Vol. 5 (new series), No. 2, 2005, p. 18-26
I noticed your comment that: “A regional coordinator commented, the model is definitely in line with the principles of our program and empowering the community.” I think this captures the essence of why empowerment works particularly well with organizations that are philosophically in alignment with many of the empowerment evaluation principles. The match has a powerful synergistic effect.
I liked the balanced reporting when you state they “reported both alignment and misalignment with the principles.” There is no perfect system or application. Success and effectiveness run along a continuum.
I could not agree more with your fundamental thesis: “Trustworthy relationships must be established for empowerment to occur.” It is all about relationships and trust – the rest as they say is commentary, from the design to data collection and analysis to reporting. They all rest on honesty and respect and trust (in academe and in the community).
I think collaborative, participatory and empowerment evaluators would agree with you (at least for the most part) that:
“Community ‘champions’ committed to their communities, who use empowering processes and have good networks and communications skills, are vital partners.”
As well as,
“Ongoing commitment by senior management to nurture and support empowered staff by providing them with the resources necessary to remain so, is a ‘must have’ ingredient to avoid empowerment fade.”
Great contributions Wayne – many thanks.
– David
PS By the way we will be continuing this dialog about the role of the evaluator in collaborative, participatory, and empowerment evaluations at the next AEA annual meeting.
Collaborative, Participatory and Empowerment Evaluation TIG Poster
Hi Nicole
I like your comment that in times of staff turnover “it can be worthwhile for the evaluator/facilitator/trainer to serve as a repository for the site’s evaluation information.”
I always make sure the group I am working with draft a mini-report of what they have accomplished and where they want to go for exactly that reason.
In addition, it is a way of honoring the community’s hard work. It protects them as well. It keeps a running record of work-to-date and existing agreements. It provides sponsors, reporters and other with a “slice of life” concerning the community’s accomplishments – enabling the group to respond to accountability related requests in a timely manner. A report (web based such as a blog or web page as well as a traditional written report) ensures a measure of transparency about what they are doing with the larger community.
A report or document summarizing the groups efforts is also a check in to see if things are on track. Finally, I like traditional and virtual reports because they ensure some stability in the process – preventing an articulate, persuasive, but uninformed citizen from taking over the group’s agenda. The report can be used to socialize them into the group before they run off with their own agenda or try to hijack the group’s agenda.
We will be talking about the role of the evaluator in collaborative, participatory, and empowerment evaluations at the next AEA annual meeting – come join us to pursue this discussion.
– David
Hi Wayne: This sounds like a really engaging project. And the tip on getting key community champions and management involved is an important one. On a side note I’ve run into the issue of staff turnover impacting participatory evaluations in some of the work I’ve done with community based organizations. At times like this I’ve found that it can be worthwhile for the evaluator/facilitator/trainer to serve as a repository for the site’s evaluation information, so that they can bring the new staff up to date on the evaluation process and plans, and begin engaging them as soon as they come on board.
Thanks–
Nicole