My name is Stanley Capela. I am an applied sociologist who currently is the Vice President for Quality Management and Corporate Compliance Officer for HeartShare Human Services of New York. I am also a Commissioner, Team Leader and Peer Reviewer for the Council on Accreditation. I have been an internal evaluator for the past 43 years and have learned what the ingredients are for a successful internal evaluation. This brief blog is my recipe.
Ingredient 1: The Role of Senior Leadership
I have been very fortunate in that senior management has a clear understanding of the evaluation process and the role it plays ensuring quality services. Ensuring understanding and buy-in from senior leadership is critical to success.
Ingredient 2: Communication
Senior management plays a key role in communicating your role within the organization. Ensure everyone from senior management to line staff understand that program evaluation helps to identify strengths and challenges in program performance, and address questions and concerns about your role and responsibilities.
Ingredient 3: Utilization Focused
Michael Patton’s Utilization Focused Evaluation book was the first book I read when I started my career, and it has stuck with me – if the information is not useful and utilized you are wasting your time. The key ingredient is to have a contextual understanding of the organizational and program environment, that you are giving stakeholders what they need, and you are best able to communicate results in a way stakeholders can understand.
Ingredient 4: Empowerment
The key ingredient is to engage line staff and clients in the evaluation process, so that they feel empowered to use and reflect on the information. When the internal evaluator actively engages line staff as well as clients in identifying strengths and challenges it makes the evaluation, relevant, useful and come to life. David Fetterman’s work on Empowerment Evaluation provides a viable approach that might be worth a review.
Ingredient 5: Contextual
As a peer reviewer I have learned it is important to have a contextual understanding of the environment in which you are evaluating. The way I evaluate an organization in North Dakota and Illinois is different from the way I would review a program within HeartShare.
Ingredient 6: Let Go of Your Ego
During the course of my career, I have had to deal with resistance to evaluation. The key ingredient is not to take things personally. Listen to what is said, assess how you can communicate your message in a way folks understand your role and more importantly look at it as a positive learning experience.
If you have any comments or questions, I can be reached at Stanley.capela@heartshare.org.
The American Evaluation Association is hosting Internal Evaluation (IE) Topical Interest Group Week. The contributions all this week to AEA365 come from our IE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Hi Stanley,
I am currently working on my PME degree through Queen’s University (Kingston, Ontario). I am taking a course that explores program inquiry and evaluation. I have read many articles on the subject over the past few weeks including one by Michael Patton, so I am very familiar with his perspective that data collection and analysis must be useful and also utilized. However, I have struggled with this concept somewhat because although it sounds like an excellent idea…I am really not sure that it is the evaluator’s obligation or duty to enforce the findings of the evaluation. And, even if one could argue that it is the evaluator’s duty, how can the evaluator actually enforce it? I find myself relating more to C.H. Weiss’s view where she states, ” So while I thought it was reasonable to ask evaluators to work hard at communicating their findings, I also counselled that they should not hold out unrealistic expectations for use.” (Weiss, 1998, p.22)
I think it is at this crossroads that I found your post to be insightful and helpful. (With 43 years of evaluation experience, I am quite sure I can learn a few things from you!) In particular, I think your 4th and 6th ingredients resonated with me the most. I have always valued collaboration, but empowerment is really key I think. On your recommendation, I read some of David Fetterman’s work on the subject. He states, “Empowerment evaluation is the use of evaluation concepts, techniques, and findings to foster improvement, and self-determination “(Fetterman, 1994a). I think if stakeholders leave the evaluation process with a sense that they are taking control of their own destiny, they will be more motivated to adapt or change…and actually utilize the findings of the evaluation.
The 6th ingredient mentions that an evaluator needs to let go of his/her ego…This is great advice. As a person who often takes feedback very personally, I think I need to be far more objective (and even detached) from the evaluation process. Rather than truly listening to suggestions, I find myself on the defensive too often. Rather than taking comments personally, I need to do as you suggest and, “Listen to what is said, assess how [I] can communicate [my] message in a way folks understand [my] role and…look at it as a positive learning experience.”
Thank you for sharing your years of experience and insights,
Marci.