My name is Susan Wolfe and I am the owner of Susan Wolfe and Associates, LLC, a consulting firm that applies Community Psychology principles to strengthening organizations and communities. Prior to initiating my consulting practice, I was employed as an internal evaluator in more than one organization.
Have you ever had a job where they sent you to evaluate a multi-site program or initiative, and find that there was no clearly defined single intervention, no specific goals or objectives, and the performance measures lacked established norms or benchmarks? This has happened to me on more than one occasion. In each case I managed to produce a useful report. How did I do it?
Lesson Learned: Sometimes you are unable to convince the powers that be that you need to first address evaluability. If this happens, describe the evaluation challenges and how they will limit what you will be able to do in writing and negotiate a longer timeline for the project. Such projects can become quite complex and you will need extra time.
Hot Tip: Consider using a comparative case study approach that utilizes quantitative, qualitative and participatory methods. After completing a case study of each site, you can then summarize common activities and outcomes. You can also determine which sites showed better outcomes and which did not, and identify successful strategies and barriers to success.
Rad Resource: Case Study Research: Design and Methods. Fifth Edition (2014) by Robert K. Yin.
Hot Tip: Identify the common core elements for the program or initiative across sites. Make sure one of your recommendations includes the development of Specific, Measureable, Attainable, Realistic and Time Bound (SMART) objectives and development of a framework or model of change.
Rad Resource: The Community Toolbox (one of my favorite resources) provides instruction and tools for Developing a Framework or Model of Change.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.