RoE TIG Week: Incorporating Research on Evaluation into your Evaluation Projects by Miriam Jacobson

I am Miriam Jacobson and I am an evaluator at ICF, where I evaluate workforce development programs and programs for low-income families. As a member of the evaluation field, I am interested in studying best practices for conducting evaluation and helping our field learn how to become more effective. To do this, I have gathered research on evaluation data in my evaluation projects that I hope will contribute to building an evidence base for evaluation practice. I want to share ideas for other evaluators who may be interested in getting involved in this type of work, including those who work outside of academic settings.

Lessons Learned:

• Many evaluation projects include opportunities to study evaluation! Most evidence-based programs we evaluate will include at least one program activity related to evaluation, such as continuous quality improvement or student assessment. As an evaluator, you can identify those components and think about how studying them could inform others who conduct evaluation.
• Evaluation clients or program leadership may be interested in RoE if it helps guide their own work. For example, when synthesizing information from foundation grantees’ evaluation reports, an evaluator could systematically assess the methods used in the reports to help the funder understand their grantees’ data collection strategies and challenges.

Hot Tips:

• Choose research topics that relate to practical issues you experience when conducting evaluations. Not only would studying those topics help you understand how to improve your evaluation work, but it might address questions that others in the field are interested in as well.
• Find small ways to include RoE in your evaluation plans. For example, you could include a few RoE items in your data collection measures. One evaluation team adapted their data collection procedures to test out different methods of obtaining a high survey response rate during their evaluation.
• Plan for dissemination up front by including within the initial scope of the project a sharing of findings with the field of evaluation. Meanwhile, if you are planning to report your findings in a journal article or other academic venue, you should thoroughly document methods you use to collect and analyze your RoE data to address potential questions of other researchers.
• Look for research funding opportunities that can help you add RoE to your ongoing evaluation work, such as the William T Grant Foundation or the Patient-Centered Outcomes Research Institute (PCORI).

---

The American Evaluation Association is celebrating Research on Evaluation (RoE) TIG Week with our colleagues in the RoE Topical Interest Group. The contributions all this week to aea365 come from our RoE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org.