Hello! My name is Dana Linnell (formerly Dana Wanzer). I’m an assistant professor of psychology in evaluation at University of Wisconsin-Stout where I teach evaluation, statistics, and research methods and I’m the program director for our MS in Applied Psychology program, which has a concentration in evaluation. Today I’ll be sharing a simple framework that may be useful for defining various terminology related to research, evaluation, and research on evaluation.
People have been struggling to differentiate evaluation from research for decades, and many different definitions of evaluation exist (see my blog post and journal article for more info on defining evaluation). In general, evaluators and researchers differ in whether they think evaluation and research are similar yet distinct activities, or whether evaluation is a type of research. This makes it difficult to communicate our work as evaluators and argue that evaluation is its own distinct profession.
When it’s hard to describe the differences between research and evaluation, it makes defining “research on evaluation” even more challenging. We are combining two terms that some people use to mean the same thing into one term! This challenge is a perennial issue within the Research on Evaluation TIG. When I was on its leadership team from 2017-2019, we received numerous proposals to the conference that we did not consider to be research on evaluation. This led us to have a discussion amongst TIG members on how we define research on evaluation.
Add to the challenge that there are many other similar terms, including research evaluation, evaluation research, meta-research, and meta-evaluation. In a recent Eval Cafe webinar, I described these definitional issues and presented this framework for helping us clarify our terminology by separating the activity being done and the thing being examined:
Just like the lines between research and evaluation may be blurry, I think the lines between these four things can be equally blurry. For example, meta-evaluation may be evaluating an evaluation but it may also be synthesizing the results across a multitude of evaluations, akin to a meta-analysis which may be more research-like in its activity. Furthermore, some of us in the field are doing meta-research on evaluation (meta-RoE) which is the activity of researching the thing of research on evaluation! To say this can get complicated is an understatement, but hopefully this can help us start to achieve clarity in our conversations regarding these topics.
I’d love to hear from you whether you find this framework useful in differentiating all these research- and evaluation-related terms in our field! Please add your comments and questions below.
The American Evaluation Association is hosting Research on Evaluation (ROE) Topical Interest Group Week. The contributions all this week to AEA365 come from our ROE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.