We, Dana Linnell and Bianca Montrosse-Moorhead, explored this question in an interview study. We interviewed 40 professionals doing applied work (e.g., evaluators, researchers) about their journey into the field, applied practice, and professional identity. This study was published in Evaluation in January 2024. Here’s what we found!
1. Evaluators have unique journeys into the field. Evaluators in our study entered evaluation through their job, a graduate program, or a combination of the two. Other professionals entered their fields through personal interests or values, life circumstances, their job and/or graduate program.
Lesson Learned
This finding suggests to us that the evaluation field should attend more closely and broaden pathways into evaluation. For example, since many evaluators enter through graduate school, more undergraduate programs could help increase the likelihood of students considering graduate school for evaluation specifically.
2. There is nuance in how professionals describe and think about the jurisdictional boundaries between evaluators and other applied practitioners. Similar to Wanzer (2021) examining similarities and differences among evaluation and research, this study found agreement in some similarities and differences among evaluators and other applied professionals. For example, participants agreed that applied professionals use systematic inquiry and strive to make a difference and participants. They also agreed that evaluators are different in important ways. They are likely to pursue different study purposes compared to other professionals, include more participatory approaches, engage in different dissemination practices, and have different professional spaces (e.g., AEA, evaluation journals).
Lesson Learned
The way that evaluators and researchers describe evaluation and research (as found in to Wanzer [2021]) mimics the ways they describe evaluators and researchers. There is likely a reciprocal relationship here: the way people identify affects the work they do, and the work they do affects how they identify.
3. There are some misconceptions other professionals have regarding evaluators and evaluation. Despite some common conceptions, there were some interesting ideas non-evaluators mentioned that we contend are misconceptions about our field. For example, one participant thought that evaluators are more prone to qualitative methods, whereas another though they were more prone to quantitative methods! One thought that evaluators are more prone to causal inference research designs. Another thought that evaluators prioritize the paying client to the exclusion of others.
Lesson Learned
This suggests we need to better communicate what evaluation is and who evaluators are to other fields and professions (such as described by Mason [2022] and Montrosse-Moorhead et al. [2017]). Otherwise, people outside of evaluation may determine this for us.
Rad Resource
Interested in learning more? Check out the manuscript preprint, research materials, and more on our project’s page on the Open Science Framework (OSF).
Our next step is analyzing the rest of the interview data and writing a paper examining values, valuing, and social justice practices among evaluators and other similar professionals. We’ll post it on our OSF too! We hope to do a larger survey study in the future.
The American Evaluation Association is hosting Research on Evaluation (RoE) Topical Interest Group Week. The contributions all this week to AEA365 come from our ROE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
I love this post and this contribution to the continued conversation about the differences between evaluators and applied practitioners (e.g., researchers). Our team plans to use this as a discussion topic for our weekly meeting.