RoE TIG Week: Knowing Ourselves: Driving the Field Forward Through Self-Discovery by Maggie Schultz Patel

image of maggie

Hi there! I’m Maggie Schultz Patel, an independent evaluation consultant and doctoral student of Research Methods and Statistics at the University of Denver.

Program evaluation is still a relatively new discipline, and there is still a need to increase our knowledge of ourselves as a field. Further, according to a recent needs assessment, there are key research questions about the field that evaluators would like to see answered through systematic inquiry. By and large, these questions are in relation to research on evaluation’s impact, methods, and context.  While debates on the merits of various methodological approaches are rampant and well-documented, it is less clear how often these approaches are used or how those methodological decisions are made in practice. Though we all may feel certain we have a grasp on when and why we tend to use particular methods as a field, we all know the pros and cons of generalizing too heavily from our own perspectives.

Lesson Learned

A systematic review of peer-reviewed evaluation literature suggests that there have been few systematic examinations of methods use. As part of my dissertation research, I am developing a mixed-method study to interrogate whether there are observable trends in the use of methods as well as to explore what goes into the selection of methods by evaluators. This will include a systematized review of evaluations published in the past ten years, potentially some latent growth curve modeling of observed trends to explore growth trajectories and inter-/intra- group differences, and interviews with practitioners to shed insight into their decision-making processes when selecting evaluation methods. This research on evaluation is expected to have important implications for the field; findings will document and contextualize the ebb and flow of methodological trends, as well as bolster future efforts to balance methodological rigor with client need. Ultimately, my hope is that the findings will help us move beyond favored methods by encouraging reflective and responsive practice in the field.

Hot Tip

If you’re interested in learning more about this or other exciting research on evaluation, look out for sessions sponsored by the RoE TIG at AEA’s Evaluation 2020.


The American Evaluation Association is celebrating RoE TIG Week with our colleagues in the Research on Evaluation TIG. All of the blog contributions this week come from our RoE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

2 thoughts on “RoE TIG Week: Knowing Ourselves: Driving the Field Forward Through Self-Discovery by Maggie Schultz Patel”

  1. Dr. Sondra LoRe

    Hi Maggie!
    Thank you for the great post! And thank you also for drawing attention to Jennifer C. Greene and colleagues. Such important work. It made me feel young to read your words, “program evaluation is still a relatively new discipline”-b/c to me I sometimes feel the years I have spent doing one form or another of evaluation makes me (and the discipline) “vintage” (in a good way). So you brought a smile to my face. I like to think of program evaluation, not as a new discipline but one that continues to develop. Growing as it demonstrates its value and use, to people and programs.
    Thank you again for a great post!
    Cheers,
    Sondra

    Sondra LoRe, Ph.D
    Manager | National Institute for STEM Evaluation and Research (NISER)
    Adjunct Professor | Evaluation, Statistics, and Measurement Program, Department of Educational Psychology & Counseling

    The University of Tennessee, Knoxville
    Office of Research & Engagement
    114 Philander P. Claxton Education Building
    slore@utk.edu

    1. Hi Dr. LoRe,

      Thank you for taking the time to comment! I appreciate your perspective; I don’t really think of our field as “new” per se (even I’ve been practicing for 10 years!); I think you’re right that to be more precise, what I meant is that it is still developing and fighting for recognition.

      Best,
      Maggie

Leave a Reply to Maggie S Patel Cancel Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.