Greetings! My name is Amy Jersild. I am an independent evaluation professional working internationally and a PhD candidate in Western Michigan University’s Interdisciplinary PhD in Evaluation program. I am excited to share the purposeful sampling approach I adopted for a 3-paper study on the professionalization of evaluation practice. Such a sampling strategy may be of interest for colleagues in conducting their own evaluations or research on evaluation.
There has been considerable thought and energy given to the development of the evaluation field worldwide in the past two decades, particularly since the adoption of the Sustainable Development Goals by the UN and the 2014 resolution on evaluation. Particularly key has been the focus on evaluation associations, which have grown exponentially across the globe and are regarded as key actors in promoting the evaluation field. There is a body of research on the professionalization of the evaluation field, as well as theories of change developed on how evaluation can become a professional practice, particularly in the Global South. With the interest to contribute toward this body of research and practice, my research explores the following areas: the phenomenon of independence and how it is experienced within different contexts by both commissioners and independent evaluators; the state of discourse on what the evaluation field should look like as a profession; and the role of evaluation associations in advancing evaluation practice.
In understanding these phenomena, I developed a heterogeneous sample of 4 countries: Brazil, Ghana, Indonesia and Uganda. My intent in sampling was to document the diversity, and to explore patterns or themes that cut across that diversity on dimensions of interest. My approach across the 3 studies began with developing partnerships with the leadership of evaluation associations within each of the 4 countries. They recommended an initial list of people within the categories of association leadership, key experts, evaluation commissioners and independent evaluators — most of whom I was able to interview via videoconference. I then employed a snowball sampling strategy to further grow my sample, and received introductions to other evaluators, commissioners, and key experts. Over the course of one year, I interviewed 68 people across the 4 countries, ranging from 15 to 18 for each country.
Lessons Learned
Purposeful sampling in qualitative inquiry is fundamentally different from statistical reasoning of probability sampling. In my aim to explore at some depth among 4 diverse country contexts my research does not aim to draw generalizations. Rather, I aim to explore trends emerging from the heterogeneity, and to engage in the creative endeavor of theorizing based on those emerging trends. My sample is also based on scope and feasibility, and based on the expressed interest of the associations within those countries to engage and collaborate.
The International Organization for Cooperation in Evaluation (IOCE), a membership organization that represents and provides support to regional and national evaluation associations, has a membership database. This database was a critical resource in designing my heterogeneous sample. Determining which associations self-identified in their registration as engaged in advocacy was my starting point. I engaged with each association to determine those that would both be interested to participate and which collectively would form a heterogeneous grouping. Through the development of a working relationship, the buy-in of each association enabled further access to other evaluation professionals.
Rad Resources
Michael Q. Patton’s book Qualitative Research and Evaluation Methods details 40 sampling strategies for use in qualitative studies. Sara Vaca’s imaginative infographic on Patton’s 40 purposeful sampling strategies provides a handy visual reference.
Michael Q. Patton’s YouTube channel features a video on purposeful sampling in qualitative inquiry which gives an excellent overview.
The American Evaluation Association is hosting Research on Evaluation (RoE) Topical Interest Group Week. The contributions all this week to AEA365 come from our ROE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.