I am Travis R. Moore, a PhD Candidate in the Civil Society and Community Research program at the University of Wisconsin-Madison. Lori Bakken is a professor in the Department of Civil Society and Community Studies at the UW-Madison and an Evaluation Specialist for the UW Division of Extension, and Luke Carmichael Valmadrid is an EQUITY-concentrated Master of Public Health student at the University of North Carolina at Chapel Hill. Our evaluation efforts span community-based participatory approaches to evaluations related to food systems, food insecurity, and obesity. Together, we are exploring how gatekeeping and gatekeeping practices influence evaluation.
What is gatekeeping? Formally, gatekeeping is the behavior of controlling or limiting access to information or people. However, in our interviews over the past year with evaluators in an array of evaluation roles (e.g., funding evaluator, community-based evaluator, consultant), we know that this definition is insufficient and unspecified for the evaluation field.
Evaluation is inherently a relational activity. Evaluators constantly interact with stakeholders to build relationships while making value judgments on programs, practices, and policies. In the process, evaluators encounter people or structures that limit or facilitate access to key stakeholders or data. Sometimes evaluators are gatekeepers. These behaviors are often in conflict with AEA’s Guiding Principles for Evaluators which encourage stakeholder inclusion, respect for people, integrity, and equity, among other ethical practices. But why and how does this happen?
Gatekeeping in evaluation is a result of needing to maintain control and ownership over information or people. In some respects, controlling access to stakeholders or data serves as a protective measure to preserve the integrity of a community (e.g., community-chosen liaison). In other respects, however, gatekeeping behavior complicates the politics of membership, participation, and collaboration; it makes evaluation projects more difficult to manage. More importantly, gatekeeping creates a culture of distrust and power imbalance, all of which may have important implications for evaluation outcomes.
Lessons Learned:
Our research reifies the importance of context and evaluator reflexivity in evaluation projects. Moreover, examining the intersection of context and specific evaluation practices may lead to important insights about how evaluators mediate, moderate or mitigate the evaluation process by including or excluding certain stakeholders and transforming or perpetuating injustices.
Hot Tips & Rad Resources:
What can an evaluator do to make sure they are successfully accounting for various aspects of gatekeeping culture in their evaluation project(s)? While our research group continues to understand the influence of gatekeeping in evaluation, one way to navigate your current stakeholder dynamics in the meantime is to conduct a stakeholder analysis.
How might evaluators themselves assess their role in evaluation projects? Conducting a context analysis (see also Rog’s (2012) dimensions of context) alongside a stakeholder analysis may help with taking perspective.
Get Involved
If you would like to participate in our research, or simply find out more about our research, please send me an email at trmoore2@wisc.edu.
The American Evaluation Association is celebrating Research on Evaluation (RoE) TIG Week with our colleagues in the RoE Topical Interest Group. The contributions all this week to aea365 come from our RoE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org.
Hello Travis,
It can be tricky having to maintain confidentiality but also using the information that can help the bigger picture. I agree that it can create distrust for future programs or evaluations to be conducted.