My name is Art Hernandez. I’m an evaluator and college professor and have been a member of AEA since 2006.
Lessons Learned
Evaluation decision making (judgment) is as susceptible to inconsistency and inaccuracy as any other type of decision. Some acknowledged sources of inaccuracy include cognitive bias, known and unknown interdependence of information, training bias, poor model specification, and cultural ignorance or bias. However, evaluators face additional challenges related to authenticity which is arguably more important in assigning definitions and judgments about value, which is the essence of the work we do.
Both evaluators and funders are at risk for inaccuracy and interactions between errors on the part of both often result in complex challenges to authenticity. This situation is exacerbated by consensus judgments by the field without benefit of feedback from the individuals and communities about which those judgments are made. These often-unrecognized decisions/judgements/presumptions can lead to false confidence and ongoing practice which sustains those erroneous judgments through a self-reinforcing, circular mechanism resulting in the persistence of inappropriate conclusions which are often detrimental to marginalized groups and contribute to historic and ongoing injustice. The sources of inauthenticity are difficult to identify and even more difficult to remediate given the usual criteria for the determination of “expertise”: training, credential, prior work, or reputation as basis for judgment about evaluator competence or expertise. Unfortunately, this tendency almost always substitutes for consideration of the full knowledge necessary for authentic evaluation. Authenticity requires evaluation be informed and guided by community knowledge rather than simply knowledge about the community. In other words, evaluation requires cultural inclusion rather than merely cultural “competence or responsiveness.”
In essence, the identification of right/authentic perspectives or “worldviews” from which to make evaluation process decisions and judgments requires the expertise of individuals from the community of interest who identify with the applicable culture and identity with and fully participate in the lived experience of that community. While a given process model may be intuitively appealing, it is important that funders and evaluators be well informed about the potential of that model to be authentic in both process and outcome and the assessment of potential, of necessity depends on the expertise of lived experience as well as any technical evaluation or inquiry expertise.
Evaluation inquiry is generally focused on valuation or determining worth as a matter of fidelity, consequence, or gain, putatively in the interest of the community “on” which evaluation work is conducted. However, it is also clear that evaluation proceeds in the interest of funders who, among other things, seek to demonstrate effect or influence or evaluator who seeks to demonstrate capacity or competence. This means that regardless of the work, there is always some degree of conflict of interest that must be managed if it is to be conducted ethically.
While there is no ready formula for addressing these concerns at a minimum, funders must support and facilitate necessary collaboration with communities as a matter of planning, implementation, and meaning-making as well as recognize that authenticity requires the work must advance at the pace of the community rather than the evaluator or funder. In addition, Evaluators must consider and begin to develop processes for inclusion and collaboration that provide for genuine collaboration and which recognizes the influence of that inclusion and collaboration on potential community experts, and safeguard against likely influences of positionality and power to ensure as is possible, that assumptions, frameworks, methods, understanding, and knowledge dissemination are consistent with the values and cultural identity of the community as state of the art professional and academic standards on an ongoing basis.
The American Evaluation Association is celebrating AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to AEA365 come from AEA’s MSI Fellows. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.