Thank you for subscribing to AEA365! AEA is aware of the issue impacting email receipt of aea365 blog posts. Our RSS Feed provider has announced a reduction in maintenance which has unfortunately resulted in sporadic email receipt of daily posts. AEA is transitioning to a new provider and intend to roll out a new, improved service in early September 2022! Until then, please continue to regularly check back to aea365 for daily tips, tricks, and resources for evaluators by evaluators. You may also add AEA365 blog’s feed URL – https://aea365.org/blog/feed/ – to your favorite RSS! Stay tuned for additional details! For questions, please contact AEA at info@eval.org.
Good day! I am Art Hernandez, a Professor at the University of the Incarnate Word in Texas. As an evaluator and teacher of evaluation, I am interested in the processes of cultural responsiveness in theory and practice, especially concerning measurement and assessment. Currently, I am a member of the Expanding the Bench™ Advisory Board and a mentor for the Leaders in Equitable Evaluation and Diversity (LEEAD) Program. I also have the honor of serving on the Joint Committee on Standards for Educational Evaluation.
Lessons Learned:
While evaluation is frequently seen as a mechanism for accountability or as a means for demonstrating influence or even effect, in most cases it is not recognized as a tool or advocacy unless it is specifically described as such. What is missed is that regardless of purpose, evaluation results are often the basis for important recommendations/decisions about activities and people. This is true for many reasons but, probably mostly because of the tendency of people to interpret observations in ways that fit their pre-existing assumptions, world views or “theory.” This tendency enables otherwise good people to make judgments that result in unfair, biased, and injurious consequences all the while grounded in “evidence-based decision making.” This fundamental source of bias and invalidity which compounds and sustains existing social and individual injustice. This is amply demonstrated by a quick examination of social consequences where some individuals are subject to scorn or criminalization of behavior that does not result in the same for others. We see this in politicians, people of wealth, individuals from historically advantaged groups or people with and from other “established” positions of privilege, where the “rules” seemingly don’t apply and judgments/punitive consequences (when they occur at all) are long in coming and quite less severe than for others. Simply put, evaluation results are always important levels for movement and change regardless of resistance or success and it is important that those levers be well constructed, and the consequences of any resulting movement be thoroughly considered as a matter of process as well as outcome.
Culturally Equitable and Responsive Evaluation is an important tool to counteract this result but, it is equally susceptible to the same prejudice and bias and must be as carefully constructed and monitored. Good evaluation of any method, considers multiple perspectives, grounds itself in ongoing reflection, is subject to change in process given new insights or information, is open to differences from what is expected, and is absolutely committed to represent what is the experience of all involved in ways that are accurate and authentic. This is more likely in instances where evaluators recognize their responsibility and their power. What we do always matters, even in those instances where evaluation seems perfunctory or otherwise seems unlikely to be the case. How we design and execute our projects should stand the scrutiny of counterfactual explanations, alternative theory, different experience or philosophy, as well as our own sense of competence and ethics. It is this rather than the nature of data or methods that constitute “rigor” in inquiry.
Hot Tip:
Diversity in the constituency of evaluation/research/inquiry teams is a key element in maximizing the likelihood that multiple perspectives are included in the work. Member-checking and other efforts to ensure identification of authentic information and interpretations which involve all those party to and informed by evaluation is another key requirement for good and responsible evaluation work.
Rad Resources:
There are any number of recent texts and publications and presentations which focus on the importance of considering multiple perspectives and including all relevant voices. The discerning evaluator will seek sources which emphasize the sources of “error”, “noise”, invalid instrumentation, and prejudiced perspectives and methods for example much of the work focusing on CREE.
The American Evaluation Association is hosting Latina/o Responsive Evaluation Discourse TIG Week. The contributions all this week to AEA365 come from LA RED Topical Interest Group members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.