Hi! I’m Lyn Boswell, a member of the Program Design Topical Group. I have worked abroad in monitoring and evaluation (M & E) in various cultures.
In the U.S., designing metrics to determine effectiveness of programs requires avoiding different “bias minefields” in achieve objectivity and validity. For example, the controversial topic of “defunding the police” is laden with much bias. As a result, it is impossible to create program design with any objective metrics.
Having the goal to “defund the police” is a political posture. It does not result in realistic measurements except for reducing budget and personnel. Once that’s done, the goal is considered achieved. But, is that really the goal? Absolutely not. The question is about whether the program, defunding in this case achieves the intended goal of a problem to which we would like a resolution. I say this because we all know that defunding is not a solution for most law enforcement.
Thus, “defunding the police,” which is heavily laden with bias would be less biased with clear goals such as:
- less lethal methods used for unarmed individuals;
- less lethal methods towards armed or unarmed mentally ill; and
- more de-escalation training.
Notice, that race is not tied with any of these goals. However, skin color has often been the “motive” for defunding the police movements. From an evaluation perspective, it is not a valid measurable goal.
- As a professional program designer and evaluator, the most important information needed is discerning what is the “real goal” you are trying to measure?
- You may need to peel back multiple layers of bias to understand what goal needs to be achieved (i.e., individual, organization, or community). Then, you can talk about solutions that are achievable/measurable.
Consider your own personal biases an evaluator. You may find the results of the Harvard Implicit Bias Test interesting: https://implicit.harvard.edu/implicit/takeatest.html
How have your biases impacted your own interpretations of project goals and outcomes?
The American Evaluation Association is celebrating Program Design TIG Week with our colleagues in the Program Design Topical Interest Group. The contributions all this week to aea365 come from our Program Design TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.