We are Dyani Bingham, Desiree Restad, Kelley Milligan, and Allyson Kelley, AKA PLLC Public Health Consultants, Evaluators, and Scientists working with tribal communities across the U.S. How do evaluators support communities becoming stronger, more confident, and reclaiming their rights as first peoples or Indigenous peoples in the US? Through empowered data collection.
One government project. Five years. 150 data indicators. Reported on annually.
We know the project well. The demanding quantitative evaluation requirements of federal or state agencies that are meant to define a program’s success or failure. The narrative is rigorous and often deficit focused. Evaluations are often defined by a top-down approach. This approach tells us what success should look like. The illusion of success is wrapped up in the copious amounts of data we collect because we are told to do so. As evaluators, we may experience feelings of fear, guilt, or shame because we cannot collect all the data, meet all the demands, or fit the 150 data indicators into a nice box. Many programs are dependent upon this false narrative, the illusion that the quantity of data collected matters more than the quality of data.
After a cumulative 50 years of community-centered evaluation with Tribal communities we are sharing our lessons learned, our hot tips and cool tricks for empowering data collection. We have heard about the history. Helicopter researchers and evaluators, coming in to collect data, and the damaging impact of deficit focused publications and reporting. This is all so disempowering.
We see agencies and sometimes evaluators collaborate with Tribal communities to collect data that never is seen again. Sometimes the results of data collected, the findings that could empower and change everything never make it to the community. Data collection occurs because it is a requirement for funding: collect data or lose funding. This needs to change. Data is a powerful tool. It signifies need, direction, growth, and success. Data can be empowering for the communities we work with, and it can also be harmful. As evaluators, we must understand the context and engage communities in conversations about data collection. We need to collect meaningful data, reflective of community values, and focus on the quality of data over quantity. Community-centered evaluation presents a unique opportunity redefine success, identify opportunities, and empower others.
Hot Tips
Five tips for empowering data collection in communities and questions for evaluators.
- Acknowledge the history of wicked data collection in the US with Tribes. Helicopter researchers and evaluators traumatized and eroded trust. As an evaluator how do you acknowledge this history? Is it a statement? An approach? An awareness of position and power?
- From the beginning, be inclusive. A lot of times the wrong people are at the table. Who are the right people and where might we find them?
- Know what data is important to collect in communities. Focus on strengths. Present data that is relevant to communities. What is the critical evaluation question the community wants to know? Nothing else matters… that much.
- Don’t be afraid of the data. Look at the data as your helper that will guide future efforts. What do evaluators do when they are overwhelmed, anxious, or uncertain? Look to the community, they are experts.
- Take ego out. Have cultural humility. Use a culturally centered approach. Evaluators must ask themselves, “Who am I and why am I here?” ” What is the system, the structure, the power?”
150 indicators to collect. 50 years with communities. 5 years of programming. This is what we know for sure. Empowered data collection is an approach, a mindset, an orientation to reclaiming the past and telling stories of hope for the future.
The American Evaluation Association is hosting Indigenous Peoples in Evaluation (IPE) TIG week. All posts this week are contributed by members of the IPE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.