Hello, AEA Community! My name is Jen LoPiccolo and I’m an evaluator with BECOME, a social change agency in Chicago with a mission to nourish communities affected by poverty and injustice to make their vision of a thriving community a reality. I’m excited to share why I believe the field of evaluation can be catalytic in creating a more just world.
Evaluation teams often hold a variety of skillsets: critical thinking, statistical analysis, conceptual / design skills, report writing, facilitating, presenting, community engagement and more. Additionally, our work often occurs in partnership with a unique intersection of stakeholders including community participants in social service programs, non-profit or government program providers, philanthropic leaders, key decision makers, and families. This interesting proximity to multiple groups positions us as evaluators to bridge the gaps for social change and inform powerful structures on community insights and learnings.
Given our positionality, it is imperative that we recognize the power and responsibility we have to our communities. We are charged with taking data from collection through analysis, report writing, and utilization. However, we do not always have a process that allows people experiencing a program to hold us accountable through providing them with an opportunity to collectively interpret the data.
HOT TIP: Involve community in each step of the process by including them from the beginning of the evaluation design by forming a committee. Community members benefiting from or interested in joining a program bring a critical lens to developing evaluation questions, thinking through methods that are responsive to their community’s cultural logistics, and identifying gaps that evaluators may overlook.
HOT TIP: Contextualize your data for your audience. Often in Chicago communities and elsewhere, the various audiences reading a final evaluation reports, building strategy from the findings, and making key decisions from key recommendations may be disconnected from the community participants impacted most by the program.
RAD RESOURCE: In the article Eyes on the Prize: Multicultural Validity, Dr. Kirkhart illustrates why validity requires congruence between an evaluation theory and its cultural context.
In addition to thinking about the context surrounding our evaluation theory and situating our data, it is equally important to consider the broader narrative about the participants in programs we are evaluating. For example, while working with a fatherhood program on the southside of Chicago, it’s critical to consider the way negative stereotypes may come up and be conscious of framing our report narrative from a strengths-based and human-centered approach.
You can look forward to hearing from James tomorrow on how to integrate social justice into your work.
The American Evaluation Association is celebrating Multicultural Issues in Evaluation (MIE) TIG Week. The contributions all this week to aea365 come from our MIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Jen,
First off I want to say thank you for posting about such an important topic.
I completely agree with you that as evaluators, we are the bridge between the communities we are evaluating and the higher structure. This puts evaluators in an important position that can cause a number of positive social changes that can impact a number of peoples lives. In addition, I also agree with your statement about finding a process where evaluators can be held accountable. I believe that evaluation data should be accessible and shared with all parties in order for learning to occur. I have learned that having stakeholders involved in the evaluation process is called collaborative approach to evaluation. This collaborative approach can be very beneficial for both evaluator and participants as it can provide a number of learning opportunities for both parties. Furthermore, through this collaborative approach evaluators can ask questions that reflect the cultural demands of the program and like you said fill in some of the gaps that evaluators may overlook. This not only helps the stakeholder but also allows the evaluator to see the program through a different lens.
Additionally, your point about contextualizing data for your audience is important to make those changes that are going to be positive for the community and programs being evaluated. Too often higher society makes changes to programs without ever realizing or understanding who the program is. The people reading data and evaluation reports are too far removed from the program that they cannot see the impacts of decisions they are making. I think you are correct in that evaluators should provide context to its audience in order to bridge that gap. Furthermore, I do believe that a way to bridge that gap can be through the questions being asked which lends back to your first “hot tip”.
Lastly, I really appreciated your comment on a strengths based approach to the framing of reports. Our biases and our potentially negative stereotypes, whether conscious or not, can influence decisions being made about the report. I agree that evaluators need to present their findings in ways that like you said are human- centered.
Thank you for your post.
Carlo Iacono
Hi Jen,
My name is Voula and I am a graduate student completing a Professional Masters of Education at Queen’s University in Kingston, ON. As a part of the curriculum, I am currently completing a course on Program Inquiry and Evaluation. As a requirement of the course, I have been introduced to the AEA365 Blog.
While reading through the various articles, I found your article very interesting. While I understand that evaluators have an extensive knowledge regarding evaluation programs, I can see how individual bias can affect evaluation. Although the intention is not for the bias to affect the evaluation, I can see how it can be evident in an evaluation from an individual who does not have experience in the program.
I love the idea of including a human-centred approach in the evaluation as many of the community members have experience attending the programs and can be of great value in providing input for making the program function at a greater capacity.
I also believe that it is important for the people who attend the program to hold the evaluator accountable because the people who experience the program may have a different account of the program than the people who are making decisions and I feel that it is important for the program participants to have a voice and one way for the community members to have a voice is to form a group of members that are interested in providing input for the evaluator to understand the purpose of the program and what is needed in order for the program to continue to assist the members.
Hello Jen,
My name is Rachel, and I am a currently a student in a Masters of Education program at Queen’s University in Ontario Canada. I am taking a course on Program Inquiry and Evaluation. Through the course we have been introduced to the AEA365 blog and have been asked to engage with an article and author that we found interesting.
I feel that you have highlighted a perspective on evaluators that I had not considered before reading your article. It is quite true that the skill set that evaluators bring to social programs juxtaposed with the unique position that they are in working with stakeholders and communities is special. It seems that evaluators have the opportunity to be bridge-builders between existing programs and the communities that the programs serve.
Bringing the community into the fray and involving them empowers their voices and gives them an opportunity to be heard. Social change and justice is brought about when the voices of the underrepresented or disenfranchised are heard. In turn, this empowerment can create and foster engagement in communities.
Your Hot Tip of creating committees of to help design evaluation is a great idea. Those most affected by social programs can be key contributors to creating and asking effective questions and not missing gaps. Program users offer a unique perspective that an independent evaluator who is not a member of the community may lack.
Additionally, if the evaluator follows the evaluation through to the implementation stage, creating a committee that helps to oversee this process could further ignite interest, commitment and an obligation to see an evaluations goals through to program change.
Thanks for your post – I thoroughly enjoyed it!
Rachel