AEA365 | A Tip-a-Day by and for Evaluators

TAG | feedback loops

AEA365 Curator note: Back in January, AEA365 readers asked to read about how evaluators deliver negative findings to clients and stakeholders. This week, we feature 5 articles with four evaluator perspectives on this topic. 

Hello AEA 365 readers! I’m Glenn Landers, the Director of Health Systems at the Georgia Health Policy Center (Andrew Young School of Policy Studies, Georgia State University). A large portion of our work is evaluation, and we’ve been fortunate enough to work in every state and many of the territories. No one likes being the bearer of bad news, but sometimes it can’t be helped.

Recently, I was engaged in a developmental evaluation of a collective impact initiative that was intended to last ten years with ample funding. Five months in, we realized the initiative was in trouble. One year in, the project was basically over. Several techniques helped incorporate the bad news into the process as learning.

Hot Tip:

Evaluation Advisory Groups! We always try to have an advisory group made up of those whose work is being evaluated and those who will use the products of the evaluation. This way, we can test what we are learning with a small group for feedback before sharing with a wider audience.

Hot Tip:

Feedback loops! We also set up several feedback loops with the funder, the facilitator, and the work’s steering committee. This way, we shared information in small packets and gained the benefit of group sense making so that everyone understood why things weren’t working as planned.

Hot Tip:

Evaluation as Learning! We were fortunate to have a project sponsor who was interested in learning from what was not working just as much as what was working. Knowing this upfront helped us to be more comfortable in being candid.

Lesson Learned:

There’s nothing that can be substituted for being present with the people who are doing the work. Relationships and trust develop over time. The more present you are with them, the more they will be able to be in a position to hear the results – whether good or bad.

What’s worked for you in delivering bad news?

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hi! I’m Melissa Rivera, an evaluation consultant with Habitat for Humanity International.

I provide evaluation consulting to local Habitat organizations and am pleased to share with you a conclusion we’ve reached about feedback loops as an approach to increasing community engagement.

Over the past 40 years, Habitat for Humanity has worked with people around the globe to help build or improve a decent and affordable place they can call home. The most important element of our mission is the partnership between Habitat and the homeowner, and we continuously seek to keep homeowners and their input at the center of what we do. Through Habitat for Humanity’s Neighborhood Revitalization approach, Habitat engages with local residents to help them respond to community aspirations, revive their neighborhoods and enhance their quality of life.

Over the past three years, we have piloted a systematized approach — feedback loops — to collect and analyze data and share findings with community residents to co-create strategies and projects that address their aspirational goals. We started with 12 local Habitat organizations in the United States.

Our goal was to examine if integrating standardized feedback mechanisms increases the level of resident engagement in these neighborhoods. We evaluated as follows:

  • Explored whether technology that allows for real-time feedback could help facilitate this process.
  • Developed a rubric to track the progression of integrating feedback loops.
  • Evaluated data associated with the feedback loop prototype.
  • Completed focus groups and key information interviews.
  • Synthesized all data to provide quarterly updates on the level of resident engagement.

Lessons Learned:

  • Feedback loops work!
  • Having a rubric to monitor the coalition’s integration of feedback loops helps Habitat provide clarity and to track continuous progression.
  • The feedback loop methodology can increase or sustain the level of community resident engagement. This occurred in each of the 12 pilot communities. As an example, prior to the pilot, 11 residents of a Missouri neighborhood were engaged in community projects. Now, more than 46 residents are actively participating.
  • Integrating the feedback loop methodology during community conversations can change the dynamic between community residents and the people and agencies that support them. One example from our pilot is a Georgia neighborhood where community residents said they felt as if they did not have a major role in decision-making regarding community improvement.  Now community residents and partners are both co-creating projects and planning on their future efforts with equal input.

These learnings provide insight for Habitat for Humanity on how collecting shared metrics has strengthened the evaluation of community engagement. The methodology helps us to continuously refine programs, projects, or systems.  Our next step is to undertake a detailed exploration of the sustainability of feedback loops.

Rad Resource: Survey Gizmo has built in reporting mechanisms that enabled community residents to see how their community responded. Another great technology is Scantron’s Class Climate product. This system is especially helpful for agencies that work in communities where residents prefer to provide their feedback through a paper form.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Archives

To top