Hi, I’m Chari Smith with Evaluation into Action. I work with a range of nonprofits and foundations in the Northwest area.
Evaluation is a learning opportunity. Nonprofits need help to set up their organizations so that they can integrate program evaluation into their daily activities. It is a critical piece to ensure they can do program evaluation long-term.
Portland Homeless Family Solutions (PHFS) is a great example of a nonprofit that achieved integrating evaluation into the organization long-term. In 2013, I worked with PHFS to create a realistic and meaningful evaluation plan for their shelter program. During that process, I learned the case managers were not consistent in how and what they documented. A key part of the plan was standardizing the data collected, so they aligned to their goal: Families get housed.
Today, PHFS continues to use the evaluation plan. Here is an example of how they use the data: They track the families’ length of stay in the shelter. Data showed an increase in the length of stay. In the past, it had been 32 days on average for about 4-5 years. Then it increased – families were staying in shelter an average of 75-90 days.
They investigated why that change occurred. Turns out some families in the shelter have more barriers to housing than others, and need more one-on-one case management than other families. A program change was made based on data. A single person was dedicated to help the families identified has having more barriers, and provide more one-to-one case management. Average days in the shelter decreased to 57.
Hot Tip: To engage nonprofit organizations, ensure anyone who is a part of data collection, analysis, reporting, communicating and/or usage is a part of the planning process. A good place to start is to administer an evaluation opinion survey, including questions that will provide insight into their perspective on program evaluation topics. Questions may include:
- What do you think the program goals are?
- What impact do you think the program has?
- Do you have concerns about evaluation?
- What do you hope to learn?
Then, use their answers to build a process that addresses those responses, and at the same time, will build bu- in to doing program evaluation. They start to see the value in doing program evaluation as a learning opportunity, not a burden.
Lessons Learned: It took three years for PHFS to migrate from managing data in spreadsheets to a database solution. It’s a challenge to find a database vendor that is the right fit in terms of costs and products.
Rad Resource: The Organizational Capacity to Do and Use Evaluation is one of my favorite issues of New Directions in Evaluation Journal. Loaded with case studies, great to learn from.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Chari,
I am a student in the process of completing a Master’s of Education at Queen’s University. In one of my courses, we have been studying evaluation and the value of evaluation within organizations.
Your article jumped out at me as it highlights non-profits which is of interest to me as well as tangible evaluation use within these organizations. Throughout my studies so far about evaluation, it seems that having internal motivation is a contributing factor to the success of a program. Do you have any recommendations of ‘how’? I find myself going back to the ‘how’ question often. It is one thing to evaluate and ensure a program runs smoothly; but, it is another to be inclusive and ensure that the stakeholders are respected and the policy makers are too. What would be your recommendations for overcoming resistance? Perhaps bolstering motivation through committees would ensure that evaluation is tackled collaboratively and all voices are being heard.
Motivation does contribute directly and indirectly within an organization and especially in evaluation. What you said in your hot tip – “To engage nonprofit organizations, ensure anyone who is a part of data collection, analysis, reporting, communicating and/or usage is a part of the planning process” is something that I will take with me in my career. Receiving positive feedback and contributions from members of an organization helps the programs to move forward as well as enhancing value in the work being done.
Thank you for this great article.
Katie
Hi Chari,
I enjoyed reading your post, and hope that my response still finds its way to you, given you wrote it almost a year ago.
Non-profit organizations and social justice issues are a passion of mine. I desire to help people but want to do it in a way that is helpful, not harmful. I’ve seen, heard, and read many examples of people with the best of intentions believing they are helping, but are actually hurting. I recently attended a workshop on that exact issue titled, “Helping without Harming.” The biggest think I took away from that workshop was the importance of working alongside people and finding out their needs, without assuming them for them – it’s not about what we think they need. I see this spilling over into program evaluation where evaluators must have open eyes and be prepared to hear from all views and all those involved in the program.
Your hot tip: “To engage nonprofit organizations, ensure anyone who is a part of data collection, analysis, reporting, communicating and/or usage is a part of the planning process” speaks to the heart of how to work alongside a non-profit organization during an evaluation. Thank you for sharing this excellent tip and reminder! I completely agree with you that if evaluators can get program users, participants, practitioners, and stakeholders on board with the evaluation, then they will see it as beneficial and not a burden. Which brings me to my question for you, regarding your post. Have you ever had a situation where those involved in the program were mistrusting or resistance to your presence? If so, how did you handle it? Was there something you did beyond simply involving them in the planning process?
Thanks again,
Kim V.
Great post, Chari! I’m curious, what database system is PHFS using?