Hello! We are Laura Beals, Director, and Aaron Gunning, Database Implementation Manager, of the Department of Evaluation and Learning at Jewish Family and Children’s Service (JF&CS), a multi-service nonprofit located near Boston, MA. We are part of an internal evaluation team responsible for collecting, analyzing, reporting, and using client-level data at the agency, including implementation and administration of our electronic client management system. Our programs span many types of services—from intensive long-term one-on-one case management to short-term information and referrals—across several target populations—new parents and their children, older adults and family caregivers, children and adults with disabilities, and people experiencing poverty, hunger, or domestic abuse.
Literature across the fields in which we work continue to reinforce the importance of supporting clients’ individual goals (e.g., parents of young children). As evaluators, we also know that data related to individualized goals can be difficult to collect uniformly and analyze meaningfully. After reviewing current literature, brainstorming with staff, and reflecting on the data architecture of our client management system, we recently revamped how we record and report on individual client goals to align with best practices for service delivery and evaluation use.
Hot Tips:
Connect case notes to service plans. Many of our staff now create a service plan for each of their clients in which they lay out agreed upon goals. Then, each time a staff member documents their work with a client in a case note, they are prompted to reflect on which, if any, goals they worked on during that session. This allows us to create reports that examine the relationships among dosage (i.e., effort devoted to a goal), goal progress, goal characteristics (e.g., category of goal), and client characteristics (e.g., age, disability, etc.).
Create a consistent set of close-ended goal categories and goal attainment responses. For example, in one division, goal categories include: Benefits, Education and Employment, Finances, etc. The attainment categories include “Met,” “Not Met,” and “Continuing”—we purposively steered programs towards using a very simple goal attainment scale which allows for more uniformity in data collection and easier aggregation in reporting. This then allows program leadership to understand what kind of goals are being set and accomplished across programs.
Rad Resources:
Though Googling will reveal many resources on the topic, here are a few that have been particularly helpful:
- The 1968 paper entitled “Goal attainment scaling: A general method for evaluating comprehensive community mental health programs” by TJ Kiresuk and RE Sherman that is the foundation of much of today’s work on this topic.
- King’s College London has created several tools for Goal Attainment Scaling in Rehabilitation.
- Evaluation of Employment Coaching for TANF and Related Populations, a project by Mathematica Policy Research for the Office of Planning, Research, and Evaluation has several relevant reports, including Self-Regulation and Goal Attainment: A New Perspective for Employment Programs and New Perspectives on Practice: A Guide to Measuring Self-Regulation and Goal-Related Outcomes in Employment Programs.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Good morning Laura and Aaron,
I just want to thank you for sharing your experiences as internal evaluators for the multi-service nonprofit. I am an educator, currently working with students and teachers in a school, Kindergarten to Grade Six. I can relate to your role as an evaluator in regards to your statement, “Literature across the fields in which we work continue to reinforce the importance of supporting clients’ individual goals” (Beals and Gunning, 2018). I can see how beneficial the recording document that you have modified will align with providing services and improve evaluation use.
As a Classroom Support Teacher, I am currently responsible for providing additional literacy support for struggling students. Within my role, I use an assessment tool to help identify areas of need and provide targeted interventions to support students’ growth. Creating a document similar to yours, I can better involve students in identifying areas to target our literacy interventions and improve the use of the evaluation tool. Including goal attainment categories would also allow, us to provide efficient feedback to classroom teachers regarding student progress towards mastery. Although the attainment categories would not provide teachers with specific feedback regarding student growth, it would supplement the necessary communication between Classroom Support Teachers and classroom teachers during an intervention cycle.
Thanks again for providing such a practical document that will provide me with an outline to improve evaluation use in education!
Dear Laura and Aaron,
I am leaving a comment as I am intrigued by your work and how to use connect case notes to service plans. I am currently taking a course in Program Inquiry and Evaluation from Queen’s University in Kingston, ON, Canada and part of my research is to reach out to a contributor at AEA 365. I also teach ESL/ELL to adult learners and many have literacy, employment and behavioural classroom management differences. I have often thought about notes per learner case but never thought of creating case notes with goal progression for a learner. We often use words like Beginning, Continuing and Developing but I like the idea of Met, Not Met and Continuing. I will check out the articles on goal attainment scaling. What education theory is most effective would you say in evaluating programs with learners who have their own goals? Thank you for your article.
Best,
Katherine