Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
We are Eden Kyse and Erin Bunger Johnson, Director and Associate Director of the Center for Research and Evaluation on Education and Human Services (CREEHS) at Montclair State University.
Program evaluation occurs in real-world, messy contexts. Doing evaluation from within a public institution adds an additional layer of context.
How do we maintain a focus on stakeholder-engaged, culturally responsive, technically sound, and actionable evaluation that contributes to community well-being while attending to these multiple, sometimes competing contexts?
Reflecting on our desire to do the most good as evaluators, with our colleagues, we set out to document our organizational values and integrate them into the daily work of the Center. These values serve as our “north star” when the seas of real world context get turbulent.
Lessons Learned:
In order to articulate and integrate values as evaluators, we have learned there is a continuous cycle of prioritizing, developing, integrating, and checking (like our projects!), with reflection built in at each phase.
Hot Tips:
- Prioritize what makes your approach unique and what you want to emphasize
First, it is important to determine what makes you or your organization unique. What is it that you do or how you do it that sets you apart from others? We found it helpful to start with AEA’s Guiding Principles for Evaluators, relevant posts on AEA365, and other resources for experiences and examples of value setting. This helped us determine what to emphasize about our practice. We workshopped wording to best represent our priorities concisely and approachably.
- Develop your values in ways that are aligned with your values
For us, it was important to engage our stakeholders. Over several weeks, we developed a set of value statements in a working group of Center staff and students. Drafts of the statements were shared with the larger Center team and with current and former clients, staff, and partners for input. Anonymously, reviewers were asked how statements reflected their experiences working with the Center and for suggestions. After further refinement, the statements were formalized on the Center’s website and built into proposal and reporting templates.
- Integrate and institutionalize your values into everyday actions
Build a values check into all stages of a project. Our values are relevant throughout our work, including at all stages of an evaluation. Questions we ask at each stage of a project include:
- Proposal development: Is it likely that this work will contribute to community improvement?
- Evaluation planning: Whose voices should be heard?
- Instrument design: How can we increase access for respondents to engage in the evaluation?
- Data collection: In what location, language, and format will data be collected?
- Data analysis: Why do outcomes vary across demographic or geographic subgroups?
- Reporting: How can findings be communicated to increase likeliness of application?
Embed values checks into routine reflection time. In addition to project activities, we learned to ground Center administrative and professional development opportunities in our values. This helps keep the “north star” in focus. At monthly meetings, we invite team members to shout out a colleague who has demonstrated Center values in their work. We also hold time for smaller groups to develop resources specific to value-related activities. When considering professional development resources, we ask staff to link growth in the identified area to Center values.
- Check yourself: Empower others to challenge you and be willing to act when you have not upheld your values
Despite best attempts, faltering happens. We learned to encourage and empower others to call us out, validate each other’s challenges, and work together to find solutions and continuously improve.
Should you find yourself swept away by those turbulent forces of competing context, come back to your values as “north star” and navigate an adjusted route to better accomplish your intended approach.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.