We are Rodney Hopson, former (2012) AEA President and Professor of Evaluation (Educational Psychology: QUERIES) at the University of Illinois-Urbana Champaign, and Aneta Cram, Ph.D. student at Victoria University of Wellington in New Zealand and AEA Program Chair for the Indigenous Peoples in Evaluation Topical Interest Group.
In a recently published article for Evaluation Matters – He Take T? Te Aromatawai, the journal associated with the Aotearoa New Zealand Evaluation Association (ANZEA), we reflect on the challenges and lessons learned in developing program theory, the role of culturally responsive evaluation (CRE), and the program evaluation standards (PES) that are employed or need to be employed during the development of program theory. See our article here, collaborated with Marvin Powell, Asia Williams, and Akashi Kaul, all from George Mason University.
- While we evaluators recognize that program theory is integral to the evaluation process, navigating the development of program theory is much more difficult, especially for evaluators who concern themselves with balancing stakeholder engagement, dialogue, and reflection through a culturally responsive lens. We reflect on PES (U2: Attention to Stakeholders, F3: Contextual Viability, P1: Responsive and Inclusive Orientation, P2: Formal Agreements, A4: Explicit Program and Context Descriptions) as important considerations we realized in working with stakeholder groups to facilitate our evaluation process and lessons learned.
- The negotiation of role and expectation by the evaluation team was a challenge that raised curiosity and frustration among stakeholders. Even with a Memorandum of Understanding, evaluator roles and expectations are and can be questioned throughout an evaluation. We reflect on PES (U3: Negotiated Purposes, U4: Explicit Values, F2: Practical Procedures, P6: Conflicts of Interests, A6: Sound Designs and Analyses) to assist us in recognizing and documenting larger power dynamics that undergirded our evaluation exercise, the strategic priorities of the organization, and the subtle issues among staff members at the table.
- Engage early and engage regularly with clients! Building trust and being seen is essential to good evaluation practice.
- Even though it is fairly intuitive, do not underestimate the value of relationships and building a strong foundation with clients and other stakeholder groups early on in the evaluation process.
- We, as evaluators, are constantly learning. Regardless of the stage in the evaluation process, regular reflection is key to ensure that we are being responsive to the stakeholders and culture that underlie the evaluand/program.
- The third edition of the Program Evaluation Standards is a core resource for evaluators. A checklist developed by Western Michigan University Evaluation Center may prove useful for those looking for a thinner and practical resource.
- The Center for Culturally Responsive Evaluation and Assessment, an international community of scholars/practitioners, serves as a portal for organizations and individuals seeking to better understand and apply cultural responsiveness. Send an abstract to the CREA VI by February 7, 2020 for its conference in Chicago, IL this fall.
This week, we’re diving the Program Evaluation Standards. Articles will (re)introduce you to the Standards and the Joint Committee on Standards for Educational Evaluation (JCSEE), the organization responsible for developing, reviewing, and approving evaluation standards in North America. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.