This post is brought to you by Alyssa Na’im of the National Science Foundation’s ITEST Learning Resource Center at the Education Development Center, and several members of the ITEST Community of Practice: Angelique Tucker Blackmon, Innovative Learning Concepts, LLC; Araceli M. Ortiz, Sustainable Future, Inc.; Carol Nixon, Edvantia, Inc.; Pam Van Dyk, Evaluation Resources, LLC; and Karen L. Yanowitz, Arkansas State University. ITEST stands for Innovative Technology Experience for Students and Teachers, and was established in 2003 to address concerns about the growing demand for science, technology, engineering, and mathematics (STEM) professionals in the U.S. The ITEST program helps young people and teachers in formal and informal K-12 settings build the knowledge and skills needed to succeed in a technologically rich society.
Rad Resources: We hosted a session (slides) at Evaluation 2010 and a webinar for the ITEST community (slides) that explored issues relating to culture, context, and stakeholder engagement in evaluation and wanted to share these insights with the AEA365 community.
Lesson Learned: Evaluators’ understanding of stakeholders’ cultural contexts should frame the way they engage and communicate with stakeholders as well as inform their professional practice.
The definition of “culture” includes not only typical reference to beliefs, social norms, and practices of racial, ethnic, religious, and/or social groups, but also references to values, goals, and practices of an institution or organization as well as those of a particular field or discipline. This provides a macro-level definition, reflecting all stakeholders involved in a program and its evaluation. Stakeholders are those who are invested in the program and are affected by its outcomes. We identify stakeholders as belonging to one or more of three groups: decision makers (e.g., funders, principal, director), implementers (e.g., staff, teachers), and recipients (e.g., students, parents, community).
Responding to stakeholders and involving them in the evaluation requires the evaluator to balance multiple goals.
- Stakeholders may have different evaluation needs. Funders typically are more interested in summative results showing impact of the program, while implementers are often additionally concerned with formative information to guide program development.
- Evaluation design and data collection methods should accommodate not just language, age, and developmental requirements, but also situational contexts.
- Communication and reporting depends on the needs of the stakeholder. While all stakeholders should receive some information regarding program outcomes, what they receive, when they receive it, and the degree of detail that they receive depends on the goals of the program.
Acknowledging that each stakeholder’s perspective emerges from their culture and context, and striving to better understand their perspectives enhances evaluators’ abilities to relate to and engage multiple stakeholders in the evaluation process. Evaluators must be active, reactive, and adaptive participants in the evaluation to effectively engage all stakeholders.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.