Hi from Jane Fields and Tim Sheldon, Research Associates at the Center for Applied Research and Educational Improvement (CAREI) at the University of Minnesota. CAREI is an independent research and evaluation center in the College of Education and Human Development (CEHD) at the University of Minnesota. As part of our work at CAREI, we serve as external evaluators for EngrTEAMS, a five-year, $8 million project funded by the National Science Foundation. EngrTEAMS is designed to increase students’ learning of science content, as well as mathematical concepts related to data analysis and measurement, by using an engineering design-based approach to teacher professional development and curriculum development (https://sites.google.com/a/umn.edu/engrteams/).
Hot Tip: To make evaluation more relatable to clients, connect evaluation to problem-solving in many fields.
Hot Tip: Shared language enhances collaboration.
Bransford & Stein (1984) state that “All areas of study, including biology, psychology, economics, physics, and chemistry, involve a host of core concepts and theories that people have found to be helpful for conceptualizing (defining) and solving important problems” (p. 20). With EngrTEAMS, we recognized early on that the steps in an evaluation cycle (see figure below from the CDC) are closely aligned to the steps of the engineering design process used in the project (see figure below from EngrTEAMS). Thus, with project staff, we talk about how when we, as evaluators, engage stakeholders and describe the program, we are essentially defining and learning about the problem. Focusing the evaluation design is similar to the planning stage in the engineering design process. Gathering credible evidence, justifying conclusions, and using the lessons learned are similar to the solution phases in the EngrTEAMS design process of trying, testing, and deciding. According to Thomas & McDonagh (2013), “Shared language is critical to collaboration…Developing and nurturing a shared language is an essential element to enhance communication and collaboration” (p. 46). When we were able to demonstrate that the problem-solving process used in evaluation was similar to that used in their engineering design framework, program staff easily saw the connections and were even more receptive to the evaluation side of the work.
Design Process
Rad Resources:
- Bransford & Stein’s (1984) The IDEAL Problem Solver: A Guide for Improving Thinking, Learning, and Creativity.
- Thomas’ (2013) Shared language: Towards more effective communication.
Samples of cycles in other fields:
- Scientific method graphic: http://www.softwaretestingclub.com/profiles/blogs/the-science-of-exploratory-testing
- Continuous improvement cycle graphic: https://leankit.com/learn/kanban/continuous-improvement/
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.