AEA365 | A Tip-a-Day by and for Evaluators

Jul/17

9

Aligning concepts and language demystifies evaluation by Jane Fields and Tim Sheldon

Hi from Jane Fields and Tim Sheldon, Research Associates at the Center for Applied Research and Educational Improvement (CAREI) at the University of Minnesota. CAREI is an independent research and evaluation center in the College of Education and Human Development (CEHD) at the University of Minnesota. As part of our work at CAREI, we serve as external evaluators for EngrTEAMS, a five-year, $8 million project funded by the National Science Foundation. EngrTEAMS is designed to increase students’ learning of science content, as well as mathematical concepts related to data analysis and measurement, by using an engineering design-based approach to teacher professional development and curriculum development (https://sites.google.com/a/umn.edu/engrteams/).

Hot Tip: To make evaluation more relatable to clients, connect evaluation to problem-solving in many fields.

Hot Tip: Shared language enhances collaboration.

Bransford & Stein (1984) state that “All areas of study, including biology, psychology, economics, physics, and chemistry, involve a host of core concepts and theories that people have found to be helpful for conceptualizing (defining) and solving important problems” (p. 20).  With EngrTEAMS, we recognized early on that the steps in an evaluation cycle (see figure below from the CDC) are closely aligned to the steps of the engineering design process used in the project (see figure below from EngrTEAMS). Thus, with project staff, we talk about how when we, as evaluators, engage stakeholders and describe the program, we are essentially defining and learning about the problem. Focusing the evaluation design is similar to the planning stage in the engineering design process. Gathering credible evidence, justifying conclusions, and using the lessons learned are similar to the solution phases in the EngrTEAMS design process of trying, testing, and deciding. According to Thomas & McDonagh (2013), “Shared language is critical to collaboration…Developing and nurturing a shared language is an essential element to enhance communication and collaboration” (p. 46). When we were able to demonstrate that the problem-solving process used in evaluation was similar to that used in their engineering design framework, program staff easily saw the connections and were even more receptive to the evaluation side of the work.

The EngrTEAMS Engineering
Design Process

An example Evaluation Cycle developed by the CDC (https://www.cdc.gov/eval/framework/)

 

 

 

 

 

 

 

 

 

 

 

Rad Resources:

Samples of cycles in other fields:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

 

No tags

No comments yet.

Leave a Reply

<<

>>

Archives

To top