A warm hello from the University of Mississippi! I am Sarah Mason, Director of the Center for Research Evaluation, here to talk about research-based strategies for building evaluation-specific situation awareness skills.
What is situation awareness?
Generally speaking, situation awareness is the ability to notice—and correctly interpret—what’s happening around you. Let’s say you have a client who only sees evaluation as a check-the-box exercise; or you’re working in a unique and different culture. Situation awareness helps you detect, understand, and respond to this information.
Situation awareness in evaluation
Evaluators have recognized the importance of situation awareness for some time—the 2018 list of AEA evaluator competencies, for example, dedicates an entire competency domain to context, and encourages evaluators to understand and respond to the contexts in which they work. Despite this, there are few opportunities for evaluators to explicitly build their skills in situation awareness. This might be because evaluation trainers see situation awareness as something that is hard to teach—best learned through experience—or simply because there is so much ground to cover in evaluation training. Regardless the reasons, there are few options for the evaluator who wants to build skills in this domain.
So how do we build skills in situation awareness?
In early 2017, I began exploring strategies for building SA skills, with the hope that I might be able to apply what I learned to evaluation.
Hot Tip:
Research suggests we need (at least) three things if we want to build our situation awareness skills:
- First, we need exposure to a LARGE number of relevant situations, such as evaluation cases or scenarios. People with good situation awareness typically build their skills over years by building up a mental bank of scenarios they use to create mental representations about how things work in the world. If we want to recreate this process, we need exposure to a wide range of evaluation scenarios—many more than we would typically experience in a standard workshop or college class.
- Second, we need repeated opportunities to practice interpreting new situations.
- And finally, we need to receive feedback on how accurate our interpretations are—so that over time we can learn how and why our assessments of a situation were on (or off) track.
Rad Resource
Drawing on this research, I partnered with software development firm Matilda Tech to build EvalPractice — an online platform that combines these three features (i.e. real-world scenarios, practice and feedback) into training modules for evaluators.
In 2018, we trialed our first module, Understanding Evaluation Contexts, and found that evaluators significantly improved their foundational situation awareness skills over the 6-week program. Many participants also said they had become more intentional in thinking about the situations they encountered—and had become more proactive in troubleshooting potential problems before they arose. Get in touch if you’d like to learn more about our work.
The American Evaluation Association is celebrating SEA Affiliate Week with our colleagues in the Southeast Evaluation Association. All of the blog contributions this week come from our SEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.