Happy Saturday! I’m Elizabeth DiLuzio, AEA365 Curator + Contributor and Evaluation + Strategy Manager at Good Shepherd Services. Like many of you, I have been giving a lot of thought to our role as evaluator during COVID. More specifically, as our country shifts from stay-at-home initiatives to restore and reconstruct, how can we assist governments, schools, foundations, and organizations as they make the difficult decisions that lie ahead?
The good news for us is that we have a lot to offer during this time. In fact, (most of) our political leaders have set the precedent that we use data to drive our next steps. So, today, I want to explore four key characteristics of a data-driven initiative and the related questions that we as data leaders need to be posing in order to move forward effectively.
Like any good initiative, it’s important to begin by identifying the organization’s goals and anti-goals in the coming months. Instead of the HiPPO (Highest Paid Person’s Opinion) making these decisions, it’s critical to convene large groups of diverse people and ask the questions in order to find our answers.
Questions to Ask: What is it that we want to accomplish? What are our goals? Our anti-goals?
It’s easy to fall into the trap of basing our decisions on the data that’s available. Instead, it’s important to start from an inquisitive place. Christian Rudder, author of Dataclysm, speaks truth when he states that “behind every number there’s a person making decisions: what to analyze, what to exclude, what frame to set around whatever pictures the number paint. To make a statement, even to make just a simple graph, is to make choices, and in those choices human imperfection inevitably comes through.”
This is why, once goals are identified, we approach the next phase from a place of inquiry instead of a place of instant discernment. Here, too, we need to convene large groups of constituents to decide how we will measure the success of our goals (and what it looks like when we have reached our anti-goals).
Questions to Ask: What do we need to know in order to make this decision? What will the signals be when we are endangering our goals?
At a minimum, this aspect of a data-driven culture entails openness about the aggregate data that is impacting leadership’s decisions. This means regular and diverse communication with constituents. At its most mature, this characteristic calls leaders to share the deidentified raw data they are using to make their decisions via an open data platform so that any constituent can do their own analysis at any time. This is a continuation of the spirit behind creating participatory goals and metrics – if people have access to their own data, they can not only be informed but help to make informed decisions in the future.
Questions to Ask: What channels of communication do we have available to us? What data are we willing and able to share?
Even with the most brilliant minds working within the most perfect data-driven culture, the decisions that are made will not be perfect. There will inevitably be mistakes in the modeling, in the interpretation of the data, and in the decision-making. What matters is how you coach leadership through the response to these mistakes. To respond with an iterative, learning mindset is key here. This is where data-driven cultures are created and strengthened.
Questions to Ask: What is possible and palatable right now?
Now it’s your turn: What are some other important considerations we need to take into account as evaluators in the next phase of COVID? Share with us in the comments below or in our Evaluators’ Slack Channel, where you can comment, share links, and even upload resources. It’s easy to join and free to use. We’ll see you there!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.