Hi everyone! I’m Yvonne M. Watson, a Program Analyst in U.S. EPA’s Evaluation Support Division (ESD) and Chair of AEA’s EPE TIG. I’m primarily responsible for managing the division’s internal evaluation capacity building efforts. I read Moneyball: The Art of Winning an Unfair Game by Michael Lewis to understand the importance of a culture that values performance measurement. The book focuses on the life of Billy Beane, former general manager of the Oakland A’s baseball team and his efforts to use unconventional stats to select members of his team. Though I did not inherit my grandmother’s passion for baseball, this book helped me gain some important insights that can be applied to the measurement and evaluation of our programs.
Lessons Learned:
- Leadership and Using Performance Data and Evaluation Results. Billy Beane used baseball stats to make decisions about the players that were drafted, traded or let go. He also demonstrated the critical role of leadership in convincing organizational members to try a new measurement approach. Our organizations have to take the next step in moving from merely collecting information to routinely analyzing performance data and using evaluation results to inform the decisions we make about our programs. Equally important is the task of asking the right questions and collecting the right data to accurately communicate performance. These data can either tell a story of fact, fiction, drama, poetry, or comedy.
- Data Quality. Michael Lewis notes that “…inadequate data led the people who ran major league baseball teams to misjudge their players and mismanage their games.” Similarly, inaccurate and poor quality performance data and evaluation results can lead program managers and decision-makers to misjudge, mismanage, over-value or under-value programs. Investing in the infrastructure to support good quality performance data and evidence needed for evaluation is crucial.
- Insiders and Outsiders. In baseball there are insiders (players, managers) and outsiders (statisticians). Our organizations and programs can benefit from program insiders who have a wealth of subject matter expertise and knowledge about a program’s goals, clients and underlying assumptions and outsiders – analysts, evaluators etc. external to the program who bring a fresh perspective at how to measure or evaluate a program. These individuals can help broaden our thinking about our program’s design, measures, processes and procedures.
- Thinking Differently. Billy Beane exhibited a willingness change things that no longer worked. In short, he learned to adapt. Likewise, our organizations – if they are to succeed, must adapt. We must challenge ourselves not to rely on what we “know or see” but be willing to dig deeper, challenge the status quo and look at our problems, programs and solutions differently.
Rad Resource: Moneyball: The Art of Winning an Unfair Game by Michael Lewis.
The American Evaluation Association is celebrating Environmental Program Evaluation Week with our colleagues in AEA’s Environmental Program Evaluation Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.