Using Performance Management Principles and Metrics to Build Evaluation Capacity by Disa Patel

Hi! I’m Disa Patel and I work at the Wisconsin Department of Children and Families (DCF). One of my favorite roles is building evaluation capacity throughout the department. Currently, I’m leading a project called Metrics that Matter to help my division’s teams identify, well, metrics that matter to them.

Cool Trick:

Talk in a language that makes sense to your audience.

I’m in the Bureau of Performance Management so it makes sense that we have a history of gathering and tracking data to gauge our performance (it’s in the name :)). DCF’s performance management system has evolved away from a PerformanceStat model for various reasons in recent years. To leverage the history of performance management at DCF, the Metrics that Matter project is using performance management principles to introduce evaluation concepts.

  1. Use logic models to visualize ‘why’ programs (and teams that support those programs) work.
    • I love logic models, but before I get into how I used them, I want to acknowledge what I learned from Nicole Robinson, founding Board President of Milwaukee Evaluation! Inc. Logic models can aid white supremacy (intentionally and unintentionally) because they superimpose artificial linearity onto the change process that supports white supremacist goals and aims. Programs are complex and beautifully chaotic. Logic models force us to count things that are measurable and that moves us away from the deeper, more complex social change goals that need to be adopted but are difficult to measure and depict on a logic model. If we’re using logic models, we need to be intentional and clear on what we are and are not measuring.
    • Logic models can help tie together what we do with what we expect to see accomplished. It answers the questions, What do we have? What are we doing? and What are we impacting?
    • I worked with IT, Finance, and program support staff on their various roles and responsibilities. It’s often hard to connect our work to the bigger vision of our organizations, but taking time to visualize how what we do impacts the children and families of Wisconsin was extremely beneficial.
  2. Performance management is distinct from evaluation as it tells us how well we are reaching our goals, not how well programs are functioning.
    The GAO Program Evaluation: Key Terms and Concepts factsheet helps to clarify these differences (shout-out to recent AEA365 post for sharing this resource).
Program evaluation and performance measurement are distinct but complementary. (Diagram showing comparison.)

Hot Tip:

Don’t call it Metrics that Matter

While Metrics that Matter is a great name and I would love to see it expand as I created it, especially because you can use M&M candies if we were in person, it just didn’t work. People jumped to metrics when we wanted people to take a step back, connect their work to outcomes, identify goals, and THEN work on metrics. Having “metrics” in the title took away from this work.

Lessons Learned:

  • *Before* you start to build a logic model – scope the team. We found out months later that a team pre-determined their scope because the timeline was too tight for specific staff to be involved. With my limited understanding of the work the team does, I didn’t catch how the scope and focus were not aligned.
  • Get excited! People often find data and metrics dull, and I can’t blame them. But finding things that get people excited, like problems they want fixed in their work, is a great way to build excitement in a group.

Rad Resources:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

1 thought on “Using Performance Management Principles and Metrics to Build Evaluation Capacity by Disa Patel”

  1. Hi Disa,
    Thank you for your blog post. I’ve been studying evaluation this term through my Masters in Education program, and I also work for a provincial government agency in Ontario. I can relate to the different requests and requirements related to both performance management and program evaluation. I too have had experiences where people jump in to the metrics without thinking about the program goals and outcomes. It’s a great idea to include the program team and to discuss the project scope before planning a logic model and I thought the GAO program evaluation document was very helpful too. I might send it around to my colleagues before we discuss planning our next project. And yes, while ‘Metrics that Matter’ seems like a flashy title, perhaps a re-brand to something like ‘More than Metrics’ would be a better fit. You’d still be able to include M&Ms at your in-person events too! Thanks again for an informative post.
    Dan

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.