MN EA Week: Craig Helmstetter on ‘Evaluating’ your community

Hello. I’m Craig Helmstetter, a Senior Research Manager at Wilder Research, a division of the Amherst Wilder Foundation in Saint Paul, Minnesota (right across the Mississippi from Minneapolis).

Remember that quartet that was playing as the Titanic sank? How good were they?

If your answer was “Who cares?!! The ship was sinking!!”, then you probably already get my point. Evaluators are generally good at gaging the effectiveness of specific programs on the specific lives of the specific individuals served by those programs. But we aren’t always so good at tracking the broader community changes that also impact the lives of those same individuals.

Maybe we should be.

Hot tip: Community indicators projects can be a great complement to more granular program-by-program evaluations. Indicators projects can provide context to other evaluations, and can provide a platform to help engage and inform broader community improvement efforts.

Lessons learned: Involve your stakeholders. Early in my own indicators work I thought that I was smart enough to pick indicators all by myself. That is a good way to get ignored. Wilder’s community indicators project, Minnesota Compass, now enjoys some success due to the involvement of over 500 project advisors. First they helped us shape the project. Now they are the project’s champions.

Hot tip: Resist the temptation to include too much data in your indicators work. A couple well-chosen indicators say way more than does a laundry list of every available data source. (And picking those few well-chosen indicators is a great way to engage your advisors.)

Lessons learned: Dig deeper. It is often not enough to look at overall trends. Where possible, cross-tabulate by race, income, gender, and place.

In Minnesota, for example, we take pride in our nation-leading workforce participation rate. However, there is a really embarrassing under-belly to that same indicator – namely that we have the biggest Black-White employment gap in the nation. Both facts are good for evaluators to know. The first is particularly relevant if you are trying to compare the success rate of employment programs in Minnesota to those located elsewhere in the U.S., and the second is relevant if you are evaluating the success rates of programs participants by race.

And don’t even get me started on how important it is to get information like this in front of policy-makers, planners, and the broader public.

Rad Resource: The Community Indicators Consortium’s website includes links to indicators projects throughout the nation, as well as “how to” webinars and publications.

Twin Cities Hot tip: Looking for some nightlife while you are in town for the AEA meeting? First Avenue, made famous in Prince’s movie Purple Rain, is right down the street from the conference.

The American Evaluation Association is celebrating Minnesota Evaluation Association (MN EA) Affiliate Week with our colleagues in the MNEA AEA Affiliate. The contributions all this week to aea365 come from our MNEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.