I’m John Cosgrove, an evaluator who is committed to utilization-focused evaluation. I am currently working with community colleges around the country to improve evaluation efforts and the use of data for continuous improvement. Clients indicate they want evaluation and data to drive continuous improvement and decision-making. Although a good place to start, data collection alone won’t get the job done. In her excellent article, Data Don’t’ Drive, Alicia Dowd reminds us that data alone won’t lead to continuous improvement.
I remember sitting in a faculty session at the University of Michigan Assessment Institute and listening to Richard Alford discuss the Craft of Inquiry. It was the end of the day and with all apologies to Dr. Alfred, I must admit I was thinking more about crafting dinner plans than inquiry, but then he made a very simple, yet powerful statement: “You don’t make the pig fatter by simply weighing it every day”.
Assessment, evaluation, data collection—whatever you want to call it—must be more than keeping score. If we don’t learn something and then take action from what we learn, we are simply recording data for the sake of recording data. As colleges are further inundated with the call for evaluation data from stakeholders, including legislators and funding agencies, they would do well to remember to structure such efforts with a meaningful culture of inquiry.
People engaged in the development of public questions and the thoughtful interpretation of data will drive continuous improvement. We should expand evaluation efforts to determine not only what works, but why it works. We offer the following framework help link questions, data collection, interpretation and action.
- INQUIRE—What Do We Want To Know? Define the specific evaluation questions.
- DISCOVER—What Do We Know? Identify data sources and methods of data collection.?
- INTERPRET—What Does the Data Tell Us? Work with stakeholders to analyze and interpret results/data.
- DEVELOP—What Actions Need To Occur? Use results to develop strategies for continuous improvement and further evaluation.
Rad Resources:
- Data Don’t Drive: Building A Practitioner-Driven Culture of Inquiry To Address Community College Performance. A Lumina Foundation for Education Research Report, 2005.
- Specify The Key Evaluation Questions. Better Evaluation: Sharing Information To Improve Evaluation, 2014
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: District Operations Manager at Mannich Consulting Services (3 Positions) - Jobs in Nigeria
Well put, John. Although for some reason after reading your post, I wanted to eat some pork…
Go Giants!
Thank you for this sage reminder and for sharing Alicia Dowd’s piece; it was new to me.
Well said, John. I like to think of it as a “culture of learning”. Community Colleges, as well as all higher education institutions need to learn how to learn. They need to become better at (as you say) using data to make needed changes. This is especially true now as community colleges are being threatened by lower government funding, competition from online programs, and the speed of change in the economy and jobs.
Insightful and thought-provoking. Nice piece