Claire Tourmen on Introducing Evaluation

My name is Claire Tourmen and I am an assistant professor in education science, in France (AgroSup Dijon). I study evaluators’ practices, skills and training. I’m interested in explaining evaluation to people who don’t know it (people involved in an evaluation for the first time, students etc.) it and I’ll share with you a simple way of doing it. It was used by one of my professors (Gérard Figari) and I find it really didactic.

Hot Tip: When introducing the concept of evaluation, I begin by asking a simple question: “If I say that it’s too cold in this room, what did I do?” I try to make them find the main operations involved in an evaluation: the final operation is to assert a judgment, such as “It is too cold”. It is easy to find. To be able to do it, I had to gather some data (by any means: I checked a thermometer, I shivered, I saw people shivering etc.). Also easy to find. The point is that I had to interpret these data to make my judgment. How did I do it?

Then I ask people a second question: “For example, if I saw that the temperature was around 15°C (or 59°F), what does it means? Is it cold or not?” The answer they always give is: “It depends!” People understand that, to be able to judge any object, you need to compare gathered data (for instance, 15°C (or 59°F), and I introduce the concept of indicator) to other elements (then I introduce the concept of standards) that give a value to it.

We finally work on the different types of standards you can use to evaluate:

1. general/legal norms and rules (15°C, or 59°F, is too cold compared to what is expected as a temperature in this kind of room);
2. objectives (It is too cold because I turned my heater on and I was expecting 19°C, or 66°F) or people’s needs (It is too cold because my audience shivers and finds it too cold to seat quietly) ;
3. what is usual or considered as normal and acceptable (It is too cold because, in this season, the average temperature is around 19°C, or 66°F, in this kind of room);
4. other data on the same object (It is too cold because I went in this room 30 minutes ago, the temperature was hotter and I didn’t expect such a difference).

I conclude by saying that whatever object you evaluate, you need to be clear on what standards you can use (as a basis of comparison) and what data you need to collect to effectively make your judgment.

Links: Stufflebeam (1980) Evaluation in Educational Decision Making (in French): http://bit.ly/Levaluation

And if you want to know more about evaluation in France, please visit the French Evaluation Association (SFE) website and guidelines (translated in English): http://bit.ly/frenchevalassociation – click on “la charte votée en 2003 – version anglaise –“ at the bottom of the page.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.