Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hi again, this is Burt Perrin. I am blessed to live in one of the world’s largely unknown, and spectacularly beautiful spots, the foothills of the Cévennes mountains in France, not far from the Mediterranean. Every turn on almost any road brings another gorgeous, and often quite different, view.
As a mountainous area, there are quite a few microclimates. It is generally recognized that temperature tends to decrease with increased altitude. At the right time of the year, one might see dead flowers that have gone over at the bottom, in full magnificent bloom higher up, and a bit of a further climb flowers that are still far from opening. But microclimates are affected by more than altitude, and not always in the way one might expect. Sometimes, but not necessarily always, it might be warmer higher up, and colder lower down.
The photo to the right is where I live. As you can see, the valley is quite steep. This means that in winter, when the sun is low behind the hills, there is only a limited period of direct sunlight, which may last all day on the Causse (the name for the plateau at the top). Thus, with sun on the Causse all day but only for a couple of hours in the valley, it might be a bit warmer higher up. This changes at night, when the temperatures can drop quickly at higher elevations. Cloud cover, wind, and other factors can also influence the weather (I’ve been focusing here on temperatures, but precipitation and other weather phenomena can also vary) in ways that might be hard to predict.
Similarly, while descending to a lower altitude is usually associated with a rise in temperature, this is not always the case. To get to our nearest market town, we first must climb and then descend through a series of switchbacks. Usually, we often find much hotter weather. But not always! On some, but not other, days, a cold sink, full of moist air, can result in cooler temperatures. Often this may burn off by mid or later morning. But not always! For various other reasons that are often difficult or impossible to understand, “normal” differences, where some areas generally are warmer (or drier, or wetter) than others may not apply on a given hour, day, or even week.
So, what might this mean for evaluation, even of very different types of undertakings such as social programs or health policies? If this were a workshop or a classroom, I would start with a small-group exercise asking participants to identify at least five implications of the above discussion for evaluation. As this is not feasible in this forum, I’d still like to suggest to you, dear reader, to consider this. Without giving you the answers here are some hints: when are mean scores meaningful or not? How do we deal with variability and, indeed, inconsistency?
Burt Perrin (Burt@BurtPerrin.com), formerly from Canada and currently living in France, has been engaged in evaluation internationally for many years, including as a leader and founding member of the CES, EES, and AEA. He has authored other AEA365 blog posts available here including about the backfire effect (Facts and evidence make no difference – or worse), about bureaucracy and evaluation (Can evaluation help make bureaucracy more responsive – or is it part of the problem?), and about Spurious Precision – Leading to Evaluations that Misrepresent and Mislead, with more to come.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.