Michael Matteson on the Hawthorne Effect and Empowerment Evaluation

I’m Michael Matteson and I’m pursuing a graduate degree at the University of Wollongong (Australia).

Lessons Learned – What’s the Hawthorne effect? Why would it matter in Empowerment Evaluation? Most people will remember hearing of the “Hawthorne effect “based on a major study of organizational and environmental effects on productivity which gave confusing and contradictory results over a long period. Commentators suggested that any effects of the experiments were a result, not of the researcher’s manipulation of variables, but of staff feeling important because they were being observed. Stephen Draper gives a definition of the Hawthorne effect that I find useful:

An experimental effect in the direction expected but not for the reason expected; i.e. a significant positive effect that turns out to have no causal basis in the theoretical motivation for the intervention, but is apparently due to the effect on the participants of knowing themselves to be studied in conjunction with the outcomes measured (Draper, 2009).

Looking at this in terms of Empowerment Evaluation, I’ve come to feel that the evaluation team’s experience of the evaluator is a major part of their experience of the evaluation. This makes it a legitimate part of the process use of the evaluation, which is the mechanism expected to enable the empowerment result.

If so, it’s important to clarify what the effect is in each situation. This will depend on what the evaluator is doing, including the atmosphere they’re providing, and the extent to which the evaluator’s involvement is part of the positive reinforcement that team members’ experience, along with their own decision-making, in the course of the evaluation.

The Hawthorne effect can be expected to modify results outside of the conscious parameters of the investigation unless consciously allowed for. In the case of data gathering, I have decided to combine observation of classes run by staff who were part of the evaluation team with observation of classes that weren’t, hoping that any Hawthorne effect in the result-gathering will be canceled out by a parallel Hawthorne effect in my observation of the non-participants.

Hot Tip: Impression management, common in focus groups, and based on Irving Goffman’s work, may be relevant here.

Resource: Draper’s article explains the issues involved and the many uses of the Hawthorne experiment’s continuing legacy. Draper, S.W. (2009, Dec 23) The Hawthorne, Pygmalion, Placebo and other effects of expectation: some notes

Lessons Learned: While the most common explanation of the Hawthorne experience is some kind of “people felt important” effect, Paul Blumberg’s 1969 Industrial Democracy: The Sociology of Participation (Schocken) already argued, based on the original research, that the most likely factor was the level of the participants’ involvement in decision-making. This aspect has been consistently ignored.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.