Frances Julia Riemer on Ethnographic Evaluations

I am Frances Julia Riemer from Professional Evaluation and Assessment Consultants (PEAC).  PEAC specializes in school-based evaluations, regularly incorporating ethnographic methods in our work. I’ve also conducted several full-scale ethnographic evaluations of government and non-governmental funded initiatives in and outside the US and in and outside schools.  These were both summative and formative; they illuminated why and how project activities worked, and equally often, how and why they didn’t.

Ethnography does not begin with a blank slate. “The ethnographer enters the field with an open mind, not an empty head.  Before asking the first question in the field, the ethnographer begins with a problem…data collection techniques, [and] tools for analysis” (Fetterman, 2009, p. 11).  This problem-centered approach, coupled with ethnography’s focus on “what’s happening here” expands an evaluation from the question of “is an intervention working” to include information about ‘how’, ‘what’ and ‘why.’

Lessons learned:

Ethnography’s strengths are in its capturing of implicit cultural meanings, its examination of what people do and what those doings mean to them, and its situating those practices within a larger theoretical context.  But while “the idea that a social setting ought to seem more complex rather than greatly oversimplified as a result of ethnographic inquiry is a step in the right direction” (Wolcott, 1999, p. 181), translating these complexities into the sound-bite language of policy makers and the logistical vernacular of project directors can pose a challenge.

Hot Tip: A series of executive reports, crafted to specific constituencies, outlining relevant evaluation findings in non-technical language, is a good way to communicate with policy makers and practitioners.

Hot Tip: Ethnographic evaluations are most useful when a wide-ranging grasp of an intervention and its relationship to society is important to judging program quality.

Lessons learned:

Doing ethnography is time consuming and typically involves longer interactions to understand the view of participants.  In order to conduct an ethnographic evaluation within a more limited timeframe, a more radical approach to data collection is necessary.  This often includes the use of a team of investigators and a clearly focused inquiry, including the identification of tasks to be undertaken.

Hot Tip: Don’t miss the roundtable Ethnographic Evaluation: A Realistic Choice or a Contradiction in Terms? at the AEA conference on Friday, November 4, 1:35 PM.

Rad Resources:

Fetterman, D. (2009). Ethnography Step by Step. Thousand Oaks, CA: Sage.

Schensul, J.J., LeCompte, M.D., Hess, A.G., & Nastasi, B.K. (1999). Using Ethnographic Data: Intervention, Public Programming, and Public Policy (Ethnographer’s Toolkit). Lanham MD: Altamira

Spindler, G., & Spindler, L. (1987). Interpretive Ethnography of Education: At Home and Abroad.  Hillsdale, NJ: Lawrence Erlbaum Associates.

Wolcott, H. F. (1999). Ethnography: A Way of Seeing. Lanham MD: Altamira.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.