Observation And Participant Observation: What Should We Observe? Part 3 by Beverly Peters

Hello again! I am Beverly Peters, faculty member in Measurement and Evaluation at American University. This is the third article in a 4 part series on Using Observation and Participant Observation for Monitoring and Evaluation. In the previous article of this series, we discussed how evaluators might get to the so-called backstage of participant observation, where they are accepted as insiders and are able to gain an emic, or insider, understanding of a project and its stakeholders.

Let’s back up a bit. Most of us probably do not even think of our observation and participant observation activities as data collection. This is problematic if we do not carry out observation and participant observation systematically. If we observe and record only certain aspects—perhaps cherry picking the best or the worst parts of the project—our data will not be balanced or useful.

What should we observe when using observation and participant observation? The simple answer is that we need to observe everything that is relevant to our project. However, staying focused on our evaluation design is actually harder than it seems, even with the best of evaluation plans. It is very easy to collect data that is not useful to our evaluation, and this will ultimately make our data unmanageable.

Carrying out observation and participant observation is a balance in collecting relevant data that you planned to collect, knowing what data is extraneous and not collecting it, and recognizing what new data is important. While we use our evaluation design or logic model as an observation plan, at the same time, we need to make sure that we are open to new and unexpected observations and findings that are relevant to our evaluation and could otherwise go unnoticed. After all, this is one of the strengths of observation and participant observation as data collection methods.

The anthropology and qualitative methods literature give guidance on what researchers should consider observing. An anthropologist, James Spradley (2016), told us that we should consider cultural behavior, or what people do; cultural knowledge, or what people know; and cultural artifacts, or what people make and use. Spradley also told us to consider objects, acts, activities, events, time, actors, feelings, and space.

Merriam and Tisdell (2015) add that we should observe:

• The physical setting, physical environment, and context,
• The participants, who is on the scene and what their roles are,
• Activities and interactions, what is going on, and if there is a sequence of activities,
• Routines,
• Informal and unplanned activities,
• Conversations, the context of the conversations, who is talking to whom,
• Nonverbal communication, including physical cues and dress, and
• Physical space

The above is potentially a lot to observe! When I engage in observation or participant observation, I start with my logic model and evaluation statement of work, making sure that I am very familiar with these. I make a table using the observable aspects that Spradley suggested, above. This table helps to illustrate how Spradley’s different observable aspects interact with each other, for instance, how activities coincide with use of space and feelings. I also add a page to my notebook that includes the list from Merriam.

My logic model, evaluation statement of work, Spradley’s table, and Merriam and Tisdell’s list can be very helpful in keeping me on track so that I do not collect too much data, and at the same time making sure that I do not miss relevant data. But again, it is all a balance that comes with practice, a well-defined logic model or evaluation statement of work, and being open to everything that is happening around you.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.