Hello again! I am Beverly Peters, and I teach in American University’s Measurement and Evaluation Program. This is the second article in a 4 part series on Using Observation and Participant Observation for Monitoring and Evaluation. In the previous article of this series, we discussed the differences between observation and participant observation, and how evaluators use these tools in their work.
We focused on participant observation, and learned that this tool can be particularly useful for evaluators interested in gathering rich, emic data; that is, when we get to what anthropologists call the “backstage.” At the backstage, evaluators are accepted (in as much as possible) as insiders; at this place, stakeholders are open and upfront with us about their perceptions and project operations.
We know that anthropologists take weeks or even months to move to the backstage. Since evaluators do not have that luxury of time, how can we get to the backstage more quickly, without sacrificing the collection of rich, emic data?
As evaluators, we usually get entrance to a project or community through gatekeepers that provide introductions and pave the way for us to get to know people and move to the backstage more quickly. But gatekeepers are only one step that evaluators use to get to the backstage.
Hot Tips:
Moving from the front to the backstage in your participant observation activities, and collecting data that is useful for your evaluation, depends on many aspects, including, but not limited to your:
- Knowledge of the project, and its culture or ethos. While knowing about the project helps you to ask informed questions about it, being appreciative of the project culture helps you gain entry into it. For example, if you are evaluating a project that embraces sustainable agriculture and you critique this, you are likely to face challenges getting to the backstage.
- Intercultural communication skills. Knowledge of the stakeholder culture and understanding its cues are equally as important as your knowledge of the organization and its projects.
- Knowledge of local languages, if you are working in a language other than your own. Speaking another language can be an important step in being accepted into a stakeholder and project community—and it also helps you to understand how people think. When you do not speak or cannot learn another language, you need to choose your interpreters wisely, for both language and cultural knowledge, and with potential micro-politics challenges in mind.
- Ability to build rapport. This relates to your personal approach: How you approach being in an unknown situation and speaking to unknown people, and if you feel comfortable not being in control. Being a good conversationalist, being inquisitive and asking questions, and taking the time to learn and care about the project and its stakeholders can help you to build the rapport needed to move to the backstage.
- Self-reflection. Being self-aware helps you to pinpoint your potential biases and identify how these could impact your interactions with stakeholders and the data that you collect. Consider your implicit biases, and find ways to counter these actively. Reflect on who you are as an evaluator, your responsibilities to stakeholders, and the unwelcome power that these afford you in your work.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Beverly,
I’ve read the first two parts of your series of articles on Using Observation and Participant Observation for Monitoring and Evaluation. I’m currently taking a course on Program evaluations, and for my course’s project, I was wondering a lot about data gathering. As a new person in the evaluation world, I have very little knowledge of observations for evaluation purposes. Your articles gave me a lot to think about to be able to complete my assignment.
I like that you describe observation as a continuum. It is not a black or white situation, but more of a grey area. Standing between the two extremities can bring a different perspective that can add a lot to an evaluation.
You mention that evaluators can get entrance to a project or a community through gatekeepers. This backstage access generally gives more detailed information, similarly to what insiders would get. Do you think those gatekeepers can inadvertently (or on purpose) add biases to an evaluation? I feel like that person can steer the evaluator to see the program through his own eyes, bringing biases to the information provided. As I was reading your post, it was the first thing that popped into my mind. Maybe I see the biases because I tend to see the word gatekeeper with a more negative connotation. I also seem to focus a lot on the different ways biases can affect the evaluation.
On a final note, I really like your hot tips. They are all things that should be obvious, but I bet most people don’t think about it. One that was completely new to me was the ability to build rapport. I agreed when you said, “taking the time to learn and care about the project and its stakeholders can help you to build the rapport needed to move to the backstage.” I also like that you mention biases in the self-reflection section; they cannot be completely removed, but being aware of them is a good step in the right direction, along with trying to find ways to counter them.
Thank you for those great articles. I’m looking forward to the next two parts.
Emilie
Thanks for the note, Emilie. I am glad that my work is helpful!
You make some really great points. We usually have so-called “gatekeepers” in this work. Maybe the evaluation gatekeeper is a program participant or administrator, or perhaps a project funder. We recognize that the gatekeepers have their own sets of biases, and that do we as evaluators too. We aim to counter these biases through awareness and self-reflection; and we design comprehensive, multi method evaluations in concert with stakeholders to support voice for all participants.
I hope this helps to provide some perspective. Thanks again for your comment.
I have applied observation and participant
observation for data collection in a practitioners case stude of a peacebuilding network.
In the study, I wholely stepped into backstage by participating per say in one of the network’s state chapter meeting.
It was data enriching. And with my background in Sociology/Anthropology, it was natural and fulfilling!
Thanks for the comment Ogadimma! Attending a meeting is a great example of how we can use the backstage to better understand people and projects. Thanks for sharing!