Hello, we are Greg Lestikow, CEO and Fatima Frank, Project Manager of evalû, a small consulting firm that focuses exclusively on rigorous evaluations of social and economic development initiatives. We champion impact evaluation that maintains academic rigor but is based entirely on our clients’ need to improve strategic and operational effectiveness and increase profitability.
In a recent project, we were tasked with designing a qualitative instrument to complement quantitative data around the sensitive topic of gender-based violence.
Rad Resource: We approached this challenge by designing a focus group discussion (FGD) protocol informed by an article on the “Participatory Ranking Method” (PRM), in which participants rank potential indicators from most to least important. PRM acknowledges project beneficiaries as experts and recognizes the local community as capable of identifying and measuring their progress towards positive change. As such, PRM incorporates local perspectives in the construction of research instruments. By using PRM, we were able to select indicators that are meaningful to the project’s local beneficiaries (in our case adolescent girls affected by violence) and reflective of the concepts they find useful when tracking their own progress. PRM is an ideal evaluation methodology for measuring awareness of sensitive topics and tracking outcomes over time, particularly for projects that may not see any kind of impact in the short or medium term.
Hot Tips:
- Start with a participatory activity to gauge local perspectives and to understand which social practices are considered more or less acceptable in the community. In our case, we asked participants what gender-based violence meant to them.
- To facilitate ranking, show a series of cards labeled with different kinds of social practices (in our case: Shout; Insult, Threaten, Push, Hit, Beat, Kill) and have participants order them from the most to the least acceptable, asking them to explain their decisions. Alternatively, participants can free-list social practices that are common in their communities and then rank-order them.
- Include an open-ended discussion to understand which social practices are acceptable in different relational and social contexts.
Lessons Learned:
- Make sure moderator and note-taker are gender appropriate.
- If you want to obtain a broad range of perspectives but anticipate potential problems with mixing certain community members in the same FGD, create a few FGD groups and separate participants.
- Ask local evaluation or project teams about any other cultural practices to consider before an FGD. For example, in Sierra Leone we started each FGD with a prayer, as this is a standard practice when people meet.
Please share your stories on challenges, solutions, and experiences in dealing with sensitive topics by leaving a comment here or contacting us.
The American Evaluation Association is celebrating Best of aea365, an occasional series. The contributions for Best of aea365 are reposts of great blog articles from our earlier years. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello,
I’m curious whether in your experience with the “Participatory Ranking Method” (PRM), you’ve experienced challenges with obtaining meaningful participants and valuable participation from community stakeholders. In cases where you are addressing a traumatic issue, in your case gender-based violence, I’m assuming you want to engage victims or families of victims. Did you run into issues where the affected stakeholders are not willing to participate and if they do, they aren’t forthcoming with their experiences as, naturally, these experiences are traumatic events and recalling them or speaking to them is a difficult experience for the stakeholder. In such circumstances, how did you solicit meaningful participation, both by attendance of the target stakeholder and also in their participation? Are the meetings more informal or do you bring on board experts, such as psychologists and community leaders to serve as facilitators, in order to make the stakeholders more comfortable with sharing their experiences? I’d be further curious to learn about how you structured the working groups, are there any tools that you used that you’d recommend in such an exercise, be that a simple flip chart or a more developed logic model sort of exercise where you’re informing the stakeholder or what you’re trying to accomplish and how said stakeholders input informs your work, etc. Thank you!
Anna