Fatima Frank and Greg Lestikow on Using a Participatory Ranking Methodology in Focus Groups for Sensitive Topics

Hello, we are Greg Lestikow, CEO and Fatima Frank, Project Manager of evalû, a small consulting firm that focuses exclusively on rigorous evaluations of social and economic development initiatives.  We champion impact evaluation that maintains academic rigor but is based entirely on our clients’ need to improve strategic and operational effectiveness and increase profitability.

In a recent project, we were tasked with designing a qualitative instrument to complement quantitative data around the sensitive topic of gender-based violence.

Rad Resource: We approached this challenge by designing a focus group discussion (FGD) protocol informed by an article on the “Participatory Ranking Method” (PRM), in which participants rank potential indicators from most to least important. PRM acknowledges project beneficiaries as experts and recognizes the local community as capable of identifying and measuring their progress towards positive change. As such, PRM incorporates local perspectives in the construction of research instruments. By using PRM, we were able to select indicators that are meaningful to the project’s local beneficiaries (in our case adolescent girls affected by violence) and reflective of the concepts they find useful when tracking their own progress. PRM is an ideal evaluation methodology for measuring awareness of sensitive topics and tracking outcomes over time, particularly for projects that may not see any kind of impact in the short or medium term.

Hot Tips:

• Start with a participatory activity to gauge local perspectives and to understand which social practices are considered more or less acceptable in the community. In our case, we asked participants what gender-based violence meant to them.
• To facilitate ranking, show a series of cards labeled with different kinds of social practices (in our case: Shout; Insult, Threaten, Push, Hit, Beat, Kill) and have participants order them from the most to the least acceptable, asking them to explain their decisions.  Alternatively, participants can free-list social practices that are common in their communities and then rank-order them.
• Include an open-ended discussion to understand which social practices are acceptable in different relational and social contexts.

Lessons Learned:

• Make sure moderator and note-taker are gender appropriate.
• If you want to obtain a broad range of perspectives but anticipate potential problems with mixing certain community members in the same FGD, create a few FGD groups and separate participants.
• Ask local evaluation or project teams about any other cultural practices to consider before an FGD. For example, in Sierra Leone we started each FGD with a prayer, as this is a standard practice when people meet.

Please share your stories on challenges, solutions, and experiences in dealing with sensitive topics by leaving a comment here or contacting us.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.