I’m Melissa McGuire, a Partner at Cathexis Consulting Inc., an evaluation practice in Toronto, Canada.
I was inspired by Ricardo Gomez’s recent post on Q methodology to share an adaptation of Q which we’ve been using in our practice that we call Group Priority Sort.
Lesson Learned: As many evaluators have experienced, it’s important to involve stakeholders in the selection of outcome indicators to get their buy-in and commitment. That said, we also know that consulting with stakeholders can stall the process, because different indicators are important to different groups. Worse still, sometimes organizations choose too many indicators in an attempt to appease everyone, which makes for a very cumbersome evaluation.
Rad Resource: Group Priority Sort is one method to rapidly get buy-in, build community, and find out which indicators are most important to measure, taking into account the perspectives of all stakeholder groups.
It is a hands-on exercise where stakeholders with different perspectives meet in person. The stakeholders are divided into small groups to sort the potential indicators (each printed on a cue card) along a continuum of relative “importance”.
a) Consensus-based relative importance score for each indicator (for each group and averaged across groups).
b) Rich qualitative information about reasons given for relative importance of each indicator.
c) Engaged participants with a greater understanding of the perspectives of other stakeholders, greater understanding of the evaluation and greater ownership of the process.
The information obtained through Group Priority Sort can then be used to inform decisions about which indicators to ultimately include in your evaluation.
Lesson Learned: Carrying out a Group Priority Sort can be resource intensive and requires careful planning and execution. However, done right, the method makes it possible to quickly narrow down priorities in a sound and meaningful way.
If you find opportunity to use Group Priority Sort, we’d love to hear about your experience.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to email@example.com. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.