Hierarchical Card Sorting (HCS) – a simple tool for qualitative inquiry by Rick Davies

Hi, I’m Rick Davies, an evaluation consultant based in Cambridge, United Kingdom.

What is it: Card (or pile) sorting is a method used by ethnographers to elicit people’s mental models: the categories they use, what belongs to these categories, and how the categories relate to each other. Hierarchical Card Sorting (HCS) is a variant I developed for research with Bangladeshi NGOs in the 1990s.

Why use it: A HCS can help make people’s knowledge more explicit, publicly available, contestable and usable.

How is works. In a brief summary, there are four steps:

1. Listing the entities to be sorted. These can be people, organisations, locations, or events. The respondent should be expected to know something about them.

2. Sorting: This involves the reiteration of a question: “What do you think is the most significant difference between all these (entities). Please sort them into two groups of any size. Describe the difference between them and the difference it makes.” The result is an annotated tree structure, a nested classification like this one below.

3. Choosing. The respondent can then be asked a second type of reiterated question, but here the format is more flexible. “Which of these two groups do you think has been more successful?” or “Which of these two groups have you spent more time with” And in each case asking a follow up question: “Why do you think so?”

4. Comparing. A succession of binary choices will generate an overall ranking, as seen below. Rankings of entities based on different choice questions can then be plotted and compared.

How has it been used? – some examples:

• To differentiate the nature of household poverty in rural Gambia
• To differentiate possible types of implementation partnerships with local NGOs in Nigeria
• To differentiate stakeholders in the evaluation of a global health initiative

How does this compare to the Most Significant Change (MSC) method? Both take an ethnographic perspective. Both ask for the respondent’s own perspective. Both ask a descriptive question and then an explanatory question. Both make use of Gregory Bateson’s idea of information as a “difference that makes a difference” (disputed as it can be). But MSC asks about differences over time i.e., change, whereas HCS asks about significant differences which are static. While MSC is intended to be a participatory process and can be used on a large scale, HCS is best used in a one-to-one or small-group interview setting.There is some “interoperability”: A tree structure generated by a HCS can be used to analyse a set of MSC stories.

Lessons Learned:

I am repeatedly reminded by experience that it is important for the interviewer to ask “dumb” i.e., naïve questions. As in “Why is that significant”, and not hastily assume I know what the respondent is thinking.

Hot Tip: 

A detailed how-to-do-it guide is available on line: https://mande.co.uk/special-issues/hierarchical-card-sorting-hcs This includes potential problems and how to deal with them, potential uses for planning and evaluation purposes, and different ways of analyzing HCS exercise data.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.