Greetings. My name is Ricardo Gomez and I currently work as a Research and Evaluation Associate for the National Collegiate Inventors and Innovators Alliance. I am also a doctoral candidate in International Education at the University of Massachusetts Amherst, Center for International Education, and alumnus of the AEA-Duquesne University Graduate Diversity Internship Program.
The opinions of stakeholders are crucial because they can shape the direction of programs and can have an impact on program execution, scalability, and performance. Hence, as a researcher and evaluator, I have always been interested in finding ways to gauge the subjectivity (i.e., opinions, perceptions, attitudes, and motivations) of evaluation participants, and incorporate these into the different phases of my evaluation activities.
Lesson Learned – Q methodology is a powerful tool that evaluators can use to explore the perspectives of evaluation participants. First used and advanced by William Stephenson in the 1930s, Q is a research method that statistically identifies different points of view (or subjectivities) on a given topic based on how individuals sort a set of statements, about that topic.
Traditionally, evaluators have relied on interviews or surveys with Likert-type items to gauge the opinions of evaluation participants. These approaches are not without their drawbacks: the typical outcome of the analysis of Likert-type items is a description of pre-specified independent categories deemed relevant by the evaluator; and interviews can be time consuming and intrusive.
The outcome of a Q study, on the other hand, is a more authentic set of factors that capture people’s attitudes and perspectives about an issue. In Q method, a group of participants (the p-set), sort a sample of items (the q-set), into a subjectively meaningful pattern (the q-sort). Resulting q-sorts are analysed using correlation and factor analysis (q-analysis), yielding a set of factors whose interpretation reveals a set of points-of-view (the f-set).
Rad Resource: Click here www.broaderimpacts.org/aea2011 for an online example of a Q-sort process.
Lesson Learned: Q methodology is an important bridge between qualitative and quantitative methods in that it provides a means for analyzing the phenomenological world of a small number of individuals without sacrificing the power of statistical analysis.
Rad Resource – The International Society for the Scientific Study of Subjectivity (ISSSS) is the official organization committed to the ideas and concepts of Q methodology as enunciated by William Stephenson. ISSSS administers an email discussion list dedicated to exchange of information related to Q Methodology. To learn more about Q methodology, join ISSSS, or become a member of the email discussion list, please visit www.qmethod.org.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators