DUP Week: Using Single Subjects Methodology in Program Evaluation by Brian Molina

Greetings, I am Brian Molina, a graduate student at in Western Michigan University’s Industrial/Organizational Behavior Management doctoral program. I have conducted single subject research and implemented performance improvement projects across many different settings and organizations.

Lessons Learned:

• Single subject research is used to evaluate program effectiveness across large groups. The belief that single subject research can only be used with one person at a time is a common (yet understandable) misconception. The number of people in the analysis depends upon the frame of reference we are interested in evaluating. For example, we may easily choose to evaluate the performance of a single individual, a single team, a single department, a single organization…and so on.
• The methodology is friendly toward organizations with limited resources or experience in evaluation.  Evaluation can be as intimidating! For organizations that are new to the process, interpreting and understanding large group statistical analyses may be difficult. Single subject research typically results in data that show behavior change over time, which can easily be interpreted by researchers and clients alike. Easier data collection and analyses make it more likely that organizations will begin and continue evaluation of their activities.
• Single subject designs allow for more maximum flexibility in implementing program changes. Conducting research is rarely an orderly process that goes precisely according to plan. Single subject methodology accommodates this unpredictability well. Changes in behavior are rapidly observable during the course of program implementation, not simply at the conclusion of sometimes-lengthy data collection. This allows leaders to make on-the-go changes to the intervention that best serve the client, without contaminating the results of an experimental evaluation.

Rad Resources:

• Introduction to Single Subjects Methodology Design Options for Home Visiting Evaluation
• Deochand’s thesis on how to graph single subject data
• Example of program evaluation using single subjects methodology: Impact of a Positive Behavior Support System on Staff Behavior at the Boys and Girls Club

The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.