Hello. I am Tom Lyzenga, a PhD student at Western Michigan University, and chair of the Business, Leadership, and Performance TIG. I am also a training evaluator and performance consultant at a major international corporation. My tip today is about filling in blanks in an impact matrix or model.
Challenge: Sometimes program stakeholders can’t provide a complete impact matrix or model. In some cases, the intervention can be applied uniquely or optionally by each program participant, so we can’t project if or how they will apply it. In other cases, the training intervention has only high level objectives and we can’t predict either how they will be applied or what specific activities or performance impacts can be expected.
Hot Tip: You can apply Rob Brinkerhoff’s Success Case Methodology to fill in the blanks in the impact matrix AFTER the intervention has occurred. By conducting structured interviews with participants who are identified as either very or not successful, we can describe how the intervention was applied, what impact it had, and what factors contributed to the success of the application. We can also describe the relative success of the program in terms of the percentage of successful participants. As a result, stakeholders can get a sense of the value of the intervention and influence the success factors for the next intervention.
Rad Resource: The demonstration I am offering, Filling in The Blanks in Models and Matrices with Success Case Methodology, at Evaluation 2013 in Washington, DC is designed to demonstrate the use of the Success Case Method to complete impact matrices and models.
The American Evaluation Association is celebrating Business, Leadership and Performance (BLP) TIG Week with our colleagues in the BLP AEA Topical Interest Group. The contributions all this week to aea365 come from our BLP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Want to learn more from Tom? He’ll be presenting as part of the Evaluation 2013 Conference Program, October 16-19 in Washington, DC.
Hello Tom,
My name is Mona and I am currently enrolled in the Professional Master of Education program at Queen’ university.
I came across your article when it came to training evaluator and performance consultant at a major international corporation.
I think Rob Brinkerhoff’s Success Case Methodology is a great evaluation method when it comes to finding result fast and in an efficient way. By interviewing participant that were least successful followed by participant that were very successful, you are able to find out what works and essentially what does not work after the intervention had occurred. The success Case method focuses on performances and it really looks at the best and worst participant in order to identify what is working and what is not working. (Brinkerhoff, 2003) By being able to analyse the two groups of participants, you are able to make tweaks to the program and make long-term plans that will benefit future participants in the program. . “Using SCM, people can get useful and accurate information about new initiatives: What results are being achieved using new approaches, what is working, what is not, and how it can be improved.” (Brinkerhoff, 2003)
I think in order to have a successful evaluation, it is important to recognize that success is not only based on how well the participants do during the intervention but rather looking at those participants that struggled and finding out the reason why they did. That is what I think works well with the Success Case method of evaluating. The focus is on both extremes and therefore, it gives us a clearer idea of what steps to take when it comes to improving a program.
Cheers,
Mona