My name is Karlyn Eckman. I’m a researcher at the University of Minnesota Water Resources Center, and also a long-time United Nations consultant. I’d like to share an evaluation tool that we’ve borrowed from our colleagues in public health and psychology. The knowledge, attitudes and practices (KAP) study method has been used for decades in international health circles. KAP is a highly focused study of people’s knowledge, attitudes and practices surrounding a specific issue or problem. KAPs are done twice, before and after a project activity, and are useful for planning as well as evaluating outcomes. We thought that the KAP method might have some utility for evaluating environmental projects and have successfully tested it in a variety of projects in Minnesota.
We found that the social outcomes of natural resources projects programs are generally overlooked. Do audiences actually adopt and maintain a recommended “best management practice” for water quality? Does knowledge increase as a result of an environmental education program? How do opinions and attitudes change over time about stormwater runoff or invasive species? A major challenge is that the staff of public agencies charged with managing natural resources in Minnesota lack training in social science research or social evaluation methods. Since project staff have limited time and resources to devote to evaluation, we wanted a cost-effective, practical “do-it-yourself” evaluation method that could be learned with training and mentoring.
Rad Resource: The KAP approach seems to fit the bill for documenting specific outcomes and results. To date, we have conducted seven KAP studies on various programs (invasive species; nonpoint source pollution; urban stormwater; septic management; road salt application; shoreland buffers; etc.). KAP has proven to be an adaptive, flexible tool that can be used in multi-disciplinary programs. It can be scaled up or down to accommodate small or large audiences (purposive and random probability sampling). KAP can be used in combination with other research or triangulation methods, such as participant observation. Agency partners indicate that they like the method’s low cost, relative ease of use, and ability to demonstrate social results.
Hot Tips and Cool Tricks: Start the KAP study process by thinking about what you don’t know about your audience, but should. Brainstorm about gaps in your knowledge about your audience that will be needed to evaluate outcomes, and make a list of those gaps. This gap analysis list becomes the basis for building a KAP questionnaire.
Form a small team of three to five people to do the gap analysis, and to develop the KAP questionnaire. This helps to ensure that key pieces of information are not overlooked. Keep the KAP questionnaire short and very focused, ideally under twenty questions.
This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.
This sounds interesting and potentially applicable to some of the work I am involved with related to job developers working with people with disabilities. Do you have any suggestions of articles or other resources to learn more about this approach?