EPE TIG Week: Evaluating Natural Resources Projects using the KAP Study by Karlyn Eckman

My name is Karlyn Eckman and I research the human dimensions of natural resources programs in Minnesota and also in developing countries.  While working in Somalia in the 1980s I learned to use the KAP (knowledge, attitudes and practices) study method to evaluate project outcomes. Since 2006 our University of Minnesota team has conducted about forty KAP studies on a variety of environmental projects.  Most studies assess whether people have adopted and maintained recommended conservation practices, or acquired new knowledge about an environmental issue.

One such project is the Native Shoreland Buffer Initiatives (NSBI) project of the Minnesota Department of Natural Resources.  NSBI encouraged shoreland property owners on three northern Minnesota lakes to adopt conservation practices to improve water quality. Many lakes are impaired by excess nutrients such as nitrates and phosphorus from agricultural operations, or bacteria from septic systems.  The NSBI designed customized conservation messages for landowners based on specific impairments, and tested different ways of delivering those messages (radio programs, social clubs and gatherings, peer-to-peer messaging, visits by technical experts, brochures, etc.).

The KAP study provided us with an economical and focused way of planning an evaluation-ready project. We began with a “gap exercise” to review what we didn’t know about landowners, and would need to know before designing NSBI outreach efforts. Based on the gaps we identified, we developed a questionnaire that was administered as a formative evaluation. Outreach messages were based upon survey results. Two years later we repeated the survey. Data from the two surveys was directly comparable. We had solid evidence that the NSBI was not only effective in changing people’s behaviors and adopting conservation practices, but also in improving their knowledge about water quality. We also learned which message delivery approaches were most effective: peer-to-peer messaging, social gatherings and visits by technical experts.

KAP studies are effective in many environmental projects. We have used method to evaluate:

* The outcome of water quality projects on urban and suburban residents (for example, changes in knowledge and attitudes of residents about local water bodies)

* Effectiveness of training (for example whether snowplow drivers reduce the application of de-icing chemicals)

* Adoption of best practices by farmers to reduce soil erosion and agrochemical use

* Cost-effectiveness of water quality projects as measured by reduced purchases of road salt by county public works departments

KAP studies can be used at any scale, with qualitative or quantitative data, and with multiple methods.  We have used KAP studies to evaluate the human dimensions of invasive species, use of agricultural practices, recreational behaviors, shifting cultivation in developing countries, application of road salt by snowplow drivers, and many other issues. In each case the KAP study method has provided critical data demonstrating project success, which is important to donors and state agencies wanting evidence of the value of investment of public resources in environmental efforts.

Rad Resources

The final evaluation report for the NSBI project can be found here.

Photo: The author (left) and team triangulating survey data on Johnson Lake in Itasca County, Minnesota

The American Evaluation Association is celebrating Environmental Program Evaluation TIG Week with our colleagues in the Environmental Program Evaluation Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.