Hello, my name is Sue Hamann. I work at the National Institutes of Health as a Science Evaluation Officer, and I teach program evaluation to graduate students. Today I’m providing tips to novices in needs assessment (NA).
Hot Tips:
Use the original definition of needs.
- The original definition of NA is the measurement of the difference between currently observed outcomes and future desired outcomes, that is, the difference between “what is” and “what should be.” Novices often plan to address either status or desired future, but they do not realize how much more valuable it is to collect data about both status and future and analyze the difference between these two conditions. Read anything about NA written by Roger Kaufman, Belle Ruth Witkin, James Altschuld, or Ryan Watkins to get started.
Collect data using multiple methods.
- A rewarding and challenging aspect of needs assessment is that an evaluator gets to take almost all her tools out of the toolbox. From census data and epidemiologic data to document reviews to group and individual interviews, needs assessment typically requires multiple methods. The best way to start is to review the literature, both in the problem area of interest and in the evaluation journals. You can start with the New Directions for Evaluation issue (#138, summer 2013) on Mixed Methods and Credibility of Evidence in Evaluation, edited by Mertens and Hesse-Biber. Also use listservs such as AEA’s Evaltalk to discover work that has been done but not published.
Keep an open mind about the validity of qualitative data, particularly interviews.
- I am consistently surprised by the cynicism that novices display toward interviews. We all value our own opinions and complain bitterly when we think someone does not listen to us, but then we devalue the opinions of others. Novices are often unaware of the rich history of research and practice with qualitative data. There are many resources to inform the novice. A seminal paper by Nobel prize winner Herbert Simon and Anders Ericsson (Verbal Reports as Data, in Psychological Review, vol 87 (3), May 1980) from cognitive psychology is accessible and informative. The third edition of a book by Brinkman and Kvale, InterView: Learning the Craft of Qualitative Research Interviewing, just released by Sage Publications, will really help you.
Remember that needs assessment and program planning go hand in hand.
- Collecting needs assessment data is just the first step in program planning. Use Jim Altschuld’s Needs Assessment Kitor other resources to plan for the work needed to conduct this vital component of program planning and evaluation.
Rad Resources:
Coming in Fall 2014, Jim Altschuld and Ryan Watkins are editing an issue of New Directions in Evaluation dedicated to Needs Assessment.
The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.