NA TIG Week: The Needs Index, A Simple Concept Maybe? By James W. Altschuld, Hsin-Ling Hung, and Yi-Fang Lee

Hi, we are James W. Altschuld, Hsin-Ling Hung, and Yi-Fang Lee from The Ohio State University, Virginia Commonwealth University, and National Taiwan Normal University and have presented on, written about, and been involved in NAs for years.  Need is the difference between the desired or what should be state of an entity and its current situation or status. Indexes of needs often come from 5-point Likert scales for the two conditions. Subtract the second score from the first —- larger gaps are needs requiring attention.  Could the concept be simpler?   Maybe Yes.  And Maybe No.  

Maybe Yes needs assessors routinely use double scaling for assessing need.  See chapters in Rus-Eft and Sleezer’s edited 2020 book.  Double scaling fits need well.  All variants of needs indices emanate from this idea and provide utilitarian information. But be careful.

Maybe No, the ‘what should be’s’ in many assessments may be skewed, clustered on high positive scale ratings.  This comes from the pre-structuring of the instrument noted by Altschuld, Hung, and Lee (forthcoming).  Not much information is in these “what should be scores”, resulting in numerous items with sizeable discrepancies.  How to prioritize when the ‘what should be’s’ are similarly high? 

Hot Tip:

One idea would be to place survey items in categories that respondents have rank-ordered before they score the double scaled items in a category.  The biggest discrepancies in the highly ranked categories should help to pinpoint areas of greatest need.

Maybe No, is there an alternative way to word the scales in order to promote a more meaningful subtraction of them?  There is – use behaviorally anchored phrasing.

Hot Tip:

Below are two such scales.  They make the ‘what should be’ and ‘what is’ statuses concrete, not abstract in nature, and in turn so is the calculation of discrepancy. Look at this idea in comparison to the abstract/standard way of framing the scales.  While items of this nature are harder to write, the effort is worthy given the potential gain in meaning due to concreteness.

What should be	What is	Description & Source
What students in my grades should be able to do?	What students in my grade can do?	The scale was behaviorally anchored from Witkin in 1977
How much time per day should be spent studying a specific subject?	How much time is currently spent?	Scale proposed for sue across school subjects from Altschuld and Witkin in 2000

Maybe No, just for conjecture, note that in each of the scaled scores there is measurement error. When we subtract, we compound error so reliability would decrease. What does that mean for validity and the value of discrepancies between the two conditions?  How much faith should we place in decisions based on them?  We don’t have answers but think that practitioners should consider this issue and find ways in which to improve in this regard.  

Final Thought

We are not criticizing needs assessments or the methods used in conducting them, rather we are stressing critical appraisal of the process so that final results are based on firmer ground.

Rad Resources:

• Altschuld, J.W., Hung, H.L., Lee, Y.F. (in press).  What Is and What Should Be Needs Assessment Scales: Factors Affecting the Trustworthiness of Results. American Journal of Evaluation
• Altschuld, J. W., & Witkin, B. R. (2000). From needs assessment to action: Transforming needs into solution strategies. SAGE Publications, Inc.
• Russ-Eft, D. F., & Sleezer, C. M, (Eds).  (2020). Case studies in needs assessment. SAGE Publications, Inc.
• Witkin, B. R. (1977). Needs assessment kits, models, and tools. Educational Technology, 17(11), 5-18. http://www.jstor.org/stable/44421256.

The American Evaluation Association is hosting Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.