Welcome to the Needs Assessment TIG’s week on AEA 365! I’m Lisle Hites, Chair of the Needs Assessment TIG and Professor of Community Medicine and Population Health at the University of Alabama. Enjoy this week’s blogs and we look forward to seeing you at the Needs Assessment TIG sessions at AEA 2023.
Today I’m linking the overall theme of this year’s AEA conference, The Power of Story, and how it applies to the practice of Needs Assessment. That said, I’m going to further and suggest that while we are embracing “The Power of Story,” we also keep in mind one key element of validity for story: “Who Controls the Narrative?”
You may be thinking: “What does the story of a needs assessment target – be it a community, an organization, etc. – have to do with assessing needs?” Or perhaps you are thinking: “The needs are the story,” or something to that effect. Let’s explore these questions, and in doing so, I’m going to take some liberties with the purpose of story. Let’s consider the following aspects of the story we should keep in mind. Everyone knows that telling a falsehood, a lie, deliberately reporting erroneous information, is wrong. The same can be said of deliberately withholding information when we report a finding. An example: If in a needs assessment we find that 40% of men and 40% of women have hypertension…and further find that 90% of men have their hypertension under control, while only 20% of women have their under control, it would be inaccurate to report that addressing hypertension control is not a concern in this county. However, if data collected was gathered from a male only data source, this conclusion would seem accurate and likely lead to a different resource allocation than if the complete story were known. While this situation may seem highly unlikely, when multiple demographic factors, such as race, gender, age, and education are all factored in, this is a common occurrence. As you are collecting the data accordingly, you, the evaluator, are in control of the narrative. If you cannot tell the complete story, it is your duty to clearly indicate what those limitations are.
As evaluators we try to seek out the truth, do our due diligence and come to the most accurate conclusions we can. That said, evaluation budgets are rarely adequate for a truly rigorous evaluative effort. We work on compressed time frames, with limited resources, and try to accomplish Herculean tasks because…, well, because they must be done. When conducting a needs assessment, it will rarely be possible to reach out to all stakeholders. Just as, when conducting a population survey, you won’t likely reach every member of the population. Just as with purposeful stratified sampling in surveying, it is equally important that the voices you sample to inform your needs assessment are both representative of that target you’re assessing. Moreover, it is important that you include voices that represent ALL sides of the story you are looking to tell.
Lessons Learned
- The importance of gathering the complete story (essential to rigorous needs assessment)
- Any part of the story that is missed may result in an inaccurate needs assessment, which may lead to inaccurate future actions. There is no check or balance here. After data is collected, what you get is all you have. Holes are not typically noticeable.
- You are controlling the narrative (like it or not)
- Use the power of story responsibly. In the end, you have a story, and you can use the power of story to elicit change. Unfortunately, an inaccurate story may be just as powerful, or even more so, than the true story.
Rad Resource
New to Needs Assessment? Check out our TIG member Ryan Watkins’ prior post on Five Must Read Articles on Needs Assessment
The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the NA AEA Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.