AEA365 | A Tip-a-Day by and for Evaluators

CAT | Needs Assessment

Hi, I’m Jim Altschuld, an emeritus faculty member at the Ohio State University in the school of educational policy and leadership. Today I’m sharing some tips on asset-based thinking when doing needs assessments.

First, for some premises to note:

  • Needs, discrepancies between current and desirable conditions, are problems, issues, or difficulties to be resolved.
  • Assets are resources (strengths) that can be used to improve what we do and programs to be delivered.
  • Needs and assets are related, yet different, philosophies through which people view, plan, and eventually evaluate efforts.
  • Most individuals employ/favor one or the other when beginning an endeavor and those lenses color viewpoints.
  • But, consider using both at the same time in an unbiased manner to optimally develop solutions to problems.

And now for some Hot Tips:

Divide and Conquer

To avoid one philosophy from dominating, divide assessors into teams to independently investigate needs and assets.

Creatively Review Findings

Have those that identified needs review the findings for assets and vice versa for the ones about needs.

Encourage Unique Ways to Think about Assets

Consider assets and how they might be employed not just as solutions for needs, but in distinct ways that capitalize upon them. (Be open to such thinking)

Recognize the Hidden Dimension of Assets

Volunteerism, help in patient care, in-kind community service contributions, the skills and abilities of individuals, etc. may be hard to measure but nevertheless is an important asset and serves an invaluable function. Without them the fabric of our society would be much less rich.

Be Instrumental when Resolving Needs and/or Utilizing Assets

The best solutions for underlying problems are those that would be helpful in dealing several problems at the same time (the instrumental concept).  

Examples Illustrating Principles in the Tips:

Use of Public Libraries

Need:      enhancing out of school computer learning for grades K-3

Solution: design programs for children that could be adapted to help elderly populations in getting connected to the internet

Fostering Volunteerism through Recreational Businesses

Need: government services will be cut due to reduced budgets

Solution: recreational businesses offer chits for what they provide to incentivize volunteers to cover some service gaps

Community to the Rescue

Need: tiny start-up business is redoing inside of small building and owners find they don’t have the skills to do parts of the work

Solution: ask the community to participate in exchange for small vouchers ($10) to be redeemed when business opens (community eagerly jumps in, vouchers redeemed, business takes off with sense of community ownership).

 

Rad Resource: Learn more from my book, Bridging the Gap Between Asset/Capacity Building and Needs Assessment.

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

I’m Maurya West Meiers, Senior Evaluation Officer at the World Bank and coauthor of A Guide to Assessing Needs: Essential Tools for Collecting Information, Making Decisions, and Achieving Development Results.

Recently in my Virginia neighborhood, community leaders and members held a charrette. You ask, what is a charrette (or charette)? Charrettes are gatherings (often time-bound and intense, with emphasis on a final push) of stakeholders to vision and map out solutions, while at the same time trying to resolve conflicts and obstacles to those solutions. It’s said to be derived from the French word for chariot or cart, and has its history traced to the word used when 19th century Parisian architectural students put their work into a cart – often in a rush – to transport to an exhibition or evaluation.   Today charrettes are used in urban planning and design processes, usually with communities, and with groups ‘carting’ ideas and information to one another. The charrette process will often bring people and groups together with the goal of getting them moving quickly from investigation to decisions. This is especially helpful when they might otherwise get stuck in the process – sometimes for months or years.

How does it work? While there are many commonalities, there is no standard structure for charrettes. They are unique to the problem and group. They usually involve a facilitated process with larger groups breaking into subgroups. The subgroups work separately to understand the issues, brainstorm, and eventually come to decisions and design solutions. The subgroups then rejoin in a large group to share the subgroups’ work and generate dialogue and some initial decisions. Through this process, multiple interests and information are shared and explored. The process then continues, potentially over multiples sessions. Together members “work on the work” together, which is meant to enhance ownership, move past obstacles, and come to joint decisions.   In community and urban planning situations, it’s not surprising that charrettes are seen to be a good tool where consensus is needed to get to workable solutions.

For those working on needs assessments – which emphasizes determining gaps, causes to those gaps, optimal results, and potential solutions – the charrette process can be another ‘tool’ in the toolbox to explore. In my research on charrettes and related community visioning exercises, I’ve gathered a range of resources from those who have used them and hope you find them useful in your work.

Rad Resources:

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Don’t scroll down! Don’t hit back button yet, read this first! My name is Ryan Watkins and I am a professor at George Washington University. Whereas many aspects of a needs assessment parallels an evaluation, the making of recommendations about what to do next is often a new challenge for us when we move from the back-end to the front-end of project (or program) design. Needs assessments, by definition, do however require that we not only identify and measure the “needs”, but that we also make recommendations about what to in order to satisfy (or partially satisfy) those needs in the future.

The good news is that there are many tools that can help us through the challenges of making these difficult choices. And yes, these are typically very complex decisions without any clear “best” options. The nature of these decisions is that we must forecast the future and what results we can reasonably expect of various alternatives. Foresight is never an exact science and it does present some risks to our projects and institutions. But making recommendations is a distinguishing characteristic of needs assessments.Watkins

The systems theory principle of equifinality tells us, however, that for any result we want to achieve (or need we want to satisfy) there is always more than one way to get there. Therefore we must know about our options and make choices among competing alternatives – which is typically not an easy task. To succeed, and to end our needs assessment with valuable recommendations about what to do next, we should balance our rational decision-making with our value-based decision-making.

Rad Resources:

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

I’m Ingrid Guerra-Lopez, Professor at Wayne State University and CEO of the Institute for Needs Assessment & Evaluation, and I want to help you through the transition from problem identification to solution design. Needs assessment give us an excellent process to identify the critical gaps in results (what), analyze contributing root causes or factors (why), and then make suggestions on (how) to address the problems and opportunities. A useful needs assessment includes recommendations that address problems and opportunities from a systemic perspective.   However, the level of detail with which those recommended solutions should be delivered to stakeholders can vary greatly, and this in turn impacts the focus and scope of work.

A design-oriented needs assessment is focused on analysis for the purposes producing specifications for designing, developing, and implementing solutions effectively. For example, needs assessment with a knowledge and learning design orientation may be triggered by the desire to develop a learning program for a particular target group. The assumption is that the learning program is a solution to a given problem, hopefully identified through a previous and rigorous needs assessment. The focus in this case, is primarily to investigate what the solution should look like, how it should be implemented, and what will enable it to achieve useful results.

Hot Tip: Collaborate with those who will be designing, developing, and implementing solutions to understand their decision-making and information requirements. Clarifying the specific questions that must be answered for solution designers and implementers will ensure you focus on concrete and ‘actionable’ information.

Hot Tip: Ensure that you gather information from a cross-section of stakeholders to understand the specific requirements of the solution. For example, in order to define what a particular educational program should look like, include potential participants, employers of program completers, subject matter experts, instructors, administrators, etc.

Hot Tip: Balance perspectives of “must have” and “nice to have.” Our process must help us understand in concrete terms what the solution must help participants/users do and achieve, not only what they may want as solution features, characteristics, or content.

Hot Tip: Agree ahead of time on a process and criteria for integrating, and potentially weighing, the solutions requirements of the various stakeholders. At the very least, this will help you make recommendations about implementing phases in light of utility, readiness, feasibility, acceptability, resources, and other agreed upon criteria.

Rad Resources:

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

I’m Hsin-Ling (Sonya) Hung, Program Co-chair for the Needs Assessment (NA) TIG. Because of my involvement in the TIG, I have reviewed annual conference proposals since 2008. Over the years I found that some proposals only had a title associated with needs or needs assessment, but these are were not needs assessments. Since needs assessments are often misunderstood, here I share what I think are the key elements constituting a needs assessment.

Using cooking as an metaphor, even a skilled chef would not be able to prepare a tasty dish without the necessary ingredients. So, if you are going to do a basic needs assessment, certain elements must be included. To create a simple ‘recipe’ for planning and reviewing needs assessments, I’m starting with Altschuld and Kumar’s definition of needs assessment.

The fundamental of a needs assessment is to assess need (attend or resolve a problem) for improvement of organizations or systems. A need is the measurable discrepancy between two conditions—“what is” and “what should be.” Without assessing a ‘gap’ it is not a NA. A genuine needs assessment project would describe needs and the conditions associated with them.Hung

So how do we make this dish? After problems/issues have been depicted, you go through a process to understand the situation, the nature and the causes of the gap(s), prioritizing needs, making decisions about their resolution, and finally developing an action plan for improvement. All of these procedures would engage many constituencies and involve collecting much information. It might include organizing a needs assessment committee (NAC), examining root causes, prioritization, making needs-based decisions, and implementing action plan–all key parts of a simple needs assessment recipe. Detailed of all these can be found in the needs assessment kit edited by James W. Altschuld.

Main Ingredient: Identifying Needs as Discrepancy

Key Ingredients: Organizing a NAC; examining root causes; prioritizing; making needs-based decisions; and implementing an action improvement plan.

Lesson Learned: A needs statement presented in discrepancy form is essential, as well as other components presented above. Without these, the needs assessment recipe will produce an unappetizing product

Rad Resources: Check out the Needs Assessment Kit edited by James Altschuld.

Book 1 Needs Assessment, An Overview

Book 2 Phase 1, Preassessment (Getting the Process Started)

Book 3 Phase 2, Assessment (Collecting Data)

Book 4 Phase 2, Assessment (Analysis and Prioritization)

Book 5 Phase 3, Post assessment (Planning for Action and Evaluating the Needs Assessment)

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Welcome to Needs Assessment TIG week on AEA 365! I’m Lisle Hites, Chair of the Needs Assessment TIG, Associate Professor in Health Care Organization and Policy and Director of the Evaluation and Assessment Unit (EAU) at the University of Alabama at Birmingham (UAB) School of Public Health. We hope you enjoy this week’s TIG contributions and look forward to seeing you at our sessions in Atlanta. Today’s posting is about the use of a Scenario Based Needs Assessment to Assess Community Needs.

In over 15 years of working with agencies and communities to assess needs, I’ve learned there’s really no one technique that suits every situation. To illustrate this point I’m sharing a recent, somewhat unusual, community-based needs assessment we conducted that was very specifically focused on the needs of daycare centers to prepare for a potential “active shooter”. While this unfortunate scenario has been fairly well assessed for public schools and other agencies (and considerable resources have been applied to follow-up on identified needs), little attention has been payed to this highly vulnerable daycare group.

We drew our methods from the disaster and emergency preparedness field. Both Federal Emergency Management Agency and Homeland Security have developed evaluation protocols (Homeland Security Exercise and Evaluation Program) to help prepare and test plans to assess needs and prepare for disaster events. With our county’s Children’s Policy Council and local law enforcement we developed a scenario that would initiate conversations and encourage representatives of daycare centers and law enforcement agencies to identify, discuss, and capture needs. By planting evaluators within each discussion group, we captured identified needs, found solutions in some cases, developed plans for finding these, and gathered lessons learned. Altogether, we acquired a reasonably comprehensive set of immediate needs for this non-homogenous group of small daycare businesses. As a result of this scenario based needs assessment and the new connections among daycare center teams and law enforcement officers, daycare centers have a better idea of what they need to do to prepare for an active shooter event and now many have relationships with local law enforcement to begin this preparation.

Lessons Learned:

  1. Needs Assessments can be conducted in a variety of ways using existing data in new and innovative ways.
  2. During the conduct of a Needs Assessment, nothing precludes you from disseminating findings (i.e. lessons learned) and even solutions to needs at the same time.
  3. Sometimes Needs Assessments are an end as well as a means, reducing the needs they seek to assess.

Rad Resource: US Department of Homeland Security (2013). Homeland Security Exercise and Evaluation Program (HSEEP) at https://www.fema.gov/media-library-data/20130726-1914-25045-8890/hseep_apr13_.pdf

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

My name is Natalia Woolley. I am a GEDI scholar from the 2014-15 cohort, and a graduate student in the Community Health Sciences department at UCLA. As part of the GEDI program, I interned at Kaiser Permanente (KP), in the Community Benefit Department’s evaluation unit.

At KP, I provided support for the Community Health Needs Assessment (CHNA), a federally mandated process all non-profit hospitals must conduct every three years. During the internship I focused on the methods used to collect and analyze primary and secondary data. I also collaborated in the department’s efforts to ensure primary data collection methods were systematically responsive to the cultural diversity found in the communities served by KP.

Lessons Learned:

Operationalizing culturally responsive practices is a challenge. Although many scholars have defined culture and articulated its importance when conducting evaluations, it is still a challenge to operationalize some cultural concepts. Nevertheless, I believe acknowledging the challenge is an important step into making needs assessments more culturally responsive.

Successful primary data collection should be culturally responsive. Hospitals must collect primary data as part of the CHNA. This process allows hospitals to better understand the communities’ main health issues, priorities and resources. To successfully connect with community members, hospitals should ensure their outreach and engagement are culturally responsive.

Hot Tips:

Helpful Hints: Secondary data can inform culturally responsive primary data collection. Secondary data provides a great deal of information about the groups living in each community. For example, secondary analysis results can provide a snapshot of the community demographics, including the population percentage with limited English proficiency. Evaluators can use this information to include language appropriate resources in the data collection process.

However, secondary data might miss some marginalized groups. To go beyond the secondary data, it is helpful to identify and contact organizations working with marginalized groups. For instance, Los Angeles County has an extensive database of organizations providing services to groups in need (https://www.211la.org/). Another possible option is to solicit input from community health workers servicing these groups.

The American Evaluation Association is celebrating Graduate Education Diversity Internship (GEDI) Program week. The contributions all this week to aea365 come from AEA’s GEDI Program and its interns. For more information on GEDI, see their webpage here: http://www.eval.org/GEDI Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I am Ryan Watkins from George Washington University. Needs have been described and defined in many different ways of the years (see the December issue of New Directions in Evaluation for elaboration). One consequence of this often perplexing medley of definitions is that the word need has lost much of its meaning. Here I will try to help clarify some important relationships with a little additional precision to our language around needs we can greatly improve our results.

Lesson Learned:

When conducting a needs assessment…

  1. Differentiate Needs from Solutions. It is easy to get tangled up in the distinction between needs and solutions to needs. Don’t confuse what you want to accomplished (closing needs) with the activities and resources used to achieve those results (such as, homeless shelters, mobile phones, and even money).
  1. Use Need as a Noun, not as a Verb. You do NOT need to buy a new car. Nor do they need Internet access. These are options that may (or may not) help satisfy needs. Yet, by using need as a verb (or in a verb sense) we commit ourselves to one solution (a car, or Internet access) before we define the need. Rather, use need as a noun (50% reduction in gender-based violence this year) so that you have a basis for comparing potential solutions and guiding decisions.   
  1. Don’t Confuse Needs and Wants. Really strong wants or desires are frequently elevated to the status of needs through our choice of words. Just ask any 3-year-old in a toy store if they want or need a shiny new toy. Don’t confuse the two.
  1. Expand to Include Individual, Group, and Societal Needs. Maslow’s Hierarchy of Needs popularized the concept that needs are individual. Nevertheless, groups (such as, teams, organizations, and institutions) have needs, as do societies.   Quality needs assessments recognize and align needs across three levels.
  1. Balance Needs and Felt-Needs. Felt-needs are often described as those perceived by the community rather than defined by an external expert. Both views on needs can be valuable. Recognize however that while people have perceptions of needs, their perceptions may not be an accurate reflection of reality. News reports, for example, may distort peoples’ perceptions on crime rates in a city. Therefore, integrate externally verifiable measures of needs in all assessments.

Rad Resource:

This blog is based on: Watkins, R., & Kavale, J. (2014). Needs: Defining what you are assessing. In J. W. Altschuld & R. Watkins (Eds.), Needs assessment: Trends and a view toward the future. New Directions for Evaluation, 144, 19–31.

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

I’m Lisle Hites, Chair of the Needs Assessment TIG and Director of the Evaluation and Assessment Unit (EAU) at the University of Alabama at Birmingham (UAB). Today’s posting is about the use of data visualization to enhance your needs assessment.

Recently, my team worked with a state agency to help them identify potential sites for a pre-k development initiative. We used ArcGIS 10.2 Geographic Information Systems (GIS) technology to geocode and map all child care centers and grant applicants within the state. In turn, these data were displayed on an interactive, web-based map using ESRI’s ArcOnline platform. Supplemental data regarding percentage of people in poverty were added to the map to enhance the decision making process for policy makers (American Community Survey Census).

Displaying these multiple sets of data visually allowed state representatives to see the highest concentrations of four year olds in the state as well as potential gaps in service coverage by existing pre-k programs. In other words, these data were used to reduce the potential for duplication of services and to identify areas of greatest need.

Lessons Learned:

  1. Needs assessments can be conducted in a variety of ways using existing data in new and innovative ways.
  2. While state representatives had ideas of what they wanted to know, data visualization led them to refine their questions and identify additional sources of information to support their “data-driven” decision.
  3. Hardcopy paper maps of each county did not provide enough geographic detail of childcare facilities. To maximize the large amount of disparate data, an online interactive mapping platform was critical to the success of this project.

Rad Resources:

ArcGIS Online (n.d.). The mapping platform for your organization. ESRI.

ArcNews (2013, Summer). ArcGIS 10.2 brings transformational capabilities to users. ESRI.

Azzam, T., & Robinson, D. (2013). GIS in evaluation: Utilizing the power of geographic

information systems to represent evaluation data. American Journal of Evaluation, 34(2),

207-224. doi: 10.1177/1098214012461710

Evergreen, S. (2013). Presenting data effectively: Community your findings for maximum

impact. NY: Sage Publications.

United States Census Bureau (2015). American community census.

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

I’m Maurya West Meiers, Senior Evaluation Officer at the World Bank and coauthor of A Guide to Assessing Needs: Essential Tools for Collecting Information, Making Decisions, and Achieving Development Results.

I often work with groups in carrying out needs assessments, collecting data, training, facilitating retreats, etc.  So I’m always looking for facilitation tips and resources.  Today I’m sharing some favorites.

Lessons learned: 

If your end-goal in your meeting with a group is to gather data or make decisions (through focus groups, multi-criteria analysis, etc.), you’ll want to do some early rapport building to get people comfortable with one another and talking.

  • Make sure the right people are in the room. It seems obvious, but take the time to define your targets in advance and make sure that those participating are those targeted.  Be prepared to gently remove people who don’t fit your pre-defined needs.  Have another coordinator with you to help in this process.  And have your room comfortably furnished and arranged.
  • Learn the names of participants in advance and give a warm greeting when they enter.  These are common networking techniques because they work and put people at ease.
  • Use name badges and table tents. Have these items ready.  You may wish to let participants write their own names instead of pre-printing them.  Perhaps Jennifer prefers to have everyone call her Jen – so give her the chance to write her name as she wishes.
  • Get people talking early.  As people enter the room, introduce them to others – and have ideas listed on a flip chart or card that they can discuss with one another.  Keep people moving and mixing.  Use a chime or bell to signal a move.

Use icebreakers.  An easy icebreaker involves giving participants name badges and asking them to write two or three things they feel comfortable discussing with others.  Example:

Meiers

  • Energizers and games. If your group work – such as in a retreat – covers a lengthy period of time, use energizers (usually involving some movement) or games to keep people alert and engaged.  If you search for energizers on YouTube, you’ll find many ideas you can adopt and adapt for your purposes and you’ll see how they work ‘in action’ and not just on paper.  This quick and easy energizer is one such example.

Rad Resources. Here are some of my “go to” books and websites on facilitation techniques and tools.

 

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Older posts >>

Archives

To top