AEA365 | A Tip-a-Day by and for Evaluators

TAG | special needs

Greetings and welcome from the Disabilities and Underrepresented Populations TIG week.  We are June Gothberg, Chair and Caitlyn Bukaty, Program Chair.  This week we have a strong line up of great resources, tips, and lessons learned for engaging typically underrepresented population in evaluation efforts.

You might have noticed that we changed our name from Disabilities and Other Vulnerable Populations to Disabilities and Underrepresented Populations and may be wondering why.  It came to our attention during 2016 that sever of our members felt our previous name was inappropriate and had the potential to be offensive.  Historically, a little under 50% of our TIGs presentations represent people with disabilities, the rest are a diverse group ranging from migrants to teen parents.  The following Wordle shows the categorical information of presentations our TIGs presentation

Categories represented by the Disabilities and Underrepresented Populations presentations from 1989-2016

TIG members felt that the use of vulnerable in our name set up a negative and in some cases offensive label to the populations we represent.  Thus, after discussion, communications, and coming to consensus we proposed to the AEA board that our name be changed to Disabilities and Underrepresented Populations.

Lessons Learned:

  • Words are important! Labels are even more important!
  • Words can hurt or empower, it’s up to you.
  • Language affects attitudes and attitudes affect actions.

Hot Tips:

  • If we are to be effective evaluators we need to pay attention to the words we use in written and verbal communication.
  • Always put people first, labels last. For example, student with a disability, man with autism, woman with dyslexia.

The nearly yearlong name change process reminded of the lengthy campaign to rid federal policy and documents of the R-word.  If you happened to miss the Spread the Word to End the Word Campaign, there are several great video and other resources at

High School YouTube video

YouTube Video – Spread the Word to End the Word




Bill S. 2781 put into federal law, Rosa’s Law, which takes its name and inspiration for 9-year-old Rosa Marcellino, removes the terms “mental retardation” and “mentally retarded” from federal health, education and labor policy and replaces them with people first language “individual with an intellectual disability” and “intellectual disability.” The signing of Rosa’s Law is a significant milestone in establishing dignity, inclusion and respect for all people with intellectual disabilities.

So, what’s in a name?  Maybe more than you think!


· · · · · · ·

Greetings, I am Jane Eppard, Vice-Chair of the Family Hope Foundation. We founded FHF in 2009 with a mission to invest in children with special needs through family support, engaging community experts, advocacy through collaboration, and financial assistance to provide access to therapies. In a very short amount of time we have gained community support and confidence and recently won Wood TV 8’s coveted Connecting with Community award. One reason for this rapid growth is due to our investment in evaluation from the very beginning.

Hot Tips:

  • When in doubt, evaluate everything. When we first began our organization, we were not sure what we should evaluate. We decided to evaluate everything from process to programming. We evaluate processes and programs formatively and adjust where needed. We also evaluate summatively to determine the overall impact and the merit and worth of our programs.
  • When in doubt, evaluation everyone. Often, organizations only evaluate consumer satisfaction. We decided to be effective, we needed to evaluate everyone. We have board members evaluate the strategic planning process, family members evaluate our scholarship application process, we interview community partners, and gather stories of the children we serve.

Lessons Learned:

  • When you don’t have the expertise yourself, solicit board members. FHF started with a handful of passionate people determined to make an impact. We learned quickly that we needed more expertise in several areas. We were able to garner this expertise by soliciting board members with a diverse set of skills. The FHF board includes professionals with expertise in law, accounting, non-profit leadership, education, and yes, evaluation.
  • Share the data, share the need. To justify funding needs, we shared census, educational, and even anecdotal data through our initial solicitations, website, newsletters, and annual reports. We also collected data on our scholarship applications and were able to make appeals based on the data from the applications. Often this type of data is not publicly accessible, for instance, the number of therapies children need that are not covered by insurance.

  • Share the data, celebrate success! Since we gathered data on everything, we were able to show gains from our very first funding cycle. This quickly gave confidence to our donors and community members that we are indeed making an impact on children with special needs and their families.

The Family Hope Foundation increase in scholarships

  • Qualitative data garners support. Donors love success stories. For our consumers, those successes are often not statistically significant or able to be measured quantitatively. I encourage you to gather evidence in the form of interviews, stories, photographs, and open-ended survey questions. The share qualitative results are often the reason we gain new supporters.


Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Greetings. I am Jane Nell Luster, Senior Manager with the Data Accountability Center (DAC) and Assistant Professor at LSUHSC-Human Development Center. DAC is a technical assistance project funded by the U.S. Department of Education, Office of Special Education Programs. DAC’s mission is to “support the submission and analysis of high-quality IDEA [Individuals with Disabilities Education Act] data by reviewing data collection and analysis and providing technical assistance to improve state capacity to meet data requirements. The Center’s mission includes assisting the Office of Special Education Programs (OSEP) at the U.S. Department of Education by taking a leadership role in the Technical Assistance and Dissemination network to support the vision of high-quality data.” I am the internal evaluator for DAC.

Foremost in my mind is the intent to objectively conduct the formative and outcome evaluations so that the results are credible and valuable. At the recent AEA meeting in San Antonio I had the opportunity to be part of a roundtable discussion with others who serve the role of internal evaluators. We specifically talked about the need to conduct systematic, data-based inquiries which in my orientation means have the evaluation design in place prior to beginning the actual evaluative activities. Another area for discussion was the principle of integrity/honesty. The concern with this principle has to do with unintended bias. As a Center funded for a specific amount of time and as an employee of that Center, how do I ensure that I don’t conduct or report the evaluation in such a way that informs our work, informs our consumers, and maintains high levels of credibility.

Hot Tip: Credibility – As noted above, I work to have the evaluation design – methods, data sources, collection periods, etc. – formalized before beginning the evaluation. This provides transparency in the evaluation process and keeps you on track as an internal evaluator.

Hot Tip: Valuable – Timeliness. One of the difficulties I encounter as an internal evaluator, especially as that is only one of many roles, is being timely in the conduct and completion of the internal evaluation. Use your calendar to schedule time for this activity; set deadlines; and push to have this included in the agenda for internal staff and management meetings.

Rad Resource: 1997 – Communicating and reporting: Practices and concerns of internal and external evaluators in the American Journal of Evaluation, 18, 105-125. Although somewhat older than many references it validated some of the concerns I have had and also emphasized that even external evaluators may have similar concerns.

Rad Resource: AEA’s Internal Evaluation TIG. Get involved; connect and learn with others!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·


To top