Jane Nell Luster on Internal Evaluation for Special Needs

Greetings. I am Jane Nell Luster, Senior Manager with the Data Accountability Center (DAC) and Assistant Professor at LSUHSC-Human Development Center. DAC is a technical assistance project funded by the U.S. Department of Education, Office of Special Education Programs. DAC’s mission is to “support the submission and analysis of high-quality IDEA [Individuals with Disabilities Education Act] data by reviewing data collection and analysis and providing technical assistance to improve state capacity to meet data requirements. The Center’s mission includes assisting the Office of Special Education Programs (OSEP) at the U.S. Department of Education by taking a leadership role in the Technical Assistance and Dissemination network to support the vision of high-quality data.” I am the internal evaluator for DAC.

Foremost in my mind is the intent to objectively conduct the formative and outcome evaluations so that the results are credible and valuable. At the recent AEA meeting in San Antonio I had the opportunity to be part of a roundtable discussion with others who serve the role of internal evaluators. We specifically talked about the need to conduct systematic, data-based inquiries which in my orientation means have the evaluation design in place prior to beginning the actual evaluative activities. Another area for discussion was the principle of integrity/honesty. The concern with this principle has to do with unintended bias. As a Center funded for a specific amount of time and as an employee of that Center, how do I ensure that I don’t conduct or report the evaluation in such a way that informs our work, informs our consumers, and maintains high levels of credibility.

Hot Tip: Credibility – As noted above, I work to have the evaluation design – methods, data sources, collection periods, etc. – formalized before beginning the evaluation. This provides transparency in the evaluation process and keeps you on track as an internal evaluator.

Hot Tip: Valuable – Timeliness. One of the difficulties I encounter as an internal evaluator, especially as that is only one of many roles, is being timely in the conduct and completion of the internal evaluation. Use your calendar to schedule time for this activity; set deadlines; and push to have this included in the agenda for internal staff and management meetings.

Rad Resource: 1997 – Communicating and reporting: Practices and concerns of internal and external evaluators in the American Journal of Evaluation, 18, 105-125. Although somewhat older than many references it validated some of the concerns I have had and also emphasized that even external evaluators may have similar concerns.

Rad Resource: AEA’s Internal Evaluation TIG. Get involved; connect and learn with others!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.