Welcome to aea365! Please take a moment to review our new community guidelines. Learn More.

ITE TIG Week: Best Practices for Preparing to Implement a New Data Collection Software When You Think Your Legacy System is a Mess by Diamond St. Thomas

Hello, my name is Diamond St. Thomas, Principal Consultant at Facets Consulting. I began my impact measurement journey over 10 years ago as a program analyst for an arts education organization. My first task as a graduate intern was to input data into a custom database. Like so many other evaluators, you hope your “summer intern” does an accurate job at data entry, to ensure that you are able to export, analyze and report on your organization’s impact. From that first assignment, I began my journey in the strategy, data and evaluation space.

As I matured in my career, I had the opportunity to support human services organizations to implement data collection technologies. That experience led me to where I am today, which is working with nonprofit organizations to leverage technology to measure their impact. 

Hot Tips:

By spending the time to review, inventory, map and compile historical information, before you engage in a data collection technology selection, you will save a lot of time and money.

Data Review/Update: Review all the data in your legacy system, create a data inventory and make any relevant updates. Clean data is key for data migration. When conducting your inventory, ask yourselves these questions:

  • Who are we collecting data on?
  • What type of data are we collecting?
  • When did you start collecting this data? 
  • Where is this data collected? How many internal and external systems are there? Integrations? Exports/Imports? Manually? 
  • Why are we collecting this data?
  • How is this data being collected? Integrations? Exports/Imports? Manually?

Key Indicators:  Identify universal elements that are consistently tracked across all programs (i.e. demographics, income, education, employment history). Understanding your data model is critical to how you will be able to analyze and report on your data. Siloing the same data within individual programs prevents universal reporting on your key indicators of success. 

Process Mapping: Gather all relevant information about the constituent journey through the programs (i.e. recruitment, enrollment, discharge, alumni, etc.). Where does the journey begin and how will you measure their success when exiting the program. 

Artifacts: Compile all your documentation that has supported your data collection efforts in the past. It is helpful to understand your current state to inform your future state, though this is not the time where you do not want to rebuild business processes that don’t work  

Rad Resources:


The American Evaluation Association is hosting Integrating Technology into Evaluation Topical Interest Group (ITE TIG) Week. The contributions all this week to AEA365 come from the ITE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.