AEA365 | A Tip-a-Day by and for Evaluators

TAG | Data Collection

Hi, I am Kerry Bruce, Director of Results and Measurement at Pact.  I am part of Pact’s central technical team that provides monitoring and evaluation support to more than 20 country offices and more than 70 projects around the world.  In 2012 we started to roll out the use of mobile technology in our programs. We have begun integrating mobile technology into our programs by using mobile phones for baseline and endline data collection.

Hot tips:

  • Bruce Mobile phones 1Mobile technology has advanced significantly since the last time you likely considered using it and now is the time to invest in learning about this technology.  Many of the early bugs have been worked out and the commercially available platforms make collection of data via mobile phone or tablet quite easy.
  • New platforms are easy to use, there are many to choose from and most include built in dashboards that help you to review and visualize your data.
  • A careful assessment of network coverage, power and power back-up should be done before you decide on a type of phone and platform.  While you don’t necessarily need a signal to use mobile phones to collect data (you can collect data offline) you will need a phone with long battery life! Many phones are now GPS enabled—and you should consider these if you would like to collect GPS waypoints and conduct geospatial analyses.
  • Understand the skills and competencies of your data collectors.  Will they be people who are familiar with mobile phones or will they need significant training and mentoring?  What type of phone will be easiest for them to use?
  • If you are using mobile phones for data collection of a baselines survey, for example, will you have a follow on use for the phones? You’ll want to consider what type of phone will be most useful for future activities so that you can yield a higher return on investment of your initial purchase.

Lessons Learned:  Bruce Mobile phones 2

  • A careful assessment of your data collection needs, logistical issues, and possible future projects is necessary before you start utilizing mobile technology.
  • Because not everyone sees the benefits of mobile technology, a basic overview of the advantages of this innovation is helpful to get your co-workers on board.

Rad Resources:

  • Online mobile technology training for a variety of uses is available for a fee from TechChange.
  • There is a free online mobile data collection selection assistant at NOMAD.

*Thank you to Mobenzi Researcher and DataWinners (DataWinners free data collection App for Android devices built using Open Data Kit tools) for the use of their images in this post.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hello, I am Edith Gozali-Lee, a research scientist at Wilder Research. I work primarily on research and evaluation projects related to education. I am currently working on a multi-site, longitudinal study of an early childhood initiative. The study includes three cohorts of school-based preschool program children in ten schools, five cohorts of community-based child care children in homes and centers, and comparison children with and without prior preschool experience. The study follows children from preschool to third grade. That’s a lot to track, making good data collection critical from the start.

Hot Tips:

These are a few coding tips that will help to ensure good data collection tracking:

  • Anticipate the different groups ahead of time and make intuitive coding to make it easier for the following years’ data tracking and analyses
  • Use categories or codes used by schools to make data analyses process easier when you merge data that you collect with other student data collected by schools (demographic data and student outcomes)
  • Label all instruments (survey and assessment forms) with these codes prior to data collection to reduce coding work after the data collection and errors for data entry

Lesson Learned:

It is helpful to hold regular project debriefs to reflect on what works well and does not work so well. This will make the evaluation process go smoother and faster the next time around.

Rad Resources:

Practical research-based information, visit CYFERnet Children, Youth and Families Education and Research Network

Resources for research in early childhood:

We are looking forward to seeing you in Minnesota at the AEA conference this October. Results of this study (along with other Wilder Research projects and studies) will be presented during a poster session: Academic Outcomes of Children Participating in Project Early Kindergarten Longitudinal Study.

The American Evaluation Association is celebrating with our colleagues from Wilder Research this week. Wilder is a leading research and evaluation firm based in St. Paul, MN, a twin city for AEA’s Annual Conference, Evaluation 2012. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

My name is Felix Blumhardt and I am a Lead Evaluator at The Evaluation Group, an independent evaluation firm in Columbia, SC. My colleague, Michael Pesant, and I have had the pleasure of working with a Teacher Quality Partnership Project over the last year and one of the challenges that we have come across has been streamlining data collection across several school districts. The most important take away from this process has been ensuring that we communicate clearly with the people from whom we ask to provide data. Here are some tips that have helped us ensure that the data collection process goes smoothly:

Hot Tip: Customize your collection process. We developed a Data Collection Workbook that was distributed to each district. The collection tool is an excel file that has been formatted to look like a workbook. We removed gridlines and left only directions, tables, and graphs. Each worksheet clearly enumerates objectives, variables, and operational definitions. Cells that need to be filled in are highlighted; all others are locked. Once the data contact person from the district inputs the data into the highlighted cells, he/she is able to see the data instantly charted and graphed. Where applicable, conditional cells indicate whether or not objectives have been met. Because this workbook will be used from year to year (the project is five years) the districts will be able to track their own progress.

Hot Tip: Preset tables, charts, and graphs. Prior to data collection, we set up the workbook with directions, project objectives, tables, charts, and graphs. When all of the districts submitted their completed workbooks, we transferred that data into an overall project workbook. We instantly had all the tables, charts, and graphs needed for our reports. Because we linked our workbook in Excel to our reports in Word, our tables and graphs in our reports were updated. This makes for easy reporting in subsequent years.

Hot Tip: Communicate clearly. Because this Tip-a-Day is focused on communication, I would be remiss if I did not emphasize the need for continued verbal communication throughout the data collection process. We sent the initial draft of the workbook to all of the districts for feedback so that we could make sure that it was not too cumbersome and was clear to them. It’s important to assess your partners’ technological capabilities and make sure your tool is compatible. In our case, this involved making sure our workbook could run on all versions of Microsoft Excel. Once we finalized the Data Collection workbook, we held a meeting about the process and reviewed the workbook with the districts. This meeting was essential to the success of the collection process using the Workbook.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

·

<< Latest posts

Archives

To top