Hello, I’m Nichelle Walton, Senior Associate – Organizational Performance at the Rockefeller Foundation, based in New York City. In 2019, the Rockefeller Foundation launched a pilot test of interactive voice response (IVR) to enable rapid, mobile data collection about its programs. Mobile data collection is attractive as a supplement to more traditional M&E approaches because it produces more timely, actionable data – often at a lower cost. It can also gain insights about households and individuals that are otherwise difficult to reach. The Foundation wanted to learn more about what it took to implement this approach, the strengths and limitations, and when and how to recommend it to program teams.
In partnership with the Rockefeller Foundation’s Food team, a test of IVR technology was deployed via a fifteen question survey of farmers who received RF-funded services in 2018. Farmers were surveyed across three countries and value chains: Kenya (mangoes), Tanzania (maize), and Nigeria (tomatoes). Results were received and analyzed in fall 2019.
Lessons Learned:
- IVR is a great option for teams who want maximum control over the survey design, data and analysis.
- It’s fantastic for learning a few specific insights quickly and viewing those results in real-time as they come in.
- It allows you to reach rural households in local languages that would otherwise be more difficult or costly to reach.
- However, because people are less responsive to long IVR surveys, it’s recommended to keep it to 10-15 questions, which means you’re limited in the depth of insight you can glean from this approach.
- Although you can ask open-ended questions via IVR, it’s often less successful so multiple choice questions are recommended. This can be limiting for teams who want to learn more about what is or isn’t working about their program.
Hot Tips:
- If you’re interested in surveying a target group – rather than surveying all households in a particular region – you’ll have to supply the numbers for the IVR calls, so have a critical mass at the ready. You can expect that many of these numbers will no longer be accurate due to cell phone number turnaround, and that only a portion of those called will complete the survey.
- Determine your survey purpose with stakeholders and have some draft questions ready before you begin your engagement with the IVR company; this will greatly speed up the survey design process, and will mean you get your results sooner.
The American Evaluation Association is celebrating International and Cross-Cultural (ICCE) TIG Week with our colleagues in the International and Cross-Cultural Topical Interest Group. The contributions all this week to aea365 come from our ICCE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.