Hello! I am Jeff Williams, PhD Candidate in Public Policy and Public Administration at the George Washington University, member of the evaluation team at CRDF Global, and member of the Washington Evaluators. Today I’ll be sharing tips and tricks about writing surveys for communities that have English as a second language, or which don’t use English at all.
Lesson Learned—Evaluators with English as a mother tongue often are engaging populations that have English as a second, third, or fourth tongue, or have no exposure to the language at all (see chart below from the 2007 American Community Survey Report for U.S. language breakdown). Translating questions and answers can be tricky in a face-to-face interview setting, but affords much more room for nuance and clarification than does administering a survey in which there is little or no interaction between the respondent and the survey designer. In these situations, all of the clarification needs to happen before the survey leaves the shop, and that calls for more up-front investment in getting it right.
Hot Tip—Pilot the language first and then the content. Before testing the survey for program-specific content, first make sure the translation of what you wrote is what you want to say. If the testers are getting a translation that does not reflect the true questions, then any pilot feedback they provide potentially is compromised.
Cool Tricks—So where do you get the language testers?
- Use in-house expertise. If, for example, your survey is going to a Russian-speaking audience and you have 15 people in your organization who speak Russian at a variety of levels, run it by them. Do “very satisfied” and “extremely satisfied” really translate into a meaningful difference?
- Use a local university. Are there faculty that specialize in that language and have conversational as well as technical fluency? Is there a student group composed of native speakers of your survey’s target language?
- Use a local ex-pat community. Keep potential IRB issues in mind and be creative – contact local community organizing committees, restaurant owners, or others involved in the target language community to help set up an informal focus group. Bring some snacks; make some friends.
On a related note, be sure you are familiar with AEA’s position on cultural competence in evaluation.
Hot Tip—Insider’s advice for Evaluation 2013 in DC: Avoid the car rental if you can as parking is costly and a bit of a pain in DC. This is a walking city, and October is a usually a good weather period for us. Metro is clean and relatively inexpensive for longer commutes, and taxis are always available if needed in a pinch.
This is the last of three weeks this year sponsored by our Local Arrangements Working Group (LAWG) for Evaluation 2013, the American Evaluation Association Annual Conference coming up next month in Washington, DC. They’re sharing not only evaluation expertise from in and around our nation’s capital, but also tips for enjoying your time in DC. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.