AEA365 | A Tip-a-Day by and for Evaluators

TAG | cross-cultural evaluation

This is part of a series remembering and honoring evaluation pioneers leading up to Memorial Day in the USA (May 30).

I am Rodney Hopson, Professor of Education Policy and Evaluation at George Mason University and former (2012) President of AEA. Asa G. Hillard III (Baffour Amankwatia II), is one of the evaluation pioneers documented in the Nobody Knows My Name (named after a book by James Baldwin) Project that uncovers the untold contributions of African American educational researchers and evaluators in the United States during the pre-Brown v. Board era. While Hilliard’s major work did not take place pre-Brown, he is a name associated with the Nobody Knows My Name Project and is a name that all evaluators should know.

Trained as an educational psychologist (University of Denver, 1963), Hilliard’s research and practice spanned educational policy, special education, anthropology, child development, and classical African civilizations, Hilliard was one of the first African Americans to provide a keynote at the American Evaluation Association conference (in 1988). Hilliard’s presentation was later published in Evaluation Practice (the precursor to the American Journal of Evaluation) in 1989 and provided ways for evaluators to think differently about data visualization, truth and evidence and the implications for cross-cultural evaluators.   In recent years, the American Evaluation Association has sponsored Think Tank sessions at its annual conference in his honor previously co-sponsored by Indigenous Peoples, MultiEthnic and Social Work Topical Interest Groups to introduce his practice to AEA members and conference goers.

Asa G. Hilliard

Asa G. Hilliard

When names like Ralph Tyler, Robert Ingle, and Marcia Guttentag are remembered, so should those like Reid E. Jackson, Asa Hilliard, and Rose Butler Browne. Their cumulative scholarship and evaluation agenda-setting both laid a foundation for policies, legislation, and counter-narratives that challenged the racial hegemony and institutional segregation that existed in the United States and contributed to the intellectual development of democratic, equitable, and culturally responsive evaluation more generally.

References and Resources:

American Psychological Association. (2016) Featured Psychologist: Asa Hilliard, III, PhD. Retrieved from: http://www.apa.org/pi/oema/resources/ethnicity-health/psychologists/asa-hilliard.aspx

Hilliard, A. G. (1989). Kemetic (Egyptian) historical revision: Implications for cross-cultural        evaluation and research in education. Evaluation Practice, 10(2), 7–23.

Hood, S. (2001). Nobody knows my name: In praise of African American evaluators who were    responsive. New Directions for Evaluation, 92, 31–43

Hood, S. & Hopson, R.K. (2008). Evaluation roots reconsidered: Asa Hilliard, a Fallen Hero in   the “Nobody Knows My Name” Project, and African Educational Excellence. Review of      Educational Research, 78(3), 410-426.

Hood, S., Hopson, R., and Kirkhart, K. (2015). Culturally Responsive Evaluation: Theory,        practice, and future implications. In Newcomer, K. and Hatry, H (Eds.). Handbook on           Practical Program Evaluation (4th ed.) (pp. 281-317). San Francisco, CA: Jossey-Bass.

The American Evaluation Association is celebrating Memorial Week in Evaluation: Remembering and Honoring Evaluation’s Pioneers. The contributions this week are remembrances of evaluation pioneers who made enduring contributions to our field. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Greetings! I’m Molly Hamm, the Monitoring, Evaluation and Learning Coordinator at The DREAM Project, an educational non-profit organization in the Dominican Republic.

Working in a multilingual environment, I must continuously switch back and forth between languages as I complete my daily tasks. From elaborating evaluation plans and designing research instruments to facilitating focus groups and presenting results, I am constantly employing either English or Spanish based on the needs of specific audiences. This process often creates double the work on any one project, as most documents need to be designed in both English and Spanish. Additionally, translating information into multiple languages can present significant challenges when it comes to validity, reliability, accuracy, and comprehension. This post focuses on challenges related to written translations for instrument design.

Lessons Learned: Because evaluators painstakingly select wording when they are designing instruments, it can be easy to fall into the trap of trying to achieve word for word translations. However, it’s most important to focus on translating meaning. Besides the fact that there are simply no translations between languages for some ideas, you want to make certain that your tools are measuring the same constructs across translations. Wording may need to be adapted significantly to elicit desired responses from participants.

Hot Tip: Use back translation. Once you have an initial draft in the original (source) language, translate into the target language.  Clean up the target language translation, and then “retranslate” into the source language. This process enables you to see how well the meaning is retained through translation. If the back translation results in a question that is measuring something different than originally intended, continue the process until satisfied with the results.

Hot Tip: Be sure to pilot translated instruments, as these should be validated in both the source and target languages. Watch for variation in the target language across countries. Due to significant regional differences in vocabulary and even grammar, having an instrument successfully translated into a language such as Spanish for use in one country does not mean it will be well understood in another country. Always adapt your translations as necessary in a new cultural context, even when using the same language.

Resources: Check out translation tips from the University of Michigan’s Cross Cultural Survey Guidelines, Duke University’s Tip Sheet on Cross-Cultural Surveys, and University of California-San Francisco’s Annotated Bibliography for Translating Surveys in Cross-Cultural Research.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

We are Dayna Albert (Project Coordinator) and Rochelle Zorzi (Editorial Board Co-chair) of the Evaluation Stories Project, an EvalPartners Innovation Challenge recipient. Our project will soon launch an International Call for Evaluation Stories. The purpose is to:

  • Identify and share stories of evaluations that have made a difference
  • Increase the demand for and use of evaluation

Minimal literature exists on the benefits or impacts of evaluation use, particularly from the perspective of evaluation users. Furthermore, most evaluation literature is very academic. Our project will employ a story-telling format in order to better communicate the benefits of evaluation use to evaluation users.

As an international project, one of our challenges is to reach a multilingual audience despite limited translation resources. A second challenge is to explain what we mean by evaluation impact – a concept that turns evaluative thinking on its head and tends to be misconstrued.

Lessons Learned: Anticipate that people may have difficulty ‘getting’ a new concept. Words alone can be inadequate and ambiguous.

Use story to explain new concepts. Here is a story that Chris Lysy helped us develop to explain the concept of evaluation impacts.

(Click here to see the video!)image005

 

Hot Tip:

–        Follow-up with clients after an evaluation to reflect on and track evaluation impacts.

–        Act now! The Call for Evaluation Stories is a great opportunity to reconnect with a client and explore their interest in participating in the Call for Evaluation Stories

 

Rad Resource:

–        To reach a multilingual online audience, add Google’s Website Translate plug-in to your website. Albeit imperfect, it provides a free and virtually instantaneous website translation.

–        To translate a blog, paste the following code into a text widget. Insert your blog’s URL where indicated. The code is written for English (en) to French (fr) translation. For English to Spanish translation, replace ‘fr’ with ‘sp’ and ‘français’ with ‘español’.

<a href=”//translate.google.com/translate?u=http%3A%2F%2FyourblogURL&amp;hl=fr&amp;ie=UTF8&amp;sl=en&amp;tl=fr”” title=””français“”><img src=”http://yourblogURL /2010/02/icons-flag-gb.png” alt=”français” /></a>

Get Involved:

Rad Resources: See these posts for additional information on evaluation stories:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Archives

To top