My name is Girija Kaimal and I am an Assistant Professor at Drexel University. Several evaluations I have done lately have been conducted remotely. Long distance evaluations can result from a range of reasons: modest program budgets, location independence of the initiatives as well as, short turnaround times that preclude travel and on-site data collection. As a result I have often worked on wonderful projects with collaborators whom I have never actually met in person. The process has its limitations because of the overall sense of disconnect but it also has its perks because we do not waste time on commutes and optimize technology to facilitate the work.
Hot Tips:
- Communicate and document mutual expectations: Make sure all contracts and agreements are shared and expectations made explicit. This includes not just the evaluator’s deliverables but also the interim deliverables from the program staff (e.g., data requests, introduction to respondents and timelines).
- Understand your tools: Email is great for documentation, scheduling and program updates. For data collection, consider more interactive methods like video chat, Skype video, Skype audio and the ubiquitous cellphone.
- Be flexible and resourceful about communication tools and times: Plan for the different time zones around the United States and around the world when scheduling interviews with participants. Respondents have differing levels of access to web-based tools. Accommodate for the respondents’ requests and level of digital access.
- When in doubt pick up the phone: For clarifications, concerns, and, issues it is better to have a conversation, rather than going back and forth over email. A lot of time can be saved and relationships built with a conversation.
- Align evaluation with project goals: Each project is unique. Understand the specific context, culture and needs of evaluation for each project and be respectful of the staff. It usually works to position the evaluator as a friend-who-offers-constructive-feedback rather than as an external auditor.
Lessons Learned:
- Define the role of the evaluator: Identify and clarify the evaluator’s role to each study respondent. Provide time to address any questions and concerns about confidentiality (which must be ensured by the way).
- A long distance evaluation is a relationship: Listen with BIG EARS because there are few if any non-verbal social cues to understand the culture and context of the program/ project.
Rad Resources: These aea365 posts discuss online evaluations:
VSA Week: Kathleen Tinworth on An Introduction to Visitor Studies Online
Courtney Heppner and Sarah Rand on producing online evaluation reports.
MNEA Week: Beth Robelia on using online focus groups
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.