I am Anne Chamberlain, and I am a Senior Research Associate at IMPAQ International, a rapidly-growing social-science research and evaluation firm serving clients in the areas of health; labor and human services; education; survey research; and international development. Recently we have been broadening our experience in researching barriers faced by people with disabilities (PWD), and strategies being employed to minimize or eliminate those barriers. For this post, I will share a lesson-learned related to the development of a survey for a U.S. Department of Labor-funded project, Evaluating the Accessibility of American Job Centers for People with Disabilities.
How many times have you said to yourself that “next time” you will faithfully keep a log regarding decisions made and actions taken during the life of your evaluation project? Fortunately our evaluation team members kept meticulous records of the many decision points that coalesced into a survey on a very difficult topic: accessibility. The topic is relevant here, because it is so nebulous and so politically charged that the survey development process was particularly labor-intensive. When it was time to show our development process to numerous stakeholders each with their own agenda, we were able to illustrate it only because there were records of the thirty plus draft iterations; decisions made in response to multiple Offices and Administrations; activation of advice from two Technical Working Groups, etc. An illustration of our development process is below:
Lesson Learned: Keep a Log of Your Evaluation Moves
- I encourage evaluators to maintain a single document that acts as a diary, keeping track of decisions and actions, who made them, when, and why. It’s helpful if this is searchable by who, when, and why. You can add a keyword scheme so that each entry also has keywords listed, to further your organization and search functions. The log may be used by you alone, or by your whole team. If it’s the latter, make sure they respect the formatting and add a ‘field’ to record who made the entry. Finally, put this resource somewhere visible. For me, this has been the key to regular use. A simple Word document on your desktop can encourage you to record anything important before closing out of your computer at the end of the day.
The American Evaluation Association is celebrating the Disabilities and Other Vulnerable Populations TIG (DOVP) Week. The contributions all week come from DOVP members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.