Hi! This is Shelly Engelman and Brandi Campbell with The Findings Group, LLC, a private evaluation firm in Atlanta, GA.

Evaluators typically implement pre/post surveys to assess programmatic impact on participants.  However, pre/post surveys are plagued by challenges:

1. Participants have difficulty responding to the “pre” survey items because they have little knowledge of the program content and choose to leave many items blank.

2. Participants feel overburdened with the “post” survey because they answered similar items on the “pre” survey and do not fill-out the “post” survey.

3. A participant is not present for either the “pre” or “post” survey, resulting in an incomplete data set for that individual.

4. Participants gain insights into program content and see it differently than at the beginning. Known as the Response Shift Bias, participants may overestimate their initial attitudes due to lack of knowledge at baseline; after the program, their deeper understanding affects their responses on the “post” survey.

Lesson Learned: Retrospective Results – Complete and Stable

Retrospective surveys ask participants to compare their attitudes before the program to after.  Because a participant completes a retrospective survey in one sitting, responses are more complete.  Not only is there a higher completion percentage with this method, but it also has been found to reduce the Response Shift Bias in participants.

Lesson Learned: The Utility of Retrospective Results

In several of our projects, the retrospective survey had advantages over the pre/post survey.  It yielded more complete datasets and higher response rates. On the other hand, because students complete the survey after the program, they may not accurately remember their attitudes before the program.  This is especially prevalent if the program occurs over several months.  Additionally, younger participants may have trouble navigating the retrospective survey format and may require additional assistance.

Contribute to the Practice of Retrospective Surveying

We appreciate that the evaluation community has more to learn about appropriate uses for retrospective surveys. To more fully understand the differences in true pre/post vs. retrospective pre/post approaches, The Findings Group is conducting pre surveys followed by retrospective pre/post surveys on a handful of programs.  We expect to measure the differences, if any, between the two “pre” response sets.  We invite you to do the same and share your results.  We could put together a panel presentation at AEA 2014!

Hot Tips: Implementing a Retrospective Survey

It is simple to rewrite pre-post survey items for a retrospective survey.

Pre/post survey: I am confident in my ability to solve computer science problems.

Retrospective pre-survey: Before this workshop, I was confident in my ability to solve computer science problems.

Retrospective post-survey: After this workshop, I am confident in my ability to solve computer science problems.

The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

retrospective pretest · survey

Hello, we are Sheila Matano and Dani O’Neill from Carson Research Consulting (CRC). We at CRC are currently in the process of conducting a survey on relocation and housing stability. Although surveys are an incredibly useful method for gathering quantitative information from large populations, response rates vary widely. In order to promote our project, interact with the community and hopefully increase our response overall rate, CRC conducted a Community Survey Day for residents in addition to providing the survey online and mailing out paper surveys.

Lesson Learned: Planning and preparation.
Planning is essential! Staffers started planning for community survey day about two months in advance.  We had meetings to discuss logistics and created a master task list to make sure everything we needed would be ready for the day of the event. Google Drive provided a way to easily edit and update important documents that could be accessed by all team members.

Lesson Learned: Know your target audience!
Using background information on the target audience, we attempted to tailor the event to the resident’s needs. We found a central location that was easily accessible by public transportation and also offered free parking. In addition, we gave residents the option of taking the survey online, completing a paper survey or taking the survey over the phone. We wanted to make sure that respondents could choose an option that was most comfortable for them.

Lesson Learned: Advertise!
We advertised the event in three ways:

• We sent a letter to our targeted residents informing them of the event
• We created event flyers that were distributed to residents
• We also made phone calls to residents a few days before the event

Hot Tip: The Day of the event
Community Survey Day was hosted on a Saturday to give residents who work during the week the opportunity to attend, and we also offered breakfast and lunch to all attendees.  Staff interviewers were available to do survey interviews with residents from 9am to 5pm. The interviewers read questions aloud to the residents and inputted their responses directly into SurveyMonkey via laptops; residents did not use computers themselves.

Hot Tip: Having the survey online made it readily accessible to interviewers and made the process of completing surveys faster and easier than using paper surveys (although we did have paper surveys on hand in case technology failed us!).

Outcome: Overall, Community Survey Day was a success. We had the opportunity to directly interact with residents and get valuable feedback about our survey. Residents appreciated that they could come in and talk with us directly, as well as offer their suggestions about our project.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

survey

Hi, this is Tania Jarosewich of Censeo Group, a program evaluation firm in northeast Ohio, and Linda Simkin of Action Research Associates of Albany, New York. We worked on different aspects of the evaluation of KnowHow2GO, an initiative funded by Lumina Foundation to strengthen college access networks.  We are excited to share with you the College Access Network Survey, a resource that Linda helped to create as part of the Academy for Educational Development (AED) evaluation team. The network survey is a tool to gather network members’ perspectives about their engagement with a network and a network’s effectiveness and outcomes.

During implementation of KH2GO, the AED technical assistance team, with Linda’s help identified five dimensions of an effective network: network management, sustainable services systems, data-driven decision-making, policy and advocacy, and knowledge development and dissemination. This framework helped guide the development of the survey, technical assistance, and evaluation of network-building efforts.

As part of the evaluation, KnowHow2GO grantees invited members of their statewide or regional networks to respond to the survey. The Network Survey provided useful information for the foundation, initiative partners, technical assistance providers, network leaders, and network members to plan technical assistance and professional development, and allowed networks to monitor network health. With minor changes, the survey can be applied to network efforts focused on different content or service areas.

Lesson Learned: Support grantees’ Network Survey use and analysis. Network leaders focused on their work – not on evaluation. Letters that introduced the survey, an informational webinar, support monitoring response rates, and individual trouble shooting were helpful to encourage grantees to engage network members in the survey.

Lesson Learned: Provide targeted technical assistance and professional development based on survey findings. The survey results allowed technical assistance providers to target their support and helped to emphasize the usefulness of the survey instrument and process

Lesson Learned: Use network survey results to show progress towards network outcomes. Information about the strengths of each network were useful for the funder and participating networks. The survey results were triangulated with other evaluation data to provide a comprehensive analysis of growth in the network building process.

Rad Resource: You can obtain a copy of the College Access Network Survey and guidelines for its use from Carrie Warick, Director of Partnerships and Policy, National College Access Network (NCAN), WarickC@CollegeAccess.org, 202-347-4848 x203. The survey can be adapted for use with networks focused on various content areas.

Rad Resource: Keep an eye out for a longer article about the Network Survey that will appear in an upcoming issue of the Foundation Review. You can also access additional resources about the Network Survey here – handouts (free from the AEA Public eLibrary) and a Coffee Break webinar recording (free only for AEA members).

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

College Access Programs · networks · survey

I am Tarek Azzam, assistant professor at Claremont Graduate University and associate director of the Claremont Evaluation Center.

Today I want to talk about crowdsourcing and how it can potentially be used in evaluation practice. Generally speaking, crowdsourcing is the process of using the power of the many individuals (i.e. the crowd) to accomplish specific tasks. This idea has been around for a long time (e.g. the creation of the oxford dictionary), but due to recent developments in technology, the ability to access the power of the crowd has become much easier.

I will focus on just one crowdsourcing website because Amazon’s Mechanical Turk is the most widely known, used, and studied crowdsourcing site. This site helps to facilitate the interactions between “requesters” and “workers” (see figures below). A requester can describe a task (e.g. please complete a survey), set the payment and allotted time for completing a task, and determine the qualifications needed to finish the task. This information is then posted on MTurk website, and interested individuals who qualify can complete the task for the promised payment.

(Share Clip)

This facilitated marketplace has some really interesting implications for evaluation practice. For example, evaluators can use MTurk to establish the validity and reliability of survey instruments before giving them to intended participants. By posting a survey on MTurk and collecting responses from individuals with similar background characteristics as your intended participants, an evaluator can establish the reliability of a measure, get feedback on the items, and if needed translate the items into another language. All this can be accomplished in a matter of days. For me personally I’ve been able to collect 500 responses for a 15 minute survey, at a cost of 55 cents per survey in less than three days.

Hot Tip: when selecting the eligibility criteria for MTurk participants choose those with 95% or higher approval ratings.

There are other uses that I am currently experimenting with. For example:

• Can MTurk respondents be used to create a matched comparison group in evaluation studies?
• Is it possible to use MTurk respondents in a matched group pre-post design?
• Is it possible to use MTurk to help with the analysis and coding of qualitative data?

These are things that are yet to be known but I will keep you updated as we progress in exploring the limits of crowdsourcing in evaluation practice.

(Share Clip)

Hot Tip: I will be presenting a Coffee Break Demonstration (free for American Evaluation Association (AEA) members) on Crowdsourcing on Thursday April 18, 2013 from 2:00-2:20pm EDT. Hope to see you there.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Data · Evaluation · survey · technology

My name is David Fetterman.  I’m President & CEO of Fetterman & Associates, an international evaluation consulting firm (with 25 years experience at Stanford University) and past-president of the American Evaluation Association (AEA).  I am probably best known for empowerment evaluation work (helping people learn how to evaluate their own programs).  For examples see our blog and an article about empowerment evaluation in the School of Medicine at Stanford University in Academic Medicine, and the book Empowerment Evaluation in the Digital Villages:  Hewlett-Packard’s $15 Million Race Toward Social Justice, Stanford University Press

LGBT-Related Survey

One of my recent evaluations, conducted with my  Stanford School of Medicine students, focused on LGBT curricular training in medical schools throughout the U.S. and Canada.  The results – a median reported time of 5 hours of LGBT-related content in the entire curriculum – were published in this article in the Journal of the American Medical Association (JAMA). It received considerable attention in the press, in part because it is as much a human rights issue as a medical education issue. I’ll share a few tips and tricks that emerged from conducting and publishing this study.

Hot Tip:  We used an online survey program to ask Deans of Schools of Medicine to evaluate their institutions’ level of coverage of 16 LGBT related topics.  Online survey tools, such as SurveyMonkey, save time and money and sort data almost instantaneously.  Surveying Deans automatically enhances the credibility of findings (especially if findings suggest minimal coverage of the material, as in our case).

Reporting survey findings was as much a social responsibility as a scholarly one.  See Anne Dohrenwend’s challenge to speak out about gay rights in Academic Medicine.

Cool Trick: Videoconferencing programs, including Skype, ooVoo, and Google Hangouts are invaluable tools to facilitate communication with team members at remote sites.  Most team members were located across the country, completing residency requirements.  Videoconferencing allowed us to function remotely and inexpensively.

Rad Resource: The Association of American Medical Colleges maintains a curriculum management and Information (CurrMIT) database that helps you determine the coverage of specific topics in medical schools. This database was particularly useful as a form of triangulation when our reporting format – “reported hours of instruction” – was questioned in a draft of our article.

Recommended LGBT cultural competence resources:

Fenway Health

Institute of Medicine of the National Academies

Lesson Learned:  Be prepared for significant opposition to unpopular or controversial findings.  Be prepared to speak with the press.  Highlight key findings and recommendations simply and concisely and be prepared to see how journalists use the information (see example of highlighted findings in New York Times.)  Appreciate your team and enjoy the media blitz for as long as it lasts.

aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. We’re celebrating LGBT Evaluation week with our colleagues in AEA’s LGBT Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. 

blog · cultural competency · lgbt · survey · technology

This is Kari Greene with Program Design & Evaluation Services in Oregon, and Emily Greytak with GLSEN, the Gay, Lesbian & Straight Education Network in New York. We are with the American Evaluation Association’s (AEA) LGBT Issues Topical Interest Group (TIG) and are heartened to see AEA members building cultural competency around transgender-inclusivity.

Have you ever thought about what you’re really asking with “What is your gender: Male or Female?” Do you want to know how people identify to others? How others see them? What sex they were assigned at birth? This ubiquitous question and standard response options deserve more thought…

Hot Tips:

Should I ask transgender identity? Sex at birth? Current gender?

Start with asking what you really need to know and why. For example, a health program offering cancer screenings may need to know if male-identified clients have anatomy/physiology typically associated with females, so they may need breast or cervical cancer screenings. Meanwhile, a housing program might only need to assess if the outcomes are different between transgender and non-trans clients.

I can’t ask people if they’re transgender – they’ll be offended!

Including trans-inclusive items appears to be innocuous for adults and youth. Oregon tested two transgender items in the statewide health survey and respondents 18 to 80 answered easily. In fact, income and weight questions have far higher refusal rates.

I put “Transgender” on my client form but a transgender client checked the “Female” box – what did I do wrong?

Nothing! Some transgender people may identify as both female or male and transgender, so you may want a “check all that apply” gender item. Others may only identify as male or female, so you could also add a question asking sex assigned at birth. Some people don’t identify as male, female or transgender so an open option is helpful.

There are so few transgender people – why bother since I can’t use them in subgroup analysis of male/female participants?

Remember the program is already serving transgender people – they just aren’t counted. Create an analytic plan that describes all participants, and combines groups reliably and respectfully. Excluding transgender respondents sends the message that the evaluation or program is not relevant or welcoming to transgender people.

Any sample questions you suggest?

Yes, but it depends on what you need to know. There is no single “best item” for assessing transgender respondents but these resources can help!

Rad Resources:

Injustice at Every Turn: A Report of the National Transgender Discrimination Survey

Assessing Transgender Status in Surveys of Adolescents: A GLSEN Research Brief Services & Advocacy for GLBT Elders

Eval12 Session 654: Don’t Ask, Can’t Report  materials in the AEA public eLibrary

aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. We’re celebrating LGBT Evaluation week with our colleagues in AEA’s LGBT Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. 

cultural competency · lgbt · survey

My name is Terry L. Brown, and I am a doctoral candidate in the College of Education at Northern Illinois University in DeKalb, Illinois. Transgressing gender is the practice of blurring socio-constructed gender boundaries as a strategic response to heteronormative practice. Heteronormative, also referred to as gender normative, practice seeks efficiencies through standardizing and regulating sameness while simultaneously diminishing difference.  Disruptions to heteronormative cultural ways offer opportunities to re-think traditional policy and programmatic response structures and frameworks. Individuals across the gender spectrum often drive change laterally, across social, political, technological, legal and economic systems, in such a way that leads to the expansion of opportunities for those traditionally excluded.

Rad Resources – The Basics: Gender 101  (Some starting points)

• The difference between sex and gender
• The Gender Acronym Defined: LGBTTIQQ2SAAP
• Transgender Glossary of Term’s from Gay & Lesbian Alliance Against Defamation (GLAAD)
• Cisgender

Hot Tip 1:  Binary categories have long exceeded their usefulness in collecting data on sexual orientation and gender. To this end,

a)  Likert-type scales are useful for capturing nuanced data when gender positionalities are often situational and can even change dynamically from questions to question.

b)  Allow for multiple items to be selected.

c)  Employ case study to capture transformational outcomes for those who identify as LGBT.

Hot Tip 2:  Gender assumptions can be found embedded among the nouns, pronouns, adjectives and verbs we use in writing, resulting in participants hitting exit before survey completion.

a)  Make sure to go outside traditional feedback loops, to review any evaluation materials before implementation.

b)  Challenge traditional academic models of “best practice” for processes that reinforce gender preferences.

c)  Use inclusive language: “we” as opposed to he/she when preparing reports.

d)  Examine metaphors and other linguistic devices for reproducing gender- normative patterns.

Hot Tip 3:  Gender assumptions are implicit understandings which often take the form of stereotypes around masculinity and femininity. Be generous in your use of comment boxes, even in demographics items, as a method for allowing:

a)  Respondents to provide feedback on word choice, or to point out embedded assumptions;

b)  To reduce conceptual blind spots and emerge unknown positionalities;

c)  To break the urge of evaluators to neatly label and categorize; and

d)  To bring out rich data and capture human diversity.

Lessons Learned – Some common heteronormative assumptions:

• Lesbians dislike men
• All trans-folk consider themselves part of the LGBT community
• Marriage = 1 man and 1 woman
• A stay at home caregiver  = a woman
• LGBT are non-spiritual and/or non-religious
• All women, lesbian or otherwise, want to have children
• Men are born leaders
• People who identify as LGBT are making a choice
• Men who identify as gay are effeminate

Rad Resources:  Explore: Heterosexuality Questionnaire;  Straight Privilege

aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. We’re celebrating LGBT Evaluation week with our colleagues in AEA’s LGBT Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. 

cultural competency · lgbt · survey

Greetings, I am Cindy Weng, a bio-statistician II employed at Pediatrics Research Enterprise at Department of Pediatrics at the University of Utah. This post was written together with my colleagues Chris Barker, SWB project manager and Larry George, statistician at Problem Solving Tools.

I learned about this methodology through a project assigned by ASA Statistics Without Borders (SWB) in 2011. The goal of this project was to analyze under 5 years (U5) mortality of children before (“baseline”) and after (“endline”) humanitarian aid given at Afghan refugee Camps in Pakistan. Survival analysis was used to estimate the probability distribution of age at death from current status and admissible age-at-death data. Inadmissible ages at death placed the date of death after the survey dates!

The International Rescue Committee survey data contained inadmissible ages at deaths, so the Kaplan Meier nonparametric maximum likelihood estimator was, used along with estimators from current status data only.

Tips:

• Maximum likelihood and least squares estimators differ. We estimated survivor functions from baseline and endline surveys. “MLE” and “LSE” denote maximum likelihood estimation and least squares estimates. They don’t always agree, because the methods are different approaches to estimation. In particular, LSE does not respond to noise. If noise is not uniform across the sample, LSE might be incorrect. The MLE takes noise into consideration. The MLE estimates in the figure are from current status data. They agreed pretty well with the Kaplan-Meier estimators from admissible ages at deaths.

Lessons learned:

• Survey data is not always what is expected. Surveys should have cross-checking validation opportunities. Current status data provided the opportunity to make two estimates of survivor functions.
• Expect unexpected outcomes. The baseline U5 estimates are over 10%, and the endline U5 estimate is approximately 4%. Pakistan’s country U5 is 8.7%. The endline U5 estimates standard deviation is less than 0.5%. The apparent reduction in U5 appears to be primarily a reduction in deaths after infant mortality in the first year. Infant mortality was almost 4% before and after.

Resources:

• Problem Solving Tools
• Marcorin and Abackerli’s article: Field Failure Data: an Alternative Proposal for Reliability Estimation

The American Evaluation Association is celebrating Statistics Without Borders Week. The contributions all week come from SWB members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

least squares estimate · maximum likelihood estimation · statistics · survey

I’m Lija Greenseid, a senior evaluator with Professional Data Analysts, Inc. in Minneapolis, Minnesota.

A mixed-mode survey is a survey that collects data by more than one mode, such as telephone, mail, or web. PDA has conducted many mixed-mode surveys as components of our evaluations of state stop-smoking programs, such as tobacco quitlines and in-person stop-smoking programs. Let me share a little of what we’ve learned.

Hot tip: There are many different ways of mixing survey modes. One tried and true method is to mail a printed pre-notification letter in advance of contacting survey participants by telephone or email. Another commonly used design is to contact survey participants by one mode and then follow-up with non-respondents by a second mode. PDA frequently employs a design in which we contact participants by phone up to 7 times, suspend calling to send a mail survey, then continue to phone participants up to 8 more times if they do not respond to the mail survey in a week.

Lesson learned: Mixed-mode surveys can be hard to administer! A survey designer must carefully consider the timing of different modes of contact and how to coordinate the contacts so as not to overburden survey participants with multiple contacts by different modes. This can be automated or done manually with tracking spreadsheets and processes. PDA designed a custom survey system to manage the mixed-mode design described above.

Hot tip: If mixed-mode surveys are harder to administer, why bother? Mixed-mode surveys have the potential of increasing survey response rates, therefore improving survey estimates by reducing non-response and coverage error. Additionally, adding a second lower-cost mode to a phone survey can actually save money by shortening the fielding time and reducing the number of calls. In sum, mixed-mode surveys can get better data for your clients more quickly and at a lower cost. Not too bad!

Rad resource: The best place to start to learn more about mixed-mode surveys is the textbook “Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 3^rd Edition” by Don Dillman, Jolene Smyth, and Leah Christian (2009).

Rad resource: The annual conference of the American Association of Public Opinion Researchers (AAPOR) is where cutting-edge research on mixed-mode surveys is presented long before the studies are published in journals. The AAPOR 2012 Conference will be held in Orlando, Florida from May 17-20^th.

Hot tip: I’ll be discussing mixed-mode surveys in an AEA coffee break webinar on Thursday, April 26^th. AEA members, please join me to learn more!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

survey

We are Jan Losby (a CDC employee) and Anne Wetmore (an ORISE Fellow), members of the Evaluation and Program Effectiveness Team in the Division for Heart Disease and Stroke Prevention at the Centers for Disease Control and Prevention (CDC).

Surveys can be an important part of your evaluation efforts. As evaluators we have probably all heard (or said), “We need a survey!” Before launching into any survey effort make certain you know the purpose of the proposed survey. To determine your purpose, take time to answer these four simple questions:

• Hot Tip #1:  Why a survey? Is there another way—focus groups, direct observation, document review, or secondary data sources—that will provide you with the information you need for your evaluation?
• Hot Tip #2:  Who do you intend to survey?  Are you surveying program staff, partners, stakeholders, recipients, employers, providers, etc.?
• Hot Tip #3:  What do you need to know? You may have a long wish list of things you would like to know—go through your list carefully and determine which ones are “need to know” rather than simply “nice to know.”  It is important that you take the time upfront to determine if each question you are considering is absolutely essential.

A useful check can be to ask yourself:

If I know________ (fill in the blank with the information you hope to gather through the survey), I will be able to ________ (fill in the blank with what you hope to be able to do, for example measure a specific outcome).

• Hot Tip #4:  How will the survey be administered (i.e., telephone, in-person, Internet)? Your timeframe and available resources will likely affect which mode you select.  Also, consider which mode will work best for the people you intend to survey.

Rad Resources

• Program Evaluation Tip Sheet: Constructing Survey Questions A tip sheet that we developed for funded programs through the Division for Heart Disease and Stroke Prevention at CDC
• Spacing, Position, and Order:  Interpretive Heuristics for Visual Features of Survey Questions a paper from Tourangeau, Couper, and Conrad at the University of Michigan: Institute for Social Research
• Question Wording guidance from the American Association for Public Opinion Research
• Analyzing Polls: Interpreting Analysis-Questionnaire Design from the Roper Center Public Opinion Archives
• Standards and Guidelines for Statistical Surveysfrom the Whitehouse Office of Management and Budget
• Survey Research on the Research Methods Knowledge Base

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

CDC · Evaluation · survey