Hi, my name is Nancy Aguirre.  I am a research associate for an independent evaluation consultant and a Professional Expert at Mt. San Antonio College in California.  One thing that’s been tricky for me is developing scales, and it seems like so many people have a preference for 5, 7, and even 10 point scales!  My tip today is about using 7 point scales for Likert-type items.

Hot tip: When creating a Likert scale for survey instruments, it is best to use a 7-point Likert scale because it will give you more variance than a 5-point scale. But, at the same time, using anything higher than a 7-point scale might be too cognitively challenging, and people cannot mentally discriminate at such a precise level. In addition, it would reduce the reliability of your results. Bipolar scales (those with a neutral midpoint, which uses opposite ends such as “strongly agree” to “strongly disagree”) perform best with seven points. Whereas, unipolar scales (zero scale at the end, such as “not at all important”) perform best with five. Plus, offering a midpoint on a bipolar scale, indicating a neutral position, increases reliability. In addition, it is best to provide neutrality, as a mid-point, because this does not force respondents to choose one camp over another. Respondents would be forced to choose a side even if the stand they take is weak. It is important to allow respondents to voice impartiality if they are “on the fence” about a particular topic. Also, they might be inclined to acquiesce to positively worded statements, and a neutral option might reduce this effect.

Rad Resource: These two resources have been very helpful for me.  The Measurement Imperative by Wittink and Bayer in Marketing Research (2003, Vol 15, Part 3, pages 19-22).  I also use William Crano’s book, Principles and Methods of Social Research.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

measure, survey, surveys

My name is Lija Greenseid. I am a Senior Evaluator, with Professional Data Analysts, Inc. in Minneapolis, MN. We conduct evaluations of stop-smoking programs. Smokers generally have lower education and literacy levels than the general population. Therefore, we want to make sure the materials we develop are understandable to smokers.

Rad Resource: Use a “readability calculator” to check the reading-level of your written materials. I have used this with program registration forms, survey instruments, consent statements, and other materials. Not surprisingly, the first drafts of my materials are often written at a level only grad students (and evaluators) can understand. With a critical eye and a few tweaks I can often rewrite my materials so that they are at an eighth-grade reading level, much more accessible to the people with whom I want to communicate.

A good Readability Calculator can be found here:

http://www.editcentral.com/gwt1/EditCentral.html

It provides you with both a reading ease score, and a number of different measures of the US school grade level of the text.

This blog posting is rated at a high-school reading level. Do you agree?

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

esl, Instruments, Literacy, Readability, reports, Smoking, surveys

Hi, my name is Dreolin Fleischer.  I am a doctoral student at Claremont Graduate University. I would like to share resources, at different price points, I have used to capture and organize qualitative and quantitative telephone interview data.

One resource I have used in the past is Microsoft Office Access. You can create a form in Access that mirrors the interview protocol you are using. You control where each field on each form is located and you can create multiple tabs for different interview questions.  As you conduct the interview you enter the interviewee’s responses directly into the field associated with the question you posed.  Cost: $$

I have used online survey programs (e.g., SurveyGizmo, SurveyMonkey, etc.) for the same purpose. You can create an online survey that mirrors the interview protocol you are using. You log into the online survey (as if you are taking the survey yourself) and enter the interviewee’s responses directly into the survey. At the completion of the interview, you can import the data (most of these programs allow you to import into Excel or SPSS files).  Cost: Free to $$ (depending on the online survey program you use)

I’ve yet to explore the tool myself, but I heard from a colleague that Google Documents now offers a way to develop online surveys for free: http://www.google.com/google-d-s/forms/.  Cost: Free

I prefer using the aforementioned resources because:

• The forms I create help me stay organized and guide me through the interview when I am on the phone.
• I have flexibility about how I organize the questions on the form (i.e., I can cluster questions together or isolate a single question according to my preference).
• I can easily record both open-ended qualitative responses and close-ended quantitative responses using these resources.
• It saves me time because the data is immediately available in a spreadsheet/table format at the conclusion of the interview.

I also use an audio recorder to record many of my interviews.  You can purchase audio recorders that connect to your landline phone or cell phone.  Of course you should always ask the interviewee permission before recording the interview.  I have had very few people refuse to be recorded. Keep in mind that recording interviews may not be appropriate for all data collection contexts.  You must weigh the pros and cons of using an audio recorder in relation to the information you are inquiring about.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

Data, interviews, organization, qualitative, quantitative, surveys