Hi, my name is Nancy Aguirre. I am a research associate for an independent evaluation consultant and a Professional Expert at Mt. San Antonio College in California. One thing that’s been tricky for me is developing scales, and it seems like so many people have a preference for 5, 7, and even 10 point scales! My tip today is about using 7 point scales for Likert-type items.
Hot tip: When creating a Likert scale for survey instruments, it is best to use a 7-point Likert scale because it will give you more variance than a 5-point scale. But, at the same time, using anything higher than a 7-point scale might be too cognitively challenging, and people cannot mentally discriminate at such a precise level. In addition, it would reduce the reliability of your results. Bipolar scales (those with a neutral midpoint, which uses opposite ends such as “strongly agree” to “strongly disagree”) perform best with seven points. Whereas, unipolar scales (zero scale at the end, such as “not at all important”) perform best with five. Plus, offering a midpoint on a bipolar scale, indicating a neutral position, increases reliability. In addition, it is best to provide neutrality, as a mid-point, because this does not force respondents to choose one camp over another. Respondents would be forced to choose a side even if the stand they take is weak. It is important to allow respondents to voice impartiality if they are “on the fence” about a particular topic. Also, they might be inclined to acquiesce to positively worded statements, and a neutral option might reduce this effect.
Rad Resource: These two resources have been very helpful for me. The Measurement Imperative by Wittink and Bayer in Marketing Research (2003, Vol 15, Part 3, pages 19-22). I also use William Crano’s book, Principles and Methods of Social Research.
This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.
Joel Nadler gave a presentation at Evaluation 2009 about survey development and the use of midpoints. He had some great resources and a strong research-base for what he had to say. His presentation slides may be found here http://bit.ly/midpoints for those interested.
Thanks, Nancy, for starting off an interesting discussion. In my work I’ve found that the choice of scale is also dependant on the audience you are trying to reach and your information needs. We do a lot of work with individuals whose experience with surveys is limited. Even a 5 point scale can present cognitive challenges. In other cases, having a midpiont may not be as important as having a “not applicable” catagory- or both may be needed to separate folks who have knowelge but don’t have a strong opinion from folks who don’t have the information needed to give a rating. Carrying out throurough cognitive testing of items helps to determine what scales to use whith what audience.
Hi Scott,
I agree that rating scale construction should be an empirical matter, as opposed to a matter of personal preference. However, when you consider that a large number of scales are based upon Likert formats with 5 or less response categories, but are not using non-parametric procedures (which they technically should since the data are non-continuous), this is an issue. Coming from an I/O Psychology background, many (including myself in the past) have been guilty of doing this, but it is still a common practice. Regarding 4-pt scales functioning better than their 7- or 9-pt alternatives, there is mixed evidence. For example, there has been recent evidence, based off of Monte Carlo simulations, which has suggested that as the number of response options increases, both validity and internal consistency improves (see Lozano et al., 2008).
Lozano, L.M., Garcia-Cueto, E., & Muniz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. European Journal of Research Methods for the Behavioral and Social Sciences, 4, 73-79.
Best,
I would like to offer a differeent perpsective on this issue. The construction of a rating scale should be an empirical matter not a question of personal preference. That is to say a scale needs to function in a certain manner and we can test for this. I have previously written an article explaining how to do this. Contrary to popular belief and practice, 4 point rating sales often function better than 5, 7 and 10 point scales. See:
Bayley, S. 2001, ‘Measuring Customer Satisfaction’, Evaluation Journal of Australasia, vol. 1, no. 1, pp. 8-17.
Hi Nancy,
Thanks for the suggestions with scale development. I, too, feel that 7-pt Likert scales offers the “best” of both worlds; it has enough options to escape the ordinal-level of measurement and approach the interval-level (required of parametric statistics), but doesn’t have too many response options that may, as you mentioned, lead to issues discriminating between options. And while I typically include a “neutral” category when developing scales, some would actually argue that no one is ever completely neutral on a topic (now, whether they’re leaning enough to one side or the other to justify an “agree” or “disagree” response is another matter, but worth taking into consideration). Great tips!