My name is Manolya Tanyu and I am a researcher at American Institutes for Research (AIR), a behavioral and social science research organization to support education, educational assessment, health, international development, and work and training. My former organization, Learning Point Associates recently merged with AIR.
Our world is a nested one where individuals are influenced by the context and individual characteristics shape the context. In recognition of this interrelatedness, I would like to talk about an increasingly popular statistical technique, multilevel modeling (also known as hierarchical linear modeling, nested modeling, or mixed modeling) that is widely used in projects at AIR that apply quantitative techniques. So, here I share basic ideas behind multilevel models and some resources.
Hot Tip: The main idea behind multilevel modeling is that the behaviors of the individuals within the same setting (e.g., classroom) will be correlated as will the behavior of the individuals from the same setting (e.g., school). Individuals within the same groups are more likely to be similar to each other due to common history and other contextual features. Standard statistical tests depend on independence of the observations. When this independence is not observed within statistical modeling approaches, it may result in spurious significant results when all individuals from different levels (e.g., schools) are aggregated. Multilevel modeling accounts for the different dependencies.
Cool Trick: By examining the effects of individuals and context simultaneously, important information is captured that would not be revealed by either examining data at the individual level (ignoring the fact that individuals are nested within different schools, for example) or aggregating data to the group (or, school) level. In turn, researchers are able to identify which level (e.g., individual- or school-level) explains more variance in the outcome of interest.
Lessons Learned: The interrelationship between context and individual level effects has strong implications for research, policy, and practice.
Rad Resources: Several researchers are well regarded for their work in multilevel modeling and have published extensively. Among these are Stephen Raudenbush, Anthony Bryk, Andrew Gelman, and Judith Singer, to name a few.
Some widely-read publications:
- Raudenbush, S. W. & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed). Thousand Oaks, CA: Sage Publications.
- Gelman, A. & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. New York: Cambridge University Press.
- Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 24(4), 323-355.
Several software packages are also used to analyze multilevel models. The most commonly used packages include: HLM, SAS, R, MPlus, and STATA.
Some useful online resources:
The American Evaluation Association is Educational Evaluation Week with our colleagues in the PreK-12 Educational Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EdEval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.