My name is Kai Chi Yam, and I am a graduate student at Washington State University. I recently had an opportunity to evaluate a university-based mentoring program. Exploratory Factor Analysis (EFA) was used to provide score validity evidence through the examination of internal structure of study-specific measures. Such evidence can increase the utility of the instruments in evaluation, and ultimately increase the creditability and utility of evaluation results. While working with my colleagues, I found that some evaluators may be familiar with EFA, but most follow software default settings with little consideration of different procedures that can affect the results of EFA. Here are some general guidelines for appropriate use of EFA.
Hot Tip: Follow these steps when conducting an EFA:
1. Data screening: Check Chi Square (p > .05), Kaiser-Meyer-Olkin measure of sampling adequacy (MSA > .70), Bartlett’s test of sphericity, univariate and multivariate normality to determine if the data is appropriate for factor analytic procedures.
2. Extraction methods: Use principal component analysis if the rationale of EFA is purely data reduction. Use principal axis factoring if the rationale of EFA is to extract latent variables.
Note: Principal axis factoring is preferred because it takes measurement error into account.
3. Rotation methods: Use direct oblimin or promax rotation when factors are non-orthogonal. Use varimax rotation when factors are orthogonal.
Note: In most evaluations, especially when the measurements are psychological in nature, evaluators should assume factors to be non-orthogonal and proceed with direct oblimin or promax rotation.
4. Criteria for retaining factors: Decide the number of factors based on at least two criteria (e.g., scree plot, parallel analysis, conceptual meaningfulness of the factor) other than Kaiser’s rule of eigenvalue > 1.
5. Interpretation: Name each factor with theoretical (i.e., supporting theories) and statistical justifications (i.e., factor loadings). A general rule of thumb for acceptable factor loading is .40 or above.
Note: All interpretations must be theoretically justifiable! Don’t base your judgment solely on factor loadings.
Rad Resource: Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. (ISBN: 0761919503) This introductory textbook is easy to follow and requires minimum knowledge in statistics and math.
Exploratory factor analysis can be a useful tool in evaluation when study-specific measures are employed. Please note that these are general guidelines, not definitive rules. Evaluators should consult with methodologists, textbooks, or journal articles before attempting EFA.
Want to learn more about Kat Chi’s work using EFA? Join us at the American Evaluation Association’s Annual Conference, Evaluation 2010, in San Antonio this November and check out the poster exhibition on Wednesday evening.