Exploring the Multidimensional Structure of the WASI-II: Further Insights from Schmid-Leiman Higher-Order and Exploratory Bifactor Solutions
Abstract
The Wechsler Abbreviated Scale of Intelligence-Second Edition (WASI-II; Wechsler, 2011a) is a versatile and widely utilized brief intelligence measure by assessment psychologists in a variety of clinical settings. Despite the fact that a hierarchical measurement model is implied in the WASI-II Manual (Wechsler, 2011b) and the scores provided by the instrument, a hierarchical measurement model was not specified in the structural validation studies conducted by the test publisher. Due to this shortcoming, two exploratory factor analytic methods (Schmid-Leiman [SL] and exploratory bifactor analysis [EBFA]) were utilized to disclose the higher-order structuring of WASI-II variables. Results from both solutions indicated that the WASI-II provides users with strong measurement of general intelligence and suggest caution when interpreting beyond that dimension despite the provision of additional factor-level indices. Implications for clinical interpretation and appropriate use(s) of the WASI-II and other related brief measures is discussed.References
American Educational Research Association, American Psychological Association, & National Council on Measurement on Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Axelrod, B. N. (2002). Validity of the Wechsler Abbreviated Scale of Intelligence and other very short forms of estimating intellectual functioning. Assessment, 9, 17-23. doi: 10.1177/1073191102009001003
Beaujean, A. A. (2015). John Carroll’s views on intelligence: Bi-factor vs. higher-order models. Journal of Intelligence, 3, 121-136. doi: 10.3390/jintelligence3040121
Beaujean, A. A., McGlaughlin, & Margulies, A. S. (2009). Factorial validity of the Reynolds Intellectual Assessment Scales for referred students. Psychology in the Schools, 46, 932-950. doi: 10.1002/pits.20435
Beaujean, A. A., Parkin, J., & Parker, S. (2014). Comparing Cattell-Horn-Carroll factor models: Differences between bifactor and higher order factor models in predicting language achievement. Psychological Assessment, 26, 789-805. doi: 10.1037/a0036745
Beal, A. L., Willis, J. O., & Dumont, R. (2013). Psychological testing by models of cognitive ability. In D. H. Saklofske, C. R. Reynolds, & V. L. Schwean (Eds.), The oxford handbook of child psychological assessment (pp. 445-473). New York: Oxford University Press.
Brown, T. A. (2016). Confirmatory factor analysis for applied research (2nd ed.). New York: Guilford Press.
Canivez, G. L. (2016). Bifactor modeling in construct validation of multifactored tests: Implications for understanding multidimensional constructs and test interpretation. In K. Schweizer & C. DiStefano (Eds.), Principles and methods of test construction: Standards and recent advancements (pp. 247-271). Gottingen, Germany: Hogrefe.
Canivez, G. L. (2013). Psychometric versus actuarial interpretation of intelligence and related aptitude batteries. In D. H. Saklofske, C. R. Reynolds, & V. L. Schwean (Eds.). The Oxford handbook of child psychological assessment (pp. 84-112). New York: Oxford University Press.
Canivez, G. L., Konold, T. R., Collins, J. M., & Wilson, G. (2009). Construct validity of the Wechsler Abbreviated Scale of Intelligence and Wide Range Intelligence Test. School Psychology Quarterly, 24, 252-265. doi: 10.1037/a0018030
Canivez, G. L., & McGill, R. J. (2016). Factor structure of the Differential Ability Scales-Second Edition: Exploratory and hierarchical factor analyses with the core subtests. Psychological Assessment. Advance online publication. doi: 10.1037/pas0000279
Canivez, G. L., & Watkins, M. W. (2010). Investigation of the factor structure of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV): Exploratory and higher-order factor analyses. Psychological Assessment, 22, 827-836. doi: 10.1037/a0020429
Canivez, G. L., Watkins, M. W., & Dombrowski, S. C. (2016a). Structural validity of the Wechsler Intelligence Scale for Children-Fifth Edition: Confirmatory factor analyses with the 16 primary and secondary subtests. Psychological Assessment. Advance online publication. doi: 10.1037/pas0000358
Canivez, G. L., Watkins, M. W., & Dombrowski, S. C. (2016b). Factor structure of the Wechsler Intelligence Scale for Children-Fifth Edition: Exploratory factor analyses with the 16 primary and secondary subtests. Psychological Assessment, 28, 975-986. doi: 10.1037/pas0000238
Carretta, T. R., & Ree, J. J. (2001). Pitfalls of ability research. International Journal of Selection and Assessment, 9, 325-335. doi: 10.1111/1468-2389.00184
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.
Carroll, J. B. (1995). On methodology in the study of cognitive abilities. Multivariate Behavioral Research, 30, 429-452. doi: 10.1207/s15327906mbr3003_6
Carroll, J. B. (2003). The higher-stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. In H. Nyborg (Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen (pp. 5-21). New York: Pergamon.
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276. doi: 10.1207/s15327906mbr0102_10
Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J. P., & Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality, 80, 219-251. doi:10.1111/j.1467 6494.2011.00739.x
de Winter, J. C. F., & Dodou, D. (2012). Factor recovery by principle axis factoring and maximum likelihood factor analysis as a function of factor pattern and sample size. Journal of Applied Statistics, 39, 695-710. Doi: 10.1080/02664763.2011.610445
Dombrowski, S. C. (2013). Investigating the structure of the WJ-III Cognitive at school age. School Psychology Quarterly, 28, 154-169. doi: 10.1037/spq0000010
Dombrowski, S. C. (2014). Investigating the structure of the WJ-III Cognitive in early school age through two exploratory bifactor analysis procedures. Journal of Psychoeducational Assessment, 32, 483-494. doi: 10.1177/0734282914530838
Dombrowski, S. C., Canivez, G. L., Watkins, M. W., & Beaujean, A. (2015). Exploratory bifactor analysis of the Wechsler Intelligence Scale for Children-Fifth Edition with the 16 primary and secondary subtests. Intelligence, 53, 194-201. doi: 10.1016/j.intell.2015.10.009
Dombrowski, S. C., McGill, R. J., & Canivez, G. L. (2016). Exploratory and hierarchical factor analysis of the WJ IV Cognitive at school age. Psychological Assessment. Advance online publication. doi: 10.1037/pas0000350
Dombrowski, S. C., & Mrazik, M. (2008). Test review: Reynolds Intellectual Assessment Scales. Canadian Journal of School Psychology, 23, 223-230. doi: 10.1177/0829573508324458
Dombrowski, S. C., Watkins, M. W., & Brogan, M. J. (2009). An exploratory investigation of the factor structure of the Reynolds Intellectual Assessment Scales (RIAS). Journal of Psychoeducational Assessment, 27, 494-507. doi: 10.1177/0734282909333179
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272-299. doi: 10.1037/1082-989X.4.3.272
Floyd, R. G., Clark, M. H., & Shadish, W. R. (2008). The exchangeability of IQs: Implications for professional psychology. Professional Psychology: Research and Practice, 39, 414-423. doi: 10.1037/0735-7028.39.4.414
Frazier, T. W., & Youngstrom, E. A. (2007). Historical increase in the number of factors measured by commercial tests of cognitive ability: Are we overfactoring? Intelligence, 35, 169-182. doi: 10.1016/j.intell.2006.07.002
Gignac, G. E. (2007). Multi-factor modeling in individual differences research: Some recommendations and suggestions. Personality and Individual Differences, 42, 37-48. doi: 0.1016/j.paid.2006.06.019
Gignac, G. E. (2016). The higher-order model imposes a proportionality constraint: That is why the bi-factor model tends to fit better. Intelligence, 55, 57-68. doi: 10.1016/j.intell.2016.01.006
Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS–IV. Multivariate Behavioral Research, 48, 639-662. doi:10.1080/00273171.2013.804398
Glutting, J., Adams, W., & Sheslow, D. (2000). Wide Range Intelligence Test. Wilmington, DE: Wide Range.
Glutting, J. J., Watkins, M. W., & Youngstrom, E. A. (2003). Multifactored and cross-battery assessments: Are they worth the effort? In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Intelligence aptitude, and achievement (2nd ed., pp. 343-374). New York: Guilford.
Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum.
Groth-Marat, G. (2009). Handbook of psychological assessment (5th ed.). Hoboken, NJ: John Wiley.
Gustaffson, J. E., & A ̇berg-Bengtsen, L. (2010). Unidimensionality and interpretability of psychological instruments. In S. E. Embretson (Ed.), Measuring psychological constructs: Advances in model-based approaches (pp. 97-122). Washington, DC: American Psychological Association.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychomerika, 30, 179-185. doi: 10.1007/BF02289447
Jennrich, R. I., & Bentler, P. M. (2011). Exploratory bi-factor analysis. Psychometrika, 76, 537-549. doi: 10.1007/s11336-011-9218-4
Johnson, K. M., & Sandoval, J. (2014). Wechsler Abbreviated Scale of Intelligence-Second Edition. In J. F. Carlson, Geisinger, K. F., & Jonson, J. L. (Eds.), The nineteenth mental measurements yearbook (pp. 1-7). Lincoln, NE: Buros Institute of Mental Measurement.
Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39, 31-36. doi: 10.1007/BF02291575
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1-73. doi: 10.1111/jedm.12000
Kaufman, A. S., & Kaufman, N. L. (2004). Kaufman Brief Intelligence Test (2nd ed.). Circle Pines, MN: American Guidance Service.
Kaufman, J. C., & Kaufman, A. S. (2001). Time for the changing of the guard: A farewell to short forms of intelligence tests. Journal of Psychoeducational Assessment, 19, 245-267. doi: 10.1177/073428290101900305
Keith, T. Z., Caemmerer, J. M., & Reynolds, M. R. (2016). Comparison of methods for factor extraction for cognitive test-like data: Which overfactor, which underfactor? Intelligence, 54, 37-54. doi: 10.1016/j.intell.2015.11.003
Kranzler, J. H., & Keith, T. Z. (1999). Independent confirmatory factor analysis of the Cognitive Assessment System (CAS): What does the CAS measure? School Psychology Review, 28, 117-144. Retrieved from http://www.nasponline.org
McClain, A. L. (1996). Hierarchical analytic methods that yield different perspectives on dynamics: Aids to interpretation. Advances in Social Science Methodology, 4, 229-240.
McCrimmon, A. W., & Smith, A. D. (2013). Review of the Wechsler Abbreviated Scale of Intelligence, Second Edition (WASI-II). Journal of Psychoeducational Assessment, 31, 337-341. doi: 10.1177/0734282912467756
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
McGill, R. J. (2016). Investigating the factor structure of the Comprehensive Test of Nonverbal Intelligence-Second Edition (CTONI-2) using exploratory factor analysis. Journal of Psychoeducational Assessment, 34, 339-350. doi: 10.1177/0734282915610717
McGill, R. J., & Canivez, G. L. (2016). Orthogonal higher order structure of the WISC–IV Spanish using hierarchical exploratory factor analytic procedures. Journal of Psychoeducational Assessment, 34, 600-606. doi: 10.1177/0734282915624293
McGill, R. J., & Spurgin, A. R. (2015). Exploratory higher order analysis of the Luria interpretive model on the Kaufman Assessment Battery for Children-Second Edition (KABC-II) school-age battery. Assessment. Advance online publication. doi: 10.1177/1073191115614081
Murray, A. L., & Johnson, W. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 37, 487-494. doi: 10.1016/j.intell.2013.06.004
Nelson, J. M., & Canivez, G. L. (2012). Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample. Psychological Assessment, 24, 129-140. doi: 10.1037/a0024878
Nelson, J. M, Canivez, G. L, Lindstrom, W., & Hatt, C. (2007). Higher-order exploratory factor analysis of the Reynolds Intellectual Assessment Scales with a referred sample. Journal of School Psychology, 45, 439-456. doi: 10.1016/j.jsp.2007.03.003
Pierson, E. E., Kilmer, L. M., Rothlisberg, B. A., & McIntosh, D. E. (2012). Use of brief intelligence tests in the identification of giftedness. Journal of Psychoeducational Assessment, 30, 10-24. doi: 177/0734282911428193
Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine. Understanding Statistics, 2, 13-43. doi: 10.1207/S15328031US0201_02
R Developmental Core Team (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. doi: 10.1080/00273171.2012.715555
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. doi:10.1080/00223891.2012.725437
Reynolds, C. R., & Kamphaus, R. W. (2003). Reynolds Intellectual Assessment Scales professional manual. Lutz, FL: Psychological Assessment Resources.
Reynolds, C. R., & Kamphaus, R. W. (2015). Reynolds Intellectual Assessment Scales (2nd ed.). Lutz, FL: Psychological Assessment Resources.
Reynolds, C. R., Kamphaus, R. W., & Raines, T. C. (2012). The Reynolds Intellectual Assessment Scales and the Reynolds Intellectual Screening test. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed.; pp. 400-421). New York: Guilford.
Reynolds, M. R., & T. Z. Keith (2013). Measurement and statistical issues in child assessment research. In D. H. Saklofske, C. R. Reynolds, & V. L. Schwean (Eds.). The Oxford handbook of child psychological assessment (pp. 48-83). New York: Oxford University Press.
Revelle, W. (2016). Psych: Procedures for psychological, psychometric, and personality research (version 1.6.9) [Computer Software]. Evanston, IL: Northwestern University.
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98, 223-237.
doi: 10.1080/00223891.2015.1089249
Ryan, J. J., Carruthers, C. A., Miller, L. J., Souheaver, G. T., Gontkovsky, S. T., & Zehr, M. D. (2010). Exploratory factor analysis of the Wechsler Abbreviated Scale of Intelligence (WASI) in adult standardization and clinical samples. Applied Neuropsychology: Adult, 10, 252-256. Doi: 10.1207/s15324826an1004_8
Sattler, J. M. (2008). Assessment of children: Cognitive foundations (5th ed.). La Mesa, CA: Sattler Publishing.
Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53-61. doi: 10.1007/BF02289209
Silverstein, A. B. (1990). Short forms of individual intelligence tests. Psychological Assessment, 2, 3-11. doi: 10.1037/1040-3590.2.1.3
Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.
Thompson, B., & Daniel, L. G. (1996). Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56, 197-208. doi: 10.1177/0013164496056002001
Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 31, 321-327. doi: 10.1007/BF02293557
Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A view and evaluation of alternative procedures for determining the number of factors or components. In R. D. Gofï¬n & E. Helmes (Eds.), Problems and
solutions in human assessment: A festschrift to Douglas Jackson at seventy (pp. 41-71). Norwell, MA: Kluwer Academic.
Watkins, M. W. (2006). Orthogonal higher order structure of the Wechsler Intelligence Scale for Children-Fourth Edition. Psychological Assessment, 18, 123-125. doi: 10.1037/1040-3590.18.1.123
Watkins, M. W. (2013). Omega. [Computer Software]. Phoenix, AZ: Ed & Psych Associates.
Wechsler, D. (2011a). Wechsler Abbreviated Scale of Intelligence (2nd ed.). Bloomington, MN: Pearson.
Wechsler, D. (2011b). Wechsler Abbreviated Scale of Intelligence—Second Edition manual. Bloomington, MN: Pearson.
Weiner, I. B. (1989). On competence and ethicality in psychodiagnostic assessment. Journal of Personality Assessment, 53, 827-831. doi:10.1207/s15327752jpa5304_18
Wood, J. M., Tatryn, D. J., & Gorsuch, R. L. (1996). Effects of under- and overextraction on principal axis factor analysis with varimax rotation. Psychological Methods, 1, 354-365. doi: 10.1037/1082-989X.1.4.354
Woodcock, R. W., McGrew, K. S., & Mather, N. (2011). Woodcock-Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside.
Yates, B. T., & Taub, J. (2003). Assessing the costs, benefits, cost-effectiveness, and cost-benefit of psychological assessment: We should, we can, and here’s how. Psychological Assessment, 15, 478-495. doi: 10.1037/1040-3590.15.4.478
Youngstrom, E. A. (2008). Commentary: Evidence-based assessment is not evidence-based medicine—Commentary on evidence-based assessment of cognitive functioning in pediatric psychology. Journal of Pediatric Psychology, 33, 1015-1020. doi: 10.1093/jpepsy/jsn060
