Chapter Two

How to Organize a Cross-Battery Assessment Using Cognitive, Achievement, and Neuropsychological Batteries

Overview

This chapter describes the fundamental principles for organizing CrossBattery Assessments (XBAs). Clear, step-by-step instructions of the approach are presented which allow practitioners to organize assessment batteries that are appropriate to particular referral concerns and purposes of evaluation. To assist practitioners in conducting XBAs, the XBA Data Management and Interpretive Assistant (XBA DMIA v2.0) is offered. The XBA DMIA v2.0 is included on the CD that accompanies this book.

Chapter 1 described how contemporary Cattell-Horn-Carroll (CHC) theory and CHC test classifications have influenced the development of all current intelligence batteries (hereafter referred to as cognitive batteries). Although none of these batteries measures the full range of broad and narrow abilities and processes specified by the theory, all provide measurement of CHC abilities and processes, some more comprehensively than others, and most represent a significant improvement over their predecessors. For example, the WoodcockJohnson III Normative Update Tests of Cognitive Abilities (WJ III NU COG) is the most comprehensive battery of cognitive subtests currently available, followed by the Differential Ability Scales, Second Edition (DAS-II). Other batteries, although less comprehensive, offer unique features that are important for evaluating certain children (e.g., the Kaufman Assessment Battery for Children, second edition [KABC-II] is particularly

relations between CHC abilities and reading achievement (see Rapid Reference 2.1). First, narrow abilities subsumed by Ga, Gc, Glr, Gsm, and Gs displayed the most consistent significant relations with reading achievement. Measures of phonological processing or awareness (e.g., Phonetic Coding [PC], which is subsumed by Ga) showed strong and consistent relations with reading achievement across many studies, especially during the early elementary school years. Gc abilities, which were typically represented by measures of Lexical Knowledge (VL), Listening Ability (LS), Language Development (LD), and General Information (K0), were also significantly related to reading achievement. As reported in some studies (e.g., Evans, Floyd, McGrew, & Leforgee, 2001; Garcia & Stafford, 2000; McGrew, 1993; McGrew et al., 1997), the significant effects of Ga and Gc on reading were present even after the powerful effect of g was accounted for in the analyses. That is, specific CHC abilities contributed significantly to the explanation of reading above and beyond the significant and large effect of g.

Rapid Reference 2.1

Important Findings on Relations Among CHC Abilities, Neuropsychological Processes, and Reading Achievement

Many studies that included Gsm indicated that Gsm most likely contributes to reading achievement through working memory processes (Berninger, 2011; Hale & Fiorello, 2004; Semrud-Clikeman, 2005). Nevertheless, significant relations between Memory Span and reading achievement have also been documented (see McGrew & Wendling, 2010). Taken as a whole, independent, comprehensive reviews of the reading achievement literature suggest that Gsm, including working memory and memory span, contributes significantly to the prediction of reading achievement (e.g., Feifer, 2011; Flanagan et al., 2006; McGrew & Wendling, 2010).

The relationship between Glr and reading achievement is consistent across most of the school-age range (e.g., 6–13 years). Associative Memory (MA) and Naming Facility (NA) are important during the elementary years; Meaningful Memory (MM) is important at ages 9 to 13 years, particularly for reading comprehension (McGrew & Wendling, 2010). In addition, several studies found a strong relation between Perceptual Speed (P), a narrow Gs ability, and reading achievement across the school-age range (6–19 years) (e.g., Berninger, 2011; Feifer, 2012; Feifer & Della Toffalo, 2007; McGrew, 1993; McGrew et al., 1997; McGrew & Wendling, 2010). The effect of Gs was present even after the effect of g on reading achievement was accounted for in the McGrew and colleagues’ (1997) study. This finding was replicated by Evans and associates (2001), who found Gs to be significantly related to both basic reading skills and reading comprehension in the early years. Thus, as with Ga and Gc abilities, Gs abilities (viz., perceptual speed) explain significant variance in reading achievement above and beyond the variance explained by g.

It appears that Gf and Gv abilities are less related to reading achievement as compared to Gc, Ga, Glr, Gsm, and Gs abilities. The significant and most consistent Gf findings were between inductive and deductive reasoning and reading comprehension (e.g., see Flanagan et al., 2006, for a discussion). This suggests that the comprehension of text may draw on an individual's reasoning abilities, depending on the demands of the comprehension task (e.g., tasks that require drawing inferences, comparing and contrasting, and making predictions). Related to Gf, the role of executive function and reading achievement (particularly reading comprehension) has been documented in the neuropsychology literature (e.g., McCloskey, Whitaker, Murphy, &

Rogers, 2012).

Very few studies reported a significant relation between Gv and reading achievement, although McGrew and Wendling (2010) reported a consistent relationship between Visual Memory and reading comprehension at ages 14 to 19 years. This finding most likely suggests that reading comprehension is aided by visualization strategies. Nevertheless, it appears that Gv abilities may not play a significant role in reading achievement. The lack of significant Gv/reading research findings indicates that the contribution of Gv abilities, as measured by current cognitive batteries, to the explanation and prediction of reading achievement is so small that, when compared to other abilities (e.g., Ga, Gc), it is of little practical significance. However, this conclusion is based only on studies that measured Gv using current cognitive batteries. It is important not to overgeneralize this conclusion to all visual abilities. As pointed out by Berninger (1990), visual perceptual abilities should not be confused with abilities that are related to the coding of visual information in printed words (i.e., orthographic code processing)—visual processes thought to be important during reading. Indeed, Flanagan and her colleagues (2006) found in their review of the literature a consistent relationship between orthographic processing and reading achievement (i.e., basic reading skills; see Berninger, 2011, for further discussion).

Don't Forget

Narrow abilities in the areas of Gc, Ga, Glr, Gsm, and Gs (and, to a lesser extent, Gf and Gv) are important for reading achievement, as are neuropsychological processes, including attention and executive functions. The importance of orthographic processing (a Gv “ability,” although not included in CHC theory) and its relation to reading achievement is most prominent in the neuropsychology literature. A comprehensive evaluation of suspected reading disability should include measurement of the specific abilities and neuropsychological processes that are most relevant to specific areas of reading difficulty.

In summary, narrow abilities in seven broad CHC domains appear to be related significantly to reading achievement. The findings of two independent, comprehensive literature reviews (i.e., Flanagan et al., 2006; McGrew & Wendling, 2010) suggest that abilities subsumed by Gc (Language Development, Lexical Knowledge, Listening Ability, General Information), Gsm (Memory Span, Working Memory), Ga (Phonetic Coding), Glr (Associative Memory, Naming Facility, Meaningful Memory), and Gs (Perceptual Speed) are related significantly to reading achievement.

Furthermore, developmental results suggest that the Ga, Gs, and Glr relations with reading are strongest during the early elementary school years, after which they systematically decrease in strength (e.g., Flanagan et al., 2006; McGrew, 1993). In contrast, the strength of the relations between Gc abilities and reading achievement increases with age. The Gv abilities of orthographic processing and visual memory are related to reading achievement. Finally, Gf abilities appear related primarily to reading comprehension from childhood to young adulthood.

Cognitive Abilities, Neuropsychological Processes, and Math

Achievement

Similar to reading, both literature reviews (Flanagan et al., 2006; McGrew & Wendling, 2010) found that Gc, Gsm (particularly working memory), and Gs are related significantly to math achievement. In contrast to reading, stronger evidence of the relations between Gf and Gv abilities and math achievement was found (see Rapid Reference 2.2).

In some of the more comprehensive studies of the relations between CHC abilities and math achievement (e.g., McGrew & Hessler, 1995), Gf, Gc, and Gs abilities correlated consistently and significantly with basic math skills and math problem solving (see also Geary, Hoard, & Bailey, 2011). However, there were developmental differences. The Gc relation with mathematics achievement increased monotonically with age, whereas the Gs relation was strongest during the elementary school years, after which it decreased (although the relationship remained significant well into adulthood). Gf was related consistently to mathematics achievement at levels higher than all other CHC abilities (except Gc) across all ages. Also, many executive functions are considered important for math achievement, including selective attention (e.g., attention to operational signs), planning (e.g., selecting salient information from word problems), organizing (e.g., ability to set up problems effectively), and self-monitoring (e.g., checking work for errors; Feifer & DeFina, 2005; McCloskey, Perkins, & Van Divner, 2009; Meltzer, 2007). As in the reading achievement research just mentioned, certain specific abilities (Gf, Gs, Gc) were found to be related significantly to mathematics achievement above and beyond the contribution of g (e.g., McGrew et al., 1997).

With one exception (i.e., a consistent relation between Spatial Scanning and basic math skills), no significant relations between Gv and mathematics achievement were found. Nevertheless, the neuropsychology literature includes substantial coverage of the importance of visual-spatial functioning in math achievement (e.g., Feifer & DeFina, 2005; Hale & Fiorello, 2004; Mazzocco, 2012). Visual-spatial functioning would appear to align with the narrow Gv ability of Visualization (Vz), although few studies have yet to systematically examine the relationship between Vz and math achievement.

Like the Gv CHC literature, very few CHC-based studies reported a significant relationship between Glr and mathematics achievement (Floyd, Evans, & McGrew, 2003; Geary, 1993; Geary et al., 2011). According to McGrew and Wendling (2010), the Glr narrow ability of Meaningful Memory is related to basic math skills at ages 9 to 13 years and math reasoning at ages 14 to 19 years; Associative Memory and Naming Facility are related to basic math skills at ages 6 to 8 years and 6 to 19 years, respectively. Also, the neuropsychology literature emphasizes the importance of developing automatic retrieval skills in math at early ages. Therefore, fluency with math facts or the rapid retrieval of basic math facts is important in understanding math learning difficulties (e.g., Geary et al., 2011; Wright, Martland, & Stafford, 2000). Long-term memory is also important in predicting mathematical problem-solving accuracy (e.g., Swanson & BeebeFrankenberger, 2004) beyond that predicted by other abilities (e.g., Gsm, Gs).

Rapid Reference 2.2

Important Findings on Relations Among CHC Abilities, Neuropsychological Processes, and Mathematics Achievement

Don't Forget

Narrow abilities and processes in the areas of Gf, Gv, Gc, Gsm, Glr, and Gs appear to be most important for math achievement. Gf plays a more prominent role in math achievement than in reading achievement. Attention and executive functions also play a role.

Cognitive Abilities, Neuropsychological Processes, and Writing

Achievement

A review of Rapid Reference 2.3 demonstrates that several CHC domains are related to writing achievement. Specifically, researchers have documented relations between cognitive abilities and writing achievement across seven CHC domains, which are listed in Rapid Reference 2.3 (Gf, Gc, Gsm, Gv, Ga, Glr, and Gs). However, evidence from the limited number of studies in certain CHC domains clearly suggests that the consistency of relations differs markedly across areas. For instance, only one study demonstrated a relation between Gf abilities and writing achievement. Specifically, McGrew and Knopik (1993) found that fluid reasoning abilities (i.e., induction and general sequential reasoning) were related significantly to basic writing skills primarily during the elementary school years (i.e., ages 6–13) and significantly related to written expression across all ages.

Rapid Reference 2.3

Important Findings on Relations Among CHC Abilities, Neuropsychological Processes, and Writing Achievement

important during the elementary school years for both basic writing skills and written expression (primarily before age 11).

Glr Naming Facility (NA) or “rapid automatic naming” has demonstrated relations with written expression, primarily the fluency aspect of writing. Associative Memory (MA).

Gs Perceptual Speed (P) is important during all school years for basic writing and related to all ages for written expression.

Information in this table was reproduced from Flanagan et al. (2006) with permission from John Wiley & Sons. All rights reserved.

Note: The absence of comments for a particular CHC ability (e.g., Gv) indicates that the research reviewed either did not report any significant relations between the respective CHC ability and writing achievement, or if significant findings were reported, they were for only a limited number of studies. Comments in bold represent the CHC abilities that showed the strongest and most consistent relation to writing achievement. Information in this table was reproduced from Flanagan, Ortiz, Alfonso, and Mascolo (2006) with permission from John Wiley & Sons. All rights reserved.

1 Includes orthographic knowledge and knowledge of morphology, which contribute to spelling and written expression.

Similarly, the study by McGrew and Knopik (1993) provided evidence for the role of Gs abilities in writing. More specifically, this study demonstrated that the Gs cluster (comprised of measures of perceptual speed) “was significantly related to Basic Writing Skills during the school years. . .after which it decreased in strength of association” (p. 690) with age. The relations between Gs and written expression were more consistent in strength across ages. As explained by McGrew and Knopik, “Given the timed nature of the [WJ-R] Writing Fluency tests that comprises one-half of the [WJ-R] Written Expression cluster, the finding of consistently significant associations between Processing Speed and this writing achievement criterion was not surprising” (p. 692). This finding is also not surprising in light of the recent refinements to CHC theory, particularly the addition of Writing Speed (WS) as a narrow Gs ability (see Figure 1.5 in Chapter 1). The neuropsychology literature highlights the importance of an ability related to Gs known as automaticity and has demonstrated its relationship to a variety of areas necessary for effective writing. For example, automaticity in the motor component of written language is important and necessary to free up critical cognitive resources important to the writing process (Feifer & DeFina, 2002).

Berninger (2011), Floyd and colleagues (2008), and Williams, Zolten, Rickert, Spence, and Ashcraft (1993) also reported significant relations

between Gs and writing abilities. For example, the latter study demonstrated relations between the WISC-III Coding subtest (a measure of perceptual speed) and the Woodcock-Johnson Psycho-Educational Battery–Revised (WJ-R) Writing Fluency test. Likewise, Hargrave (2005) found that, in addition to other CHC broad abilities, Gs significantly predicted performance on the WJ III ACH Broad Written Language Cluster. Given these findings, it seems likely that processing speed is important in terms of writing automaticity as well as more general writing ability. Although only a few studies found a relation between Gs and writing achievement, the strength of the Gs effects demonstrated in the aforementioned studies is significant and warrants continued attention and investigation (Floyd et al., 2008).

The relation between Gv and writing achievement is sparse in the CHC literature, suggesting the need for continued study. Because only one study in Flanagan and colleagues’ (2006) review reported a significant relation between Gv and writing achievement (Aaron, 1995), it may be that Gv abilities as assessed by the major cognitive batteries do not play a significant role in writing achievement. This is not to say that Gv abilities are unimportant for writing. In fact, orthographic processing is particularly influential in basic writing tasks (e.g., spelling; see Berninger, 2009, 2011; Bruck, 1992; Moats, 1995). As defined by Aaron (1995), orthography refers to the visual patterns of the written language. However, “orthographic processing ability is not the same as visual memory even though visual memory may play a role in it” (p. 347). Specifically, some researchers have indicated that a certain type of memory for orthographic units may play a role in spelling words that cannot be accurately spelled using the rules of pronunciation alone (see Kreiner & Gough, 2007, for a more in-depth discussion). Although orthographic knowledge plays a significant role in basic writing tasks, CHC theory does not currently have a narrow ability category corresponding to this type of processing. Nevertheless, as with reading achievement, we have listed orthographic processing under Gv in Rapid Reference 2.3.

Another Gv-like ability, called visual-motor integration, is discussed in the neuropsychology and occupational therapy literature, particularly regarding its importance for writing (e.g., Feifer & DeFina, 2002; Hale & Fiorello, 2004; Miller, 2007; Volman, van Schendel, & Jongmans, 2006). However,

based on contemporary CHC theory, it is our contention that visual-motor integration tests measure a narrow ability called Manual Dexterity (P1), which is part of the broad ability of Gp (Psychomotor Abilities), not Gv. P1 is defined as the ability to make precisely coordinated movements of a hand or of a hand and the attached arm (Schneider & McGrew, 2012). It may be that the importance of visual-motor integration for writing achievement has not found its way into the CHC literature to date because tests like the Bender Gestalt-II and others have not been included in the CHC/writing achievement research. In general, many of the existing Gv abilities that comprise CHC theory (e.g., closure speed) appear to be minimally related to writing achievement. It is likely that additional Gv abilities and Gv-like abilities (e.g., orthographic processing; visual-motor integration) that are related to writing achievement will be incorporated within the CHC theoretical framework in the near future (Flanagan et al., 2006; Flanagan, Alfonso, Ortiz, & Dynda, in press).

The CHC-based research on the relations between Glr and Gc and writing achievement is also sparse. The fact that only a handful of studies have documented a significant relation between Glr and writing to date suggests that either Glr abilities are of limited importance to the writing process or the importance of Glr in writing ability has not been investigated thoroughly. Conversely, the importance of Glr for writing achievement (particularly, retrieval fluency) appears to be fairly well documented in the neuropsychology literature (e.g., Berninger, 2007; Feifer & DeFina, 2002; Miller, 2007). Another Glr narrow ability, Associative Memory (MA), appears to be involved in mapping sounds to their corresponding letters (e.g., Mather & Wendling, 2011, 2012). In terms of Gc, McGrew and Knopik (1993) and Floyd et al. (2008) found significant relations among language development (LD), lexical knowledge (VL), general information (K0), and writing abilities (i.e., basic writing skills and written expression). Although the Gc research is also limited, there are certainly stores of knowledge (Gc) that are necessary for successful writing. For example, knowledge of orthography and morphology as well as lexical knowledge contribute to spelling and written expression (Berninger, 2011; Mather & Wendling, 2011).

Despite the limited research on the relations between CHC abilities and

writing achievement, Rapid Reference 2.3 shows that Gc and Gsm displayed the most consistent significant relations with overall writing achievement. Additionally, Phonetic Coding, a narrow Ga ability, and Perceptual Speed, a narrow Gs ability, were found to have strong and consistent relations with writing achievement across many studies, especially during the early elementary school years (e.g., Berninger, Cartwright, Yates, Swanson, & Abbott, 1994; Johnson, 1993; Joshi, 1995; McGrew & Knopik, 1993). Finally, the majority of CHC-based studies that found a relationship between Gsm and writing achievement suggested that memory span is an important predictor of early writing achievement. The neuropsychology literature documents the importance of working memory in writing (e.g., Berninger, 2007, 2011; Dehn, 2012; Feifer & DeFina, 2002; Miller, 2007).

Don't Forget

Narrow abilities and processes in the areas of Gc, Ga, Gsm, and Gs have the most consistent and strongest relations to writing achievement. Orthographic processing, visual-motor integration, attention, and executive functions also play a role.

Overall, several CHC abilities and neuropsychological processes are related significantly to writing achievement. Among these, the most consistent relations appear to be with Ga (phonetic coding), Gsm (memory span), Gs (perceptual speed), and Gc (lexical knowledge, language development, and general information, the latter of which includes orthographic knowledge and knowledge of morphology). In addition, visual-motor integration (Gp) and retrieval fluency (Glr) are important. The relatively limited research on the relations between cognitive abilities and writing achievement may be related, in part, to the fact that writing research has taken a tertiary position to reading and math research. That is, although the early pioneering literature on learning disabilities emphasized both writing and reading disabilities, the subsequent learning disabilities literature has given more attention to reading than writing (Berninger, 2011). Given the importance of writing throughout one's educational (and, often, professional) careers, the field would benefit from additional research within this domain.

In summary, Rapid References 2.1, 2.2, and 2.3 presented the available literature on the relations between cognitive abilities and neuropsychological processes and reading, math, and writing achievement, respectively, based largely on two independent, comprehensive reviews of the literature

In similar fashion, with respect to legal considerations, there are times when federal or local regulations mandate that certain types of data should be collected (e.g., IQ or global ability scores from cognitive batteries). Although this most often occurs in assessments that are conducted for the purpose of gathering data to inform decisions regarding special education eligibility, many states and districts no longer mandate global ability scores for disability determination (e.g., determination of SLD). However, in those locations where global ability is still mandated or encouraged, practitioners may find it necessary to obtain the required score even though they may not consider the score relevant. For example, instead of administering a WJ III NU COG (which measures seven CHC broad cognitive abilities adequately), a practitioner may administer the Wechsler Intelligence Scale for Children– Fourth Edition (WISC-IV) and supplement it with the WJ III NU COG (e.g., in the areas of Glr and Ga), so that he or she can obtain the Full Scale Intelligence Quotient (FSIQ). Although a WJ III NU COG evaluation is more straightforward than the WISC-IV/WJ III NU crossbattery, the evaluator was constrained by the need to obtain a global ability score, that is, the FSIQ. Fortunately, because local education agencies can no longer require an ability (IQ)–achievement discrepancy for SLD determination (34 Code of Federal Regulations [CFR] §300.307 [a], U.S. Department of Education, 2005), the practice of giving certain tests for the sole purpose of generating an IQ has declined considerably in recent years.

Scenario 3: Consideration of Examinee

Characteristics

The third scenario in which decisions regarding test selection and organization may be highly subject to specific referral concerns involves testing individuals who possess characteristics that set them apart from the mainstream. For example, practitioners are often called on to assess the abilities of individuals who have sensory or perceptual impairments (e.g., deafness, blindness), who have fine-motor impairments (e.g., individuals with cerebral palsy, tremors, seizure activity), or who come from culturally and linguistically diverse backgrounds. Obviously, if an individual is unable to manipulate objects because he or she cannot see or hold them, test selection