ASSETS OF THE KTEA-3

STRENGTHS AND WEAKNESSES OF THE KTEA™-3 AND WIAT®-III 297

word and nonsense word reading, oral and silent reading fluency, phonetic decoding, phonological processing, and rapid naming and a criterion referenced letter naming measure. This is much broader coverage than is found on most broad spectrum achievement tests and many reading tests. Reading comprehension is now measured with two subtests: Reading Comprehension and Reading Vocabulary (which we consider an extremely valuable addition to the KTEA-II selection of subtests). Reading comprehension questions are read by the examinee and themselves pose appropriate reading challenges. In a few cases, the questions appear to be more di cult than the passages (which would be a problem for a test of content knowledge but adds to the depth of measurement of a reading comprehension test). Unlike some other tests (see, for example, Keenan & Betjemann, 2006), very few KTEA-3 reading comprehension questions can be answered correctly on the basis of prior knowledge and deduction without reading the passage. Reading Comprehension also includes some direction-following items, similar to “Turn around in your chair and then stand up.”

Three subtests now measure aspects of written expression. The Written Expression subtest format of having the examinee fill in missing letters, words, punctuation, and sentences in a story book as the examiner tells the story and gives instructions, and then write a summary of the entire story is engaging, and it produces useful data, including the error analysis. There are four di erent Written Expression levels, three of which require separate storybooks. The need to keep three di erent storybooks in stock is a mild but worthwhile inconvenience111111111 . The artwork in the test easels and the storybook is attractive and appears to appeal to the students whom we have tested. The Spelling subtest begins with letteror word-writing items with pictures and continues with words presented in the traditional and e ective format of the examiner saying the word, using it in a prescribed sentence (printed on the record form), and repeating the word. The Writing Fluency subtest (new to the KTEA-3) asks the examinee to write “good, short” sentences about what is happening in each picture in the response booklet. After the sample and teaching items, the examinee is told to “Keep your sentences short and work as fast as you can.” We strongly approve of fluency tests that explicitly tell the examinee to work fast (which is not the case with many tests).

The extensive error analysis procedures do require some time and attention if done by hand, but they provide genuinely useful information if the norms are used. The error categories are detailed and educationally meaningful. Error analysis worksheets are included in the KTEA-3 flash drive. The norms tables for the error analyses (Technical & Interpretive Manual, Appendix H) are clear and helpful. Breaux (2015) provides clear instructions on how to perform error analyses on subtests that use item sets when examinees reverse to a lower level item set. Examiners can use the analyses when necessary and omit them when there are no unanswered questions about remediation, as in the case of a high score or when an examinee’s weakness in a domain is obvious, extreme, and pervasive. Unlike error analyses on many tests, the KTEA-3 weakness-average-strength categories are based on test norms rather than

298 ESSENTIALS OF KTEA™-3 AND WIAT®-III ASSESSMENT

arbitrary cuto s. It is essential that examiners use the norms, which requires some initial bullying of graduate students and supervisees. Raw scores in error categories (like any raw scores) can be very misleading.

During the KTEA-3 standardization, the KTEA-3 and KABC-II (Kaufman & Kaufman, 2004b) were administered to 99 examinees and the KTEA-3 and Di erential Ability Scales (DAS-II; Elliott, 2007) to 122 examinees. Data from those studies were used to create tables of correlations between the several composite ability scores of the KABC-II and DAS-II and all of the KTEA composites and subtests. Those correlations can be used to look up predicted achievement scores and significant di erence and base rate data for predicted and actual achievement. These easily used tables are very valuable. Our only complaints are that there are also tables for simple di erences, which might inspire someone to actually use the simple-di erence method (see McLeod, 1968, 1974), and that similar tables are not provided for the Wechsler Preschool and Primary Scale of Intelligence (4th ed.) (WPPSI-IV; Wechsler, 2012a), Wechsler Intelligence Scale for Children (4th ed.) (WISC-IV; Wechsler, 2003), Wechsler Adult Intelligence Scale (4th ed.) (WAIS-IV; Wechsler, 2008), and other popular tests such as the Woodcock-Johnson IV Tests of Cognitive Abilities (WJ IV COG; Schrank, McGrew, & Mather, 2014a). Those data would be very helpful to examiners who administer the KTEA-3 while another examiner selects and administers a test of cognitive ability. The WISC-V Technical and

Interpretive Manual (Wechsler, 2014b, pp. 96–101) provides WISC-V correlations

111111111

with the KTEA-3.

Two Forms

The availability of two forms (A and B) of the KTEA-3 is extremely helpful in many situations. As discussed below, with very few exceptions, the two forms are statistically equivalent, so examiners bent on retesting students can alternate forms. An alternate form is also extremely useful when there is reason to doubt the validity of scores in a recent assessment.

We do need to o er one caveat. Even with alternate forms, individually administered achievement tests, such as the KTEA-3, WIAT-III, or WJ-IV ACH (Schrank, Mather, & McGrew, 2014) are not well suited for measuring short-term academic progress, especially in the higher grades. Table 5.1 shows the raw scores equivalent to a standard score of 100 by Winter norms for grades 1 through 12 on five of the KTEA-3 subtests without time limits or weighted or transformed scores. In grades 1 through 4, expected annual growth on these five subtests was 4.5 to 11 raw score points. In grades 5 through 8, however, the average student gained only 1.5 to 5.5 raw score points per year. In high school, expected annual growth was only 0 to 3.5 raw score points. With such small anticipated annual gains to maintain a standard score of 100, it could be di cult to distinguish growth or stagnation from random variation in test scores, especially in the higher grades. This is not a flaw of the KTEA-3, which

STRENGTHS AND WEAKNESSES OF THE KTEA™-3 AND WIAT®-III 299

Table 5.1 Raw Scores Corresponding to Standard Scores of 100 by Winter Grade Norms on the KTEA-3 Form A: Five Untimed Subtests Without Weighted or Transformed Scores

Note: From Tables D.2 of the KTEA-3 Technical & Interpretive Manual (Kaufman & Kaufman, 2014b). Raw scores with tenths shown as decimals111111111 were interpolated.

is better than many individually administered achievement tests in this regard, but a warning to be very cautious in using any individually administered test for measuring annual academic progress. Curriculum-based measurement (CBM) can be used as frequently as desired and may be a better way of measuring short-term progress. Group, multiple-choice achievement tests that take hours or days to administer have more items and, therefore, expected growth of more items per year than do individually administered achievement tests. The KTEA-3 does provide examiners with an additional way to track growth across time: Growth Scale Values (GSV). The KTEA-3 GSVs are similar to the GSV scales on the WIAT-III and the Woodcock Reading Mastery Tests-Third Edition (WRMT-III; Woodcock, 2011). The KTEA-3 GSVs are equal-interval scale and they can be used to monitor progress over time.

Standardization

Please see Chapter 2 for detailed information on the standardization of the KTEA-3, which is also described at length in Chapter 1 of the Technical & Interpretive Manual (Kaufman & Kaufman with Breaux, 2014b). The large, national sample of students tested in the fall and in the spring was stratified to match then-current U. S.