308 ESSENTIALS OF KTEA™-3 AND WIAT®-III ASSESSMENT

Language Fundamentals–Fourth Edition (Wiig, Semel, & Secord, 2003) Formulated Sentences subtest, which had correlations of .64, .53, and .47 with the KTEA-3 Oral Expression, Written Expression, and Listening Comprehension subtests, respectively. The correlation between the KTEA-3 Oral Language composite and the WIAT-III Oral Language composite was .73 with correlations of .48 between the Listening Comprehension subtests and .56 between the Oral Expression subtests of the KTEA-3 and WIAT-III.

Similarly, the reliabilities are low and the Standard Errors of Measurement high for the age-norm sample for Oral Language (SEm 4.85 to 6.34) and Oral Fluency (SEm 6.99 to 8.81). Therefore, di erences needed for statistical significance between these tests and between these and other tests are large.

Administration and Scoring

Error analysis for Letter & Word Recognition and Nonsense Word Decoding requires phonetic transcription of the examinee’s pronunciations. Unless the examinee reads very rapidly, this is not di cult for examiners with some background in reading instruction or speech pathology, but might be challenging for those lacking this background. One reviewer (RD), for whom this description is valid, has had great di - culty accurately recording the phonemic responses of several students. Thankfully, the Administration Manual (Kaufman & Kaufman with Breaux, 2014c, p. 28) notes, “Recording oral responses verbatim may be111111111 recommended or required on some subtests, depending on the purpose of the evaluation and the depth of information required. Use of an audio recorder during these subtests is highly recommended to ensure accuracy of recording and scoring.” We hope that examiners will record the oral responses even though the need to audio record responses increases the amount of equipment needed for test administration. A pocket-size audio recorder is not much to add to the kit along with a machine to play the audio files.

Weighted raw scores for Reading Comprehension, Written Expression, Listening Comprehension, Word Recognition Fluency, and Oral Expression are taken from conversion tables in Appendix C of the Technical & Interpretive Manual based on the item set(s) administered and the number of items passed within the set(s) and there are also Raw Score Lookup tables in Appendix C for Writing Fluency, Object Naming Fluency, and Letter Naming Fluency. The Subtest Composite Score Computation Form on the KTEA-3 flash drive should make it easy to enter the correct raw scores and weighted raw scores, but examiners new to the concept of item sets sometimes encounter di culties.

We do not find significant di culties with scoring most of the Written Expression and Oral Expression items, but some of our graduate students, workshop attendees, and colleagues report uncertainty in scoring some items, even with the explanations, examples, and Language Glossary in the Scoring Manual (Kaufman & Kaufman with Breaux, 2014d).

STRENGTHS AND WEAKNESSES OF THE KTEA™-3 AND WIAT®-III 309

Test Items

Some potentially ambiguous or confusing items on the KTEA-II were improved in the KTEA-3. However, several of the KTEA-3 Reading Comprehension and Listening Comprehension items list as “incorrect” responses that appear to us to be merely incomplete. Some of these are marked (say, Tell me more), but several are not and querying is permitted only for the responses so marked on the easel. Laconic examinees with adequate reading comprehension might be penalized on those items.

Interpretation

As noted above, under Test Development and in Tables 5.2 and 5.3, examiners must be very cautious when interpreting scores on subtests that limit possible high scores for older examinees and possible low scores for younger ones. Near-perfect raw scores producing only modestly high standard scores and near-zero raw scores with moderately high standard scores must be interpreted with extreme care and may require assessment with another instrument. When an examinee’s skills are extremely weak, criterion referenced assessment may be more useful than norm-based assessment.

If the six Academic Skills Battery (ASB) subtests for older students or the five for kindergarten or three for prekindergarten are not administered, then subtests and other Composites cannot be compared to the ASB. However, subtests and composites can still be statistically compared to each other111111111 .

Appendix G in the Technical & Interpretive Manual lists di erences needed for statistical significance and base rates for di erences between Composites (Tables G.3 and G.7 for grade and age, respectively) and between subtests (Tables G.4 and G.8), which is essential information. However, in Tables G.4 and G.8, it would have been helpful to highlight the two subtests that make up each composite. Most examiners want to know whether the components of a total score are su ciently consistent with one another to permit confident interpretation of that total score. This is another instance in which examiners may want to print out and store copies of tables, which they could highlight manually. We find it helpful to use the blank space in the “Composite” columns on the Subtest & Composite Score Computation Form (which must be printed from the KTEA-3 flash drive for each examinee) to note whether the di erence or di erences between the two or three subtests in each composite are significant and unusual (base rate). We write either “p < .05” or “n.s.” and either “f ≤ 10%” or “f ≥ 10%” in the blank space for each composite.

As noted above, the comparison between the Reading Comprehension and Listening Comprehension subtests is extremely valuable diagnostically because the formats of the two subtests are so similar and the correlation between them is fairly high (e.g., .61 in grade 5), so a di erence of 22 points has a base rate ≤10% in grades 1 through 6. The comparison between Written Expression and Oral Expression is somewhat less helpful because of substantial di erences in test format and a lower