382 ESSENTIALS OF KTEA™-3 AND WIAT®-III ASSESSMENT

(in item type, stimulus presentation, response format, etc.) to one already tested in an equivalency study, then an additional study is not required because the prior research findings are assumed to generalize to similar tests. The primary goal of the equivalency studies has been to maintain raw score equivalence between paper and digital administration and scoring formats. The standard criterion that has been used to ensure equivalence in all studies is an e ect size of 0.2 or smaller, which is equal to three standard score points or fewer on the KTEA-3 and WIAT-III. E ect size is the average amount of di erence between scores on Q-interactive and paper administrations, divided by the standard deviation of scores. Equivalency of raw scores indicates that the norms, reliability, and validity information obtained using the paper and pencil tests can be applied to Q-interactive results.

For the KTEA-3 and WIAT-III, equivalency study results suggest that the scores obtained from the digitally adapted administrations are comparable to those obtained from traditional paper based administrations. No equivalency studies were conducted specifically for the KTEA-3 because all subtests were similar (in item type, stimulus presentation, response format, etc.) to subtests already included in previous equivalency studies. However, a technical report specific to the WIAT-III (available at www.helloq.com/research) outlines a study conducted for Oral Reading Fluency and the Sentence Repetition component of Oral Expression. The results indicated nonsignificant score di erences with e ect sizes of –0.05 to 0.11 for Oral Reading

Fluency (Rate, Accuracy, Fluency scores) and –0.08 for Sentence Repetition. In

111111111

addition, a preliminary readability study was conducted to ensure that readability was not a ected by tablet presentation of text. For Word Reading and Pseudoword Decoding, the text size of the words/pseudowords were slightly reduced in size in order for the stimuli to fit on the tablet screen. For Oral Reading Fluency, no text size adjustments were needed because each reading passage in the printed stimulus booklet fit on the tablet screen without the need for scrolling. The results of this readability study indicated nonsignificant score di erences with e ect sizes of 0.18 for Word Reading/Pseudoword Decoding and –0.11 for Oral Reading Fluency. In summary, none of the score di erences were statistically significant at the .05 level, and all e ect sizes were 0.11 or smaller, well within the tolerance limits for equivalency.

A variety of clinical assessments also yielded comparable scores in the Q-interactive and paper administration formats, including the WAIS-IV, WISC-IV, NEPSY-II, WMS, and CELF-5.

ESSENTIAL FEATURES OF Q-INTERACTIVE

This section reviews the key terminology used in Q-interactive, and then describes the essential features of Q-interactive’s two main components: the Central website and the Assess application.

Q-INTERACTIVE FOR KTEA™-3 AND WIAT®-III 383

Key Terminology

Understanding the key terminology that is used throughout Q-interactive, as explained in Rapid Reference 7.1, is helpful for new users. Some of the terms will be unfamiliar to new users because they are specific to the technology (e.g., Assess, Central). Other terms may di er from what is typically used in the KTEA-3 and WIAT-III because they are intended to be generic enough to apply across a diverse array of clinical assessments (e.g., pertaining to mental health, personality, behavior, neuropsychology, etc.); for example, the term client is used instead of student or examinee.

Rapid Reference 7.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ............. . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Key Terminology in Q-interactive

00

Q-i: Q-interactive is also referred to as “Q-i” for short.

Applications

Assess: The iPad application through which examiners can administer and score assessments

Central: The website (www.qiactive.com)22222222111111111 through which examiners create clients, assign assessments, and create reports for a specific assessment

Devices (iPads)

Client Device: The iPad used to present stimuli to the examinee

Examiner Device (also referred to as Practitioner Device in the Assess settings): The iPad used by the examiner to administer and score

Features and Actions

Home Screen: The entry point on Assess, where all assessments currently on the iPad can be accessed

Assessment: A battery of subtests assigned to a particular examinee for a particular testing event

Card: All of the Assess content that appears within one frame on the iPad without having to swipe forward or backward in the app (e.g., most subtests have one card for each item, while all items of the Spelling subtests are presented on a single card)

Contextual Events: Common events that examiners often want to track, such as the number of times a student says, “I don’t know,” does not respond, or requires repetition of an item prompt

Pop-over: A window that opens to display new information within Assess Swipe: To move from one card to the next, swipe your finger horizontally across

the screen

Q-INTERACTIVE FOR KTEA™-3 AND WIAT®-III 385

Assess was designed to use similar design features and functionality across tests to make it easier for practitioners to learn and transition between tests. These basic features, found on the home screen and within the assessment screens, will be reviewed next.

Home Screen: Interview, Battery, and Results Tabs

On the home screen, information is organized under three tabs: Interview, Battery, and Results. The Interview tab shows basic demographic information (birth date, gender) for the examinee as well as the date and time of the assessment. This tab also provides the option to write Notes relevant to the assessment (this feature can be used before or after the assessment and all notes will sync back to Central) or remove the assessment from the iPad.

22222222111111111

The Battery tab lists the subtests in the current assessment battery. Subtests already administered are grayed out, and the current subtest has a play button arrow. Tapping on the current subtest will take you to the last viewed screen in the assessment.

Q-INTERACTIVE FOR KTEA™-3 AND WIAT®-III 387

Global Bar

After beginning an assessment in Assess and leaving the home screen, a black bar with icons (referred to as the global bar) will remain at the top of each item or instruction screen. The global bar includes seven icons that provide features common to all subtests (see Rapid Reference 7.2 for a list of features). These features are meant for use while administering an item, so the global bar features are not available at other times (e.g., while reviewing an item already administered or previewing an item not yet administered).

Notes may be written by the examiner using a finger or stylus at any point before, during, or after an assessment session (see Notes button on Interview tab of Assess home screen to access Notes before or after an assessment session). During an assessment, in the upper right-hand corner of every screen is a pencil icon, which opens up an area for writing notes about the current item/subtest or the assessment as a whole. A drop-down menu in the top left allows you to switch between notes. This large recording area is the size of the iPad screen, so item responses that would require more than a few lines of writing or which would benefit from the large area, such as a drawn image, should be recorded here. As with verbatim responses, you can start another page of notes if you need more writing space, by tapping the pencil icon with a plus symbol. All notes are available for later review on Assess by touching the Notes icon and navigating to the appropriate note, by tapping the “All Notes” button under the Results tab of your assessment, or on111111111the Central Website by clicking on Notes when viewing the assessment overview.

Rapid Reference 7.2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ............. . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Global Bar Features

The global bar includes seven icons. These icons and their features are explained here in the order of their appearance in the figure (left to right).

Home (Q) Takes you back to the Assess home screen, where you can switch between assessments, edit your battery, or view general information and results for an assessment.

Review pane (or “All to Do”) This drop-down lists all subtests and all items in the battery, allowing you to navigate between items and subtests for review. A red circle counter will display how many items in the battery need scoring or review (“All To Do”), and each of these items is flagged. These “to do” items must be resolved before an assessment can be considered complete and ready for reporting.

(continued)

Q-INTERACTIVE FOR KTEA™-3 AND WIAT®-III 389

(Continued)

Manual discontinue (Hand icon) Ends administration of the subtest or section. Once manually discontinued, a subtest cannot be resumed. Use with caution: If the discontinue point has not been reached, the subtest will be invalidated.

Notes (Pencil) Opens up a window in which to take handwritten notes about the battery as a whole, the subtest, or the current item. You may navigate between these three options using a drop-down menu. The note may be flagged for later review by clicking the exclamation point. Notes are stored on Assess and Central under the specific assessment for which they were taken.

Picture-in-Picture (Double screen) This pop-over displays a smaller version of what is being displayed on the client device. Only select subtests currently offer this feature.

About (?) This pop-over displays the current version of the App software, which is useful to note if you have any technical difficulties.

Observational Checklist (KTEA-3 only) The KTEA-3 includes an additional clipboard icon in the Global Bar (before the Notes pencil) for responding to a qualitative

22222222111111111

observations survey regarding general and subtest-specific behaviors. Based on the examiner’s qualitative observations, a table will be included in the score report to suggest possible areas of cognitive processing weaknesses to investigate further.

Beginning of Subtest Coverpage

A coverpage appears at the beginning of each subtest or subtest section. This coverpage informs the examiner that a transition will be made. If you decide not to proceed with the subtest or section, you may go back to the Assess home page to modify the battery but do not swipe past the coverpage. Once you swipe past the coverpage, the subtest becomes billable (the usage will be reflected in your account) and the subtest can no longer be edited out of the battery.

Item Card

Subtests are administered using item cards, which may include one or more of the following features. Images showing examples of each feature are included as well.

•Administration prompts: text that is read aloud to the examinee is presented in a teal-colored font

•A button to present the item stimulus (audio or visual)

•Scoring options

•Response capture

Q-INTERACTIVE FOR KTEA™-3 AND WIAT®-III 391

The item bar, found just below the global bar, appears on each Item Card when administering items in Assess. The item bar indicates the item(s) currently being administered. Rapid Reference 7.3 lists the other primary features of the item bar.

Rapid Reference 7.3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ............. . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Item Bar Features The item bar includes three primary features/icons. These features are explained here in the order of their appearance in the figure (left to right).

Special instructions (Speech Bubble) This pop-over contains instructions and prompts for addressing common “If-Then” scenarios during administration of the subtest (e.g., If the examinee does X, then do/say Y).

22222222111111111

Score Displays the current raw score for any item(s) on the current card. Clicking on it will typically display a pop-over that includes more information, such as the total number of correct items and incorrect items on a card that has multiple items.

Flag for Review (Flag) Clicking the flag will change the flag’s color from white to red, indicating that the item needs review (even if it is scored). Flagged items are easily accessed later in the Review sidebar and from the home screen’s “Results” tab.

End of Subtest Card

After an examinee discontinues or completes a subtest, the End of Subtest Card displays the following data:

•Standard scores

•Raw scores (total, item-level, and weighted, if any)

•Administration time

•Number of errors by error/skill category

392 ESSENTIALS OF KTEA™-3 AND WIAT®-III ASSESSMENT

Learn More

. . . ............. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ............ . . . . . . .

In order to keep this Essentials book within its page limits, only a preview of this Q-interactive chapter is provided in print. Chapter 7 may be viewed in its entirety by accessing the content online in the book’s Digital Resources.

TEST YOURSELF

.. . ............. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ............ . . . . . . .

1.Tablet-based assessments can be conducted on both Q-interactive and Q-global.

(a)True

(b)False

2.Equivalency study results suggest that the raw scores obtained from the digitally adapted administrations of the KTEA-3 and WIAT-III on Q-interactive are comparable to those obtained from traditional paper-based administrations.

(a)True

3.After an examinee discontinues or completes a subtest, the End of Subtest Card displays the following data:

(a)Standard Scores

(b)Raw Scores

(c)Administration time

(d)All of the above

4.The digital administration process on Q-interactive makes examiner errors nearly impossible to make, so practicing the test administration is not necessary even for first-time users.

(a)True

(b)False

5.Which of the following statements about modifying an assessment battery (reordering, removing, or adding subtests) is false?

(a)Before administration of the assessment begins, a battery can be modified in Central or in Assess.

(b)During administration in Assess, the examiner can modify subtests that have not yet started (i.e., progressed past the Beginning of Subtest Card).

(c)During administration in Assess, all subtests can be modified.

(d)After administering the complete battery, additional subtests can be added in Assess.

Q-INTERACTIVE FOR KTEA™-3 AND WIAT®-III 393

6.Benefits of conducting assessments on Q-interactive’s Assess application include which of the following?

(a)Automatic timing of timed subtests

(b)Audio recording of subtests that have oral responses

(c)Picklists for subtests that have common correct and incorrect responses

(d)All of the above

7.Even using Q-interactive, paper response booklets are required for administering subtests with written responses.

(a)True

(b)False

8.The Global bar in the Assess application provides access to all of the following features except:

(a)Review pane or “All to Do” items

(b)Subtest information (i-button)

(c)Notes

(d)Manual discontinue

(e)Audio recordings

9.Which two of the following types of data are not saved when the assessment is synced back to the Central website?

(a)Audio recordings

22222222111111111

(b) Verbatim capture entries

(c) Notes

(d) Examinee’s touch responses

(e) Picklist responses

10. Which of the following statements about Q-interactive is false?

(a) Assess is an iPad app for administration and scoring

(b) Score reports are generated using the Assess application.

(c) An assessment is initially created on the Central website.

(d) Assessment data collected in Assess is synced (saved) to the Central website.

Answers: 1. b; 2. a, 3. d; 4. b; 5. c; 6. d; 7. True; 8. e; 9. a, d; 10. b.