Словари и журналы / Психологические журналы / p109Journal of Occupational and Organizational Psychology

109

Journal of Occupational and Organizational Psychology (2002), 75, 109–114

© 2002 The British Psychological Society

www.bps.org.uk

Short research n ote

Exercise order and assessment centre performance

Peter Bycio* and Baniyelme Zoogah

Williams College of Business, Xavier University, USA

Ratings from an operational assessment centre (AC) were examined as a function of the order that candidates participated in the assessments. Assessment scheduling had a signiŽ cant multivariate effect on the preconsensus exercise-based ratings. The effect was small, however, accounting for at most 1% of the rating variance. Differences in exercise order were seen as being unlikely to result in serious unfairness to applicants, although programme designers should be sensitive to the possibility of early vs. late-day performance differences.

The assessment centre (AC) method has been used for many years to help make a variety of employment decisions (Thornton, 1992). Use of the method typically requires that candidates participate in a series of assessments, including job-related situational exercises. Assessor judgments are then made based on their observations from these multiple sources.

As with most tests, it is important that the AC experience be as standardized as possible. However, there are many ways in which non-standardization can creep in to potentially create unfairness (Cohen, 1978). These include design issues such as differences in the order that candidates participate in the assessments and variations in the length and nature of the breaks between assessments.

Cohen and Sands (1978) evaluated the impact of the order in which candidates participated in the exercises. Although they did not find significant differences, at least two aspects of their study may have masked a true effect. First, their order manipulation involved only two exercises in which assessees either completed the In-basket or a Leadership problem first. Also, the sample size was small (N=67) resulting in a relative lack of statistical power. In the present investigation, exercise order was re-examined using a substantially stronger manipulation involving 10 different orders of 9 assessment activities. Moreover, a large sample size was used, which increased statistical power.

Although specific hypotheses were not advanced, given only one previous study in the area, some empirical work supports the notion that exercise order could be a concern. For example, AC participation is anxiety provoking, even for otherwise well-adjusted candidates and these anxiety levels correlate negatively with overall AC performance (Fletcher & Kerslake, 1993). Practice in situational exercises has been proposed as a way to prevent undue anxiety and promote more accurate assessment

*Requests for reprints should be addressed to Peter Bycio, Williams College of Business, Xavier University, 3800 Victory Parkway, Cincinnati, Ohio 45207-5163, USA (e-mail: bycio@xu.edu).

110 Peter Bycio and Baniyelme Zoogah

(Fletcher & Kerslake, 1993). Thus, in terms of exercise order, candidates might benefit from a calming familiarity effect when they perform sequentially in similar assessments relative to those who face frequent changes. For example, since assessee performance has been shown to vary with exercise form (individually oriented role plays vs. group discussions) (see Schneider & Schmitt, 1992) the potential impact of this type of change across orders is worthy of investigation.

Many other kinds of fundamental change are typically embedded within exercise orders, which could differentially impact candidate anxiety and performance. These include whether the assessor is a passive observer or active participant (Zedeck, 1986) and the degree to which the assessment and/or required candidate response is standardized (Thornton, 1992). One might also wonder if performance in a given assessment is enhanced when preceded by a lunch break? The 10 orders examined in this study allowed for an empirical evaluation of the potential impact of many of these variables.

Method

Subjects

Data from a 1-day AC designed to meet the selection and developmental objectives for the position of manufacturing supervisor were studied. A group of 12 candidates per AC were evaluated by five assessors. The sample consisted of ratings of either 1862 or 1914 candidates (depending on the variables involved) who were evaluated during the first 3 years of the programme.

AC activities

There were nine major AC activities, excluding lunch. Five were situational exercises including two group-oriented forms: the Problem Solving Group (PSG) and the Human Relations Group (HRG) and three individually oriented ones: the 24-item In-basket and subsequent Interview (IB-IN), the Role-Play (RP), and the Interview (INTER). See Bycio, Alvares, and Hahn (1987) for further details concerning the exercises.

The four remaining activities included the completion of a Personal Information Questionnaire (QUEST) that was the basis for questions during the INTER. A self-administered (no assessor present) 1-hour Videotape exercise (VIDEO) was used where candidates responded to a questionnaire concerning job-related situations. The responses were available at the consensus meeting, although no performance rating was made. Finally, an AC coordinator administered two paper-and-pencil tests concerning mechanical comprehension (MEC) and numerical (NUM) ability. These results were available at the consensus meeting as well.

Exercise order

Within a given run of the AC, each of the 12 assessees experienced the programme in a different order except for candidates 1 and 7 as well as 4 and 10 for whom the sequence was the same. Potentially important variations with respect to form were present in that some had a series of individually oriented assessments back-to-back. Moreover, some orders had periods free from the direct scrutiny of assessors interspersed between assessor-present activities, whereas others began the day with three consecutive assessor-present exercises. Finally, some candidates completed the paper-and-pencil tests sequentially while others did not. The specifics of the 10 different orders are shown in Table 1.

Assessor ratings

Each of the five assessors saw a given candidate in a different exercise and rated eight managerial abilities along with overall performance in the activity. At a subsequent meeting, assessors

Table 1. Variations in exercise order

Note. PSG=Problem-Solving Group; IB-IN=In-basket and interview; QUEST=Personal Information Questionnaire; HRG=Human Relations Group; MEC=Mechanical Test; NUM=Numerical Test; INTER=Interview; VIDEO=Videotape; RP=Role-Play. An assessor was present for the PSG, IB-IN, HRG, INTER, and the RP.

aTwo of the 12 candidates experienced the program in this order.

order Exercise

111