Appendix A: developing proficiency descriptors

This appendix discusses technical aspects of describing levels of language attainment. Criteria for descriptor formulation are discussed. Methodologies for scale development are then listed, and an annotated bibliography is provided.

Descriptor formulation

Experience of scaling in language testing, the theory of scaling in the wider field of applied psychology, and preferences of teachers when involved in consultation processes (e.g. UK graded objectives schemes, Swiss project) suggest the following set of guidelines for developing descriptors:

•Positiveness: It is a common characteristic of assessor-orientated proficiency scales and of examination rating scales for the formulation of entries at lower levels to be negatively worded. It is more difficult to formulate proficiency at low levels in terms of what the learner can do rather than in terms of what they can’t do. But if levels of proficiency are to serve as objectives rather than just as an instrument for screening candidates, then positive formulation is desirable. It is sometimes possible to formulate the same point either positively or negatively, e.g. in relation to range of language (see Table A1).

An added complication in avoiding negative formulation is that there are some features of communicative language proficiency which are not additive. The less there is the better. The most obvious example is what is sometimes called Independence, the extent to which the learner is dependent on (a) speech adjustment on the part of the interlocutor (b) the chance to ask for clarification and (c) the chance to get help with formulating what he/she wants to say. Often these points can be dealt with in provisos attached to positively worded descriptors, for example:

Can generally understand clear, standard speech on familiar matters directed at him/her, provided he/she can ask for repetition or reformulation from time to time.

Can understand what is said clearly, slowly and directly to him/her in simple everyday conversation; can be made to understand, if the speaker can take the trouble.

or:

Can interact with reasonable ease in structured situations and short conversations, provided the other person helps if necessary.

205

Appendix A: developing proficiency descriptors

Table A1. Assessment: positive and negative criteria

•Definiteness: Descriptors should describe concrete tasks and/or concrete degrees of skill in performing tasks. There are two points here. Firstly, the descriptor should avoid vagueness, like, for example ‘Can use a range of appropriate strategies’. What is meant by strategy? Appropriate to what? How should we interpret ‘range’? The problem with vague descriptors is that they can read quite nicely, but an apparent ease of acceptance can mask the fact that everyone is interpreting them differently. Secondly, since the 1940s, it has been a principle that distinctions between steps on a scale should not be dependent on replacing a qualifier like ‘some’ or ‘a few’ with ‘many’ or ‘most’ or by replacing ‘fairly broad’ with ‘very broad’ or ‘moderate’ with ‘good’ at the next level up. Distinctions should be real, not word-processed and this may mean gaps where meaningful, concrete distinctions cannot be made.

•Clarity: Descriptors should be transparent, not jargon-ridden. Apart from the barrier to understanding, it is sometimes the case that when jargon is stripped away, an apparently impressive descriptor can turn out to be saying very little. Secondly, they should be written in simple syntax with an explicit, logical structure.

•Brevity: One school of thought is associated with holistic scales, particularly those used in America and Australia. These try to produce a lengthy paragraph which

206