Assessment

The User Guide for Examiners produced by ALTE (Document CC-Lang (96) 10 rev) provides detailed advice on operationalising constructs in tests, and avoiding unnecessary distortion though test method effects.

9.3Types of assessment

A number of important distinctions can be made in relation to assessment. The following list is by no means exhaustive. There is no significance to whether one term in the distinction is placed on the left or on the right.

Table 7. Types of assessment

9.3.1 Achievement assessment/proficiency assessment

Achievement assessment is the assessment of the achievement of specific objectives – assessment of what has been taught. It therefore relates to the week’s/term’s work, the course book, the syllabus. Achievement assessment is oriented to the course. It represents an internal perspective.

Proficiency assessment on the other hand is assessment of what someone can do/knows in relation to the application of the subject in the real world. It represents an external perspective.

Teachers have a natural tendency to be more interested in achievement assessment in order to get feedback for teaching. Employers, educational administrators and adult learners tend to be more interested in proficiency assessment: assessment of outcomes, what the person can now do. The advantage of an achievement approach is that it is close

183

Common European Framework of Reference for Languages: learning, teaching, assessment

to the learner’s experience. The advantage of a proficiency approach is that it helps everyone to see where they stand; results are transparent.

In communicative testing in a needs-oriented teaching and learning context one can argue that the distinction between achievement (oriented to the content of the course) and proficiency (oriented to the continuum of real world ability) should ideally be small. To the extent that an achievement assessment tests practical language use in relevant situations and aims to offer a balanced picture of emerging competence, it has a proficiency angle. To the extent that a proficiency assessment consists of language and communicative tasks based on a transparent relevant syllabus, giving the learner the opportunity to show what they have achieved, that test has an achievement element.

The scales of illustrative descriptors relate to proficiency assessment: the continuum of real world ability. The importance of achievement testing as a reinforcement to learning is discussed in Chapter 6.

9.3.2 Norm-referencing (NR)/criterion-referencing (CR)

Norm-referencing is the placement of learners in rank order, their assessment and ranking in relation to their peers.

Criterion-referencing is a reaction against norm-referencing in which the learner is assessed purely in terms of his/her ability in the subject, irrespective of the ability of his/her peers.

Norm-referencing can be undertaken in relation to the class (you are 18th) or the demographic cohort (you are 21,567th; you are in the top 14%) or the group of learners taking a test. In the latter case, raw test scores may be adjusted to give a ‘fair’ result by plotting the distribution curve of the test results onto the curve from previous years in order to maintain a standard and ensure that the same percentage of learners are given ‘A’ grades every year, irrespective of the difficulty of the test or the ability of the pupils. A common use of norm-referenced assessment is in placement tests to form classes.

Criterion-referencing implies the mapping of the continuum of proficiency (vertical) and range of relevant domains (horizontal) so that individual results on a test can be situated in relation to the total criterion space. This involves (a) the definition of the relevant domain(s) covered by the particular test/module, and (b) the identification of ‘cut-off points’: the score(s) on the test deemed necessary to meet the proficiency standard set.

The scales of illustrative descriptors are made up of criterion statements for categories in the descriptive scheme. The Common Reference Levels present a set of common standards.

9.3.3 Mastery CR/continuum CR

The mastery criterion-referencing approach is one in which a single ‘minimum competence standard’ or ‘cut-off point’ is set to divide learners into ‘masters’ and ‘non-masters’, with no degrees of quality in the achievement of the objective being recognised.

The continuum criterion-referencing approach is an approach in which an individual ability is referenced to a defined continuum of all relevant degrees of ability in the area in question.

184

Assessment

There are in fact many approaches to CR, but most of them can be identified as primarily a ‘mastery learning’ or ‘continuum’ interpretation. Much confusion is caused by the misidentification of criterion-referencing exclusively with the mastery approach. The mastery approach is an achievement approach related to the content of the course/module. It puts less emphasis on situating that module (and so achievement in it) on the continuum of proficiency.

The alternative to the mastery approach is to reference results from each test to the relevant continuum of proficiency, usually with a series of grades. In this approach, that continuum is the ‘criterion’, the external reality which ensures that the test results mean something. Referencing to this external criterion can be undertaken with a scalar analysis (e.g. Rasch model) to relate results from all the tests to each other and so report results directly onto a common scale.

The Framework can be exploited with mastery or continuum approach. The scale of levels used in a continuum approach can be matched to the Common Reference Levels; the objective to be mastered in a mastery approach can be mapped onto the conceptual grid of categories and levels offered by the Framework.

9.3.4 Continuous assessment/fixed point assessment

Continuous assessment is assessment by the teacher and possibly by the learner of class performances, pieces of work and projects throughout the course. The final grade thus reflects the whole course/year/semester.

Fixed point assessment is when grades are awarded and decisions made on the basis of an examination or other assessment which takes place on a particular day, usually the end of the course or before the beginning of a course. What has happened beforehand is irrelevant; it is what the person can do now that is decisive.

Assessment is often seen as something outside the course which takes place at fixed points in order to make decisions. Continuous assessment implies assessment which is integrated into the course and which contributes in some cumulative way to the assessment at the end of the course. Apart from marking homework and occasional or regular short achievement tests to reinforce learning, continuous assessment may take the form of checklists/grids completed by teachers and/or learners, assessment in a series of focused tasks, formal assessment of coursework, and/or the establishment of a portfolio of samples of work, possibly in differing stages of drafting, and/or at different stages in the course.

Both approaches have advantages and disadvantages. Fixed point assessment assures that people can still do things that might have been on the syllabus two years ago. But it leads to examination traumas and favours certain types of learners. Continuous assessment allows more account to be taken of creativity and different strengths, but is very much dependent on the teacher’s capacity to be objective. It can, if taken to an extreme, turn life into one long never-ending test for the learner and a bureaucratic nightmare for the teacher.

Checklists of criterion statements describing ability with regard to communicative activities (Chapter 4) can be useful for continuous assessment. Rating scales developed in relation to the descriptors for aspects of competence (Chapter 5) can be used to award grades in fixed point assessment.

185

Common European Framework of Reference for Languages: learning, teaching, assessment

9.3.5 Formative assessment/summative assessment

Formative assessment is an ongoing process of gathering information on the extent of learning, on strengths and weaknesses, which the teacher can feed back into their course planning and the actual feedback they give learners. Formative assessment is often used in a very broad sense so as to include non-quantifiable information from questionnaires and consultations.

Summative assessment sums up attainment at the end of the course with a grade. It is not necessarily proficiency assessment. Indeed a lot of summative assessment is normreferenced, fixed-point, achievement assessment.

The strength of formative assessment is that it aims to improve learning. The weakness of formative assessment is inherent in the metaphor of feedback. Feedback only works if the recipient is in a position (a) to notice, i.e. is attentive, motivated and familiar with the form in which the information is coming, (b) to receive, i.e. is not swamped with information, has a way of recording, organising and personalising it; (c) to interpret, i.e. has sufficient pre-knowledge and awareness to understand the point at issue, and not to take counterproductive action and (d) to integrate the information, i.e. has the time, orientation and relevant resources to reflect on, integrate and so remember the new information. This implies self-direction, which implies training towards self-direction, monitoring one’s own learning, and developing ways of acting on feedback.

Such learner training or awareness raising has been called évaluation formatrice. A variety of techniques may be used for this awareness training. A basic principle is to compare impression (e.g. what you say you can do on a checklist) with the reality, (e.g. actually listening to material of the type mentioned in the checklist and seeing if you do understand it). DIALANG relates self-assessment to test performance in this way. Another important technique is discussing samples of work – both neutral examples and samples from learners and encouraging them to develop a personalised metalanguage on aspects of quality. They can then use this metalanguage to monitor their work for strengths and weaknesses and to formulate a self-directed learning contract.

Most formative or diagnostic assessment operates at a very detailed level of the particular language points or skills recently taught or soon to be covered. For diagnostic assessment the lists of exponents given in section 5.2 are still too generalised to be of practical use; one would need to refer to the particular specification which was relevant (Waystage, Threshold, etc.). Grids consisting of descriptors defining different aspects of competence at different levels (Chapter 4) can, however, be useful to give formative feedback from a speaking assessment.

The Common Reference Levels would appear to be most relevant to summative assessment. However, as the DIALANG Project demonstrates, feedback from even a summative assessment can be diagnostic and so formative.

9.3.6 Direct assessment/indirect assessment

Direct assessment is assessing what the candidate is actually doing. For example, a small group are discussing something, the assessor observes, compares with a criteria grid, matches the performances to the most appropriate categories on the grid, and gives an assessment.

186