Assessing end-of-school attainment

Is there a 'best' way to establish the levels of knowledge, understanding and skill that students have attained by the end of Year 12? This is a question Dr Gabrielle Matters and I considered in our recent review of senior assessment and tertiary entrance processes for the Queensland Government.

Some people we consulted clearly believe there is. For some, the answer is a traditional, externally developed and marked three-hour written examination. For others, the answer is at the opposite end of the spectrum – teacher devised and evaluated assessment activities only. We concluded that, in the current Queensland context, neither of these extremes is adequate.

For the purposes of certifying attainment in a subject, assessment processes must satisfy a number of requirements. They must provide dependable, high-quality evidence of different kinds of learning that can be compared directly across teachers and schools; the process for arriving at students' subject results must be straightforward and transparent; and results must be reported on a scale that is fine enough to distinguish meaningful differences in student attainment.

We began our review by recognising that different assessment methods are appropriate for different kinds of learning. For example, students' abilities to design an investigation of a topic and to collect, analyse and interpret relevant information generally are best assessed over a period of time through direct observations of their work. Classroom teachers are in the best position to evaluate such evidence. On the other hand, students' factual and procedural knowledge in a subject usually can be assessed fairly and accurately by administering a well-designed test.

So, rather than beginning with a preference for a particular approach to assessment, we began by recognising that subject syllabuses identify different kinds of learning which are best assessed in different ways. In all subjects, there is a need for teacher observations and judgements. But in most if not all subjects, a significant proportion of what students are expected to learn can be assessed efficiently and reliably with a common externally developed test.

We also recognised that a feature of current processes in Queensland is the central role that teachers play in assessing their own students' performances. In important ways, our recommendations in Redesigning the secondary–tertiary interface maintain and strengthen this role. To appreciate how they do this, an understanding of common senior assessment processes is required.

In many parts of Australia, teachers' assessments in each subject in each school are scaled statistically to that group's distribution of results on an external examination. Each student's scaled school assessment is then added to their exam score to obtain their subject result, which is then used in the calculation of their tertiary admissions rank (ATAR). The scaling ensures that, as a group, students do not receive teacher assessments better or worse than the group's performance on the exam – thus giving the exam (and the narrower set of syllabus objectives it generally assesses) precedence over teachers' assessments.

In Queensland, teachers currently report performances in senior subjects using five levels of achievement. However, for students seeking a tertiary admissions rank (OP), teachers also provide finer-grained indicators of achievement in a subject. In each subject in each school, these indicators are scaled statistically to that group's distribution of results on the Queensland Core Skills (QCS) Test prior to being used to construct OPs. This scaling ensures that the group's achievement indicators in a subject are no better or worse than the group's performance on the QCS Test.

In contrast, our recommendations are designed to support teachers to make reliable assessments of particular aspects of learning and achievement that can be compared across schools and used directly in the calculation of students' subject results and thus tertiary admissions ranks – without prior scaling against an external test or examination.

A first requirement is the use of more tightly specified assessment activities in each subject for the purposes of certifying student attainment. We envisage these activities sitting alongside and complementing the day-to-day assessments that teachers make to inform classroom decision making and to provide ongoing feedback to students and parents. For most subjects, four kinds of activities appear to provide adequate coverage of the syllabus objectives. We recommend that three of these activities be designed and marked by teachers. However, if students' results are to be compared directly across schools, then the broad kinds of assessment activities that teachers use, the conditions under which students complete these, and the marking schemes for judging and recording performances on these activities will need to be consistent across schools. To meet this requirement, we recommended that the Queensland Curriculum and Assessment Authority (QCAA) develop broad specifications and marking schemes for the three school assessments and also set and mark an external assessment in each subject.

A second requirement is a redesigned moderation process. Under our recommendations, this will include checking and endorsing teachers' proposed assessment activities prior to use. Checks will consider consistency with QCAA specifications and the level of demand or challenge of teachers' proposed assessments. Moderation also will include post-assessment confirmation that teachers have applied the relevant marking scheme accurately and consistently. We envisage moderation meetings in which teachers from different schools meet to undertake ‘blind' re-marking of samples of student work on each of the three school assessment activities. If a school is judged to have been too generous or too harsh in its marking, then it will be required to re-mark all student work on that activity.

Finally, it is essential that the process for arriving at a student's subject result is straightforward and transparent. We propose that, in each subject, each of the three school assessments be marked out of 10 by teachers using a marking scheme provided by QCAA, and the external assessment activity be marked out of 30. These marks will then be added to obtain a subject result out of 60. Only this total mark will be reported on a student's senior certificate, with Queensland universities then using these subject results in their calculation of tertiary admissions ranks.

This is an ambitious proposal. It places a higher level of trust and confidence in teachers' judgements and requires the redesign and strengthening of Queensland's system of school-based assessment.