Shifting the focus of NAPLAN
The decision to move NAPLAN online provides an opportunity to place less emphasis on comparing the performances of schools and more emphasis on supporting student learning.
An intriguing discrepancy has emerged between the performances of Australian secondary students in the National Assessment Program – Literacy and Numeracy (NAPLAN) and their performances in the OECD's Programme for International Student Assessment (PISA). Since 2008, the average performance of Year 9 students on NAPLAN has remained relatively stable. But over the same period, the performances of 15 year olds in PISA have declined steadily. What are we to make of this?
One suggested explanation is that schools are more focused on improving their NAPLAN results than they are on PISA. According to this explanation, average scores on NAPLAN are being maintained because schools are responding to the publication and comparison of their NAPLAN results on the My School website, with PISA providing the more accurate picture of (downward) trends in literacy and numeracy levels.
This explanation has some support in international experience. Some states in the United States have seen improvements on state-wide tests that are used to hold schools accountable, with no parallel improvement in those states' performances on the sample-based, low-stakes National Assessment of Educational Progress (NAEP). The conclusion is that improvements on the state-based tests reflect ‘score inflation', with NAEP providing the more accurate indication of trends over time.
A second explanation is that NAPLAN and PISA assess different skills. NAPLAN assesses basic literacy and numeracy skills and PISA assesses students' abilities to apply these skills to complex, real world problems. Under this explanation, for whatever reason, the basic skill levels of Australian secondary students are being maintained over time, but their abilities to apply these skills are in decline.
A third possibility is that both these explanations are correct: because schools are placing greater emphasis on the basic skills on which they are being publicly compared, performances on these skills are being maintained at the expense of the higher level skills assessed by PISA. If this is true, we may face twin challenges: significantly improving students' literacy and numeracy skills and placing less emphasis on comparing schools and more emphasis on improving learning.
The decision to move NAPLAN online may assist in addressing the first of these challenges. The introduction of computer adaptive tests that are better targeted on students' current skill levels should provide better information about where individuals are in their long-term literacy and numeracy development. This is desirable because it is now clear that students in the same year of school are widely dispersed in their skill levels. Some Year 9 students perform at the same level as some Year 5, and possibly some Year 3, students. Computer adaptive testing recognises that learning occurs on a continuum and so provides better information about where individuals are in their learning and for setting challenging personalised targets for further growth.
However, online computer adaptive testing is only a first step. A longer-term vision would uncouple NAPLAN tests and NAPLAN reporting from year levels entirely. In other words, there would be no such thing as a ‘Year 3', ‘Year 5', ‘Year 7' or ‘Year 9' test. Instead, a student's performance on NAPLAN tasks would result in a NAPLAN score as at present, together with a conclusion about the absolute proficiency level that the student had reached, regardless of their age or year level – for example, ‘achieved Reading Band 8 and working towards Band 9'. By foregrounding the NAPLAN score scale and proficiency bands, NAPLAN would model and promote a growth mindset in assessment, an approach that follows naturally from recognition that learning occurs on a continuum and that a single year level test is inappropriate for most students. It would then be unnecessary to restrict NAPLAN testing to particular years of school.
An advantage of this approach is that it would provide a framework of proficiency levels against which all students could challenge themselves and monitor their progress, a little like the grades against which students monitor their progress in music. There would be explicit recognition that students in the same year of school are at different points in their learning and may be progressing at different rates. Individual students would attempt online assessments when they felt ready to demonstrate achievement of the next proficiency level, rather than in specified assessment periods. In this way, students may be encouraged to aim for progressively higher levels of literacy and numeracy and to take greater responsibility for monitoring their own progress. In contrast, national minimum standards tied to year levels do little to challenge all students to higher levels of achievement.
The monitoring of literacy and numeracy achievement against a set of absolute proficiency levels would require a shift in thinking on the part of students, teachers and parents who are used to interpreting test performances only in terms of year level expectations. Because the performances of all students would continue to be reported on the current NAPLAN score scale, it would still be possible to calculate year level averages, to show how students perform in relation to their year group, and to identify levels of proficiency that, ideally, all students should reach by particular times in their schooling. However, the primary focus in reporting NAPLAN results would be on the proficiency level an individual had reached and the progress they had made over time.
By shifting the focus of NAPLAN in this way, it also may be possible to downplay school comparisons based on year-level means (a statistic that is strongly correlated with students' socioeconomic backgrounds) and to make greater use of NAPLAN's ability to compare schools based on the value they add, reflected in the progress students make. Although, in general, encouraging comparisons and competition between schools is an ineffective route to better national performance.
The decision to move NAPLAN online provides a unique opportunity to shift the focus of assessment from common year-level tests and low national minimum standards to the monitoring of each student's progress against challenging personal targets. It also introduces an opportunity to place less emphasis on comparing schools and more emphasis on supporting learning. The reversal of Australia's long-term decline in PISA may depend on seizing these opportunities.
