What’s happening with PIRLS?
In December this year, the results from the most recent administration of the Progress in International Reading Literacy Study (PIRLS) will be released – in a different way than ever before due to interruption of the cycle because of the COVID-19 pandemic.
PIRLS is an international comparative study of student achievement at Year 4. It is directed by the International Association for the Evaluation of Educational Achievement (IEA) and is managed in Australia by ACER. PIRLS 2021 is the fifth such study since PIRLS was first conducted in 2001, and Australia has participated in the 3 most recent cycles – PIRLS 2011, 2016 and 2021.
As a Southern Hemisphere participant, Australia usually conducts the field trial and main data collection for PIRLS in the year preceding Northern Hemisphere participants, e.g., in 2020 for the PIRLS 2021 cycle. However, many Australian schools, particularly those in Victoria, had periods of extended school closures in 2020, which contributed to the decision to delay Australia’s main data collection for PIRLS until 2021. In Australia, testing was still carried out in the latter part of the school year, as has been the usual practice, just 12 months later than orginally planned and with a different cohort of students.
For some countries, however, testing was carried out with the students originally sampled in their Year 4 classes at the beginning of the following school year, when they had already started Year 5. Because of this, there is some doubt about whether results from these groups of students will be comparable. In addition, not all participating countries’ results will be published in December due to further delays to their testing and later delivery of their data.
What is the goal of PIRLS?
PIRLS aims to provide the best policy-relevant information about how to improve teaching and learning and to help young students become accomplished and self-sufficient readers, by assisting countries to monitor and evaluate their teaching of reading across time.
Students in the fourth year of schooling typically have gained most of their reading skills in a multitude of environments – at school and at home; in different classrooms with different teachers. In order to reflect this situation, PIRLS collects a rich array of background data from students, schools and teachers, and also collects data about the education systems themselves.
It’s important to identify those children who are most at-risk of not being able to read well at this critical stage of their education. At Year 4, students are moving from the stage of learning to read, to the stage of reading to learn. If a student is not able to glean what is required from a text, and more and more of their learning is text-based, then they are unlikely to be able to keep up with what they are supposed to be learning, regardless of the subject.
What does PIRLS measure?
The reading assessment in PIRLS is organised around two dimensions:
- the purposes for reading
- the processes of comprehension that readers use in understanding the texts and their related questions.
The 2 purposes for reading identified in PIRLS are reading for literary experience (narrative fiction) and reading to acquire and use information (non-fiction). These 2 purposes account for most of the reading activities done by young students in and out of school.
Within each of these 2 major reading purposes, 4 processes of comprehension are also assessed:
- focusing on and retrieving explicitly stated information
- making straightforward inferences
- interpreting and integrating ideas and information
- examining and evaluating content, language and textual elements.
Overall, half of the PIRLS assessment focuses on reading for literary experience and half on reading to acquire and use information.
How are results collected?
Within Australia, a stratified random sample of 281 primary schools participated in the data collection for PIRLS 2021 – similar to the sample in 2016. The stratification of the sample ensured that the PIRLS sample was representative of the Australian Year 4 population (according to jurisdiction, school sector, geographic location of each school and socioeconomic category for the area of each school). At each sampled school, at least one intact Year 4 class was selected to participate in PIRLS 2021. If the school had combined classes, such as Year 3s and 4s, then multiple classes were selected to have the equivalent of a whole class of Year 4 students. This resulted in a sample of 5,488 Year 4 students. Statistical weighting enables these students to represent the total Australian student population at Year 4.
Given the difficulties faced by schools during the testing period, we are very grateful to the sampled schools, teachers and students who did participate, and extend our heartfelt thanks.
How did Australia perform in previous PIRLS cycles?
The mean score for Australian students on PIRLS 2016 was 544 points, a statistically significant 17 points higher than in PIRLS 2011. Australian students were outperformed by students in 13 countries, including England, Northern Ireland, Singapore and Hong Kong, who all tested in English.
Australian students showed a relative strength in the ‘Literary experience’ reading purpose. Australia’s score on the ‘Informational’ reading purpose (reading to acquire and use information) was not different to Australia’s overall reading score.
As well as reporting scores on the point scale, PIRLS reports the percentage of students who attained the international benchmarks. These benchmarks are developed using scale-anchoring techniques and describe 4 levels – Advanced, High, Intermediate and Low. Within Australia, 81% of Australian students achieved the intermediate international benchmark, which has been set as the proficient standard for Australia.
What should we expect from Australia in PIRLS 2021?
There have been 4 NAPLAN assessments since the last cycle of PIRLS – in 2017, 2018, 2019 and 2021, with 2020 missing due to the COVID-19 pandemic. During that time, there has been no real change in the average scores of either Year 3 or Year 5 students.
We have yet to see the impacts of the pandemic and the resultant loss of school time on these young learners. Over the 2020 and 2021 school years, many schools moved between physical, hybrid and online learning environments as they balanced the safety needs of students and staff with the need to provide an effective learning environment. Some schools and some teachers were more prepared than others to do this, and the support from home also varied widely. In some states of Australia there was little to no impact of the pandemic on schools during the 2020 and 2021 school years, while in others there was a great deal of time spent out of the classroom learning environment.
So far, the international research on the effects of the pandemic have pointed to learning loss, particularly in the early years. Researchers in the USA found that many children in Year 2 and Year 3 in their study lost momentum on fundamental skills such as reading, with the difficulty in creating a language-rich environment on Zoom being one of the primary reasons (Domingue et al., 2021). The impact is expected to be most severe on reluctant readers – boys, those from homes in which parents do not read regularly, and those from a disadvantaged background. In another study conducted in the UK, attainment gaps were found for both Year 1 and Year 2 students, with the most profound effects on students from a disadvantaged background (Rose et al., 2021).
In Australia, there are already noticeable issues in regards to disadvantaged students. While the national average proportion of students attaining the proficient standard is 81%, 89% of students from ‘more affluent’ schools achieve this compared to 69% of those attending ‘more disadvantaged’ schools. Will the COVID-19 disruption make this gap even worse? We’ll find out in December.
References
Domingue, B. W., Hough, H. J., Lang, D., & Yeatman, J. (2021, March). Changing Patterns of Growth in Oral Reading Fluency during the COVID-19 Pandemic. (Policy Analysis for California Education, Working Paper). https://edpolicyinca.org/sites/default/files/2021-03/wp_domingue_mar21-0.pdf (PDF, 4.23 MB)
Rose, S., Badr, K., Fletcher, L., Paxman, T., Lord, P., Rutt, S., Styles, B. & Twist, L. (2021). Impact of School Closures and subsequent support strategies on attainment and socio-emotional wellbeing in Key Stage 1. Research report. Education Endowment Foundation. https://d2tic4wvo1iusb.cloudfront.net/documents/pages/projects/Impact-on-KS1-Closures-Report.pdf?v=1638448453 (PDF, 8.32 MB)
