The Research Files Episode 60: Australian teenagers and financial literacy

This podcast from Teacher is supported by the University of Newcastle, Australia – top 150 in the world for Education.

From Teacher magazine, I'm Rebecca Vukovic and you're listening to an episode of The Research Files.

In today's world, young people must be financially literate to perform common, day-to-day tasks, like using a debit card or choosing a mobile phone plan. But, how financially literate are teenagers in Australia? And, how do they compare to their international peers?

The OECD's Programme for International Student Assessment (or PISA, as it's commonly referred to) includes an optional assessment that examines 15-year-old students' understanding of money matters and financial literacy. This week, the Australian Council for Educational Research released a report that analyses Australian students' performance in the latest PISA Financial Literacy survey. It's titled PISA 2018: Financial literacy in Australia.

A total of 117 000 students from 20 countries and economies participated in the financial literacy survey, including over 9400 Australian students from 740 schools.

In today's episode, I'm joined by one of the report's co-authors, Lisa DeBortoli, a Senior Research Fellow at ACER. We discuss some of the key findings to come from the report and what students were required to do for this assessment. You'll also hear Lisa share more about the proficiency levels, what they mean, and the kind of knowledge and skills that students are typically capable of displaying at each level, as well as some practical examples. Let's jump in.

Rebecca Vukovic: Lisa DeBortoli, thanks for joining Teacher magazine.

Lisa DeBortoli: Thank you for having me, I'm looking forward to speaking to you about financial literacy today.

RV: Of course we're here to talk about the PISA 2018 Financial Literacy in Australia report, which was released this week. To kick things off, could you share what some of the main goals of PISA are?

LD: The focus of PISA is different to other kinds of assessments – it looks at young adults, those students who are nearly at the end of their compulsory education. And, what we do is, we look at their ability to apply their knowledge and skills to real life situations, rather than just looking at what factual and procedural knowledge they can produce and have learned in school. So, in that context, PISA tries to answer a number of important questions that are related to education and these include questions such as: How well prepared are young adults to meet the challenges of the future? What skills do young adults have that will help them adapt to change in their lives? Will they be able to analyse, reason and communicate their ideas effectively? It also extends a little bit further in terms of schooling. So, what are some ways of organising schools and school learning so that they can be more effective than others? What influence does the quality of school resources have on student outcomes? And, more importantly too, PISA actually looks at the educational structures and practices that can maximise the opportunities of students from disadvantaged backgrounds.

RV: There's obviously a lot to it but let's chat now about the financial literacy assessment and go over some of the Australian results in an international context. Could you run me through some of the key findings for Australian students and how they compared to their international peers?

LD: Sure. Similar to other international studies, results are reported on a continuous scale of performance. So, PISA results are reported as average scores and they provide a summary of student performance and they also allow for comparisons of relative standing between countries and different sub groups of interest. In the PISA 2018 Financial Literacy assessment the highest performing country was Estonia who achieved an average of 547 points, while the lowest performing country was Indonesia, with an average of 388 points. Australian students achieved an average score of 511 points.

To give you some perspective around these results, it's possible to estimate the score point difference that's associated with one year of schooling in Australia, which is, on average, around 31 points. So, what does that mean? Australian students were performing about one year of schooling lower than Estonian students and almost four years of schooling higher than Indonesian students.

In addition to Estonia, there were three other countries – Finland, Canada and Poland – who outperformed Australia. Two countries – the United States and Portugal – performed on par with Australia. And there were also 13 countries that performed lower than Australia. And these included countries such as Latvia, the Russian Federation, Spain, Italy, Serbia, Peru, and Georgia.

RV: Really interesting and I'd like to drill down a little bit further now. Could you explain what some of the key findings are in a national context?

LD: So, results for a number of demographic groups were reported in the Australian PISA reports so that they can allow for nationally comparable reporting of progress towards the Alice Springs (Mparntwe) Education Declaration. So when we report results for different demographic groups, we cover a range of the groups. To start off with, for gender, there were no significant gender differences between males and females in the 2018 financial literacy assessment. Internationally, Australia was one of 14 countries where female and male students performed on par with each other. In three countries – Bulgaria, Indonesia and Georgia – the female students outperformed the boys, while in Italy, Peru and Poland, the male students outperformed the females.

Now, given there was no gender differences in the average scores of Australia, there's also another way to report the results and that's looking at the students that are the high performers and the low performers. So, the low performers are those students in the financial literacy assessment that achieve lower than 400 points and the high performers are those that score over 625 points. So, in terms of the proportions of low performers for male and females in Australia, they were about the same (15 per cent for female students and 16 per cent for male students). The proportions of female and male students in Australia who were also high performers was also about the same (13 per cent for female students and 16 per cent for male students). I can talk about some of the other demographic groups, would you like me to do that?

RV: Yeah that would be really interesting.

LD: So, in terms of socioeconomic background – before I tell you the results for this group, I'll provide you with some information about how this construct is measured in PISA because usually socioeconomic status is a broad concept that summarises many different aspects of a student, a school or a school system. So in PISA, a student's socioeconomic status is measured by the PISA Index of Economic Social and Cultural Status. This index is basically a composite score that is based on: the highest occupational status of parents, the highest educational level of parents and years of education, and the [availability] of household possessions, which is used as a proxy for family wealth. And typically when we report results for socioeconomic background, we distinguish groups that are the most disadvantaged, the socioeconomically average students and also the least disadvantaged students.

So in PISA, in Financial Literacy, the least disadvantaged students scored on average 89 points higher than the most disadvantaged students. These results are striking given the difference in performance between these groups is almost three years of schooling. And if we look at it in terms of the high and the low performers – for the most disadvantaged students, only six per cent of the high performers, whereas 26 per cent were low performers. And this picture is very different for the least disadvantaged students where we find 25 per cent of students that are high performers and seven per cent of students that were low performers.

RV: So those were some key results for socioeconomic background but I understand that you also looked at other demographic factors like geographic location and Indigenous background. The report says for geographic location, students in metropolitan schools performed significantly higher than students in provincial and remote schools, and students in provincial schools performed significantly higher than students in remote schools. For Indigenous background, Indigenous students performed significantly lower than the non-Indigenous students, by 86 points, which is around two and three-quarter years of schooling.

There are lots of statistics that go into a lot more detail in the full report so for any listeners interested in learning more, we'll link to the report in the transcript of this episode at teachermagazine.com.au. Moving on now, I'd like to speak now about what it was that students were actually required to do as part of this assessment. I understand all students completed a two-hour cognitive test that consisted of financial literacy questions, as well as reading and mathematical literacy questions. Could you explain a little more about what this involved?

LD: As you mentioned, students completed a two-hour test where students were presented with a series of units. Now these consisted of a stimulus – which is typically a short written passage or text that accompanied a table or graph or diagram – along with a series of questions. Questions were asked in different response formats so that the full range of cognitive abilities and knowledge for the financial literacy could be assessed. So just over half of the questions in the financial literacy assessment had a multiple-choice format. There were simple multiple-choice questions where students are asked to select one correct response from among four possible options, or where students had to select an answer from a selectable element such as a graphic or a text. There were also complex multiple-choice questions where students were asked to respond by selecting the correct response to each of the number of statements or questions.

The other kinds of question format in the financial literacy test were constructed response questions and these are questions where students are required to generate their own answers. Some of these responses to these questions have a limited range of acceptable answers, typically numbers, and these were coded automatically, while there were some other responses that required an extended answer. Sometimes this ranged from writing a short explanation to showing the method and thought process that students used in reaching their response. Now for these types of questions, the responses had to be coded by trained experts who selected the code that best captured the response provided by a student to an item and then each code was then converted to a score for that question.

I should also add that the financial literacy test is computer-based so the item writers were able to take advantage of creating questions that used interactive features such as a slide bar and running simulations. Not only can we get more information about how students go about finding a solution to these questions, but these kinds of questions also keep the students engaged in the test.

RV: And then after the cognitive test students also completed a suite of three student questionnaires. What was the purpose of gathering this information? What did it tell you?

LD: That's right, students completed three questionnaires. These consisted of the internationally standard student questionnaire, which all participating countries get their students to complete. This questionnaire collects information on students and their family background, aspects of students' lives such as their attitudes towards learning, their habits and life in and outside of school, and also looks at aspects of their motivation and engagement.

In Australia, we also asked students to complete two internationally optional questionnaires. The first one was the Information and Communications Technology Questionnaire which basically collects information on the availability and use of ICT, students' perceptions of their competence in completing tasks and also their attitudes towards computers. The second optional questionnaire, the Financial Literacy Matters Questionnaire, really looked at the financial perspective, asking students about their attitudes towards money and behaviours and their experiences and familiarity with money matters.

Now, collecting this background information provides a greater depth and understanding of students because it provides us with a wealth of reliable and valid non-cognitive outcomes. These type of data can inform policy and research in their own right and also what I think is more important is these data provide a context through which we can interpret scores from the cognitive test, both within and also across education systems.

RV: School principals were also asked to fill out a questionnaire, weren't they? Why was this?

LD: So principals from participating schools were asked to complete a school questionnaire that collected descriptive information about the school. So, things such as: the quality of the school's human and material resources, decision making processes, instructional practices and school and classroom climate. Similar to the data collected in the student questionnaires, these data help us to understand student performance and also help us to understand what's happening within school systems as well.

RV: We'll be back after this quick message from our sponsor.

You're listening to a podcast from Teacher magazine, supported by the University of Newcastle, Australia, a leader in postgrad study. Education graduates enjoy a 96 per cent employment rate and median starting salary of $85,000, which is above the national average. With multiple degrees available, including master's in leadership and management in education and special and inclusive education, the University of Newcastle offers flexible options including face-to-face and online modes of delivery so you can fit study around your lifestyle.

RV: I'd like to talk about the proficiency levels now. Could you explain what the proficiency levels are and what they mean for this assessment?

LD: So proficiency levels are another way of reporting student performance. So while I mentioned before, mean scores provide a comparison of student performance on a numerical level, what the proficiency levels do is provide a description of what students know and what they can do. So the financial literacy proficiency scale is divided into five levels and that spans from Level 1 (the lowest level) to Level 5 (the highest). So those students who are placed at Level 5 scoring 625 points or above, as I mentioned before, are considered the high performers, where students who are placed below Level 2 and they've scored 400 points or lower, are considered low performers. Now the descriptions of each of these levels are based on framework-related cognitive demands – they're imposed by tasks that are located within each level to describe the kinds of skills and knowledge needed to successfully complete those tasks and which can then be used as characterisations of the substantive meaning for each level.

RV: I'd like to delve in a bit deeper now because I'm interested in hearing more about the knowledge and skills that students are typically capable of displaying at each level. So let's start with Level 5, the high performers. What are some things that students at this level can typically do?

LD: So these high performers really have demonstrated that they have a high level of skill and knowledge. So they can apply their understanding of a wide range of financial terms and concepts to contexts that may only become relevant to their lives in the long term. They can analyse complex financial products and can take into account features of financial documents that are significant but unstated or not immediately evident, such as transaction costs. They can also work with a high level of accuracy and solve non-routine financial problems, and they can describe the potential outcomes of financial decisions showing an understanding of the wider financial landscape, such as income tax. Would you like me to take you through a released item that if answered correctly, would demonstrate what a student at Level 5 would be capable of doing?

RV: Yeah that would be fantastic!

LD: So this question comes from the unit ZCycle, which is about a hypothetical smartphone app about bike sharing. So in this question, or in the stimulus initially, students are first introduced to a text that explains how the bike sharing scheme works and how the membership fee in the scheme is managed through the app. Prospective bike users should understand there is an annual and monthly membership fee and that they may be asked to pay additional costs for each ride, depending on the ride duration. So this question is an example of a relatively common fee structure, combining fixed and variable costs, that students may encounter not only in bike sharing schemes, but also in some other plans such as mobile plans.

So students have an opportunity to try the app when they're doing the assessment to see how different uses of the bike scheme would affect the final cost. So the question asks students to compare the cost of using the bike sharing scheme for six months or one month. In considering the response, they need to include having relatively short rides during the week and for longer rides during the weekend. So the students, using the interactive features in the assessment, they need to work out the cost of an annual membership, including short and long rides, and they also need to calculate the cost of a six month membership, including those short and long rides. As students are actually progressing through this question, they need to recognise that the annual membership is cheaper than a six month membership with the extra rides and to achieve a correct response, students have to indicate two things – that the one year membership is less expensive, and the cost difference for Julie (as an example of a rider), what that difference would actually be.

RV: And Lisa, Level 3 on the scale is considered a middle performer. I'm wondering then, what are some of the key skills that students are displaying at Level 3?

LD: The students at Level 3 can typically apply their understanding of commonly used financial concepts and terms and products to situations that are relevant to them. They begin to consider the consequences of final decisions and they can make simple financial plans in familiar contexts. They can make straightforward interpretations of a range of financial documents and can apply a range of basic numerical operations including calculating percentages. They can also choose the numerical operations needed to solve routine problems in relatively common financial literacy contexts such as budget calculations.

There's an example of a Level 3 item called Motorbike Insurance. So students in this scenario were provided with the following details about a motorbike insurance policy and Steve's plans about his insurance. So the stimulus that students are given is: ‘Last year, Steve's motorbike was insured with the PINSURA Insurance Company. The insurance company covered damage to the motorbike from accidents and theft of the motorbike. Steve plans to renew his insurance with PINSURA this year but a number of factors in Steve's life have changed since last year.' So this question is presented to students as a complex multiple-choice. Students are provided with three statements and they're asked how each factor in the statement is likely to affect the cost of Steve's motorbike insurance.

So the statements that the students were given were: ‘Steve replaced this old motorbike with a much more powerful motorbike. Steve has painted his motorbike a different colour. Steve was responsible for two road accidents last year.' So for each statement, students needed to indicate whether the factor increases the cost, reduces the cost or has no effect on the cost. This question relies on students understanding that the higher the risk exposure with regards to measurable criteria, the more it will cost them to buy appropriate insurance.

RV: And Lisa, Level 1 is on the other end of the scale and is considered a low performer. I'm wondering then what are some skills that students at Level 1 are typically displaying?

LD: Look, the low performers are those students that really are only demonstrating basic financial literacy skills and they're not yet able to apply their knowledge to real life situations involving financial issues and decisions. These students can identify common financial products and terms and interpret information relating to financial concepts, but only basic financial concepts. They can recognise the difference between needs and wants and they can make simple decisions on everyday spending. They can also recognise the purpose of everyday financial documents, such as an invoice, and apply single and basic numerical operations.

An example of a question that a low performer would typically be able to respond to is an item called Invoice. So in this question, the students are asked to interpret a financial document, an invoice, and identify its purpose. There are no calculations required to respond to this item. As a stimulus, the students are shown an invoice, the invoice is addressed to Sarah Johnson from the company Breezy Clothing and it lists three items of clothing in a table format showing the product code, quantity, unit cost and total cost. The question is a simple multiple-choice and students are actually asked why was this invoice sent to Sarah? And they can answer one of the four responses:

• Because Sarah needs to pay the money to Breezy Clothing.
• Because Breezy Clothing needs to pay the money to Sarah.
• Because Sarah has paid money to Breezy Clothing.
• And lastly, because Breezy Clothing has paid the money to Sarah.

RV: It's really interesting to hear those specific examples linked to the proficiency levels. I guess, it makes it really clear for me exactly what students could demonstrate in this assessment. So just finally Lisa, for teachers and school leaders listening to this podcast, what are some of the key points they can take away from reading this report?

LD: Overall, I think Australian students have performed really well in financial literacy. Of the 20 participating countries and economies, only four countries outperformed Australia and Australia outperformed 13 countries and economies. In terms of the highest and lowest performing students, 14 per cent of Australian students were high performers and 16 per cent of students were low performers. For the groups of students that have demonstrated the high skills of knowledge in financial literacy, they are to be congratulated.

On the other hand, I think there are a group of students who are the low performers who haven't demonstrated the financial literacy skills that are essential to perform the necessary financial operations and engage in financial activities in everyday life. In particular: the 26 per cent of students from the most disadvantaged backgrounds, the 39 per cent of Indigenous students and the 39 per cent of students who attended schools in remote areas. I think it's important to consider what the impact of this will be for these young students to meet the challenges for the future and how the financial literacy skills and knowledge for these students can be improved.

That's all for this episode. We'll include a link to the full report we've been discussing today in the transcript of this episode at teachermagazine.com.au. That's where you'll also find all of our articles, videos and infographics for free. And while you're there, be sure to sign up to the Teacher Bulletin to have new content delivered straight to your inbox each week.

This podcast from Teacher is supported by the University of Newcastle, Australia. A world leader in postgrad education. Apply now and maximise your potential.

References

Thomson, S., De Bortoli, L., & Underwood, C. (2020). PISA 2018: Financial Literacy in Australia. Australian Securities and Investments Commission. https://research.acer.edu.au/ozpisa/48