The Research Files Episode 65: How educators are using research

This podcast from Teacher is supported by Bank First. Bank First is a customer owned bank, committed to supporting the financial wellbeing of the education community since 1972.

Thanks for downloading this episode of The Research Files from Teacher magazine. I’m Dominique Russell.

You’ll be well aware that here at Teacher we are all about promoting quality teaching and leading and assisting school improvement at a grassroots level. We aim to support educators by sharing stories using evidence-based approaches which you can trust and adapt for use in your own school settings. We also ask you how you use Teacher content in your practice in our annual reader survey, and so many of you always tell us about how stories we’ve shared have motivated school change, or inspired discussion with colleagues.

But on a broader scale, how are teachers and school leaders accessing and using research and evidence? What challenges do they face when doing so? And, what enables quality use of research and evidence?

The research team for the Monash Q Project is looking at all of these questions in a report they have just released. The report shares data from a survey they conducted of teachers and school leaders on how they’re using research and evidence in their work. The Monash Q Project is a much larger project than this one report – it’s a five year partnership between Monash University and the Paul Ramsey Foundation with the broader aim of improving the use of research evidence in schools.

I’m joined today by two co-authors of this report on research and evidence use – Jo Gleeson, a Research Fellow on the Q Project, and Lucas Walsh, a Professor of Youth Policy and Education Practice, and one of the chief investigators on the Q Project. Let’s kick off the episode with a bit more background information on the project from Lucas.

Lucas Walsh: Well the Q Project is a five year partnership between the Monash Faculty of Education and Paul Ramsey Foundation. And over the five years we’re covering four strands that are intersecting. The first strand is looking at how to conceptualise how evidence is used, because this is the main focus of the work – we’re looking at how evidence, research evidence in particular, is used by teachers.

So we started the project by looking at how it’s conceptualised in other fields to get a, sort of a, more contextual, grounded understanding. The second strand is looking at what does it look like in practice within the context of teachers within Australia. The third strand is how we develop it through professional learning. And in response to your question about what phase we’re at, we currently think of ourselves as moving from understanding to improvement. So the phrase of the project we’re currently in is looking at improvement and what it might look like, both at a practitioner level, but also at a systems level.

And the fourth and final strand involves embedding it within those systems, but we’ve already started those conversations with a number of jurisdictions and with overarching Commonwealth bodies.

DR: Obviously this research report that we’re looking at right now, it focuses on the survey that you’ve conducted. So, can you go over who exactly was involved in this survey, when it was conducted, and how the participants for it were chosen?

Jo Gleeson: Sure. So we started a research phase – what Lucas referred to as strand two – we started that research phase in schools last year. So our first activity was the survey. And we administered that to schools that had volunteered to participate with the Q Project. And we were right in the middle of recruiting schools when COVID-19 hit. And so that curtailed, I guess, the number of schools that we were able to recruit.

But I’ve got to be honest with you, the response that we had from teachers wanting to be involved in the first survey, despite being in lockdown, shifting their practices to online, was incredible. We had about 130 responses from teachers that we had recruited ourselves and then we worked with some recruitment agencies to help us get some other teachers and hence we got nearly 500 teachers and school leaders from the four states in which the Q Project is participating which is New South Wales, Victoria, Queensland and South Australia. So it was an incredible response.

And then when we invited those people that had completed a survey, did they want to participate in an interview? So we went to our Q partner schools, and again, just an incredibly generous response, that 29 teachers and school leaders put up their hand. And in fact others tried to, but, you know, we had to put a stop to getting it.

But just incredible in terms of how open they were, how generous they were with their time, and also their honesty. Some of the insights that they provided, which as I discussed with you are coming out in an upcoming paper around their thoughts about what quality research use means, just fantastic.

DR: Absolutely. And so part of the survey, you asked the participants what types of research and what types of evidence they value most. So I’m interested in what they said about that and then also if the types of research and evidence that they valued most, did that actually translate into what they utilised most as well?

JG: We asked for their views as to what type of evidence that they consulted most frequently. Now, actually, it’s non-research related types of evidence that they frequently use. Student data, policy and curriculum documents, guidance from official bodies were consistently – it didn’t matter whether it was a teacher or a school leader, it didn’t matter what state they came from, what type of school environment they came from – they were the three top types of evidence that they used.

Research-related resources, like research disseminated from a university or university-based guidance or advice was not used, or not consulted often, when compared with those other data types. So there was some interesting contradictions in the data, if you like. We asked a question, whether you had used research in the last 12 months. And 70 per cent of the teachers and the school leaders said ‘yes I had’. So you would expect research-related evidence types to be consulted more frequently than they were, but they weren’t.

So that started throwing up, ‘what’s going on here?’ So we suspect that teachers and school leaders think that they’re using research, but it might not be, or they might use research once relative to using other types of evidence far more often.

LW: And it’s useful to note that, you know, educators who are not necessarily research engaged are only not consulting research very often, but they’re using these other evidence sources and types, including student data, far less as well. And I mean it’s useful to clarify here that our definition of research evidence, our working definition, is evidence that’s been generated through systematic studies undertaken by universities or research organisations. The kinds of things you might see reported in books, reports, articles, research summaries, training courses and the like.

DR: And so thinking now about the research and the evidence that they’re accessing and that they’re turning to most frequently. Did you gain any data and any insight on how they used this in practice?

JG: So they’re using it most often – you would think, when we talk to teachers, or when you talk to different people involved in the education system, what comes to mind when you think of using research is like ‘okay, I’m going to change the way we go about literacy’, for instance. Or, ‘I’m going to put in a new way of conducting a particular teaching practice’.

So it’s usually viewed as something big, as something quite formal, as something quite direct. But in fact teachers are saying to us ‘when I use research, I use it mainly to have discussions of best practice with my colleagues, or I use it to improve my own knowledge, or to reflect on my own, personal either teaching approach or maybe something that I’ve experimented with’.

So teachers and school leaders are telling us that they use it quite differently to how you might first imagine that they would use it.

DR: And something else that I found interesting was obviously with the hundreds of people that you surveyed, you had quite a range of educators that got back to you and provided their insights. So you had school leaders, teachers and also people with vastly different levels of experience. And you go into and the differences between their responses, so things like their confidence level. So can describe to me what some of the differences that you found?

JG: So the main difference is when you look at school leaders versus teachers. On every aspect of research use, school leaders have more positive beliefs in the value of research. They feel more confident in their capacities to find research, interpret it, use it in practice; they have far stronger beliefs about their school’s environment, or their school’s support for research use than teachers.

So in every way, that was probably the biggest difference coming out. We didn’t find any differences between school types. So if a school – and we had such a range of schools, there were tiny little schools with only like 30 children in them in really remote parts of the country, and then we had big, huge schools in metro areas – we would have thought that there would have been differences by school type, but there really weren’t. So it didn’t matter where the school was located, what socio-economic demographic it was, whether it was primary or secondary, didn’t matter at all. But school teachers and leaders were the biggest difference.

We did see some differences, too, the more experienced a teacher was in terms of years of service, the more confident that they felt, certainly than teachers with less than five years’ experience. That was interesting to us. We would have thought, particularly if the teacher had gone through post-graduate type qualifications, that they would feel confident in their capacities to use research, even if they didn’t have many years of service. But no, that was quite surprising actually. But they were the main differences across the survey data.

LW: …We saw a clustering of the use of evidence around solving a particular problem, although we did see quite a spread across other uses as well. And really that the reflections that you just pointed out, Jo, in terms of sometimes teachers used it in reflective ways and how it’s used is also varied. So you’ll get ad-hoc pick up what’s in front of me, or something that I’ve just heard at a conference, versus more targeted approaches which tend to be more focused at the leadership level, which Jo has summarised quite neatly.

JG: There’s probably two other points. We asked one particular question which was whether you believe that your own teaching experience and observation should be prioritised over the use of research. Teachers, overwhelmingly, believed in that far more than school leaders did. So there’s something happening there with either time, or role, or confidence. Not sure what it is and it’s certainly something that we’re looking to investigate more this year, that belief of experience and knowledge over research use.

The other thing that came out, just looking at teachers and school leaders again. Teachers are really social creatures, and teachers, as opposed to school leaders, wanted to use social and relational ways of finding, interpreting, using research more than school leaders did. They really relied on recommendations from their colleagues, or their personal networks, word of mouth, that type of thing.

LW: And where school leaders were more oriented towards what we call credibility factors. Another thing that we haven’t raised yet that has been implicit in what Jo has been saying is the powerful role of context. And it’s not just context in the sense of those relational things that Jo was talking about, in where teachers are, but the context in which the particular challenges that they’re facing within their school.

Coming up, we’ll hear about the implications of this report, and also what factors support an educator’s access to and use of research. But first, here’s a quick message from our sponsor.

You’re listening to a podcast from Teacher magazine, supported by Bank First. Bank First is proud to be the bank that supports your purpose. They’ve been financially empowering educators to realise their dreams since 1972, and understand that your passion, dedication and expertise make a real difference to people’s lives. Visit bankfirst.com.au to find out how they can help you reach your financial goals.

DR: And so I’d like then now to turn to the implications of these survey findings. You detail a few in the report, so could you describe those for me?

JG: Sure. Well the first thing is that you have to look at Q’s aim. As a project, our aim is to really stimulate conversation around quality research use. And for people to start asking questions of their own practice or the practices, if they’re a school leader, of their staff, or if they’re a policy maker, of their jurisdiction. And it’s really trying to ask questions, ‘well, what’s happening here?’, and ‘how can we either work individually or collectively to improve research use?’

So I think one of the implications of the paper overall is that there’s things that are going on here that can be improved. Irrespective of whether you are a teacher or a school leader or what state you came from, there were positive perceptions of the benefit of using research. And even though it’s not in this report, when [we’ve looked at qualitative findings] and we’ve interviewed teachers, there’s this overwhelming desire to use research. Even if teachers and school leaders don’t feel they’re very good at it, they’re quick to tell us, ‘I really want to do this,’ right? So I think that’s a positive implication, that there is a real desire to use research.

But – and there is a ‘but’ – and the ‘but’ is that there are things stopping them doing it. Now, whether that’s their own confidence level, whether that’s the culture of the school that they’re in, that the culture might not be very supportive, or it might not be very open-minded to experimenting with different knowledge, with different research.

Or there are physical barriers. There are real barriers. Teachers are telling us, particularly teachers as opposed to school leaders, ‘I don’t have the time to do this. I can’t find the time to really go and look at different research sources, of reading that. Even if I did have the confidence, even if I did have the ability, I don’t have time.’

Some teachers are telling us, ‘hey, I want to make time for this and I want to learn, but I don’t have access’. And other teachers are saying ‘I just don’t know where to find it’. So, they want to go there, but they don’t know where to start looking. So there are some real knowledge gaps, there’s some skill gaps, there’s some capability gaps, and then there are some real challenges around time and access that need to be addressed.

So I think, irrespective of who you are – whether you are an individual teacher yourself, or a school leader, or you are in a department of education or a professional learning provider – there’s things in here, in this report, that teachers and school leaders are saying that they need help with and that they should be getting that help.

LW: So in summary, we know that teachers are keen to use research evidence. We know this occurs across systems… the small or large schools that Jo referred to. We also know there are differences within schools between school leadership and teachers in general. We also know that there is a variety of ways that evidence is used and types of evidence that is used, and we’re getting greater clarity about what’s being used more and what the barriers and opportunities are there.

There’s a deeper, wider question there about understanding the teacher profession as being made up of many things, but it includes mindsets and skillsets, dispositions and attitudes across the life course of a given teacher; and our work is looking at one particular strand of that, that’s the use of research evidence.

But the data can yield something much wider about how teachers understand and develop and grow over time as professionals, and will understand better what kinds of things work within schools and across school settings in improving teacher practice.

DR: You describe the quality use of research evidence [in your report] and you say that it’s: ‘the thoughtful engagement with and implementation of appropriate research evidence, supported by a blend of individual and organisational enabling components.’ So would you be able to go over what these enabling components are?

LW: Well, so there’s a range of them, and Jo, jump in whenever you like. But we’re looking at different dimensions of teachers’ professional lives. And enabling components at an individual level look at things like: the skillsets, the knowledge and capabilities that are required to thoughtfully engage and implement appropriate evidence; there’s the mindset component, which are these dispositions and attitudes as well as values that are required to thoughtfully engage with and implement appropriate evidence; as we’ve already highlighted, relationships are super important, the interpersonal processes and connections that teachers require to be able to thoughtfully engage with and implement appropriate research evidence. And Jo’s already highlighted how teachers look to trusting relationships with colleagues for ideas, advice and reflection.

At a wider organisation level, there are enabling components that we’ve identified and these would include things like leadership. So, does the school’s vision, commitments, and does it role model and support the thoughtful engagement and implementation of research evidence? That enabling culture needs to be there and it often starts at the top in the school leadership.

The culture’s obviously important, as we’ve highlighted. Does the organisational ethos, the values of the school, the kinds of norms set up, do they foster thoughtful engagement and implementation of research evidence? It’s something that’s around us all the time, but is not necessarily foregrounded or identified routinely as part of daily practice.

The other thing that Jo’s also highlighted is, is the infrastructure in place? Are the organisational structures, resources, processes there to enable engagement and thoughtful use of research evidence to take place? And these include issues such as access that Jo’s already mentioned.

Finally there’s a third level which is at the system level. We understand that the schools exist within complex ecosystems within their own communities and they’re also embedded within this wider system. And so it’s why we’ve identified the system level as being very important because they set the tone for what’s important in schools, they set the tone for what’s expected of teachers. And we would argue that the thoughtful engagement and use of research evidence needs to be a priority and those kinds of signals are sent out at the system level.

JG: Probably all I’d add to that is the – so, Lucas has described the enablers. Right at the core are two very important things as well that do enable research use.

Lucas referred to before, the importance of context. Teachers and school leaders are telling us very, very loudly, that: ‘if the research is not suited to my context, or I can’t adapt it for my context, I won’t use it’. And, in fact, it can act very much as a deterrent to using it in the future.

So when we talk about appropriate evidence, we’re not trying to get into debates about, you know, has it gone through some trial versus something else? Yes, we want good, credible evidence, but we absolutely are highlighting the importance of context – and that does enable it. And somehow, part of what we want to do is, how do we help to get realistic, usable, context-fitting, research into the hands of teachers?

The second thing is around thoughtful engagement. Teachers and school leaders are telling us that to use research well, it takes time and it takes consideration, and it takes questioning. That’s probably the theme that teachers and school leaders are telling us most. You’ve got to question it, right? Can I use this? Will it work?

And, in fact, not that it’s in this report, but teachers and school leaders when research is not being used well, they describe situations where it hasn’t been questioned, where the latest fad has just been picked up and plonked down in place. So, thoughtful engagement, the questioning and the contextual relevance are enabling factors themselves.

LW: So you ask the question, you know: Is the research evidence credible? Is it fit for purpose? And, do I have the skills and environment and support that I need in order to be able to do that well?

JG: …We would love if any schools want to, you know, through listening to this, or want to participate in Q Project activities that are coming up, we would really welcome that. The more people that want to get involved, fantastic.

That's all for this episode. To stay up to date with our Research Files series, be sure to subscribe to our podcast channel on Apple podcasts, Spotify or SoundCloud, so you can be notified of any new episodes.

You’ve been listening to a podcast from Teacher, supported by Bank First. Bank First is proud to be the bank that exists to serve the education community. Visit bankfirst.com.au to find out how they can help you reach your financial goals.

References and related reading

Rickinson, M., Gleeson, J., Walsh, L., Cutler, B., Cirkony, C., & Salisbury, M. (2021). Research and evidence use in Australian schools: Early insights from educators. Q Survey Summary 01/2021. Q Project, Monash University. DOI: 10.26180/14234009

Quality Use of Research Evidence (QURE) Framework Report. Monash Q Project.

Q Suite of resources for educators on research and evidence use.