Before we jump into this episode, are you subscribed yet to the free weekly Teacher bulletin? Get a weekly wrap of our latest content straight to your inbox. Join our community by clicking on the sign-up button at our website, teachermagazine.com.
Thanks for downloading this podcast from Teacher. I’m Dominique Russell. It’s well-known from existing research that boys tend to have more positive attitudes towards STEM than girls do at school. We also know that students’ perceptions of the classroom environment have a powerful impact on their engagement and their learning outcomes.
A new study has investigated the classroom experiences of boys and girls participating in integrated STEM projects (or iSTEM) to see how they differ, and also to better understand how the classroom experiences of females might impact their participation in STEM. Specifically, the research team were eager to understand the emotions of the boys and girls in the classroom and how this could play a role in their learning and engagement.
It’s the first time classroom emotional climate has been considered in research on gender differences in STEM learning. So, in order to investigate this, a research team from Curtin University in Western Australia needed to develop and validate a new research tool. In this episode I’m joined by the lead author, Rekha Koul, to find out more about the process of developing and validating this tool, and, of course, their research findings. Let’s jump in.
Dominique Russell: Thank you for joining us on this episode of The Research Files, Rekha. I thought to get things started, it would be great if you could introduce yourself to our listeners and also give us a bit of an overview of your study that we’ll be talking about in this episode.
Rekha Koul: Thank you, Dominique, first of all, for having me and showing your interest in this paper. My name is Rekha Koul. I am Professor of Science Education at Curtin University, currently acting as Deputy Head of School, as well as I'm leading the STEM Research Group.
The STEM Research Group within the School of Education has a long history within Curtin. We had a Science and Mathematics Education Group which was called SMEG and was a Postgraduate Research Centre. And our expertise was in science and mathematics education. For a very long time it had an ARC [Australian Research Council] Centre for Excellence status, but overtime the funding dried out.
But as we focused on science and mathematics education research, at the same time we were known around the world as Mecca of learning environment research. And that's where I started my PhD study in science learning environments. And as I progressed in my career, the areas of learning environment which I felt were not touched, I started looking into [those] areas. Like, we looked at students’ perceptions of assessment.
Long story cutting it short, nearly half a decade back, I felt we have been focusing quite a lot on the achievement. Not much focus has been given to emotions in the classroom and coming from science and now moved into the STEM area along the way, we applied for an ARC Discovery Grant. And we got it and the journey of this research started.
And in this study, although we endeavoured to look at the students’ emotions in STEM classrooms – but obviously being a female and knowing what's happening in the STEM area, my interest was also to look at the gender differences. And this paper is one of the papers from that study. It was a larger study – by now we have nearly 6 research papers/book chapters for this study, but gender differences came out from this study.
DR: And so why was it important for you and your colleagues to conduct this research?
RK: When I looked at the thought of the classroom environment and emotions in the classroom, I found out that worldwide there was no research tool available to measure the students’ emotions. There was one study done in US which is known as 7 Cs (a Ferguson study funded by Gates Foundation), and they had come up with the – they called it 7 Cs study, but the tool was not validated.
In learning environment research what we have done, in the last 40 years, have made these inexpensive tools available to teachers where they can measure the students’ perception and see where they are standing. So, to make a tool available, we had to go ahead with validation of this tool. And we changed it slightly (the 7 Cs) – although we use the Ferguson model as the basis of this study – but the study went through rigorous statistical analysis and multiple stages.
… And the 7-scale study moved into an 8-scale questionnaire, and we added a scale of motivation in it. To start with it was care, control, clarity, challenge, consultation, consolidation, and collaboration. But we did add motivation in it.
At one stage, that is soon after – if you would know, that WA, we closed down for a very short time during the COVID break (only for 10 weeks). And I did use it to measure the emotions of our university-going students during close-down and after close-down. So, we also validated it for use with the university students. So this tool has gone through rigorous analysis and is ready to be used by anyone that wants it now.
DR: So we'll get to the findings shortly, but beforehand I thought it would be great if we could have a sound understanding of the process of developing and validating this questionnaire for this research. So can you talk me through what kind of work this involved for yourself and your colleagues?
RK: As I said, I didn't want to reinvent the wheel, so I just wanted to draw on whatever was available globally. And the only tool we could find was the Ferguson study where they had developed this 7 Cs questionnaire. We took that tool and did some review of literature, wherever/whatever had happened, and we found one more study where emotions were studied – but not in depth, it was a surface level study. So, took both of them and we developed a questionnaire.
This questionnaire was run through many experts in learning environment research and asked them to review them for language and their comments. And we had a focus group, online, because many people worldwide came – we had one colleague from Netherlands, Professor Theo Wubbels – and we discussed it in detail. And next was, make those changes, and then we went to 8 schools in Perth and pretested this questionnaire with students from years 7, 8 and 9.
And choice of classes was very important for us. Senior High School students are very much siloed, so we couldn't have had iSTEM students. And, keeping in view, STEM is not a subject still in schools – we talk so much about STEM, our government has invested so much in STEM, but it's still not a subject. So we looked at the schools where either within the science stream or math streams, they were running an iSTEM-type project. And it became really very, very difficult for us to get more schools participating. Mainly schools have research fatigue, that's another factor. But we were able to get around 700 students and then, you know, parental permissions and all, but we were lucky enough to get around 700 students signing in for this study.
In pretesting we did focus groups with students, discussed the questionnaire after it was tested on them and incorporated their feedback. We ended up with a from 49-item to 71-item questionnaire, and then we administered it on the larger population. And finally, was confirmatory factor analysis and other analysis and we ended up with a 41-item questionnaire. I really wanted to keep it short and crisp. Well, I didn't play a role in that, it was the statistics which told us what is to be done. Because we know student attention span is shortening, so the shorter the item, more reliable we would get the answers.
And now it is ready. We have this questionnaire. It has a good factor structure. It has excellent reliabilities. We established the reliability and validity and only after that, we went on to other statistical analysis where we looked at their perceptions and even the gender gap and other things.
… As I said, it has a scale on care which is saying – the questions around it are, does the student feel that the student is cared [for] enough by the teacher? Control – how much control does the teacher apply in the classroom? Is there clarity in instruction? Do students feel challenged enough when they are undertaking iSTEM activities? Is there any agency given to them – means are they consulted when the activity is developed or when they are assessed? Because it's all activity-based where we are looking at the skills development, is there consolidation at the end? And most of these activities are group-based activities, so how well is that collaboration working?
In addition to these, as I said, we added a scale of motivation. So how motivated do they feel while doing these activities? And what we did, we also added a scale from TOSRA (Test of Science-Related Attitudes) which was developed way back in ’82 by Professor Barry Fraser who was my partner on this project – and even the ARC Grant. [The scale] was attitude to science which we adopted as attitude to STEM.
And strangely enough – now I'm going to come back to the females’ attitudes – the differences between males and females were highest for attitude to STEM, which we can discuss later why it could be (because I haven't explored it enough in this one study). But for the scales of control and challenge, females had better scores than the males. That means they didn't feel that the activities were challenging enough, but they appreciated the control in the classroom. For other scales, they were less motivated … which is a concern and I'll talk about it later.
DR: How interesting. Is there any other ways that your study and your research is quite distinctive from others?
RK: It is. Keeping in view, again, I'm going back to the history of learning environments, we were looking at the gender differences in classrooms in our research, but what we found – keeping the primacy of iSTEM in schools, there were no studies done in gender differences in STEM.
Going back to science or mathematics– the studies done in 70s and 80s were showing a larger gender gap. But as we came to early 2000s, the gender gap was narrowing. But when STEM came into being (where UNESCO standards of getting those 21st Century skills in students; and teachers started implementing integrated project-based, problem-based activities in the classroom) nothing was done around that. There were some qualitative studies, but not much of quantitative, nothing which would give you rigorous breakdown of breaking the responses on item-wise, individual-wise. And that's what the study did, and it's explained in detail in the paper we are talking about today.
DR: And so, tell me about the findings for this particular research paper.
RK: Our findings, to keep it very short, is overall females had negative perceptions about STEM, which is not new. But here what we could do – because we had these 8 scales of emotions – we could see where they have more negative perceptions and where they don't. For control and challenge as I initially told you, they had slightly higher scores than the males, but they were not statistically significant (but had slightly higher scores).
For other scales: care, clarity, consultation, consolidation and collaboration and motivation; they had lower scores than the males. Then the control variable which we used (the attitude to STEM), the gap was the widest. And that's what it is. It wasn't surprising, but we need to look at it and consider what can be done ahead.
DR: What are some implications, then, of those findings that you've just shared for classroom teachers around Australia?
RK: More deeply thinking about it, I think we are not doing something. We are doing something wrongly. It's been more than 2 decades that the Australian government has been pumping millions of dollars in STEM education and professional development of our teachers. If I look at it, why we are investing in STEM, is that 79% of jobs or careers need STEM-related skills and it has increased since 1990s. So we do need more STEM workers and if 50% of the population has negative perceptions, Australia will be short of workers. That's the main thing.
And at the same time, career inspirations are built in primary schools. So where are we not doing the thing right? Our government is investing money. Eighty-five per cent of our primary school teachers are females. So do we need a new framework for developing the PDs for training STEM teachers?
And if we look at my paper itself, I've talked about the sort of activities which were done in those classes was making generators, making cranes and pulleys and testing for mechanical advantage, designing and making a wave generation, designing and manufacturing a remotely operated vehicle, designing a robot and designing and making and testing of robotics or hydraulic arms. Maybe our girls are not liking it. Maybe our females are not conversant with running these activities, so we need to rethink and, you know, celebrate what girls do well. That's my take from it and I'm currently writing another ARC grant application where I am focusing on the professional development needs of our teachers.
DR: Is there room for any other further research in this area, or further use of the questionnaire that you developed as well?
RK: First, [the] use of this tool – it’s available for teachers, teacher educators, researchers. I'm getting quite a few requests from around the world (‘can we use this tool?’) and it’s available. In addition to that, we can use it for the evaluation of STEM programs around the country. I would like to see correlation between Classroom Emotional Climate and student achievements. Difference between the students’ perception of emotions and teachers’ perception – teachers could be thinking, ‘I'm caring for the students’, but where is that middle ground? Unless teachers realise that, we find that difference and find space where this gap can be narrowed. And once that gap is identified, we can have an action research which can be an iterative study and improve the emotions and improve the eventual achievement, which maybe we'll look at the cognitive achievement or skill development. And again, that's a very loose feel because when we talk about the skills, we don't give our teachers skills [on] how to measure them. And those would be lots of projects coming out of the study.
DR: For the teachers or anyone listening to this episode now who are wanting to access the questionnaire to have a look at it in a bit further detail, where should they go to be able to have a look at it?
RK: Send me an email and I'll share the questionnaire with them. That's not a problem, and even if they want some more support with how to use it, they want some help with the data analysis I can provide for that. I would love to do that.
That’s all for this episode. Thanks for listening. You can contact Rekha Koul at R.Koul@curtin.edu.au. I’ll leave that email, too, in the transcript of this podcast episode which you can find under the podcast tab over at our website, teachermagazine.com. We’ll be back with a new episode very soon – don’t forget to follow our podcast channel on Spotify or Apple podcasts to make sure you never miss a new episode.
Are you currently subscribed to the weekly Teacher bulletin? It’s a free weekly wrap of our latest content straight to your inbox. Join our community by clicking on the sign-up button at our website, teachermagazine.com.
References
Koul, Rekha B., McLure, Felicity I. & Fraser, Barry J. (2023) "Gender differences in classroom emotional climate and attitudes among students undertaking integrated STEM projects: a Rasch analysis", Research in Science & Technological Education, 41:3, 1051-1071, DOI: 10.1080/02635143.2021.198185
This research found that many project-based STEM activities involve making generators, making cranes and pulleys, designing and manufacturing a remotely operated vehicle, and designing robots. In this episode, Rekha Koul asks whether girls are interested in these activities. As a classroom teacher, reflect on the iSTEM activities you offer to your female students. Do these activities match their interests?
