Research Files Special: Dr Rukmini Banerji on monitoring schooling and learning across India

• Listen to and download all the Teacher podcasts for free on iTunes and Soundcloud

Hello and welcome to Teacher's podcast series, The Research Files – I'm Jo Earp. This month, I'm with Rukmini Banerji, Director of the ASER Centre in India – the research and assessment arm of non-government education organisation Pratham. For 10 years she has led the Annual Status of Education Report study, which monitors schooling and learning across the country. In July, she will become the new CEO of the Pratham Education Foundation.

Jo Earp: Dr Rukmini Banerji joins me now from Delhi, welcome to The Research Files.

Rukmini Banerji: Hi.

JE: As I mentioned in the intro there, you've been leading the Annual Status of Education Report study since 2005. Can you give our listeners an overview of the scale and scope of this study?

RB: Sure. So, we call it the Annual Status of Education Report in English, but the name that we use in India is ASER and it actually means 'impact'. The reason we started doing this big exercise was we hoped it would have an impact across the country.

What it is, is actually a very simple thing. It's a very simple test of reading; there are five tasks and we think that kids will be able to do at least one of those - starting from reading letters, to words, to simple sentences, to a little story (an eight line story), that's what the reading test is. The arithmetic test is similar – it has number recognition, subtraction problems and division problems.

... The way we set it up was that, India is a very big country as you know and we have something like 600-plus districts. Just to give you an idea, each district has anywhere between 1000 and 3000 schools. So, we wanted this to happen, not just in kind of a national sense, but also to have traction in every single district.

We picked rural districts – which are almost 575 out of our 600 - and we looked for a local institution or organisation in each district, which could be a college, an NGO, a university, a women's group – any organised group that actually was interested in thinking about their kids. A sample of about 30 villages is chosen from the census lists and people will go out to these 30 villages and do a sample of about 20 households from each village.

So, if you put it all together it becomes quite huge, but you have from district level onwards estimates of basic reading, arithmetic and of course schooling that is then put together in the big aggregate report, which is called the ASER Report. We bring it out every year in mid-January, so that the data for the current school year is available within the school year itself.

Interestingly, unlike many other [student achievement] studies, this one is a household survey. We do this outside the school for a variety of reasons - one of the big reasons is that we want to get everybody involved. We think that education, schooling and learning are very important issues which need to concern all citizens and not just those that are in education.

JE: Of course, alongside of increasing attendance and enrolments - that's obviously extremely important - the next challenge is improving learning outcomes once children get to school. Again, can you give us an overview of the challenges that India is facing.

RB: If I even start with enrolment, we find that between government, parents and communities we've managed to enrol a very, very large number of children into our schools.

So, the current estimates are that in the age group of 6 to 14, which is the elementary school age group in India, we have more than 96 per cent of kids who are actually enrolled in school. So, in some sense, even though there's a little bit left still – you know the last mile is always very hard - we've got the bulk of our children enrolled and attached to some school. So, really now the next challenge is, what is happening in school and how can we increase the value of every year that children spend in school.

In 2005 there were not very many pieces of data that you could look at to see where we were at, so when the ASER report in 2005 and 2006 came out the country was really quite shocked. We all assumed because our children are in school they must be learning. We had this feeling that things are not as good as they need to be, but we didn't have a really concrete sense of where we were at.

And so the first report, and every subsequent report, shows numbers of the following type; that in Fifth Grade, so after five years of formal school, we have about half our children who are not yet able to read simple stories, simple texts, which you could say that's Second Grade level.

If you look at arithmetic, most of our textbooks are [linked to the] curriculum framework – kids are usually expected to be able to do operations with numbers up to 100, especially addition and subtraction, by the time they finish Second Grade. Our data shows that by Fifth Grade, about half of the children are only getting up to Second Grade level - which means that half our children are two, three or more years behind where they need to be.

The implications of this are many, of course. One of them is that our teaching in our schools is still quite traditional and so typically the way a teacher will teach is if she is a Third Grade teacher, she will teach from the Third Grade textbook. Similarly if you're in the Fifth Grade, they teach from a Fifth Grade textbook. But if you look at the distribution of children's learning levels, it's probably the case that only 10 per cent of the kids in any class are really at the level at which they can benefit from the teaching from the textbook – which means every year you're leaving more and more children behind and not spending time in helping them catch up. So, by the time you reach Fifth Grade you have this situation.

We feel this is one of the biggest challenges. While we want our children to reach curriculum and grade level expectations, as a huge country with very large numbers of children, we might have to spend some time and quite a bit of energy in just helping them catch up and build the basic skills first.

JE: Of course, to do this and to lift those learning outcomes, you need to know where children are at in their learning. That involves assessment. You'll be speaking at the ACER Research Conference later this year about your work in assessment. It's been termed 'citizen [-led] assessment' hasn't it?

RB: The reason we developed the approach the way we did is because we felt that, given the realities of our context, our country, our communities, we needed to take a slightly different path than maybe what is being taken in countries which have much more developed systems of delivery in education and have a different level of parental education and so on.

So, a couple of facts about India, some of which I'm sure you and your listeners know. Of the children who are in school today – especially those in rural areas – 50 per cent of the mothers have probably not had any schooling or very little schooling, which means that the ability at home to help children with learning is very low. You add to that, in our survey we actually look at what reading materials you have at home. Less than 20 per cent of our rural households have any printed material at home, other than religious books (let's say).

... Now, if I go to the school side, we have many types. The government schools are obviously the ones that are listed and we know where they are and so on. But, we have a kind of a growing, mushrooming, private school sector – often little mom and pop-type outfits which are growing in low income areas particularly, where parents feel [if they] pay fees and send their kid to a private school, probably he or she will get a better education. ... Parents often are not in a position to be able to judge what a 'good' school is, in terms of learning.

These private schools are often not on any official list, so if you had to sample schools you would probably get a biased sample which was only really covering government schools. In many states in northern India this sector of private schools is quite large and growing quickly, and [is] very unregulated. You may have 40 or 50 per cent of children going to these kinds of schools.

The third reason is that there is varying attendance in schools on any given day. So, you may have states in which 80 to 85 per cent of kids are to be found in school on any given day, but you also have states where this number may be as low as 50 per cent. So, you could do a school-based sample, [and] even if you had a full list of schools you may miss children because they're absent from school that day. ... So, for a variety of community-based reasons as well as the reasons I just laid out, we felt that a household survey would be the best way to come up with an estimate of where all children are.

JE: Thanks for your time today, before I let you go I wonder if I can get you can look forward 10 years. You're a firm believer that there's no 'silver bullet', if you like, when it comes to improving learning in primary schools, but what's your hope for the future?

RB: I'm also quite a hopeful person. I feel that when you actually acknowledge and see a problem, then there are many different solutions that come up. And one of the things that our ASER Report has done is to actually make a new problem visible to many people.

If the problem is visible, and I find that across the country now more and more people are accepting that you need to have course work on schooling, but you need to also now invest time and energy in figuring out what to do with learning, I think that increasingly more and more solutions will emerge. And that if 10 years later you were to be having this conversation with me, we would hopefully be talking ... I'm sure we'll still have problems, but I hope it will be a completely different and a new kind of problem.

JE: Dr Rukmini Banerji, we welcome you to Australia in August, of course, for the ACER Research Conference. Until then, good luck for the future and thank you for sharing your work with Teacher today.

RB: Thank you so much.

For more information on the research discussed in this podcast and to access other articles and videos - visit www.teachermagazine.com.au, or join our community on social media via Facebook and Twitter.

• Make sure you don't miss a Teacher podcast, subscribe for free by visiting acer.ac/teacheritunes or www.soundcloud.com/teacher-ACER.