This report was originally published 15 May 2015.

Summary

This Learning Curve presents findings from a pilot study undertaken in 2013 which measured student engagement, wellbeing and quality teaching in a group of NSW government secondary schools. These three aspects of schooling are increasingly seen as essential means of improving learning outcomes for students. The pilot study used Tell Them From Me (TTFM), a survey tool developed by Dr Doug Willms from the University of New Brunswick in Canada, a leading expert in the field of student engagement.

What is student engagement?

Student engagement refers to the extent to which students identify with and value schooling outcomes, and participate in academic and non-academic school activities¹. Measures of engagement may include: affective dimensions such as enjoyment of school and relationships with teachers and peers; cognitive measures such as academic performance or attainment; and behavioural dimensions such as attendance and participation in school activities².

Why is student engagement important?

Research over the last 30 years³ has increasingly shown that student engagement is not only an important outcome in itself, but it is also directly related to academic performance and future outcomes. For example, a 2009 American study of 78,106 students in 160 schools across eight states, found that a one-percentage point increase in a student’s engagement was associated with a six-point increase in reading achievement and an eight-point increase in maths achievement scores⁴.

Other studies of student engagement have shown that increased student engagement has a flow-on effect in regard to educational and occupational success many years into the future. For example, an Australian study which used data from the Childhood Determinants of Adult Health study and a school engagement index, found that each unit of school engagement was independently associated with a ten per cent higher chance of achieving a post-compulsory school education at some point during the next 20 years, including as a mature age student. This was true over and above the influence of family background and personality⁵.

Measuring student engagement

Student engagement can be difficult to measure as there are multiple definitions of engagement, and many potential instruments to measure it. These measures include: student self-report instruments where students respond to set questionnaires; teacher report instruments where teachers respond to questionnaires about students; and observational measures which involve direct observation of the behaviour of individuals, targeted students or classrooms. Of these, student self-report instruments are the most utilised measure and are usually viewed as the most effective way of measuring engagement as they do so across behavioural, emotional and cognitive spectrums⁶.

Tell Them From Me survey

The self-reporting student survey Tell Them From Me has been used extensively in Canada for a number of years. It uses three sets of measures of student engagement: social engagement; institutional engagement; and intellectual engagement. Sub-categories of engagement sit under each of these broader categories, shown in the table below⁷.

The survey also measures drivers of student outcomes and other indicators, including wellbeing⁸.

Key Findings

1. Students become less engaged in the middle year of secondary school (the ‘Year 9 dip’)

The most noticeable pattern in the NSW data is a dip in engagement in Year 9. This is especially noticeable when compared with similar Canadian data (Figure 1). This Year 9 ‘dip’ is present in a number of measures including academic interest, positive behaviour and homework behaviour. It also appears in the proportion of students with a positive sense of belonging – one of the key measures of social engagement.

Social engagement is manifest in students’ willingness to work with others, and their ability to function in and contribute to social institutions. When students feel a sense of belonging at school, their engagement is often enhanced; when they don’t, behavioural problems often follow⁹.

2. Girls engage with school differently from boys

The TTFM pilot survey found that gender appears to make a difference to levels of student engagement. For example, girls are more likely than boys to comply with behavioural expectations at school, but are less likely to feel a positive sense of belonging. Girls are also more likely to report moderate or high levels of anxiety at school than boys, peaking in Year 9.

Some studies of overall school engagement show that girls are more engaged than boys. For example, the findings of an Australian Longitudinal Study of Australian Youth Survey (LSAY) showed that girls in Australian secondary schools are more satisfied than boys on measures of general satisfaction, attitudes towards their teachers and their sense of achievement; and that girls score slightly higher on general happiness measures at school than boys¹⁰.

However, on indicators of social engagement and emotional wellbeing, differences are often found between genders along the lines of those found in the pilot study. . For example, data from the Programme of International Student Assessment (PISA) 2012 reveal that girls are more anxious about maths than boys, even when performing similarly well. Girls are also less likely to engage in mathematics activities such as mathematics competitions¹¹. A 2013 longitudinal study on how school belonging changes over the course of high school also found girls’ school belonging tended to decline, whereas boys’ school belonging remained the same¹². The NSW pilot study did not show girls’ school belonging declined at a more rapid rate than boys, although it was consistently lower than boys'.

The TTFM pilot survey also found gender differences in some, but not all, measures of intellectual engagement. For example, TTFM measures the level of interest a student has in a range of subjects as well as the amount of effort they spend studying for that subject or the degree to which students are trying hard to succeed. Boys are notably more likely to express interest in maths, while girls tend to be more interested in English.

3. SES is strongly related to engagement, but some schools defy the trend

Teacher-student relations

The Tell Them From Me data shows a clear gap, on average, in teacher-student relationships between high and low SES students (Figure 6). The NSW pilot study additionally shows a dip in Year 9 in teacher-student relations among both high SES and low SES students, consistent with the dip observed in other measures such as sense of belonging. In Australia and internationally, there is a clear relationship between the two¹³.

Results from international surveys suggest that students who are in schools where teacher-student relations and the learning climate are poor are more likely to have low levels of engagement with and at school. They are also more likely to arrive late for school, skip classes or days of school, report a weak sense of belonging and hold negative attitudes towards school¹⁴. As students progress through secondary school, they also face increasing complexity. Students consistently say that what most helped them thrive in spite of these challenges was the quality of the relationships they developed with adults in their schools¹⁵.

International studies also show that levels of engagement vary among schools and suggest that the role of the classroom teacher may be as important, or even more important than students’ family background¹⁶.

Expectations for success

Results from the NSW pilot study also show that low SES students are more likely to believe that teachers have low expectations for them, whereas high SES students are more likely to believe teachers hold higher expectations for them. Research has shown that if teachers hold low expectations for students, students achieve less¹⁷.

4. Teaching practices make a difference

The Tell Them From Me survey allows us to ask questions relating to quality instruction across all secondary school years. One of the measures relating to quality instruction is ‘Effective Learning Time’ which captures whether important concepts are taught well, class time is used efficiently, and whether homework and assessments support class objectives. Initial analysis from the NSW pilot study shows that students who agreed that these things happened in their classes, had, on average, higher performance, regardless of SES.

CESE’s first publication, which analysed effective teaching practices using data from PISA and Trends in International Mathematics and Science Study (TIMSS)¹⁸, found that some teaching practices are associated with significantly higher levels of student achievement. This is in line with Hattie’s extensive analysis of ‘what works’ in education¹⁹. CESE's research, following Hattie, confirmed the academic value of:

• mastery learning (the provision of explanations of what it means to ‘master’ the material being taught)
• feedback that is clear, purposeful, meaningful and compatible with students’ prior knowledge
• structuring class sessions to entice, teach, and listen to students, and
• questioning of students.

5. Non-metropolitan students have lower aspirations than metropolitan students, regardless of SES and performance

The NSW data shows a consistent gap in aspirations between provincial and metropolitan students regardless of SES. Where performance data is also available, analysis confirms these findings among high-performing students.

A key feature of education data in Australia is the lower post-school aspirations of regional and remote students when compared with their metropolitan counterparts²⁰. Results from PISA 2012 show that high-performing 15-year-old students who attend schools in non-metropolitan Australia are less likely to aspire to go to university. The gap in aspirations between metropolitan and non-metropolitan high performers is particularly pronounced among boys.

Conclusion

Student engagement is an important measure not only in itself, but also in relation to student outcomes both now and in the future. The TTFM pilot study of student engagement, wellbeing and quality instruction provided a chance to analyse state-wide trends in relation to student engagement.

The main findings from the pilot study are that NSW school students disengage with school during the middle years of high school; that girls ‘do’ school better than boys but have less of a sense of belonging than boys; that SES is strongly related to engagement, but some schools defy the trend; that teaching practices such as the effective use of classroom time make a difference regardless of SES status; and that non-metropolitan students have lower post- school aspirations than metropolitan students, regardless of SES and achievement.