Do innovative school reforms boost pupil performance?

A well-functioning compulsory education system is essential to a country’s performance, but how is publicly funded education best provided? And how can innovative policies boost the performance of the state-maintained school sector, especially when reaching out to disadvantaged pupils?

Pursuing answers to these questions has led some countries to initiate reforms to their school systems, which have led them away from the conventional ‘local’ or ‘community’ school model. For instance, Sweden introduced a voucher system in the early 1990s that enabled pupils to attend privately owned schools (‘free’ schools) funded by public money. Public schools in municipalities with the largest growth in free schools exhibited positive test score gains relative to similar schools in other municipalities, suggesting that the increased competition brought about by free school growth had a positive impact on public-school performance (see Bergstrom and Sandstrom 2005, Björklund et al. 2005, Bohlmark and Lindahl 2015).

Another significant reform has taken place in the US where, since 1992, charter schools have changed the way many pupils are being educated. Charters are typically newly established schools that enjoy greater autonomy than the rest of the public school system. Like free schools, they are privately owned but publicly funded. While initial evidence on charters was mixed, recent studies, which typically compare individuals who ‘win’ a place at an oversubscribed charter via lottery with those who ‘lose’ a place, suggest that charters can improve test scores and impact positively upon medium-term outcomes such as college enrolment (see Abdulkadiroglu et al. 2011, Angrist et al.2013, Fryer and Dobbie 2014). The positive effects are most pronounced for charters located in urban areas, serving less-privileged students and adhering to the ‘no excuses’ model, which stresses behavioural norms and work ethic (see Fryer and Dobbie 2013).

While these reforms may appear to be far-reaching – around 16% of Swedish pupils attend free schools, and 2.5 million US pupils attend charters – they pale in comparison to the reforms that have recently been taking place in England.

Academy schools

Introduced in the 2002/03 school year by the Labour government of the time, academy schools enjoy more autonomy than traditional community schools. The original (‘sponsored’) academies are able to operate outside of local authority control and are managed by a team of independent co-sponsors who delegate management of the school to a largely self-appointed board of governors responsible for hiring staff, performance management, the curriculum, and length of the school day. Unlike most charter schools, academies are typically conversions from pre-existing schools.¹

The initial programme was relatively modest in size and aimed at improving performance in failing schools. However, the growth of academies following the election of the coalition government in 2010, which quickly widened the remit of the programme with the Academies Act, has been striking. The Academies Act enabled a wider range of schools to gain academy status; notably, it allowed high-performing schools to convert to an academy without entering into a sponsoring relationship. The result is that 60% of England’s secondary schools now have academy status.

While it is too early to assess the impact of the academies programme as a whole, Eyles and Machin (2015) have looked at the effect on test scores of attending one of the 106 sponsored academies that opened between the 2003 and 2009 academic years. As previously noted, the schools that became academies under the Labour government were somewhat atypical. Naïve comparisons of test scores of pupils attending these schools with those attending other state-maintained schools can lead to biased estimates of the causal effect of academy attendance on test scores for two reasons. First, schools that converted in this period were poorly performing schools deemed to be failing. Second, the decision to attend an academy school is non-random, meaning that unobservable characteristics correlated with both academic achievement and the decision to attend an academy may confound the causal effect we wish to estimate.

To avoid these pitfalls, outcomes were compared for pupils who were enrolled in an academy school prior to conversion, but who took their exams in the school post conversion, with pupils attending schools that later became academies under the same Labour government programme. Focusing on students enrolled in the academy before conversion ameliorates concerns related to non-random school choice. Similarly, studying pupils that sit their exams at a school that later becomes an academy as a control group nets out unobservable confounders (e.g. ‘ethos’ to become an academy) at the school level.

The findings are that academy attendance can lead to sizeable gains in pupil achievement. Figure 1 shows event study estimates of the effect of academy attendance on standardised tests (Key Stage 4, KS4) taken in the final year of compulsory schooling. The y-axis denotes standard deviation changes.

In Figure 1, the academy conversion effect is allowed to vary by the number of years since the academy converted, with the year of conversion denoted as year c.

• Therefore, pupils who sit their exams in an academy four years post conversion exhibit test scores that are on average around 0.2 standard deviations higher than pupils in similar schools that become academies later.

Figure 2 splits the academy conversion effect by predecessor school status, that is, according to whether pupils attend a community school that converts to academy status or whether they attend any other type of state-maintained school that converts. The contrast is important because it is community school conversions that led to the greatest gains in autonomy.

Figure 2. Effect of academy attendance on standardised tests, split by predecessor school status

As can be seen, it is pupils who attend community schools that become academies before they sit their final exams who are driving the overall results.

• This suggests that large gains in autonomy enable schools to boost performance.

Autonomy

The question of how autonomy is used to improve performance is an important one. The Department for Education’s (2014) survey of academy schools sheds light on the question of ‘Do Academies Make Use of Their Autonomy?’ According to the survey, the most prominent changes are those in school leadership, the procurement of services that were previously provided by the local authority, and curriculum changes. Our research highlights the role of head teacher turnover. In the initial year of becoming an academy, schools are 60% more likely to change their head teacher than schools that have yet to convert.

The future

Given that academy schools now dominate the educational landscape in England, it is natural to ask to what extent the above effects can be meaningfully extrapolated to the surge of academies that opened post-2010. Although the coalition government continued with the sponsored academies programme, most of the growth of the academy sector (almost 80% in the preceding five years) has come from high-performing schools, known as ‘converters’, voluntarily converting to academy status without entering into a sponsoring relationship.

Along with Olmo Silva (Eyles et al. 2015), we document significant differences between schools that became sponsored academies and those that have opened as converters. We show that post-2010 converter academies are very different in terms of pre-conversion characteristics when compared to schools that become sponsored academies. Prior to conversion, converter academies tend to have high levels of attainment and few pupils eligible for free school meals. On the other hand, those becoming sponsored academies are drawn from the bottom of the attainment distribution and serve many disadvantaged pupils.

Whether the differences between converter academies and sponsored academies are manifested in differences in performance effects is unclear; similarly, the question of the extent to which a successful, but small-scale, intervention can be rolled-out with equal success on an almost national scale remains open.

Conclusion

The academies programme can be seen as the latest in a series of attempts to find innovative schooling strategies that boost the performance of state-maintained schools. Like Sweden and the US, the English education system is moving beyond the traditional state-maintained school. But compared with reforms in those countries, it is doing so on a scale and at a pace that are unprecedented. Whether the early successes of sponsored academies translate into success of the wider programme remains in doubt. What is clear is that the academies programme, and some aspects of its mode of operation, can offer important lessons regarding the optimal provision of state-maintained education.

References

Abdulkadiroglu A, J Angrist, S Dynarski, T Kane and P Pathak P (2011), “Accountability and Flexibility in Public Schools: Evidence From Boston’s Charters and Pilots”, Quarterly Journal of Economics, 126, 699-748.

Abdulkadiroglu A, J Angrist, P Hull and P Pathak (2014), “Charters without Lotteries: Testing Takeovers in New Orleans and Boston”, NBER Working Paper 20792.

Angrist J, S Cohodes, S Dynarski, P Pathak and C Walters (2013), “Stand and Deliver: Effects of Boston’s Charter High Schools on College Preparation, Entry and Choice”, NBER Working Paper 19275.

Bergstrom F and M Sandstrom (2005), “School Vouchers in Practice: Competition Will Not Hurt You”, Journal of Public Economics, 89, 351–80.

Bjorklund A, M Clark, P Edin, P Frederiksson and A Krueger (2005), The Market Comes to Education in Sweden: an Evaluation of Sweden’s Surprising School Reforms , Russell Sage Foundation, New York.

Bolhmark A and M Lindahl (2015), “Independent Schools and Long-run Educational Outcomes: Evidence from Sweden’s Large-scale Voucher Reform”, Economica, 82, 508–551

Department for Education (2014), “Do Academies Make Use of Their Autonomy?”https://www.gov.uk/government/publications/do-academies-make-use-of-their-autonomy

Dobbie W and R Fryer (2013), “Getting Beneath the Veil of Effective Schools: Evidence from New York City”, American Economic Journal: Applied Economics, 5, 58–75.

Dobbie W and R Fryer (2014), “The Medium-Term Impacts of High-Achieving Charter Schools”,Journal of Political Economy, forthcoming

Eyles A and S Machin (2015), “The Introduction of Academy Schools to England’s Education”, Centre for Economic Performance, Discussion Paper No 1368

Eyles A, S Machin and O Silva (2015), “Academies 2: The New Batch”, Centre for Economic Performance, Discussion Paper No 1370.

Fryer R (2014), “Injecting Charter School Best Practices into Traditional Public Schools: Evidence from Field Experiments”, Quarterly Journal of Economics, 129, 1355–1407.