On 6 December the results of PISA 2015 will be published for the 72 participating counties. Kirsty Williams, the Cabinet Secretary for Education will be making a statement in Plenary on Wales’ outcomes.
It is generally accepted that Wales has performed comparatively poorly in previous cycles of PISA and the results have been the catalyst to extensive changes to the education system in Wales. This blog gives some information on the PISA process, how Wales has performed in previous years, its impact on policy and criticisms of the scheme.
About PISA
The OECD’s Programme for International Student Assessment (PISA) is a triennial survey that aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. PISA 2015 was the sixth study to be conducted. Wales first took part in 2006 making these the fourth set of data for Wales.
PISA tests three ‘domains’, reading, maths and science. Each survey has a particular focus, in 2015 this was science. The questions require pupils to demonstrate knowledge, apply learning and solve problems through analysis and logical thinking. PISA avoids emphasis on the assessment of factual recall and information retrieval, and focuses more on knowledge application. PISA does not test how well a student has mastered a school’s specific curriculum. As the OECD’s chief education analyst, Andreas Schleicher says ‘the knowledge economy no longer pays you for what you know... it pays you for what you can do with what you know.’
Schools are selected to be nationally representative. Stratifiers include school size and type (whether independent, maintained, special, middle, secondary, single sex or mixed etc) and by region, whether urban or rural, language and attainment. The identity of schools is not disclosed.
How Wales performed in previous years
The first set of results in 2006 [PDF 62.7KB] showed that Wales was below average in maths and reading. In 2009, Wales’ results [PDF63KB] were lower in all domains compared with 2006. Wales’ score in 2012 [PDF 170KB] had fallen further in Mathematics and Science since 2009. The score had risen in reading, but was still lower than the 2006 result. Wales’ ranking had fallen in all three domains since 2009. The table below shows Wales’ mean scores.
In 2012, Wales had lower scores in each of the domains than its UK counterparts. The table below shows Wales and UK mean scores.
Source: Extracted from Tables B1, C1 and D1, National Foundation for Educational Research, Achievement of 15-year-olds in Wales: PISA 2012 National Report
The Welsh Government response to PISA
While the 2006 results were considered disappointing, they were not seen as the ‘main catalyst to improve standards’ [PDF 655KB]. However, that changed with the publication of the 2009 results. Described by Leighton Andrews, then Minister for Children, Education and Lifelong Learning as ‘extremely disappointing’ he said that PISA was a ‘wake-up call to a complacent system’ [PDF 79KB].
These results coupled with evidence from Estyn that schools were not performing as well as they should led the Welsh Government to begin large-scale school improvement reform. A range of policies were introduced with the aim of improving the school system. These began with Leighton Andrews’ 20 point action plan [PDF 79KB], ultimately leading to the current ‘tripartite reform’ of the curriculum, initial teacher training and teacher development which his successor, Huw Lewis said were ‘triggered by the shock to the system’ the PISA results delivered.
An ambitious target?
After the 2009 results, the Welsh Government set an aim that Wales would be in the top 20 PISA nations by the 2015 cycle. At that time, of the 67 countries taking part, Wales was ranked 38th for reading, 40th for maths and 30th for science.
The 2012 results placed Wales 41st in reading, 43rd in maths and 36th in science (of 68 nations).
In October 2014, the Welsh Government’s long-term vision for education, Qualified for Life [PDF 2.3 MB] amended the PISA target to an ‘ambition’ to achieve scores of 500 in each of reading, mathematics and science in the PISA tests 2021, rather than a position in the rankings relative to other countries. Huw Lewis, then Minister for Education and Skills said that the 500 score would make sense to teachers and young people, whereas the top 20 score was not of use to the wider system.
PISA 2015 results
In December 2013, Huw Lewis, then Minister, said that the PISA 2012 results were ‘not good enough’, but also that ‘we are too early into our school improvement journey to expect to see a step change in performance in this last round of PISA assessment.’
In January 2016, he told the Fourth Assembly’s Children, Young People and Education Committee that he was ‘confident’ there would be ‘an uplift’. Although he qualified this saying that we would have to to regard the results in December as a ‘point along a journey’.
In August this year, the new Cabinet Secretary, Kirsty Williams, suggested that she did not expect an improvement between the 2012 and 2015 results. She said that there was a need to be realistic about how quickly the education reforms can embed within a system and when there will be an effect of the reforms in PISA results.
Criticisms of PISA
Kirsty Williams has confirmed that she will continue to support Wales’ involvement in PISA although she said there was still a ‘job of work to do’ to convince everyone in the sector that PISA matters and benefits Welsh pupils.
While PISA is seen by many as an influential element in policy making, there are those who are less convinced of its benefits. In an open letter to the OECD’s Andreas Schleicher (May 2014), a number of world-wide academics expressed concerns about PISA. These included the dangers of over-testing, the short-term emphasis on measurable results versus longer-term issues like citizenship and caring, and an emphasis on education as a means of job prospects rather than the wider goals of education.
The OECD responded that PISA opens up perspectives to a wider range of policy options that arise from international comparisons and that it assesses a range of learning outcomes and their contexts.
Huw Lewis said ‘whilst always remembering that PISA does not encompass everything about a good education but it certainly does tell us things that are very, very important’.
Some argue that the format of PISA tests and students’ familiarity with the types of question will affect the test results. In February 2014, the Welsh Government announced that PISA-based tests for schools would be made available in Wales. Individual schools, or clusters of schools, were able to take part in individual school-based PISA-type assessments, to benchmark themselves against the highest performing nations in the world. This was a voluntary option. 89 secondary schools took part in the 2015 PISA-Based Tests. Changes to the curriculum, GCSEs and the Welsh Baccalaureate will assess learners on the same types of skills that the PISA assessments test.
Whatever the results show on 6 December it is clear that Wales will continue to feel the impact of PISA for some time.
Article by Sian Hughes, National Assembly for Wales Research Service
