Resisting the Seduction of the Global Education Measurement Industry: Notes on the Social Psychology of PISA

Gert BiestaDepartment of Education, Brunel University London

e-mail: gert.biesta@brunel.ac.uk

Author detailsGert BiestaDepartment of EducationBrunel University LondonUxbridgeUB8 3PHUnited Kingdom

Resisting the Seduction of the Global Education Measurement Industry: Notes on the Social Psychology of PISA

Gert BiestaDepartment of Education, Brunel University London

e-mail: gert.biesta@brunel.ac.uk

AbstractThe question I raise in this paper is why measurement systems such as PISA has gained so much power in contemporary education policy and practice. I explore this question from the bottom up by asking what might contribute to the ways in which people invest in systems such as PISA, that is, what are the beliefs, assumptions and desires that lead people to actively lending support to the global education measurement industry or fall for its seduction. I discuss three aspects of what, in the paper, I refer to as the ‘social psychology’ of this dynamics, highlighting the seductive nature of numbers, measurement and comparison, the persistence of technological expectations about education and its workings, and the reference to social justice as a key motivator for wanting to know how systems work and perform. I raise critical questions with regard to each of these aspects and, through this, suggest ways towards a more grown-up response to the difficult question of providing good education for everyone rather than engaging in an unsustainable race for the top.

KeywordsPISA, Measurement, Social Psychology, Education Policy, What Works, Social Justice

Introduction: The Global Upscaling of Education PolicyIn his 1987 book The Struggle for the American Curriculum (Kliebard 1987) Herbert Kliebard presented the forging of the American curriculum as the outcome of a struggle between a range of groups and parties that all had different interests in what the curriculum was supposed to represent and bring about. He thus showed that the curriculum should never just be understood in rational terms – as an answer to Herbert Spencer’s question ‘What knowledge is of most worth?’– but always also as the result of social and political struggle. If Kliebard still worked in a tradition that saw the curriculum mainly as a national project, the contemporary dynamics of curriculum making are increasingly taking place at a transnational and global level (see, for example, Priestley and Biesta 2013). It thus becomes important to ask what the dynamics of the struggle for the ‘world curriculum’ are, also because at the level of official politics, curriculum matters tend to remain the prerogative of national governments.1

One important factor in the recent global ‘upscaling’ of education policy is the role of transnational players (see also Rivzi and Lingard 2010). Whereas some of these players – such as, for example, the World Bank – tend to intervene quite directly in national educational policy, often because policy and money come as

1 This is for example the case in Europe where the European Union has no official say in education policy, although through its (in)famous ‘open method of coordination’ it does exert a significant influence on policy in the European member states – see, for example, Porte and Pochet 2012).

1

one package (see Ball 1998 for an early analysis of these dynamics), the more remarkable impact has been the result of measurement systems that, at least in their stated intention, did not seek to influence or change education policy directly but ‘just’ wanted to provide information upon which national education policy makers could make decisions about the desired shape, form and direction of their education system. Although such systems have been around for a while – IEA, the International Association for the Evaluation of Educational Achievement responsible for such studies as TIMMS, PIRLS and CIVED, emerged in the late 1950s and became a legal entity in 1967 – PISA, The Programme for International Student Assessment, run by the OECD since 2000, has not only become the most visible of these systems but most likely also the most influential (see Hopman, Brinek and Retzl, 2007; Pereira, Kotthoff and Cowen 2011; Pons 2011; d’Agnese 2015).

The OECD, the Organisation for Economic Development and Cooperation, was founded in 1961 with 18 member states plus the USA and Canada, and has over time grown to its current size of 34 member states. Its stated ‘mission’ is “ to promote policies that will improve the economic and social well-being of people around the world” (OECD website; accessed 17 July 2015).2 The following statement provides a comprehensive overview of what the OECD is, how it sees it remit, and what its main areas of concern are.

The OECD provides a forum in which governments can work together to share experiences and seek solutions to common problems. We work with governments to understand what drives economic, social and environmental change. We measure productivity and global flows of trade and investment. We analyse and compare data to predict future trends. We set international standards on a wide range of things, from agriculture and tax to the safety of chemicals.

We also look at issues that directly affect everyone’s daily life, like how much people pay in taxes and social security, and how much leisure time they can take. We compare how different countries’ school systems are readying their young people for modern life, and how different countries’ pension systems will look after their citizens in old age.

Drawing on facts and real-life experience, we recommend policies designed to improve the quality of people's lives. We work with business, through the Business and Industry Advisory Committee to the OECD, and with labour, through the Trade Union Advisory Committee. We have active contacts as well with other civil society

2 The OECD is the successor of the OEEC, the Organisation for European Economic Co-operation, which was established in 1948 with the aim to contribute to administering the Marshall Plan, aimed at the reconstruction of Europe after the Second World War. While the Marshall plan had a strong focus on rebuilding the economy, there was also a clear ideological focus towards rebuilding democracy across Europe (see, for example, Milward 1984).

2

organisations. The common thread of our work is a shared commitment to market economies backed by democratic institutions and focused on the wellbeing of all citizens. Along the way, we also set out to make life harder for the terrorists, tax dodgers, crooked businessmen and others whose actions undermine a fair and open society.(OECD website; accessed 17 July 2015).

As significant issue for the focus of this paper is that the OECD sees itself as a forum where governments can work together to share experience and seek solutions. In this regard the OECD) has no direct power to set policy but works through a logic that is sometimes referred to as ‘soft law’ (see, for example Abbott and Snidal 2000; see also footnote 2).

Referring to soft-law as a ‘logic’ is meant to highlight that, unlike hard law which, amongst other things, can be enforced and comes with a system of sanctions, soft law ultimately depends on ‘investment’ from the bottom up. It depends, in other words, on the willingness of actors to actively invest in the suggestions, communications, interactions, and information generated and orchestrated by such organisations as the OECD. The power of soft law, to put it differently, is predominantly rhetorical, and although rhetoric can be described as the art of persuasion, persuasion is never a one-directional process but depends as much on the skills of the rhetoricians as on the contribution of their ‘audience’ – that is the degree to which audiences wish to invest their beliefs in what is on offer and act on those beliefs. It is, therefore, as much as matter of identification as it is a matter of persuasion.3

In this paper I would like to look at the phenomenon of PISA – which I take to stand for similar phenomena in the global upscaling of education policy – precisely with regard to the question why actors are willing to invest in PISA and everything that comes with it, thus giving it the kind of power it appears to exert on current education policy in many countries around the world. I refer to my particular interest as that of the social psychology of PISA and similar phenomena, as I’m interested in the perceptions, beliefs and actions of those who in some way are at the receiving end of PISA. I want to shed some light, in other words, on the psycho-social dynamics around PISA, particularly because I assume that the power of PISA has a lot to do with expectations about education and its manageability that have been around in education for a long(er) time – expectations that we could also see as desires about education and its manageability (on this see also Meirieu 2007). It is the persistence of some of these expectations and their underlying desires that is interesting as well in explaining the dynamics around PISA and similar systems (see d’Agnese 2015; see also Au 2011; Alexander 2011).

The paper is organised in an introduction, three sections and a conclusion. In the first section to follow I discuss questions about the purpose – or as I prefer: telos

3 To highlight that rhetoric requires active identification of those being addressed, is a key insight of what is known as ‘new rhetoric’ – see, for example, Rutten and Soetaert (2015).

3

(Biesta, 2015a) – of education, which has to do with the wider theme of values and measurement. Next I look at the issue of educational improvement, and particularly the enduring existence of technological expectations about education. Thirdly I raise some questions about the social justice rationale for systems such as PISA, highlighting the tension between beliefs and social structures and arrangements. In the concluding section I return to the idea of the social psychology of PISA and the socio-psychological dynamics surrounding it, suggesting the importance of distinguishing between what Bainbridge and West (2012, 6) have called the syntax and the semantics of education, and making a case for a grown-up response to PISA and similar systems.

Measuring What We Value or Valuing What We Measure?The most visible way in which systems such as PISA are seductive is in that they seem to provide clear, unambiguous and easy to digest and to communicate information about the apparent quality of educational systems, particularly with regard to their ‘performance’ (put in quotation marks because the very idea that the quality of education systems has to do with its performance is already problematic – see below). Quantitative measures that can easily be transformed into league tables and into clear statements about gain and loss between different data-trawls which, in turn, provide a clear basis for policy makers to set targets for ‘improvement’ (see below) – such as gaining a higher place in the league table than apparent competitors, increasing national performance by at least a certain number of points, or articulating the ambition to score ‘at least above average’ – give PISA a simplicity that is absent in complicated discussions about what counts as good education. Yet there are obviously a number of issues here that need exploration and that have been discussed fairly widely in critical literature about PISA and similar systems.

One discussion concerns what I have suggested (Biesta, 2010a) to refer to as the technical validity of the measurements of PISA, which is the question whether what is being measured is an accurate representation of what is supposed to be measured. Next to the technical flaws that can be found in PISA’s methodology – a topic of ongoing discussion (see, for example, Goldstein 2004; Kreiner and Christensen 2014) – there is the in my view more important issue of what I have termed the normative validity of the PISA measurements, which has to do with the question whether what is being measured is focusing on an acceptable and justifiable operationalization of what education is supposed to be and do. One obvious issue here is that PISA only measures performance (or achievement) in a small number of school subjects (mathematics, science and reading) and only focuses on performance at one particular point on the educational career of young people (at the age of 15). While it may be useful to have such information available, even the suggestion that what PISA measures provides a valid indication of the quality of education – both as system and as practice – is difficult to accept. The fact that PISA seems to function in this way and is accepted as a good indication of the quality of education thus indicates, in my view, the seductive power of numbers and of league tables.

While the argument I am putting forward here (and have put forward elsewhere; see most notably Biesta 2010a; 2015a) is not an argument against measurement

4

in education – albeit that there are issues about how strict we define measurement and what it is we seek to measure (see below) – the big question is whether we are measuring what we value about education, that is, whether we seek to establish whether education is indeed doing what we hope and expect from it, or whether we have created or are creating a situation in which we have come to value what is being measured. The power ascribed to PISA suggests that the latter situation is the reality we are currently in (see also d’Agnese 2015). In terms of the ‘social psychology’ that is at stake here, there is not only the seduction of numbers – that is, the idea, that numbers are more accurate and objective than say narrative accounts of education quality – but also, so I wish to suggest, the role of fear, and particularly the fear of being behind and the fear of being left behind. The latter fear explains the belief and investment in league-tables which, just in their form, seem to communicate the idea that some (some education systems, for example) are better than others and that – and the ‘and that’ is key here – those who do not perform in the way those at the top do, are lacking and are lagging behind. The question whether it would be desirable to perform as those who end up at the top of the league table is a question often not asked – having a league table seems to make it ‘obvious’ that the top position is the most desirable position – but should, of course, be asked, not only with regard to the question which measures identify a certain system as ‘top’ but also with regard to the question what it takes to end up at the top of the list and whether this makes it worth striving for the top position.4

I am not claiming any original insights with regard to my concerns about the logic of league tables, but do wish to highlight the ways in which league tables work, the kind of desires they generate and tap into, and how this may blind us from asking the questions that need to be asked first, such as ‘What are the criteria upon which something appears at the top of a league table?,’ ‘What does it take to perform “top” in terms of those criteria?’, and ‘Is that a price we are willing to pay to become “top”?’ All these questions are obviously normative questions that can only be answered in relation to clear articulations of the values we hold about education and thus in relation to views we have about good education.

Two points are important here. The first is that although education can legitimately be expected to have a concern for what is often referred to as ‘academic achievement’ – although the reduction to achievement in only a very small number of subjects is in no way an accurate representation of the rather wide domain of subjects schools should take responsibility for – this is not the only thing that schools ought to do, nor is it the only thing that schools actually do. As I have suggested in several of my publications, schools not only have a

4 The question ‘what it takes’ is important because I believe that to a very large extent it is possible to reconstruct and control education systems so that they end up at the top of measurement systems such as PISA. But to do so may require very strong and problematic interventions – such as taken children away from their family environment if there were a concern that this was impacting negatively on their ‘performance,’ or bribing students so that they work hard, or focusing all the energy on training young people for the PISA tests rather than providing them with a proper education.

5

potential role to play in the qualification of children and young people – that is the ways in which schools provide them with the knowledge, skills and dispositions that qualify them to act in a range of different vocational and social domains – but also play an important role in introducing children and young people in social, cultural, religious, vocational and political traditions, and always also ought to be concerned about the formation of the child and student as person, with the ultimate aim to help children and young people to lead independent, responsible and meaningful lives. In addition to qualification, schools also need to be concerned, therefore, with what we could call socialisation and subjectification or, in wider terms, the formation of the person (see Biesta, 2010a).

This already indicates that when we wish to engage in comparisons about the quality of education systems, we should not only look at achievement and performance in the domain of qualification, but should also be interested in the domains of socialisation and subjectification. The reason I use the word ‘interested’ is because I do not wish to suggest that in response to the recognition of the fact that schools are about more than qualification we should ask for the development of a whole battery of measures to pin down what is being achieved across the whole range of areas of concern. That this is not a good idea is partly because there is the question whether achievement across this range can be adequately assessed and evaluated in terms of measurement and partly because it may well be that the work we do in education in relation to these domains may not always benefit from the demand to visibility. After all, sometimes it is important not to put our students under constant pressure to provide evidence of their progress; there are things that just need to be left with them and where we need to trust and hope that something good may emerge, particularly with regard to the key task of the school to contribute to the ability of children and young people to figure out their own lives and take responsibility for this.

The suggestion that what matters in education is not just qualification but also socialisation and subjectification/the formation of the person not only means that we need to take a broader view than what PISA and similar systems seek to represent – which is not to suggest, in itself, that such information may not be useful, but it is not a valid definition of what education ought to be concerned about – but also means that we need to understand that this broader agenda for education raises some serious questions about how we keep the three domains of educational purpose in a meaningful balance. After all, what we seek to achieve in one domain will not automatically support what we seek to achieve in another domain – a focus on academic performance may often communicate ‘competition’ as a value whereas socialisation into the culture of democracy requires that we highlight the importance of ‘collaboration’ and perhaps even ‘compassion’ and empathy.’ This shows that even if we were to try to measure (or in a broader and more ‘relaxed’ sense: would try to assess) what education systems achieve across this broad spectrum, this cannot be done by just adding up what can be measured/assessed in relation to each of the domains. There is also the really difficult question of what we do with ‘trade offs’ and ‘check and balances.’

6

This relates to the second point to make here, which is that it is also a mistake to think that the quality of education – both as practice and as system – only is to be found in its performance understood at the achievement of students on certain tasks and tests. While this is part of the picture, it is important to bear in mind that the quality of education also legitimately has to do with the particular processes and ways of working, doing and interacting we utilise. The reason for this is well-known but often forgotten and has to do with the fact that, unlike where it concerns interactions in the physical world, the way in which we, as educators, act and interact, is never just an intervention upon an object, but always a meaningful act. The key point here is that students perceive this meaning and are influenced by this as well. The standard example here is the role of punishment in education, which may well be an effective means to intervene in certain behaviour, but while it may be effective to change behaviour we must bear in mind that punishing always also communicates a message about the relationship between educator and child, namely that force or violence may be permissible. This, in a nutshell, shows why in talking about the quality of education, we also need to pay attention to processes and practices in themselves. And the ‘in themselves’ here means that this is not just a matter of effectiveness in bringing about certain results, outcomes or achievements, but always also has to do with the educative quality of our ways of working.

Connecting this to what I have said before, it means that we should never just focus on performance of education systems but should always also ask how this performance is brought about, where the key question is whether this is done in a way that is morally acceptable and educationally meaningful. That some high performing education systems seem to be in countries where there are also large numbers of young people with severe psychological problems, seems to indicate that at least in some cases the price that is being paid to end up at the top of the list is humanely and educatively too high.

The fact that education systems can be managed towards certain ‘outputs’ brings me to the second issue I would like to discuss, which is the belief in the manageability of education.

Making Education Work: The Persistence of Technological Expectations5

Measurement systems such as PISA are often justified with reference to the suggestion that if we have better knowledge about the relative performance of education systems we can then intervene to drive up such performance, particularly in order to produce better ‘outcomes.’ Interestingly the focus of attention often turns to performance on tests that are supposed to measure such outcomes rather than on the outcomes themselves. Although for some this may be a subtle and perhaps negligible difference, or merely a question of technical validity, it is interesting – and also worrying – to notice that systems such as PISA often result in the ambition to drive up test scores, rather than on wider questions about the improvement of education of which such test scores may or may not be a valid indicator. This inevitably raises value-laden questions along the lines discussed in the previous section, that is, both with regard to what

5 See also Biesta (2015b).

7

education is supposed to bring about, and about the moral and educative quality of the ways in which education seeks to bring this about.

Underlying all this is the belief that once we know how the education system performs and, more specifically, once we know how education systems work, it should be possible to intervene in some way in order to make the system perform better – which is often called improving achievement or, particularly in the English context, ‘raising standards.’6 Yet the idea that it should be possible to intervene in some way in order to increase performance hides a set of rather complicated issues. Key in this is a long-standing theme in the discussion about school education, namely whether it is possible to think of such education in terms of a relationship between inputs and outcomes in such a way that the ‘right’ inputs will generate the desired outcomes. This way of thinking about the working of education – to which I have referred as a production metaphor of education or, in Aristotelian terms, the idea of education as poiesis, a process of making things (Aristotle, 1980) – reveals the existence of what might be termed technological expectations about education. This has to do with the idea that education in some way can be made to work as a more or less perfect input-outcome machine (see also Smeyers and Depaepe 2006).

The scholars of German geisteswissenschaftliche Pädagogik – the name under which education established itself as an academic discipline in the German speaking world in the beginning of the 20th century – already indicated the existence of these expectations and also already vehemently rejected that the logic of education could be understood in this way. Whereas part of their rejection was based on the idea that technological expectations are not appropriate in the human domain where interaction is based on meaning and interpretation – and insight they based on the work of Dilthey and his distinction between Naturwissenschaft and Geisteswissenschaft (see Biesta, in press) – the argument against a technological configuration of education was also based on the conviction that education ultimately ought to promote the freedom and independence of the one being educated, and therefore should never treat the student as an object of interventions, but always needs to see students as subjects of their own actions and initiatives. There was, in other words, both a factual and a normative part to the rejection of the technological view of education.

That technological expectations about education are persistent can be seen in two recent developments, the first being the school effectiveness movement and the second the rise of the idea of evidence-based education where the task of research is seen as that of identifying ‘what works’ so that the job of educational practitioners is to implement what research has shown to be ‘working.’ One flaw with the suggestion that it would be possible to identify strategies that ‘work’ is that education never works in an abstract sense but always works for something. While this may be acknowledged in some of the work done under the heading of

6 The phrase ‘raising standards’ has always puzzled me, as a standard is simply an indicator used to asses the performance of something, rather than that it refers to the performance itself. In this regard just raising the standard is actually a rather pointless exercise.

8

‘what works’ – albeit that often a simplistic taken for granted notion of the purpose of education in terms of producing a narrow set of learning outcomes seem to inform the endeavour – the key point I have made in the previous section, namely that education always works and ought to work in relation to a number of different domains and that these domains are always to some degree in tension with each other, what may work in relation to one domain may well have adverse effects with regard to another domain, is seldom ‘on the radar.’

Yet the point I would like to highlight here – in addition to the important normative questions – is about the belief that education in some way functions as what I tend to refer to as a ‘quasi-causal’ system. The ‘quasi’ is relevant here because very few educational researchers and practitioners, when pushed, would express a belief in education as a perfect causal system, but even if this view is rejected, there is still a widespread belief that in some way teaching (and other ‘input’ factors such as curriculum and pedagogy) are ‘interventions’ onto the student that, if well organised, crafted, targeted and executed, may increase the likelihood of bringing about some results on the side of the student. There is, in other words, an underlying belief in an input-processing-output model of education, even if it is granted that education seldom works in a perfect manner.

One useful way to raise questions about this belief is to point to the fact that the kind of deterministic causality that seems to be the reference point for these expectations only happens in physical nature under very specific circumstances. As I have discussed elsewhere in more detail (Biesta 2010b), making use of the vocabulary from systems theory and complexity, deterministic causality is actually only found in closed systems – that is, systems that are closed off from intervening factors in their environment – and where the interactions between elements in such systems do work in the way of linear causation, in most cases having to do with material interaction (such as billiard balls or, in the case of Newton, planets). When we look at the conditions under which systems operate in a strongly causal way, we can begin to see that education systems do not generally tend to function under these conditions. There are three important differences. The first is that education systems tend to be open systems. Students spend only a limited amount of their time in schools – although the amount of time is, of course, a significant chunk of their lives – and are therefore subject to a range of other influences. Secondly, education systems do not operate in terms of physical push and pull, but rather through communication and interpretation. They are, therefore, semiotic systems. Thirdly, unlike billiard balls or planets that are often used as examples of deterministic causality, students, and also their teachers, have an ability to think and this thinking influences their actions. Education is therefore a recursive system where the ways in which the system evolves over time feed back into the system.

One reason why I find it useful to think of education in this way – as an open, semiotic, recursive system – is because it makes it possible to indicate with much more precision why such systems differ from causal deterministic systems. Now one could argue – and in my view this point is correct – that if education systems are open, semiotic recursive systems it is very unlikely that such systems will operate in any predictable way at all. But the second reason why I find it useful

9

to think in this way about education systems and their working is that it gives us a much more precise indication of what needs to be done if we wish to make such system function in more predictable ways. In order to achieve this we need to reduce the openness of the system, we need to limit the semiosis of the system – the freedom of interpretation – and we need to limit the recursivity of the system – the freedom of thinking and acting. In a sense this is what we do in order to make education work in some way. A school building is an example of how we reduce the openness of the system, as is a timetable, a compulsory schooling age, and so on. A curriculum, but also the assessment we link to it, is an attempt to reduce interpretation – by communicating that the school is not about anything and by sanctioning, through assessment, those interpretations that we think are correct, right, appropriate, and so on. In addition we also reduce the recursivity of the system, most notably by influencing the interpretations the actors in the system have about their role – as education only is possible when teachers have a sense of what it means to teach and students have a sense as to what it means to be a student (see also Fenstermacher 1986, particularly on the idea of ‘studenting’).

The third reason why this particular understanding of the dynamics is useful, is because it makes visible in quite an elegant way what price we must pay if we want to ‘push’ education towards predictable performance. The approach shows, in other words, that it is possible to turn education into a deterministic input-output machine, but the price to pay for this is a radical reduction of its openness – the school as a total, 24/7 institution – and a radical control over interpretation and self-interpretation. These characteristics are generally not seen as the characteristics of good, autonomy-enhancing education, but rather provide a description of brainwashing techniques. Viewed in this way, they immediately show that turning education into a perfect machine is technically possible, but morally and educatively undesirable – and most would (hopefully) add that this is therefore an option that should be rejected.

In terms of PISA and similar systems, one thing this discussion shows is the problem with approaching education as a system that produces certain outcomes – and in this regard the notion of ‘performance’ is problematic. The production metaphor is, to put it differently, inappropriate if we seek to understand how education ‘works.’ But what the ideas presented in this section also make visible is that it is possible to turn education into something that produces pre-specified results, but that such a desire always comes at a price. The approach presented in this section provides a very clear way to make visible what price this is and where it is being paid. Key here is that this is not a black and white issue because good education is also not about total openness. What it rather shows is that education operates on a continuum of ‘open’ versus ‘closed’ and that somewhere on that continuum there is a tipping point where education becomes uneducative and also – particularly with regard to systems that put enormous pressure on children and students to ‘perform’ – morally reprehensible. PISA and similar systems not only feed into a whole tradition that sees education through the metaphor of production and control, but also gains support from those who believe that education can be understood in this way and perhaps even more so from those who believe that education ought to be

10

understood in this way. And the question is to what extent systems such as PISA push education beyond the tipping point, perhaps even against the intentions of those who ‘invest’ in it – which brings me to my third and final point.

A social justice rationale?There is one more belief that plays a role in the way in which people invest in PISA and similar systems and thus provide such systems with power. This has to do with an important justification for assessing the relative performance of educational systems, namely the suggestion that it is only when we know how systems perform and how they perform in comparison to each other, that we can work towards equality of opportunity for. I refer to this as the social justice rationale of PISA, and it is a rationale we should be taken seriously, as there is ample evidence to show that education does play a role in the allocation of life-opportunities and that these opportunities are not distributed in an equal manner. But the question is, of course, how education plays this role.

The observation I wish to make here is that the widespread belief in the importance of equal opportunities – which is used as a rationale for measuring the performance of systems in order to see whether they all perform in the same way, in order then to justify interventions so as to make sure that they all perform in the same way – seems to take a rather individualistic outlook on the social ‘returns’ of education. Such an outlook suggests that (1) we should have a system that functions in the same way everywhere and that (2) it is then up to individuals to make use of this system so as to, individually, maximise the return they will receive on their ‘investment’ in education.

The flaw in this way of thinking can perhaps best be demonstrated by an example from England where it was recently argued by a politician that all students should have the opportunity to achieve the grades in secondary school that would allow them entry into ‘top’ universities such as Oxford and Cambridge, and that this politician was going to act to make this possible. What was remarkably absent in this suggestion was a consideration of what would happen if the policy was indeed going to be successful so that all students coming from secondary school would be qualified to gain a place at Oxford or Cambridge. Whereas this was clearly the ambition of the policy, the implication of its success would be that Oxford and Cambridge would jointly have to become as big as the current UK Higher Education system and that all other universities would have to be closed. This example quickly shows the individualistic bias in the discourse of equal opportunities and the absence of an understanding of the structural issues at play.

The more general emphasis on students, their motivation, the qualities they need to posses in order to be successful, and so on – including the rise within the measurement industry to measure and assess all these kind of student characteristics as well – all hide the way in which education very often remains a societal ‘sorting machine’ through which only a very small minority can escape the wider dynamics of social reproduction. This is why I think that we should be careful in just accepting the social justice argument for systems such as PISA, as it too easily results in a situation where individuals carry the blame for much

11

deeper structural problems in society – the whole discourse around ‘student motivation’ is, in my view, one example of what the problem here is. But again the belief in equal opportunity is strong and widespread and tends to be a strong motivator for people to ‘invest’ in systems such as PISA – financially and rhetorically – not only by those at the conservative end of the political spectrum where one may expect an approach to education as a system that ought to reproduce social stability, but also at the progressive end of the political spectrum where education is seen as a vehicle for progress and emancipation.

Concluding comments: On syntax, semantics and grown-up-nessIn this paper I have tried to explore some aspects of what I have termed the ‘social psychology’ of PISA and similar systems, focusing on a number of beliefs that seem to motivate people to ‘invest’ in such systems, that is, to provide active support for what they do and stand for. One is the belief that it is possible to express the quality of education systems in simple quantitative measures and to do so comparatively, that is, on the belief that it is normal and inevitable that not all education systems are equally good. In light of this I have indicated some of the technical and normative problems with regard to assessing the quality of educational systems and have argued that it is quite unlikely that this can be done – and should be done – in the crude and narrow way in which systems such as PISA approach this issue.

A further belief that seems to play a role in the reason why people are inclined to support systems such as PISA has to do with ideas about the functioning of education, and particularly ideas about how education can be managed to work better – where better is often, implicitly or explicitly, defined as ‘more effective.’ I have shown that technological expectations about education are quite persistent, and I have also shown that they are based on assumptions about the workings of education that are at least unhelpful as they seem to contradict some of the basic aspects of education, including the fact that education is an interaction between meaning-making human beings, not an intervention onto abstract objects. The alternative view I have put forward helps to shed light on how we can make education work, immediately also showing that any attempt to make education work – that is, to move it more in the direction of producing pre-determined outcomes – always comes at a price. I have also argued that we should not be wiling to pay any price for making education work according to what we desire from it, partly for educative reasons and partly for wider moral and political reasons.

The third belief that seems to motivate people to invest in systems such as PISA has to do with a social justice rationale, that is, with the idea that all education systems should perform in a similar way so as to create full equality of opportunity. Here I have exposed the individualistic bias of this assumption, and have pointed at the practical limitations of the equal opportunity argument, particularly the fact that if all students would really be successful in seizing their opportunity, the system is most unlikely to cope with this situation, and is also unlikely to accept it.

12

Looking at the discussion around measurement and improvement of education in this way begins to shed some light on the bigger question that motivated this paper: Why do people invest in systems such as PISA? This question, as I have argued, is particularly relevant given that PISA in itself has no power (and may not even claim to have any power) but nonetheless has become powerful because people seem to believe in it, or seem to have beliefs that lead them to support systems such as PISA. The ‘social psychology’ surrounding PISA thus reveals a complex ‘web’ of desires, beliefs and expectations, some of which are explicit but other are perhaps more implicit. Philippe Meirieu, as I have already indicated in the introduction to this paper, refers to the desire to bring education under control as an ‘infantile’ desire (see Meirieu, 2007) as it seems to ‘forget’ a number of important facts about education, such as that education is basically a matter of interaction between humans, not an intervention upon objects, and that education is ultimately aimed at helping children and young people to exist in the world in their own way, and not in the way decided by others. Perhaps there is therefore also something infantile in the desires invested in systems such as PISA – both in the way in which they seem to want to make the workings of education completely transparent, and in the way in which, through comparison and league tables, they wish to create a clear hierarchy, where some are at the top and others at the bottom in such a way that they should be blamed and are being blamed for being there.

One way to go beyond systems such as PISA is therefore to ask what a grown-up response to the complexities of education might look like, one where we do expect that education may make a difference, but where we do not think that through measurement and interventions based upon such measurements, we can make education perfect. This is not to give up on the imperative for social justice, but it is to locate this imperative somewhere else and not immediately ‘fall’ for the logic of systems such as PISA. In trying to characterise this ‘logic’ I have indicated a fundamental difference between those who approach education in terms of production metaphors – including Aristotelian poiesis – and those we reject such metaphors as an adequate and appropriate way to engage with the complexities of education. I am inclined to think that this difference highlights one of the most fundamental set of beliefs about education, that is the opposition between those who belief that education in some way ‘works’ and those who reject the working metaphor.

In a recent publication Bainbridge and West introduce the helpful distinction between approaches that seem to be after the ‘syntax’ of education and those that are interested in the ‘semantics’ of education. Whereas the first seem to look at a level where individuals only appear as elements in a system, a focus on the semantics of education highlights that education ultimately takes place in subjective lives, that is, the lives of individuals for which things mean something and whose key challenge is not about how to be effectively prompted so as to produce outcomes, but to figure out what the meaning of one’s life may be, can be and ought to be. This is perhaps another helpful way to indicate why systems such as PISA, although seductive, ultimately miss a point and, most importantly, tend to miss the point of education.

13

AcknowledgementsAn earlier version of this paper was presented at the annual conference of the European Educational Research Association in Budapest, Hungary, September 2015. I would like to thank Pádraig Hogan for organizing the session in which the paper was presented and several members of the audience for helpful feedback.

ReferencesAu, W. 2011. “Teaching Under the New Taylorism: High-Stakes Testing and the Standardization of the 21st Century Curriculum.” Journal of Curriculum Studies 43(1): 25–45.Alexander, R. 2011. “Evidence, Rhetoric and Collateral Damage: The Problematic Pursuit of ‘World Class’ Standards.’ Cambridge Journal of Education 41(3): 265–286.Abbott, K.W., and D. Snidal 2000. “Hard and Soft Law in International Governance.” International Organization 54: 421-456.Aristotle 1980. The Nicomachean Ethics. Oxford: Oxford University Press.Ball, S.J. 1998. “Big Policies/Small World: An Introduction to International Perspectives in Education Policy.” Comparative Education 34(2), 119-130.Bainbridge, A., and West, L. 2012. “Introduction: Minding a Gap.” In Psychoanalysis and Education, edited by A. Bainbridge and L. West, 1-22. London: Karnac. Biesta, G.J.J. 2010a. Good Education in an Age of Measurement. Boulder, CO: Paradigm Publishers.Biesta, G.J.J. 2010b. “Five Theses on Complexity Reduction and its Politics.” In Complexity Theory and the Politics of Education, edited by D.C. Osberg and G.J.J. Biesta, 5-13. Rotterdam/Boston/Taipei: Sense Publishers.Biesta, G.J.J. 2015a. “What is Education For? On Good Education, Teacher Judgement, and Educational Professionalism.” European Journal of Education 50(1): 75-87. Biesta, G.J.J. 2015b. “On the Two Cultures of Educational Research and How We Might Move Ahead: Reconsidering the Ontology Axiology and Praxeology of Education.” European Educational Research Journal 14(1): 11-22.Biesta, G.J.J. in press. “Teaching, Teacher Education and the Humanities: Reconsidering Education as a ‘Geisteswissenschaft.’” Educational Theory.d’Agnese, V. 2015. “PISA’s Colonialism. Success, Money, and the Eclipse of Education. Power & Education 7(1): 56-72.Fenstermacher, G.D. 1986. “Philosophy of Research on Teaching: Three Aspects.” In Handbook of Research on Teaching (3rd edition), edited by M.C. Wittrock, 37-49. New York: MacMillan; London: Collier Macmillan.Goldstein, H. 2004. “International Comparisons of Student Attainment: Some Issues Arising from the PISA Study.” Assessment in Education, 11(3): 319-330Hopman, S., T.G. Brinek and M. Retzl, eds. 2007. PISA zufolge PISA – PISA according to PISA. Wien/Berlin: Lit Verlag.Kliebard, H. 1987. The struggle for the American curriculum, 1893-1958. London/New York: Routledge.Kreiner, S. and K.B. Christensen 2014. “Analyses of Model Fit and Robustness: A New Look at the PISA Scaling Model Underlying Ranking of Countries According to Reading Literacy.” Psychometrika 79(2): 210-231.

14

Meirieu, P. 2007. Pedagogie: Le devoir de resister. Issy-les-Moulineaux : ESF éditeur.Milward, A.S. 1984. The Reconstruction of Western Europe: 1945-1951. London: Methuen.Pereira M.A., H.G. Kotthoff and R. Cowen, eds 2011. PISA Under Examination. Changing Knowledge Changing Texts and Changing Schools. Rotterdam, Boston, Taipei: Senser Publishers.Pons, X. 2011. “What Do We Really Learn from PISA? The Sociology of its Reception in Three European Countries (2001–2008).” European Journal of Education 46(4): 540–548.Porte, C. de la and P. Pochet 2012.”Why and How (Still) Study the Open Method of Co-ordination (OMC)?” Journal of European Social Policy 22(3): 336-349Priestley, M. and G.J.J. Biesta, eds 2013. Reinventing the Curriculum. New Trends in Curriculum Policy and Practice. London: Bloomsbury.Rizvi, F. and B. Lingard 2010. Globalizing Education Policy. London: Routledge.Rutten, K. and R. Soetaert 2015. “Attitudes Towards Education: Kenneth Burke and New Rhetoric. Studies in Philosophy and Education 34(4): 339-347Smeyers, P. and M. Depaepe, eds 2006. Educational Research: Why ‘What Works’ Doesn’t Work. Dordrecht: Springer.

15