Posts Tagged ‘PISA’

On 21 January, I attended the launch webinar of DEFI (the Digital Education Futures Initiative), an initiative of the University of Cambridge, which seeks to work ‘with partners in industry, policy and practice to explore the field of possibilities that digital technology opens up for education’. The opening keynote speaker was Andrea Schleicher, head of education at the OECD. The OECD’s vision of the future of education is outlined in Schleicher’s book, ‘World Class: How to Build a 21st-Century School System’, freely available from the OECD, but his presentation for DEFI offers a relatively short summary. A recording is available here, and this post will take a closer look at some of the things he had to say.

Schleicher is a statistician and the coordinator of the OECD’s PISA programme. Along with other international organisations, such as the World Economic Forum and the World Bank (see my post here), the OECD promotes the global economization and corporatization of education, ‘based on the [human capital] view that developing work skills is the primary purpose of schooling’ (Spring, 2015: 14). In other words, the main proper function of education is seen to be meeting the needs of global corporate interests. In the early days of the COVID-19 pandemic, with the impact of school closures becoming very visible, Schleicher expressed concern about the disruption to human capital development, but thought it was ‘a great moment’: ‘the current wave of school closures offers an opportunity for experimentation and for envisioning new models of education’. Every cloud has a silver lining, and the pandemic has been a godsend for private companies selling digital learning (see my post about this here) and for those who want to reimagine education in a more corporate way.

Schleicher’s presentation for DEFI was a good opportunity to look again at the way in which organisations like the OECD are shaping educational discourse (see my post about the EdTech imaginary and ELT).

He begins by suggesting that, as a result of the development of digital technology (Google, YouTube, etc.) literacy is ‘no longer just about extracting knowledge’. PISA reading scores, he points out, have remained more or less static since 2000, despite the fact that we have invested (globally) more than 15% extra per student in this time. Only 9% of all 15-year-old students in the industrialised world can distinguish between fact and opinion.

To begin with, one might argue about the reliability and validity of the PISA reading scores (Berliner, 2020). One might also argue, as did a collection of 80 education experts in a letter to the Guardian, that the scores themselves are responsible for damaging global education, raising further questions about their validity. One might argue that the increased investment was spent in the wrong way (e.g. on hardware and software, rather than teacher training, for example), because the advice of organisations like OECD has been uncritically followed. And the statistic about critical reading skills is fairly meaningless unless it is compared to comparable metrics over a long time span: there is no reason to believe that susceptibility to fake news is any more of a problem now than it was, say, one hundred years ago. Nor is there any reason to believe that education can solve the fake-news problem (see my post about fake news and critical thinking here). These are more than just quibbles, but the main point that Schleicher is making is that education needs to change.

Schleicher next presents a graph which is designed to show that the amount of time that students spend studying correlates poorly with the amount they learn. His interest is in the (lack of) productivity of educational activities in some contexts. He goes on to argue that there is greater productivity in educational activities when learners have a growth mindset, implying (but not stating) that mindset interventions in schools would lead to a more productive educational environment.

Schleicher appears to confuse what students learn with the things they have learnt that have been measured by PISA. The two are obviously rather different, since PISA is only interested in a relatively small subset of the possible learning outcomes of schooling. His argument for growth mindset interventions hinges on the assumption that such interventions will lead to gains in reading scores. However, his graph demonstrates a correlation between growth mindset and reading scores, not a causal relationship. A causal relationship has not been clearly and empirically demonstrated (see my post about growth mindsets here) and recent work by Carol Dweck and her associates (e.g. Yeager et al., 2016), as well as other researchers (e.g. McPartlan et al, 2020), indicates that the relationship between gains in learning outcomes and mindset interventions is extremely complex.

Schleicher then turns to digitalisation and briefly discusses the positive and negative affordances of technology. He eulogizes platform companies before showing a slide designed to demonstrate that (in the workplace) there is a strong correlation between ICT use and learning. He concludes: ‘the digital world of learning is a hugely empowering world of learning’.

A brief paraphrase of this very disingenuous part of the presentation would be: technology can be good and bad, but I’ll only focus on the former. The discourse appears balanced, but it is anything but.

During the segment, Schleicher argues that technology is empowering, and gives the examples of ‘the most successful companies these days, they’re not created by a big industry, they’re created by a big idea’. This is plainly counterfactual. In the case of Alphabet and Facebook, profits did not follow from a ‘big idea’: the ideas changed as the companies evolved.

Schleicher then sketches a picture of an unpredictable future (pandemics, climate change, AI, cyber wars, etc.) as a way of framing the importance of being open (and resilient) to different futures and how we respond to them. He offers two different kinds of response: maintenance of the status quo, or ‘outsourcing’ of education. The pandemic, he suggests, has made more countries aware that the latter is the way forward.

In his discussion of the maintenance of the status quo, Schleicher talks about the maintenance of educational monopolies. By this, he must be referring to state monopolies on education: this is a favoured way of neoliberals of referring to state-sponsored education. But the extent to which, in 2021 in many OECD countries, the state has any kind of monopoly of education, is very open to debate. Privatization is advancing fast. Even in 2015, the World Education Forum’s ‘Final Report’ wrote that ‘the scale of engagement of nonstate actors at all levels of education is growing and becoming more diversified’. Schleicher goes on to talk about ‘large, bureaucratic school systems’, suggesting that such systems cannot be sufficiently agile, adaptive or responsive. ‘We should ask this question,’ he says, but his own answer to it is totally transparent: ‘changing education can be like moving graveyards’ is the title of the next slide. Education needs to be more like the health sector, he claims, which has been able to develop a COVID vaccine in such a short period of time. We need an education industry that underpins change in the same way as the health industry underpins vaccine development. In case his message isn’t yet clear enough, I’ll spell it out: education needs to be privatized still further.

Schleicher then turns to the ways in which he feels that digital technology can enhance learning. These include the use of AR, VR and AI. Technology, he says, can make learning so much more personalized: ‘the computer can study how you study, and then adapt learning so that it is much more granular, so much more adaptive, so much more responsive to your learning style’. He moves on to the field of assessment, again singing the praises of technology in the ways that it can offer new modes of assessment and ‘increase the reliability of machine rating for essays’. Through technology, we can ‘reunite learning and assessment’. Moving on to learning analytics, he briefly mentions privacy issues, before enthusing at greater length about the benefits of analytics.

Learning styles? Really? The reliability of machine scoring of essays? How reliable exactly? Data privacy as an area worth only a passing mention? The use of sensors to measure learners’ responses to learning experiences? Any pretence of balance appears now to have been shed. This is in-your-face sales talk.

Next up is a graph which purports to show the number of teachers in OECD countries who use technology for learners’ project work. This is followed by another graph showing the number of teachers who have participated in face-to-face and online CPD. The point of this is to argue that online CPD needs to become more common.

I couldn’t understand what point he was trying to make with the first graph. For the second, it is surely the quality of the CPD, rather than the channel, that matters.

Schleicher then turns to two further possible responses of education to unpredictable futures: ‘schools as learning hubs’ and ‘learn-as-you-go’. In the latter, digital infrastructure replaces physical infrastructure. Neither is explored in any detail. The main point appears to be that we should consider these possibilities, weighing up as we do so the risks and the opportunities (see slide below).

Useful ways to frame questions about the future of education, no doubt, but Schleicher is operating with a set of assumptions about the purpose of education, which he chooses not to explore. His fundamental assumption – that the primary purpose of education is to develop human capital in and for the global economy – is not one that I would share. However, if you do take that view, then privatization, economization, digitalization and the training of social-emotional competences are all reasonable corollaries, and the big question about the future concerns how to go about this in a more efficient way.

Schleicher’s (and the OECD’s) views are very much in accord with the libertarian values of the right-wing philanthro-capitalist foundations of the United States (the Gates Foundation, the Broad Foundation and so on), funded by Silicon Valley and hedge-fund managers. It is to the US that we can trace the spread and promotion of these ideas, but it is also, perhaps, to the US that we can now turn in search of hope for an alternative educational future. The privatization / disruption / reform movement in the US has stalled in recent years, as it has become clear that it failed to deliver on its promise of improved learning. The resistance to privatized and digitalized education is chronicled in Diane Ravitch’s latest book, ‘Slaying Goliath’ (2020). School closures during the pandemic may have been ‘a great moment’ for Schleicher, but for most of us, they have underscored the importance of face-to-face free public schooling. Now, with the electoral victory of Joe Biden and the appointment of a new US Secretary for Education (still to be confirmed), we are likely to see, for the first time in decades, an education policy that is firmly committed to public schools. The US is by far the largest contributor to the budget of the OECD – more than twice any other nation. Perhaps a rethink of the OECD’s educational policies will soon be in order?

References

Berliner D.C. (2020) The Implications of Understanding That PISA Is Simply Another Standardized Achievement Test. In Fan G., Popkewitz T. (Eds.) Handbook of Education Policy Studies. Springer, Singapore. https://doi.org/10.1007/978-981-13-8343-4_13

McPartlan, P., Solanki, S., Xu, D. & Sato, B. (2020) Testing Basic Assumptions Reveals When (Not) to Expect Mindset and Belonging Interventions to Succeed. AERA Open, 6 (4): 1 – 16 https://journals.sagepub.com/doi/pdf/10.1177/2332858420966994

Ravitch, D. (2020) Slaying Goliath: The Passionate Resistance to Privatization and the Fight to Save America’s Public School. New York: Vintage Books

Schleicher, A. (2018) World Class: How to Build a 21st-Century School System. Paris: OECD Publishing https://www.oecd.org/education/world-class-9789264300002-en.htm

Spring, J. (2015) Globalization of Education 2nd Edition. New York: Routledge

Yeager, D. S., et al. (2016) Using design thinking to improve psychological interventions: The case of the growth mindset during the transition to high school. Journal of Educational Psychology, 108(3), 374–391. https://doi.org/10.1037/edu0000098

Whenever a word is frequently used in arguments trying to persuade people to believe some opinion or other, our mental twists and turns to make the opinion plausible involve shifting from meaning to meaning without realizing it. This has happened to creativity on a grand scale.’ (Perry, L. (1987). The Educational Value of Creativity. Journal of Art and Design Education 6 (3) ) quoted in Pugliese, 2010: 8)

If you take a look at the word ‘creativity’ in Google’s Ngram viewer, you’ll notice that use of the word really took off around 1950, the year when J. P. Guilford published an article entitled ‘Creativity’ in American Psychologist. Guilford’s background was in the US military. His research was part funded by the US Navy and his subjects were US Air Force personnel. His interest was in the classification and training of military recruits.

With the Cold War between the US and the Soviet Union hotting up, Guilford’s interests increasingly became a matter of national security. In 1954, Carl Rogers, argued that education tended to turn out ‘conformists […] rather than freely creative and original thinkers’ (Rogers, 1954: 249) and that there was a ‘desperate need’ for the latter. He warned that ‘international annihilation will be the price we pay for a lack of creativity’. When Sputnik scared the shit out of the American military, creativity became more important still. It ‘could no longer be left to the chance occurrence of genius; neither could it be left in the realm of the wholly mysterious and the untouchable. Men had to be able to do something about it; creativity had to be a property in many men; it had to be something identifiable; it had to be subject to efforts to gain more of it’ (Razik, 1967).

It wasn’t long before creativity moved beyond purely military concerns to more generally corporate ones. Creativity became one of the motors driving the economy. This process is tracked in a fascinating article by Steven Shapin (2020), who quotes the director of research at General Electric as saying in 1959: ‘I think we can agree at once that we are all in favour of creativity’. Since then, the idea of creativity has rarely looked back.

By the end of the century, the UK government had set up a National Advisory Committee on Creative and Cultural Education, chaired by Ken Robinson. Creativity was seen as a ‘vital investment in human capital for the twenty-first century’ (National Advisory Committee on Creative and Cultural Education, 1999). Quoting the prime minister, Blair, the report stated that ‘our aim must be to create a nation where the creative talents of all the people are used to build a true enterprise economy for the twenty-first century — where we compete on brains, not brawn’.

A few years later, in the US, the Partnership for 21st Century Skills was founded, supported primarily by the corporate community with companies like AOL Time Warner, Apple and Microsoft providing financial backing. The ‘21st century skills’ required by global employers (or more specifically that global employers wanted national governments to pay for) could be catchily boiled down to the 4Cs – communication, collaboration, critical reflection and creativity. What was meant by creativity is made clearer in Trilling and Fadel’s bible of 21st century skills (2009: 56):

Given the 21st century demands to continuously innovate new services, better processes, and improved products for the world’s global economy, and for the creative knowledge required in more and more of the world’s better-paying jobs, it should come as no surprise that creativity and innovation are very high on the list of 21st century skills. In fact, many believe that our current Knowledge Age is quickly giving way to an Innovation Age, where the ability to solve problems in new ways (like the greening of energy use), to invent new technologies (like bio- and nanotechnology) or create the new killer application of existing technologies (like efficient and affordable electric cars and solar panels, or even to discover new branches of knowledge and invent entirely new industries, will all be highly prized.

In this line of thought, creativity is blurred with ‘innovation skills’ and inextricably linked to business (and employee) performance. It involves creative thinking techniques (such as brainstorming), the ability to work collaboratively and creatively with others, openness to new ideas and perspectives, originality and inventiveness in work, and understanding real-world limits to adopting new ideas (Trilling & Fadel, 2009: 59). Although never defined very precisely, the purpose of creativity in education (as well as 21st skills more generally) is crystal-clear:

A fundamental role of education is to equip students with the competences they need – and will need – in order to succeed in society. Creative thinking is a necessary competence for today’s young people to develop. It can help them adapt to a constantly and rapidly changing world, and one that demands flexible workers equipped with ‘21st century’ skills that go beyond core literacy and numeracy. After all, children today will likely be employed in sectors or roles that do not yet exist’. (OECD, 2019: 6)

Creativity, then, has become first and foremost about the development of human capital and, by extension, the health of financial capital. In the World Economic Forum’s list of ‘5 Things You Need To Know About Creativity’, #1 on the list is ‘Creativity is good for the Economy’, #4 is ‘It’s important for leadership’, and #5 is ‘It’s crucial for the future of work’. For the World Bank, creativity is more or less synonymous with entrepreneurship (World Bank, 2010).

Given the importance that the OECD attaches to creativity, it was inevitable that they should seek to measure it. The next round of PISA tests, postponed to 2022 because of Covid-19, will incorporate evaluation of creative thinking. As the OECD itself recognises (OECD, 2019), this will be no easy task. There are problems in establishing a valid and agreed construct of creative thinking / creativity. There is debate about the extent to which creative thinking is domain-specific (does creative thinking in science different to creative thinking in the arts?). Previous attempts to measure creativity have been less than satisfactory. But none of this will stop the OECD juggernaut, and shortcomings in the first round of evaluations can be taken, creatively, as ‘an opportunity to learn’ (Trilling & Fadel, 2009: 59). There will be a washback effect, but this is all to the good in the eyes of the OECD. One of their most significant objectives in measuring creativity is to encourage ‘changes in education policies and pedagogies’ (OECD, 2019: 5): ‘the results will also encourage a wider societal debate on both the importance and methods of supporting this crucial competence through education’. To a large extent, it is an agenda-setting exercise.

Creativity’s most well-known cheerleader is the late Ken Robinson. His advocacy of creativity in education for the purposes of developing human capital is clear from his contribution to the National Advisory Committee on Creative and Cultural Education (1999) report. Subsequently, he changed his tune a little, and was careful to expand on his reasons for promoting creativity. Creativity, for Robinson, became something of broader importance than it was for those with a 21st century skills agenda. ‘There’s a lot of talk these days about 21st century skills,’ he said, ‘and I go along with a great deal of it, my only reservation about the idea of 21st century skills is that when they’re listed, they often include skills that were relevant at any time, in any century, it’s not that they’re a completely brand new set of things that people need to learn now that they didn’t have to learn before, but the context is very different’. In another interview, when pushed about creativity as an ‘essential 21st century skill’ – ‘why is creativity especially important right now?’ – Robinson again avoided going too far down the 21st century skills path. In reply, he offered a number of reasons, but the economy was the last that he mentioned. Human capital mattered to Robinson (‘any conversation about education that doesn’t take account of the economy is really, in some respects, detached and naïve from the world that we live in’ he said in another interview), but he made a point of downplaying it. As a highly accomplished rhetorician, Robinson knew how to tailor his messages for his audiences. His success and fame were due in large part to his ability to craft messages for everybody, and his readiness to allow the significance of creativity to shift from one meaning to another played, in my view, a large role in his appeal.

In ELT, there is no doubt that creativity is, as Maley and Kiss (2018: v) put it, ‘a fashionable concept’. In addition to Maley & Kiss’s ‘Creativity and English Language Teaching’ (2018), recent publications have included ‘The Creative Teacher’s Compendium’ (Clare & Marsh, 2020), ‘Hacking Creativity’ (Peachey, 2019), ‘50 Creative Activities’ (Maley, 2018), ‘Creativity in English Language Teaching’ (Xerri & Vassallo, 2016), ‘Creativity in the English language classroom’ (Maley & Peachey, 2015) and ‘Being Creative’ (Pugliese, 2010). In addition, there have been chapters on creativity in recent books about 21st century skills in ELT, such as ‘21st Century Skills in the ELT Classroom – A Guide for Teachers’ (Graham, 2020) and ‘English for 21st Century Skills’ (Mavridi & Xerri, 2020). Robinson is regularly cited.

What is striking about all these publications is that the kind of creativity that is promoted has virtually nothing to do with the kind of creativity that has been discussed in the first part of this article. The notion of language learners as human capital is absent, the purpose of creativity teaching is entirely different, and the creativity of the 4 Cs of 21st century skills has transformed into something else altogether. Even in the edited collections with ‘21st century skills’ in their titles, creativity has little or nothing to do with the creativity of the OECD. In most of these titles, ‘21st century skills’ are not mentioned at all, or only briefly in passing. In the 330 pages of Maley and Kiss (2018), for example, there are only three mentions of the term.

Instead, we have something that is not very ‘21st century’ at all. Definitions of creativity in these ELT books are very broad, and acknowledge the problems in even providing a definition. Recognising these difficulties, Nik Peachey (2019: iv) doesn’t even attempt to provide a definition. Instead, he offers a selection of ideas and activities which have something to do with the concept. Maley (in Xerri & Vassallo, 2016: 10) takes a similar approach, offering a list of attributes, including things like newness / originality, immediacy, wonder, curiosity / play, inspiration, finding / making connections, unpredictability, relevance and flow. Pugliese (2010: 114) asks teachers how they interpret creativity and this list includes problem-solving, the teacher’s aesthetic drive, a combination of the previous two, and a search for Rogerian self-actualization. Both writers focus heavily on the teacher’s own commitment to creativity. For Pugliese (2010: 12), ‘creativity is about wanting to be creative’.

In practice, the classroom ideas that are on offer can usually be put into one or more of the following categories:

  • Activities that involve the arts: drama, stories, music, song, chants, poetry and dance, etc. Maley (2018) is especially interested in poetry, and Pugliese (2010) explores music and the visual arts in more detail.
  • Activities that involves the learners in personalized self-expression, with emotional responses prioritized.
  • Activities which are in some way exploratory, unpredictable or ‘different’. The work of John Fanselow (e.g. 1987) is an important inspiration here.

The overall result is a relabelled mash-up of ideas that have been around for some time: exploitation of literature, music and art; humanistic approaches inspired by Stevick, Rinvolucri and others; a sprinkling of positive psychology; and, sometimes, suggestions for using digital technology to facilitate creative expression of some kind. I hope I am not being unfair if I suggest that the problem of definition arises because the 21st century label of creativity has been stuck on bottles of vintage wine.

Most of these writers seem content to ignore 21st century OECD-style creativity, to pretend that it is not the driver of the ‘fashionable concept’ they are writing about. However, the reason for this silence surfaces from time to time: most of these ELT writers disapprove of, even dislike, the OECD version of creativity. Here, for example, is Chris Kennedy in the foreword to Maley & Peachey (2015: 2):

It is worrying in our market-driven world that […] certain concepts, and the words used to express them, lose their value through over-use or ill-definition. […] The danger is that such terms may be hijacked by public bodies and private institutions which employ them as convenient but opaque policy pegs on which practitioners, including educators, are expected to hang their approaches and behaviours. ‘Creativity’ is one such term, and UK government reports on the subject in the last few years show the concept of creativity being used to support a particular instrumental political view as a means of promoting the economy, rather than as a focus for developing individual skills and talents.’

And here’s David Nunan (in Mavridi & Xerri, 2020: 6) dishing out some vitriol:

I have been unable to find any evidence that the ability to solve such [problem-solving 21st century-style creativity tasks] transfers to the ability to solve such problems in real life. This has not stopped some people building their careers out of the concept and amassing considerable compensation in the process. Robinson even garnered a knighthood’.

This is all rather strange. It is creativity as a 21st century skill that has made the topic a ‘fashionable concept’. The ideas of Alan Maley et al about creativity become more plausible because the meaning of the key term can shift around. His book (with Nik Peachey) was commissioned by the British Council, an organisation that is profoundly committed to the idea that 21st century skills, including creativity, are essential for young people ‘to be fully prepared for life and work in a global economy’. In this light, Maley & Peachey (2015), which kicks off with a Maley poem before the Chris Kennedy foreword, may almost be seen as a subversive hijacking, a détournement of British Council discourse. But détourneurs can be détourné in their turn …

Maley’s co-author, Tamas Kiss, on ‘Creativity and English Language Teaching’, a book which so strenuously avoided the discourse of 21st century creativity, chose to discuss this work in the following way for a webpage for his university:

‘Dr Kiss explained that creativity has been the subject of investigation in several fields including psychology and business, as well as language teaching, and is one of the ‘core skills’ of most 21st century educational frameworks:

“People have realised that traditional knowledge transfer systems are not necessarily preparing students for 21st century jobs,” said Tamas, “New educational frameworks, for example those developed by the Council of Europe, emphasize cross-cultural communication, problem-solving, and creativity.”’

The university in question is Xi’an Jiaotong University, to the west of Shanghai. In the same year as the publication of the book that Kiss co-authored with Maley, Barbara Schulte gave a conference presentation entitled ‘Appropriating or hijacking creativity? Educational reform and creative learning in China’ (Schulte, 2018). She noted the increasing importance accorded to creativity in China’s educational reforms, the country’s increasing engagement with OECD benchmarks, and the way in which creative approaches ‘originally intended to empower learners are turned into their exact opposites, constraining learners’ spaces even more than with conventional approaches’.

Creativity is a classic weasel word. Its use should come accompanied with a hazard warning.

References

Clare, A. & Marsh, A. (2020). The Creative Teacher’s Compendium. Teddington, Middx.: Pavilion

Fanselow, J. (1987). Breaking Rules. Harlow: Longman

Graham, C. (Ed.) (2020). 21st Century Skills in the ELT Classroom – A Guide for Teachers. Reading: Granet

Guilford, J. P. (1950). Creativity. American Psychologist, 5 (9): pp.444–454

Maley, A. (2018). Alan Maley’s 50 Creative Activities. Cambridge: Cambridge University Press

Maley, A. & Kiss, T. (2018). Creativity and English Language Teaching. London: Palgrave Macmillan

Maley, A. & Peachey, N. (Eds.) (2015). Creativity in the English language classroom. London: British Council

Mavridi, S. & Xerri, D. (Eds.) English for 21st Century Skills. Newbury, Berks.: Express Publishing

National Advisory Committee on Creative and Cultural Education. (1999). All Our Futures: Creativity, Culture and Education http://sirkenrobinson.com/pdf/allourfutures.pdf

OECD (2019). PISA 2021 Creative Thinking Framework (Third Draft). Paris: OECD.

Peachey, N. (2019). Hacking Creativity. PeacheyPublications.

Pugliese, C. (2010). Being Creative. Peaslake: DELTA

Razik, T. A. (1967). Psychometric measurement of creativity. In Mooney, R. L. & Razik, T. A. (Eds.) Explorations in Creativity. New York: Harper & Row

Rogers, C. (1954). Toward a Theory of Creativity. ETC: A Review of General Semantics, 11: pp. 249-260

Schulte, B. (2018). Appropriating or hijacking creativity? Educational reform and creative learning in China. Abstract from Australian Association for Research in Education (AARE) Conference 2018, Sydney, Australia.

Shapin, S. (2020). The rise and rise of creativity. Aeon 12 October 2020 https://aeon.co/essays/how-did-creativity-become-an-engine-of-economic-growth

Trilling, B. & Fadel, C. (2009). 21st Century Skills. San Francisco: Jossey-Bass

World Bank (2010). Stepping Up Skills. Washington: The World Bank

Xerri, D. & Vassallo, O. (Eds.) (2016). Creativity in English Language Teaching. Floriana: ELT Council

If you cast your eye over the English language teaching landscape, you can’t help noticing a number of prominent features that weren’t there, or at least were much less visible, twenty years ago. I’d like to highlight three. First, there is the interest in life skills (aka 21st century skills). Second, there is the use of digital technology to deliver content. And third, there is a concern with measuring educational outputs through frameworks such as the Pearson GSE. In this post, I will focus primarily on the last of these, with a closer look at measuring teacher performance.

Recent years have seen the development of a number of frameworks for evaluating teacher competence in ELT. These include

TESOL has also produced a set of guidelines for developing professional teaching standards for EFL.

Frameworks such as these were not always intended as tools to evaluate teachers. The British Council’s framework, for example, was apparently designed for teachers to understand and plan their own professional development. Similarly, the Cambridge framework says that it is for teachers to see where they are in their development – and think about where they want to go next. But much like the CEFR for language competence, frameworks can be used for purposes rather different from their designers’ intentions. I think it is likely that frameworks such as these are more often used to evaluate teachers than for teachers to evaluate themselves.

But where did the idea for such frameworks come from? Was there a suddenly perceived need for things like this to aid in self-directed professional development? Were teachers’ associations calling out for frameworks to help their members? Even if that were the case, it would still be useful to know why, and why now.

One possibility is that the interest in life skills, digital technology and the measurement of educational outputs have all come about as a result of what has been called the Global Educational Reform Movement, or GERM (Sahlberg, 2016). GERM dates back to the 1980s and the shifts (especially in the United States under Reagan and the United Kingdom under Thatcher) in education policy towards more market-led approaches which emphasize (1) greater competition between educational providers, (2) greater autonomy from the state for educational providers (and therefore a greater role for private suppliers), (3) greater choice of educational provider for students and their parents, and (4) standardized tests and measurements which allow consumers of education to make more informed choices. One of the most significant GERM vectors is the World Bank.

The interest in incorporating the so-called 21st century skills as part of the curriculum can be traced back to the early 1980s when the US National Commission on Excellence in Education recommended the inclusion of a range of skills, which eventually crystallized into the four Cs of communication, collaboration, critical thinking and creativity. The labelling of this skill set as ‘life skills’ or ‘21st century skills’ was always something of a misnomer: the reality was that these were the soft skills required by the world of work. The key argument for their inclusion in the curriculum was that they were necessary for the ‘competitiveness and wealth of corporations and countries’ (Trilling & Fadel, 2009: 7). Unsurprisingly, the World Bank, whose interest in education extends only so far as its economic value, embraced the notion of ‘life skills’ with enthusiasm. Its document ‘Life skills : what are they, why do they matter, and how are they taught?’ (World Bank, 2013), makes the case very clearly. It took a while for the world of English language teaching to get on board, but by 2012, Pearson was already sponsoring a ‘signature event’ at IATEFL Glasgow entitled ‘21st Century Skills for ELT’. Since then, the currency of ‘life skills’ as an ELT buzz phrase has not abated.

Just as the World Bank’s interest in ‘life skills’ is motivated by the perceived need to prepare students for the world of work (for participation in the ‘knowledge economy’), the Bank emphasizes the classroom use of computers and resources from the internet: Information and communication technology (ICT) allows the adaptation of globally available information to local learning situations. […] A large percentage of the World Bank’s education funds are used for the purchase of educational technology. […] According to the Bank’s figures, 40 per cent of their education budget in 2000 and 27 per cent in 2001 was used to purchase technology. (Spring, 2015: 50).

Digital technology is also central to capturing data, which will allow for the measurement of educational outputs. As befits an organisation of economists that is interested in the cost-effectiveness of investments into education, it accords enormous importance to what are thought to be empirical measures or accountability. So intrinsic to the Bank’s approach is this concern with measurement that ‘the Bank’s implicit message to national governments seems to be: ‘improve your data collection capacity so that we can run more reliable cross-country analysis and regressions’. (Verger & Bonal, 2012: 131).

Measuring the performance of teachers is, of course, a part of assessing educational outputs. The World Bank, which sees global education as fundamentally ‘broken’, has, quite recently, turned more of its attention to the role of teachers. A World Bank blog from 2019 explains the reasons:

A growing body of evidence suggests the learning crisis is, at its core, a teaching crisis. For students to learn, they need good teachers—but many education systems pay little attention to what teachers know, what they do in the classroom, and in some cases whether they even show up. Rapid technological change is raising the stakes. Technology is already playing a crucial role in providing support to teachers, students, and the learning process more broadly. It can help teachers better manage the classroom and offer different challenges to different students. And technology can allow principals, parents, and students to interact seamlessly.

A key plank in the World Banks’s attempts to implement its educational vision is its System Assessment and Benchmarking for Education Results (SABER), which I will return to in due course. As part of its SABER efforts, last year the World Bank launched its ‘Teach’ tool . This tool is basically an evaluation framework. Videos of lessons are recorded and coded for indicators of teacher efficiency by coders who can be ‘90% reliable’ after only four days of training. The coding system focuses on the time that students spend on-task, but also ‘life skills’ like collaboration and critical thinking (see below).

Teach framework

Like the ELT frameworks, it can be used as a professional development tool, but, like them, it may also be used for summative evaluation.

The connections between those landmarks on the ELT landscape and the concerns of the World Bank are not, I would suggest, coincidental. The World Bank is, of course, not the only player in GERM, but it is a very special case. It is the largest single source of external financing in ‘developing countries’ (Beech, 2009: 345), managing a portfolio of $8.9 billion, with operations in 70 countries as of August 2013 (Spring, 2015: 32). Its loans come attached with conditions which tie the borrowing countries to GERM objectives. Arguably of even greater importance than its influence through funding, is the Bank’s direct entry into the world of ideas:

The Bank yearns for a deeper and more comprehensive impact through avenues of influence transcending both project and program loans. Not least in education, the World Bank is investing much in its quest to shape global opinion about economic, developmental, and social policy. Rather than imposing views through specific loan negotiations, Bank style is broadening in attempts to lead borrower country officials to its preferred way of thinking. (Jones, 2007: 259).

The World Bank sees itself as a Knowledge Bank and acts accordingly. Rizvi and Lingard (2010: 48) observe that ‘in many nations of the Global South, the only extant education policy analysis is research commissioned by donor agencies such as the World Bank […] with all the implications that result in relation to problem setting, theoretical frameworks and methodologies’. Hundreds of academics are engaged to do research related to the Bank’s areas of educational interest, and ‘the close links with the academic world give a strong credibility to the ideas disseminated by the Bank […] In fact, many ideas that acquired currency and legitimacy were originally proposed by them. This is the case of testing students and using the results to evaluate progress in education’ (Castro, 2009: 472).

Through a combination of substantial financial clout and relentless marketing (Selwyn, 2013: 50), the Bank has succeeded in shaping global academic discourse. In partnership with similar institutions, it has introduced a way of classifying and thinking about education (Beech, 2009: 352). It has become, in short, a major site ‘for the organization of knowledge about education’ (Rizvi & Lingard, 2010: 79), wielding ‘a degree of power that has arguably enabled it to shape the educational agendas of nations throughout the Global South’ and beyond (Menashy, 2012).

So, is there any problem in the world of ELT taking up the inclusion of ‘life skills’? I think there is. The first is one of definition. Creativity and critical thinking are very poorly defined, meaning very different things to different people, so it is not always clear what is being taught. Following on from this, there is substantial debate about whether such skills can actually be taught at all, and, if they can, how they should be taught. It seems highly unlikely that the tokenistic way in which they are ‘taught’ in most published ELT courses can be of any positive impact. But this is not my main reservation, which is that, by and large, we have come to uncritically accept the idea that English language learning is mostly concerned with preparation for the workplace (see my earlier post ‘The EdTech Imaginary in ELT’).

Is there any problem with the promotion of digital technologies in ELT? Again, I think there is, and a good proportion of the posts on this blog have argued for the need for circumspection in rolling out more technology in language learning and teaching. My main reason is that while it is clear that this trend is beneficial to technology vendors, it is much less clear that advantages will necessarily accrue to learners. Beyond this, there must be serious concerns about data ownership, privacy, and the way in which the datafication of education, led by businesses and governments in the Global North, is changing what counts as good education, a good student or an effective teacher, especially in the Global South. ‘Data and metrics,’ observe Williamson et al. (2020: 353), ‘do not just reflect what they are designed to measure, but actively loop back into action that can change the very thing that was measured in the first place’.

And what about tools for evaluating teacher competences? Here I would like to provide a little more background. There is, first of all, a huge question mark about how accurately such tools measure what they are supposed to measure. This may not matter too much if the tool is only used for self-evaluation or self-development, but ‘once smart systems of data collection and social control are available, they are likely to be widely applied for other purposes’ (Sadowski, 2020: 138). Jaime Saavedra, head of education at the World Bank, insists that the World Bank’s ‘Teach’ tool is not for evaluation and is not useful for firing teachers who perform badly.

Saavedra needs teachers to buy into the tool, so he obviously doesn’t want to scare them off. However, ‘Teach’ clearly is an evaluation tool (if not, what is it?) and, as with other tools (I’m thinking of CEFR and teacher competency frameworks in ELT), its purposes will evolve. Eric Hanushek, an education economist at Stanford University, has commented that ‘this is a clear evaluation tool at the probationary stage … It provides a basis for counseling new teachers on how they should behave … but then again if they don’t change over the first few years you also have information you should use.

At this point, it is useful to take a look at the World Bank’s attitudes towards teachers. Teachers are seen to be at the heart of the ‘learning crisis’. However, the greatest focus in World Bank documents is on (1) teacher absenteeism in some countries, (2) unskilled and demotivated teachers, and (3) the reluctance of teachers and their unions to back World Bank-sponsored reforms. As real as these problems are, it is important to understand that the Bank has been complicit in them:

For decades, the Bank has criticised pre-service and in-service teacher training as not cost-effective For decades, the Bank has been pushing the hiring of untrained contract teachers as a cheap fix and a way to get around teacher unions – and contract teachers are again praised in the World Bank Development Report (WDR). This contradicts the occasional places in the WDR in which the Bank argues that developing countries need to follow the lead of the few countries that attract the best students to teaching, improve training, and improve working conditions. There is no explicit evidence offered at all for the repeated claim that teachers are unmotivated and need to be controlled and monitored to do their job. The Bank has a long history of blaming teachers and teacher unions for educational failures. The Bank implicitly argues that the problem of teacher absenteeism, referred to throughout the report, means teachers are unmotivated, but that simply is not true. Teacher absenteeism is not a sign of low motivation. Teacher salaries are abysmally low, as is the status of teaching. Because of this, teaching in many countries has become an occupation of last resort, yet it still attracts dedicated teachers. Once again, the Bank has been very complicit in this state of affairs as it, and the IMF, for decades have enforced neoliberal, Washington Consensus policies which resulted in government cutbacks and declining real salaries for teachers around the world. It is incredible that economists at the Bank do not recognise that the deterioration of salaries is the major cause of teacher absenteeism and that all the Bank is willing to peddle are ineffective and insulting pay-for-performance schemes. (Klees, 2017).

The SABER framework (referred to above) focuses very clearly on policies for hiring, rewarding and firing teachers.

[The World Bank] places the private sector’s methods of dealing with teachers as better than those of the public sector, because it is more ‘flexible’. In other words, it is possible to say that teachers can be hired and fired more easily; that is, hired without the need of organizing a public competition and fired if they do not achieve the expected outcomes as, for example, students’ improvements in international test scores. Further, the SABER document states that ‘Flexibility in teacher contracting is one of the primary motivations for engaging the private sector’ (World Bank, 2011: 4). This affirmation seeks to reduce expenditures on teachers while fostering other expenses such as the creation of testing schemes and spending more on ICTs, as well as making room to expand the hiring of private sector providers to design curriculum, evaluate students, train teachers, produce education software, and books. (De Siqueira, 2012).

The World Bank has argued consistently for a reduction of education costs by driving down teachers’ salaries. One of the authors of the World Bank Development Report 2018 notes that ‘in most countries, teacher salaries consume the lion’s share of the education budget, so there are already fewer resources to implement other education programs’. Another World Bank report (2007) makes the importance of ‘flexible’ hiring and lower salaries very clear:

In particular, recent progress in primary education in Francophone countries resulted from reduced teacher costs, especially through the recruitment of contractual teachers, generally at about 50% the salary of civil service teachers. (cited in Compton & Weiner, 2008: 7).

Merit pay (or ‘pay for performance’) is another of the Bank’s preferred wheezes. Despite enormous problems in reaching fair evaluations of teachers’ work and a distinct lack of convincing evidence that merit pay leads to anything positive (and may actually be counter-productive) (De Bruyckere et al., 2018: 143 – 147), the Bank is fully committed to the idea. Perhaps this is connected to the usefulness of merit pay in keeping teachers on their toes, compliant and fearful of losing their jobs, rather than any desire to improve teacher effectiveness?

There is evidence that this may be the case. Yet another World Bank report (Bau & Das, 2017) argues, on the basis of research, that improved TVA (teacher value added) does not correlate with wages in the public sector (where it is hard to fire teachers), but it does in the private sector. The study found that ‘a policy change that shifted public hiring from permanent to temporary contracts, reducing wages by 35 percent, had no adverse impact on TVA’. All of which would seem to suggest that improving the quality of teaching is of less importance to the Bank than flexible hiring and firing. This is very much in line with a more general advocacy of making education fit for the world of work. Lois Weiner of New Jersey City University puts it like this:

The architects of [GERM] policies—imposed first in developing countries—openly state that the changes will make education better fit the new global economy by producing workers who are (minimally) educated for jobs that require no more than a 7th or 8th grade education; while a small fraction of the population receive a high quality education to become the elite who oversee finance, industry, and technology. Since most workers do not need to be highly educated, it follows that teachers with considerable formal education and experience are neither needed nor desired because they demand higher wages, which is considered a waste of government money. Most teachers need only be “good enough”—as one U.S. government official phrased it—to follow scripted materials that prepare students for standardized tests. (Weiner, 2012).

It seems impossible to separate the World Bank’s ‘Teach’ tool from the broader goals of GERM. Teacher evaluation tools, like the teaching of 21st century skills and the datafication of education, need to be understood properly, I think, as means to an end. It’s time to spell out what that end is.

The World Bank’s mission is ‘to end extreme poverty (by reducing the share of the global population that lives in extreme poverty to 3 percent by 2030)’ and ‘to promote shared prosperity (by increasing the incomes of the poorest 40 percent of people in every country)’. Its education activities are part of this broad aim and are driven by subscription to human capital theory (a view of the skills, knowledge and experience of individuals in terms of their ability to produce economic value). This may be described as the ‘economization of education’: a shift in educational concerns away from ‘such things as civic participation, protecting human rights, and environmentalism to economic growth and employment’ (Spring, 2015: xiii). Both students and teachers are seen as human capital. For students, human capital education places an emphasis on the cognitive skills needed to succeed in the workplace and the ‘soft skills’, needed to function in the corporate world (Spring, 2015: 2). Accordingly, World Bank investments require ‘justifications on the basis of manpower demands’ (Heyneman, 2003: 317). One of the Bank’s current strategic priorities is the education of girls: although human rights and equity may also play a part, the Bank’s primary concern is that ‘Not Educating Girls Costs Countries Trillions of Dollars’ .

According to the Bank’s logic, its educational aims can best be achieved through a combination of support for the following:

  • cost accounting and quantification (since returns on investment must be carefully measured)
  • competition and market incentives (since it is believed that the ‘invisible hand’ of the market leads to the greatest benefits)
  • the private sector in education and a rolling back of the role of the state (since it is believed that private ownership improves efficiency)

The package of measures is a straightforward reflection of ‘what Western mainstream economists believe’ (Castro, 2009: 474).

Mainstream Western economics is, however, going through something of a rocky patch right now. Human capital theory is ‘useful when prevailing conditions are right’ (Jones, 2007: 248), but prevailing conditions are not right in much of the world (even in the United States), and the theory ‘for the most part ignores the intersections of poverty, equity and education’ (Menashy, 2012). In poorer countries evidence for the positive effects of markets in education is in very short supply, and even in richer countries it is still not conclusive (Verger & Bonal, 2012: 135). An OECD Education Paper (Waslander et al., 2010: 64) found that the effects of choice and competition between schools were at best small, if indeed any effects were found at all. Similarly, the claim that privatization improves efficiency is not sufficiently supported by evidence. Analyses of PISA data would seem to indicate that, ‘all else being equal (especially when controlling for the socio-economic status of the students), the type of ownership of the school, whether it is a private or a state school, has only modest effects on student achievement or none at all’ (Verger & Bonal, 2012: 133). Educational privatization as a one-size-fits-all panacea to educational problems has little to recommend it.

There are, then, serious limitations in the Bank’s theoretical approach. Its practical track record is also less than illustrious, even by the Bank’s own reckoning. Many of the Bank’s interventions have proved very ‘costly to developing countries. At the Bank’s insistence countries over-invested in vocational and technical education. Because of the narrow definition of recurrent costs, countries ignored investments in reading materials and in maintaining teacher salaries. Later at the Bank’s insistence, countries invested in thousands of workshops and laboratories that, for the most part, became useless ‘white elephants’ (Heyneman, 2003: 333).

As a bank, the World Bank is naturally interested in the rate of return of investment in that capital, and is therefore concerned with efficiency and efficacy. This raises the question of ‘Effective for what?’ and given that what may be effective for one individual or group may not necessarily be effective for another individual or group, one may wish to add a second question: ‘Effective for whom?’ (Biesta, 2020: 31). Critics of the World Bank, of whom there are many, argue that its policies serve ‘the interests of corporations by keeping down wages for skilled workers, cause global brain migration to the detriment of developing countries, undermine local cultures, and ensure corporate domination by not preparing school graduates who think critically and are democratically oriented’ (Spring, 2015: 56). Lest this sound a bit harsh, we can turn to the Bank’s own commissioned history: ‘The way in which [the Bank’s] ideology has been shaped conforms in significant degree to the interests and conventional wisdom of its principal stockholders [i.e. bankers and economists from wealthy nations]. International competitive bidding, reluctance to accord preferences to local suppliers, emphasis on financing foreign exchange costs, insistence on a predominant use of foreign consultants, attitudes toward public sector industries, assertion of the right to approve project managers – all proclaim the Bank to be a Western capitalist institution’ (Mason & Asher, 1973: 478 – 479).

The teaching of ‘life skills’, the promotion of data-capturing digital technologies and the push to evaluate teachers’ performance are, then, all closely linked to the agenda of the World Bank, and owe their existence in the ELT landscape, in no small part, to the way that the World Bank has shaped educational discourse. There is, however, one other connection between ELT and the World Bank which must be mentioned.

The World Bank’s foreign language instructional goals are directly related to English as a global language. The Bank urges, ‘Policymakers in developing countries …to ensure that young people acquire a language with more than just local use, preferably one used internationally.’ What is this international language? First, the World Bank mentions that schools of higher education around the world are offering courses in English. In addition, the Bank states, ‘People seeking access to international stores of knowledge through the internet require, principally, English language skills.’ (Spring, 2015: 48).

Without the World Bank, then, there might be a lot less English language teaching than there is. I have written this piece to encourage people to think more about the World Bank, its policies and particular instantiations of those policies. You might or might not agree that the Bank is an undemocratic, technocratic, neoliberal institution unfit for the necessities of today’s world (Klees, 2017). But whatever you think about the World Bank, you might like to consider the answers to Tony Benn’s ‘five little democratic questions’ (quoted in Sardowski, 2020: 17):

  • What power has it got?
  • Where did it get this power from?
  • In whose interests does it exercise this power?
  • To whom is it accountable?
  • How can we get rid of it?

References

Bau, N. and Das, J. (2017). The Misallocation of Pay and Productivity in the Public Sector : Evidence from the Labor Market for Teachers. Policy Research Working Paper; No. 8050. World Bank, Washington, DC. Retrieved [18 May 2020] from https://openknowledge.worldbank.org/handle/10986/26502

Beech, J. (2009). Who is Strolling Through The Global Garden? International Agencies and Educational Transfer. In Cowen, R. and Kazamias, A. M. (Eds.) Second International Handbook of Comparative Education. Dordrecht: Springer. pp. 341 – 358

Biesta, G. (2020). Educational Research. London: Bloomsbury.

Castro, C. De M., (2009). Can Multilateral Banks Educate The World? In Cowen, R. and Kazamias, A. M. (Eds.) Second International Handbook of Comparative Education. Dordrecht: Springer. pp. 455 – 478

Compton, M. and Weiner, L. (Eds.) (2008). The Global Assault on Teaching, Teachers, and their Unions. New York: Palgrave Macmillan

De Bruyckere, P., Kirschner, P.A. and Hulshof, C. (2020). More Urban Myths about Learning and Education. New York: Routledge.

De Siqueira, A. C. (2012). The 2020 World Bank Education Strategy: Nothing New, or the Same Old Gospel. In Klees, S. J., Samoff, J. and Stromquist, N. P. (Eds.) The World Bank and Education. Rotterdam: Sense Publishers. pp. 69 – 81

Heyneman, S.P. (2003). The history and problems in the making of education policy at the World Bank 1960–2000. International Journal of Educational Development 23 (2003) pp. 315–337. Retrieved [18 May 2020] from https://www.academia.edu/29593153/The_History_and_Problems_in_the_Making_of_Education_Policy_at_the_World_Bank_1960_2000

Jones, P. W. (2007). World Bank Financing of Education. 2nd edition. Abingdon, Oxon.: Routledge.

Klees, S. (2017). A critical analysis of the World Bank’s World Development Report on education. Retrieved [18 May 2020] from: https://www.brettonwoodsproject.org/2017/11/critical-analysis-world-banks-world-development-report-education/

Mason, E. S. & Asher, R. E. (1973). The World Bank since Bretton Woods. Washington, DC: Brookings Institution.

Menashy, F. (2012). Review of Klees, S J., Samoff, J. & Stromquist, N. P. (Eds) (2012). The World Bank and Education: Critiques and Alternatives .Rotterdam: Sense Publishers. Education Review, 15. Retrieved [18 May 2020] from https://www.academia.edu/7672656/Review_of_The_World_Bank_and_Education_Critiques_and_Alternatives

Rizvi, F. & Lingard, B. (2010). Globalizing Education Policy. Abingdon, Oxon.: Routledge.

Sadowski, J. (2020). Too Smart. Cambridge, MA.: MIT Press.

Sahlberg, P. (2016). The global educational reform movement and its impact on schooling. In K. Mundy, A. Green, R. Lingard, & A. Verger (Eds.), The handbook of global policy and policymaking in education. New York, NY: Wiley-Blackwell. pp.128 – 144

Selwyn, N. (2013). Education in a Digital World. New York: Routledge.

Spring, J. (2015). Globalization of Education 2nd Edition. New York: Routledge.

Trilling, B. & C. Fadel (2009). 21st Century Skills. San Francisco: Wiley

Verger, A. & Bonal, X. (2012). ‘All Things Being Equal?’ In Klees, S. J., Samoff, J. and Stromquist, N. P. (Eds.) The World Bank and Education. Rotterdam: Sense Publishers. pp. 69 – 81

Waslander, S., Pater, C. & van der Weide, M. (2010). Markets in Education: An analytical review of empirical research on market mechanisms in education. OECD EDU Working Paper 52.

Weiner, L. (2012). Social Movement Unionism: Teachers Can Lead the Way. Reimagine, 19 (2) Retrieved [18 May 2020] from: https://www.reimaginerpe.org/19-2/weiner-fletcher

Williamson, B., Bayne, S. & Shay, S. (2020). The datafication of teaching in Higher Education: critical issues and perspectives, Teaching in Higher Education, 25:4, 351-365, DOI: 10.1080/13562517.2020.1748811

World Bank. (2013). Life skills : what are they, why do they matter, and how are they taught? (English). Adolescent Girls Initiative (AGI) learning from practice series. Washington DC ; World Bank. Retrieved [18 May 2020] from: http://documents.worldbank.org/curated/en/569931468331784110/Life-skills-what-are-they-why-do-they-matter-and-how-are-they-taught