Posts Tagged ‘learning outcomes’

Definition of gritGrit book cover

from Quartz at Work magazine

 

Grit is on the up. You may have come across articles like ‘How to Be Gritty in the Time of COVID-19’ or ‘Rediscovering the meaning of grit during COVID-19’ . If you still want more, there are new videos from Angela Duckworth herself where we can learn how to find our grit in the face of the pandemic.

Schools and educational authorities love grit. Its simple, upbeat message (‘Yes, you can’) has won over hearts and minds. Back in 2014, the British minister for education announced a £5million plan to encourage teaching ‘character and resilience’ in schools – specifically looking at making Britain’s pupils ‘grittier’. The spending on grit hasn’t stopped since.

The publishers of Duckworth’s book paid a seven-figure sum to acquire the US rights, and sales have proved the wisdom of the investment. Her TED talk has had over 6.5 million views on YouTube, although it’s worth looking at the comments to see why many people have been watching it.

Youtube comments

The world of English language teaching, always on the lookout for a new bandwagon to jump onto, is starting to catch up with the wider world of education. Luke Plonsky, an eminent SLA scholar, specialist in meta-analyses and grit enthusiast, has a bibliography of grit studies related to L2 learning, that he deems worthy of consideration. Here’s a summary, by year, of those publications. More details will follow in the next section.

Plonsky biblio

We can expect interest in ‘grit’ to continue growing, and this may be accelerated by the publication this year of Engaging Language Learners in Contemporary Classrooms by Sarah Mercer and Zoltán Dörnyei. In this book, the authors argue that a ‘facilitative mindset’ is required for learner engagement. They enumerate five interrelated principles for developing a ‘facilitative mindset’: promote a sense of competence, foster a growth mindset, promote learners’ sense of ownership and control, develop proactive learners and, develop gritty learners. After a brief discussion of grit, they write: ‘Thankfully, grit can be learnt and developed’ (p.38).

Unfortunately, they don’t provide any evidence at all for this. Unfortunately, too, this oversight is easy to explain. Such evidence as there is does not lend unequivocal support to the claim. Two studies that should have been mentioned in this book are ‘Much ado about grit: A meta-analytic synthesis of the grit literature’ (Credé et al, 2017) and ‘What shall we do about grit? A critical review of what we know and what we don’t know’ (Credé, 2018). The authors found that ‘grit as it is currently measured does not appear to be particularly predictive of success and performance’ (Credé et al, 2017) and that there is no support for the claim that ‘grit is likely to be responsive to interventions’ (Credé, 2018). In the L2 learning context, Teimouri et al (2020) concluded that more research in SLA substantiating the role of grit in L2 contexts was needed before any grit interventions can be recommended.

It has to be said that such results are hardly surprising. If, as Duckworth claims, ‘grit’ is a combination of passion and persistence, how on earth can the passion part of it be susceptible to educational interventions? ‘If there is one thing that cannot be learned, it’s passion. A person can have it and develop it, but learn it? Sadly, not’. (De Bruyckere et al., 2020: 83)

Even Duckworth herself is not convinced. In an interview on a Freakonomics podcast, she states that she hopes it’s something people can learn, but also admits not having enough proof to confirm that they can (Kirschner & Neelen, 2016)!

Is ‘grit’ a thing?

Marc Jones, in a 2016 blog post entitled ‘Gritty Politti: Grit, Growth Mindset and Neoliberal Language Teaching’, writes that ‘Grit is so difficult to define that it takes Duckworth (2016) the best part of a book to describe it adequately’. Yes, ‘grit’ is passion and persistence (or perseverance), but it’s also conscientiousness, practice and hope. Credé et al (2017) found that ‘grit is very strongly correlated with conscientiousness’ (which has already been widely studied in the educational literature). Why lump this together with passion? Another study (Muenks et al., 2017) found that ‘Students’ grit overlapped empirically with their concurrently reported self-control, self-regulation, and engagement. Students’ perseverance of effort (but not their consistency of interests) predicted their later grades, although other self-regulation and engagement variables were stronger predictors of students’ grades than was grit’. Credé (2018) concluded that ‘there appears to be no reason to accept the combination of perseverance and passion for long-term goals into a single grit construct’.

The L2 learning research listed in Plonsky’s bibliography does not offer much in support of ‘grit’, either. Many of the studies identified problems with ‘grit’ as a construct, but, even when accepting it, did not find it to be of much value. Wei et al. (2019) found a positive but weak correlation between grit and English language course grades. Yamashita (2018) found no relationship between learners’ grit and their course grades. Taşpinar & Külekçi (2018) found that students’ grit levels and academic achievement scores did not relate to each other (but still found that ‘grit, perseverance, and tenacity are the essential elements that impact learners’ ability to succeed to be prepared for the demands of today’s world’!).

There are, then, grounds for suspecting that Duckworth and her supporters have fallen foul of the ‘jangle fallacy’ – the erroneous assumption that two identical or almost identical things are different because they are labelled differently. This would also help to explain the lack of empirical support for the notion of ‘grit’. Not only are the numerous variables insufficiently differentiated, but the measures of ‘grit’ (such as Duckworth’s Grit-S measure) do not adequately target some of these variables (e.g. long-term goals, where ‘long-term’ is not defined) (Muenks et al., 2017). In addition, these measures are self-reporting and not, therefore, terribly reliable.

Referring to more general approaches to character education, one report (Gutman & Schoon, 2012) has argued that there is little empirical evidence of a causal relationship between self-concept and educational outcomes. Taking this one step further, Kathryn Ecclestone (Ecclestone, 2012) suggests that at best, the concepts and evidence that serve as the basis of these interventions are inconclusive and fragmented; ‘at worst, [they are] prey to ‘advocacy science’ or, in [their] worst manifestations, to simple entrepreneurship that competes for publicly funded interventions’ (cited in Cabanas & Illouz, 2019: 80).

Criticisms of ‘grit’

Given the lack of supporting research, any practical application of ‘grit’ ideas is premature. Duckworth herself, in an article entitled ‘Don’t Believe the Hype About Grit, Pleads the Scientist Behind the Concept’ (Dahl, 2016), cautions against hasty applications:

[By placing too much emphasis on grit, the danger is] that grit becomes a scapegoat — another reason to blame kids for not doing well, or to say that we don’t have a responsibility as a society to help them. [She worries that some interpretations of her work might make a student’s failure seem like his problem, as if he just didn’t work hard enough.] I think to separate and pit against each other character strengths on the one hand — like grit — and situational opportunities on the other is a false dichotomy […] Kids need to develop character, and they need our support in doing so.

Marc Jones, in the blog mentioned above, writes that ‘to me, grit is simply another tool for attacking the poor and the other’. You won’t win any prizes for guessing which kinds of students are most likely to be the targets of grit interventions. A clue: think of the ‘no-nonsense’ charters in the US and academies in the UK. This is what Kenneth Saltzman has to say:

‘Grit’ is a pedagogy of control that is predicated upon a promise made to poor children that if they learnt the tools of self-control and learnt to endure drudgery, then they can compete with rich children for scarce economic resources. (Saltzman, 2017: 38)

[It] is a behaviourist form of learned self-control targeting poor students of color and has been popularized post-crisis in the wake of educational privatization and defunding as the cure for poverty. [It] is designed to suggest that individual resilience and self-reliance can overcome social violence and unsupportive social contexts in the era of the shredded social state. (Saltzman, 2017: 15)

Grit is misrepresented by proponents as opening a world of individual choices rather than discussed as a mode of educational and social control in the austere world of work defined by fewer and fewer choices as secure public sector work is scaled back, unemployment continuing at high levels. (Saltzman, 2017: 49)

Whilst ‘grit’ is often presented as a way of dealing with structural inequalities in schools, critics see it as more of a problem than a solution: ‘It’s the kids who are most impacted by, rebel against, or criticize the embedded racism and classism of their institutions that are being told to have more grit, that school is hard for everyone’ (EquiTEA, 2018). A widely cited article by Nicholas Tampio (2016) points out that ‘Duckworth celebrates educational models such as Beast at West Point that weed out people who don’t obey orders’. He continues ‘that is a disastrous model for education in a democracy. US schools ought to protect dreamers, inventors, rebels and entrepreneurs – not crush them in the name of grit’.

If you’re interested in reading more critics of grit, the blog ‘Debunked!’ is an excellent source of links.

Measuring grit

Analyses of emotional behaviour have become central to economic analysis and, beginning in the 1990s, there have been constant efforts to create ‘formal instruments of classification of emotional behaviour and the elaboration of the notion of emotional competence’ (Illouz, 2007: 64). The measurement and manipulation of various aspects of ‘emotional intelligence’ have become crucial as ways ‘to control, predict, and boost performance’ (Illouz, 2007: 65). An article in the Journal of Benefit-Cost Analysis (Belfield et al., 2015) makes the economic importance of emotions very clear. Entitled ‘The Economic Value of Social and Emotional Learning’, it examines the economic value of these skills within a benefit-cost analysis (BCA) framework, and finds that the benefits of [social and emotional learning] interventions substantially outweigh the costs.

In recent years, the OECD has commissioned a number of reports on social and emotional learning and, as with everything connected with the OECD, is interested in measuringnon-cognitive skills such as perseverance (“grit”), conscientiousness, self-control, trust, attentiveness, self-esteem and self-efficacy, resilience to adversity, openness to experience, empathy, humility, tolerance of diverse opinions and the ability to engage productively in society’ (Kautz et al., 2014: 9). The measurement of personality factors will feature in the OECD’s PISA programme. Elsewhere, Ben Williamson reports that ‘US schools [are] now under pressure—following the introduction of the Every Student Succeeds Act in 2015—to provide measurable evidence of progress on the development of students’ non-academic learning’ (Williamson, 2017).

Grit, which ‘starts and ends with the lone individual as economic actor, worker, and consumer’ (Saltzman, 2017: 50), is a recent addition to the categories of emotional competence, and it should come as no surprise that educational authorities have so wholeheartedly embraced it. It is the claim that something (i.e. ‘grit’) can be taught and developed that leads directly to the desire to measure it. In a world where everything must be accountable, we need to know how effective and cost-effective our grit interventions have been.

The idea of measuring personality constructs like ‘grit’ worries even Angela Duckworth. She writes (Duckworth, 2016):

These days, however, I worry I’ve contributed, inadvertently, to an idea I vigorously oppose: high-stakes character assessment. New federal legislation can be interpreted as encouraging states and schools to incorporate measures of character into their accountability systems. This year, nine California school districts will begin doing this. But we’re nowhere near ready — and perhaps never will be — to use feedback on character as a metric for judging the effectiveness of teachers and schools. We shouldn’t be rewarding or punishing schools for how students perform on these measures.

Diane Ravitch (Ravitch, 2016) makes the point rather more forcefully: ‘The urge to quantify the unmeasurable must be recognized for what it is: stupid; arrogant; harmful; foolish, yet another way to standardize our beings’. But, like it or not, attempts to measure ‘grit’ and ‘grit’ interventions are unlikely to go away any time soon.

‘Grit’ and technology

Whenever there is talk about educational measurement and metrics, we are never far away from the world of edtech. It may not have escaped your notice that the OECD and the US Department of State for Education, enthusiasts for promoting ‘grit’, are also major players in the promotion of the datafication of education. The same holds true for organisations like the World Education Forum, the World Bank and the various philanthro-capitalist foundations to which I have referred so often in this blog. Advocacy of social and emotional learning goes hand in hand with edtech advocacy.

Two fascinating articles by Ben Williamson (2017; 2019) focus on ClassDojo, which, according to company information, reaches more than 10 million children globally every day. The founding directors of ClassDojo, writes Ben Williamson (2017), ‘explicitly describe its purpose as promoting ‘character development’ in schools and it is underpinned by particular psychological concepts from character research. Its website approvingly cites the journalist Paul Tough, author of two books on promoting ‘grit’ and ‘character’ in children, and is informed by character research conducted with the US network of KIPP charter schools (Knowledge is Power Program)’. In a circular process, ClassDojo has also ‘helped distribute and popularise concepts such as growth mindset, grit and mindfulness’ (Williamson, 2019).

The connections between ‘grit’ and edtech are especially visible when we focus on Stanford and Silicon Valley. ClassDojo was born in Palo Alto. Duckworth was a consulting scholar at Stanford 2014 -15, where Carol Dweck is a Professor of Psychology. Dweck is the big name behind growth mindset theory, which, as Sarah Mercer and Zoltán Dörnyei indicate, is closely related to ‘grit’. Dweck is also the co-founder of MindsetWorks, whose ‘Brainology’ product is ‘an online interactive program in which middle school students learn about how the brain works, how to strengthen their own brains, and how to ….’. Stanford is also home to the Stanford Lytics Lab, ‘which has begun applying new data analytics techniques to the measurement of non-cognitive learning factors including perseverance, grit, emotional state, motivation and self-regulation’, as well as the Persuasive Technologies Lab, ‘which focuses on the development of machines designed to influence human beliefs and behaviors across domains including health, business, safety, and education’ (Williamson, 2017). The Professor of Education Emeritus at Stanford is Linda Darling-Hammond, one of the most influential educators in the US. Darling-Hammond is known, among many other things, for collaborating with Pearson to develop the edTPA, ‘a nationally available, performance-based assessment for measuring the effectiveness of teacher candidates’. She is also a strong advocate of social-emotional learning initiatives and extols the virtues of ‘developing grit and a growth mindset’ (Hamadi & Darling-Hammond, 2015).

The funding of grit

Angela Duckworth’s Character Lab (‘Our mission is to advance scientific insights that help kids thrive’) is funded by, among others, the Chan Zuckerberg Initiative, the Bezos Family Foundation and Stanford’s Mindset Scholars Network. Precisely how much money Character Lab has is difficult to ascertain, but the latest grant from the Chan Zuckerberg Initiative was worth $1,912,000 to cover the period 2018 – 2021. Covering the same period, the John Templeton Foundation, donated $3,717,258 , the purpose of the grant being to ‘make character development fast, frictionless, and fruitful’.

In an earlier period (2015 – 2018), the Walton Family Foundation pledged $6.5 millionto promote and measure character education, social-emotional learning, and grit’, with part of this sum going to Character Lab and part going to similar research at Harvard Graduate School of Education. Character Lab also received $1,300,000 from the Overdeck Family Foundation for the same period.

It is not, therefore, an overstatement to say that ‘grit’ is massively funded. The funders, by and large, are the same people who have spent huge sums promoting personalized learning through technology (see my blog post Personalized learning: Hydra and the power of ambiguity). Whatever else it might be, ‘grit’ is certainly ‘a commercial tech interest’ (as Ben Williamson put it in a recent tweet).

Postscript

In the 2010 Cohen brothers’ film, ‘True Grit’, the delinquent ‘kid’, Moon, is knifed by his partner, Quincy. Turning to Rooster Cogburn, the man of true grit, Moon begs for help. In response, Cogburn looks at the dying kid and deadpans ‘I can do nothing for you, son’.

References

Belfield, C., Bowden, A., Klapp, A., Levin, H., Shand, R., & Zander, S. (2015). The Economic Value of Social and Emotional Learning. Journal of Benefit-Cost Analysis, 6(3), pp. 508-544. doi:10.1017/bca.2015.55

Cabanas, E. & Illouz, E. (2019). Manufacturing Happy Citizens. Cambridge: Polity Press.

Chaykowski, K. (2017). How ClassDojo Built One Of The Most Popular Classroom Apps By Listening To Teachers. Forbes, 22 May, 2017. https://www.forbes.com/sites/kathleenchaykowski/2017/05/22/how-classdojo-built-one-of-the-most-popular-classroom-apps-by-listening-to-teachers/#ea93d51e5ef5

Credé, M. (2018). What shall we do about grit? A critical review of what we know and what we don’t know. Educational Researcher, 47(9), 606-611.

Credé, M., Tynan, M. C., & Harms, P. D. (2017). Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113(3), 492. doi:10.1037/pspp0000102

Dahl, M. (2016). Don’t Believe the Hype About Grit, Pleads the Scientist Behind the Concept. The Cut, May 9, 2016. https://www.thecut.com/2016/05/dont-believe-the-hype-about-grit-pleads-the-scientist-behind-the-concept.html

De Bruyckere, P., Kirschner, P. A. & Hulshof, C. (2020). More Urban Myths about Learning and Education. Routledge.

Duckworth, A. (2016). Don’t Grade Schools on Grit. New York Times, March 26, 2016 https://www.nytimes.com/2016/03/27/opinion/sunday/dont-grade-schools-on-grit.html?auth=login-google&smid=nytcore-ipad-share&smprod=nytcore-ipad

Ecclestone, K. (2012). From emotional and psychological well-being to character education: Challenging policy discourses of behavioural science and ‘vulnerability’. Research Papers in Education, 27 (4), pp. 463-480

EquiTEA (2018). The Problem with Teaching ‘Grit’. Medium, 11 December 2018. https://medium.com/@eec/the-problem-with-teaching-grit-8b37ce43a87e

Gutman, L. M. & Schoon, I. (2013). The impact of non-cognitive skills on outcomes for young people: Literature review. London: Institute of Education, University of London

Hamedani, M. G. & Darling-Hammond, L. (2015). Social Emotional Learning in High School: How Three Urban High Schools Engage, Educate, and Empower Youth. Stanford Center for Opportunity Policy in Education

Kirschner, P.A. & Neelen, M. (2016). To Grit Or Not To Grit: That’s The Question. 3-Star Learning Experiences, July 5, 2016 https://3starlearningexperiences.wordpress.com/2016/07/05/to-grit-or-not-to-grit-thats-the-question/

Illouz, E. (2007). Cold Intimacies: The making of emotional capitalism. Cambridge: Polity Press

Kautz, T., Heckman, J. J., Diris, R., ter Weel, B & Borghans, L. (2014). Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success. OECD Education Working Papers 110, OECD Publishing.

Mercer, S. & Dörnyei, Z. (2020). Engaging Language Learners in Contemporary Classrooms. Cambridge University Press.

Muenks, K., Wigfield, A., Yang, J. S., & O’Neal, C. R. (2017). How true is grit? Assessing its relations to high school and college students’ personality characteristics, self-regulation, engagement, and achievement. Journal of Educational Psychology, 109, pp. 599–620.

Ravitch, D. (2016). Angela Duckworth, please don’t assess grit. Blog post, 27 March 2016, https://dianeravitch.net/2016/03/27/angela-duckworth-please-dont-assess-grit/

Saltzman, K. J. (2017). Scripted Bodies. Routledge.

Tampio, N. (2016). Teaching ‘grit’ is bad for children, and bad for democracy. Aeon, 2 June: https://aeon.co/ideas/teaching-grit-is-bad-for-children-and-bad-for-democracy

Taşpinar, K., & Külekçi, G. (2018). GRIT: An Essential Ingredient of Success in the EFL Classroom. International Journal of Languages’ Education and Teaching, 6, 208-226.

Teimouri, Y., Plonsky, L., & Tabandeh, F. (2020). L2 Grit: Passion and perseverance for second-language learning. Language Teaching Research.

Wei, H., Gao, K., & Wang, W. (2019). Understanding the relationship between grit and foreign language performance among middle schools students: The roles of foreign language enjoyment and classroom Environment. Frontiers in Psychology, 10, 1508. doi: 10.3389/fpsyg.2019.01508

Williamson, B. (2017). Decoding ClassDojo: psycho-policy, social-emotional learning and persuasive educational technologies. Learning, Media and Technology, 42 (4): pp. 440-453, DOI: 10.1080/17439884.2017.1278020

Williamson, B. (2019). ‘Killer Apps for the Classroom? Developing Critical Perspectives on ClassDojo and the ‘Ed-tech’ Industry. Journal of Professional Learning, 2019 (Semester 2) https://cpl.asn.au/journal/semester-2-2019/killer-apps-for-the-classroom-developing-critical-perspectives-on-classdojo

Yamashita, T. (2018). Grit and second language acquisition: Can passion and perseverance predict performance in Japanese language learning? Unpublished MA thesis, University of Massachusetts, Amherst.

 

If you cast your eye over the English language teaching landscape, you can’t help noticing a number of prominent features that weren’t there, or at least were much less visible, twenty years ago. I’d like to highlight three. First, there is the interest in life skills (aka 21st century skills). Second, there is the use of digital technology to deliver content. And third, there is a concern with measuring educational outputs through frameworks such as the Pearson GSE. In this post, I will focus primarily on the last of these, with a closer look at measuring teacher performance.

Recent years have seen the development of a number of frameworks for evaluating teacher competence in ELT. These include

TESOL has also produced a set of guidelines for developing professional teaching standards for EFL.

Frameworks such as these were not always intended as tools to evaluate teachers. The British Council’s framework, for example, was apparently designed for teachers to understand and plan their own professional development. Similarly, the Cambridge framework says that it is for teachers to see where they are in their development – and think about where they want to go next. But much like the CEFR for language competence, frameworks can be used for purposes rather different from their designers’ intentions. I think it is likely that frameworks such as these are more often used to evaluate teachers than for teachers to evaluate themselves.

But where did the idea for such frameworks come from? Was there a suddenly perceived need for things like this to aid in self-directed professional development? Were teachers’ associations calling out for frameworks to help their members? Even if that were the case, it would still be useful to know why, and why now.

One possibility is that the interest in life skills, digital technology and the measurement of educational outputs have all come about as a result of what has been called the Global Educational Reform Movement, or GERM (Sahlberg, 2016). GERM dates back to the 1980s and the shifts (especially in the United States under Reagan and the United Kingdom under Thatcher) in education policy towards more market-led approaches which emphasize (1) greater competition between educational providers, (2) greater autonomy from the state for educational providers (and therefore a greater role for private suppliers), (3) greater choice of educational provider for students and their parents, and (4) standardized tests and measurements which allow consumers of education to make more informed choices. One of the most significant GERM vectors is the World Bank.

The interest in incorporating the so-called 21st century skills as part of the curriculum can be traced back to the early 1980s when the US National Commission on Excellence in Education recommended the inclusion of a range of skills, which eventually crystallized into the four Cs of communication, collaboration, critical thinking and creativity. The labelling of this skill set as ‘life skills’ or ‘21st century skills’ was always something of a misnomer: the reality was that these were the soft skills required by the world of work. The key argument for their inclusion in the curriculum was that they were necessary for the ‘competitiveness and wealth of corporations and countries’ (Trilling & Fadel, 2009: 7). Unsurprisingly, the World Bank, whose interest in education extends only so far as its economic value, embraced the notion of ‘life skills’ with enthusiasm. Its document ‘Life skills : what are they, why do they matter, and how are they taught?’ (World Bank, 2013), makes the case very clearly. It took a while for the world of English language teaching to get on board, but by 2012, Pearson was already sponsoring a ‘signature event’ at IATEFL Glasgow entitled ‘21st Century Skills for ELT’. Since then, the currency of ‘life skills’ as an ELT buzz phrase has not abated.

Just as the World Bank’s interest in ‘life skills’ is motivated by the perceived need to prepare students for the world of work (for participation in the ‘knowledge economy’), the Bank emphasizes the classroom use of computers and resources from the internet: Information and communication technology (ICT) allows the adaptation of globally available information to local learning situations. […] A large percentage of the World Bank’s education funds are used for the purchase of educational technology. […] According to the Bank’s figures, 40 per cent of their education budget in 2000 and 27 per cent in 2001 was used to purchase technology. (Spring, 2015: 50).

Digital technology is also central to capturing data, which will allow for the measurement of educational outputs. As befits an organisation of economists that is interested in the cost-effectiveness of investments into education, it accords enormous importance to what are thought to be empirical measures or accountability. So intrinsic to the Bank’s approach is this concern with measurement that ‘the Bank’s implicit message to national governments seems to be: ‘improve your data collection capacity so that we can run more reliable cross-country analysis and regressions’. (Verger & Bonal, 2012: 131).

Measuring the performance of teachers is, of course, a part of assessing educational outputs. The World Bank, which sees global education as fundamentally ‘broken’, has, quite recently, turned more of its attention to the role of teachers. A World Bank blog from 2019 explains the reasons:

A growing body of evidence suggests the learning crisis is, at its core, a teaching crisis. For students to learn, they need good teachers—but many education systems pay little attention to what teachers know, what they do in the classroom, and in some cases whether they even show up. Rapid technological change is raising the stakes. Technology is already playing a crucial role in providing support to teachers, students, and the learning process more broadly. It can help teachers better manage the classroom and offer different challenges to different students. And technology can allow principals, parents, and students to interact seamlessly.

A key plank in the World Banks’s attempts to implement its educational vision is its System Assessment and Benchmarking for Education Results (SABER), which I will return to in due course. As part of its SABER efforts, last year the World Bank launched its ‘Teach’ tool . This tool is basically an evaluation framework. Videos of lessons are recorded and coded for indicators of teacher efficiency by coders who can be ‘90% reliable’ after only four days of training. The coding system focuses on the time that students spend on-task, but also ‘life skills’ like collaboration and critical thinking (see below).

Teach framework

Like the ELT frameworks, it can be used as a professional development tool, but, like them, it may also be used for summative evaluation.

The connections between those landmarks on the ELT landscape and the concerns of the World Bank are not, I would suggest, coincidental. The World Bank is, of course, not the only player in GERM, but it is a very special case. It is the largest single source of external financing in ‘developing countries’ (Beech, 2009: 345), managing a portfolio of $8.9 billion, with operations in 70 countries as of August 2013 (Spring, 2015: 32). Its loans come attached with conditions which tie the borrowing countries to GERM objectives. Arguably of even greater importance than its influence through funding, is the Bank’s direct entry into the world of ideas:

The Bank yearns for a deeper and more comprehensive impact through avenues of influence transcending both project and program loans. Not least in education, the World Bank is investing much in its quest to shape global opinion about economic, developmental, and social policy. Rather than imposing views through specific loan negotiations, Bank style is broadening in attempts to lead borrower country officials to its preferred way of thinking. (Jones, 2007: 259).

The World Bank sees itself as a Knowledge Bank and acts accordingly. Rizvi and Lingard (2010: 48) observe that ‘in many nations of the Global South, the only extant education policy analysis is research commissioned by donor agencies such as the World Bank […] with all the implications that result in relation to problem setting, theoretical frameworks and methodologies’. Hundreds of academics are engaged to do research related to the Bank’s areas of educational interest, and ‘the close links with the academic world give a strong credibility to the ideas disseminated by the Bank […] In fact, many ideas that acquired currency and legitimacy were originally proposed by them. This is the case of testing students and using the results to evaluate progress in education’ (Castro, 2009: 472).

Through a combination of substantial financial clout and relentless marketing (Selwyn, 2013: 50), the Bank has succeeded in shaping global academic discourse. In partnership with similar institutions, it has introduced a way of classifying and thinking about education (Beech, 2009: 352). It has become, in short, a major site ‘for the organization of knowledge about education’ (Rizvi & Lingard, 2010: 79), wielding ‘a degree of power that has arguably enabled it to shape the educational agendas of nations throughout the Global South’ and beyond (Menashy, 2012).

So, is there any problem in the world of ELT taking up the inclusion of ‘life skills’? I think there is. The first is one of definition. Creativity and critical thinking are very poorly defined, meaning very different things to different people, so it is not always clear what is being taught. Following on from this, there is substantial debate about whether such skills can actually be taught at all, and, if they can, how they should be taught. It seems highly unlikely that the tokenistic way in which they are ‘taught’ in most published ELT courses can be of any positive impact. But this is not my main reservation, which is that, by and large, we have come to uncritically accept the idea that English language learning is mostly concerned with preparation for the workplace (see my earlier post ‘The EdTech Imaginary in ELT’).

Is there any problem with the promotion of digital technologies in ELT? Again, I think there is, and a good proportion of the posts on this blog have argued for the need for circumspection in rolling out more technology in language learning and teaching. My main reason is that while it is clear that this trend is beneficial to technology vendors, it is much less clear that advantages will necessarily accrue to learners. Beyond this, there must be serious concerns about data ownership, privacy, and the way in which the datafication of education, led by businesses and governments in the Global North, is changing what counts as good education, a good student or an effective teacher, especially in the Global South. ‘Data and metrics,’ observe Williamson et al. (2020: 353), ‘do not just reflect what they are designed to measure, but actively loop back into action that can change the very thing that was measured in the first place’.

And what about tools for evaluating teacher competences? Here I would like to provide a little more background. There is, first of all, a huge question mark about how accurately such tools measure what they are supposed to measure. This may not matter too much if the tool is only used for self-evaluation or self-development, but ‘once smart systems of data collection and social control are available, they are likely to be widely applied for other purposes’ (Sadowski, 2020: 138). Jaime Saavedra, head of education at the World Bank, insists that the World Bank’s ‘Teach’ tool is not for evaluation and is not useful for firing teachers who perform badly.

Saavedra needs teachers to buy into the tool, so he obviously doesn’t want to scare them off. However, ‘Teach’ clearly is an evaluation tool (if not, what is it?) and, as with other tools (I’m thinking of CEFR and teacher competency frameworks in ELT), its purposes will evolve. Eric Hanushek, an education economist at Stanford University, has commented that ‘this is a clear evaluation tool at the probationary stage … It provides a basis for counseling new teachers on how they should behave … but then again if they don’t change over the first few years you also have information you should use.

At this point, it is useful to take a look at the World Bank’s attitudes towards teachers. Teachers are seen to be at the heart of the ‘learning crisis’. However, the greatest focus in World Bank documents is on (1) teacher absenteeism in some countries, (2) unskilled and demotivated teachers, and (3) the reluctance of teachers and their unions to back World Bank-sponsored reforms. As real as these problems are, it is important to understand that the Bank has been complicit in them:

For decades, the Bank has criticised pre-service and in-service teacher training as not cost-effective For decades, the Bank has been pushing the hiring of untrained contract teachers as a cheap fix and a way to get around teacher unions – and contract teachers are again praised in the World Bank Development Report (WDR). This contradicts the occasional places in the WDR in which the Bank argues that developing countries need to follow the lead of the few countries that attract the best students to teaching, improve training, and improve working conditions. There is no explicit evidence offered at all for the repeated claim that teachers are unmotivated and need to be controlled and monitored to do their job. The Bank has a long history of blaming teachers and teacher unions for educational failures. The Bank implicitly argues that the problem of teacher absenteeism, referred to throughout the report, means teachers are unmotivated, but that simply is not true. Teacher absenteeism is not a sign of low motivation. Teacher salaries are abysmally low, as is the status of teaching. Because of this, teaching in many countries has become an occupation of last resort, yet it still attracts dedicated teachers. Once again, the Bank has been very complicit in this state of affairs as it, and the IMF, for decades have enforced neoliberal, Washington Consensus policies which resulted in government cutbacks and declining real salaries for teachers around the world. It is incredible that economists at the Bank do not recognise that the deterioration of salaries is the major cause of teacher absenteeism and that all the Bank is willing to peddle are ineffective and insulting pay-for-performance schemes. (Klees, 2017).

The SABER framework (referred to above) focuses very clearly on policies for hiring, rewarding and firing teachers.

[The World Bank] places the private sector’s methods of dealing with teachers as better than those of the public sector, because it is more ‘flexible’. In other words, it is possible to say that teachers can be hired and fired more easily; that is, hired without the need of organizing a public competition and fired if they do not achieve the expected outcomes as, for example, students’ improvements in international test scores. Further, the SABER document states that ‘Flexibility in teacher contracting is one of the primary motivations for engaging the private sector’ (World Bank, 2011: 4). This affirmation seeks to reduce expenditures on teachers while fostering other expenses such as the creation of testing schemes and spending more on ICTs, as well as making room to expand the hiring of private sector providers to design curriculum, evaluate students, train teachers, produce education software, and books. (De Siqueira, 2012).

The World Bank has argued consistently for a reduction of education costs by driving down teachers’ salaries. One of the authors of the World Bank Development Report 2018 notes that ‘in most countries, teacher salaries consume the lion’s share of the education budget, so there are already fewer resources to implement other education programs’. Another World Bank report (2007) makes the importance of ‘flexible’ hiring and lower salaries very clear:

In particular, recent progress in primary education in Francophone countries resulted from reduced teacher costs, especially through the recruitment of contractual teachers, generally at about 50% the salary of civil service teachers. (cited in Compton & Weiner, 2008: 7).

Merit pay (or ‘pay for performance’) is another of the Bank’s preferred wheezes. Despite enormous problems in reaching fair evaluations of teachers’ work and a distinct lack of convincing evidence that merit pay leads to anything positive (and may actually be counter-productive) (De Bruyckere et al., 2018: 143 – 147), the Bank is fully committed to the idea. Perhaps this is connected to the usefulness of merit pay in keeping teachers on their toes, compliant and fearful of losing their jobs, rather than any desire to improve teacher effectiveness?

There is evidence that this may be the case. Yet another World Bank report (Bau & Das, 2017) argues, on the basis of research, that improved TVA (teacher value added) does not correlate with wages in the public sector (where it is hard to fire teachers), but it does in the private sector. The study found that ‘a policy change that shifted public hiring from permanent to temporary contracts, reducing wages by 35 percent, had no adverse impact on TVA’. All of which would seem to suggest that improving the quality of teaching is of less importance to the Bank than flexible hiring and firing. This is very much in line with a more general advocacy of making education fit for the world of work. Lois Weiner of New Jersey City University puts it like this:

The architects of [GERM] policies—imposed first in developing countries—openly state that the changes will make education better fit the new global economy by producing workers who are (minimally) educated for jobs that require no more than a 7th or 8th grade education; while a small fraction of the population receive a high quality education to become the elite who oversee finance, industry, and technology. Since most workers do not need to be highly educated, it follows that teachers with considerable formal education and experience are neither needed nor desired because they demand higher wages, which is considered a waste of government money. Most teachers need only be “good enough”—as one U.S. government official phrased it—to follow scripted materials that prepare students for standardized tests. (Weiner, 2012).

It seems impossible to separate the World Bank’s ‘Teach’ tool from the broader goals of GERM. Teacher evaluation tools, like the teaching of 21st century skills and the datafication of education, need to be understood properly, I think, as means to an end. It’s time to spell out what that end is.

The World Bank’s mission is ‘to end extreme poverty (by reducing the share of the global population that lives in extreme poverty to 3 percent by 2030)’ and ‘to promote shared prosperity (by increasing the incomes of the poorest 40 percent of people in every country)’. Its education activities are part of this broad aim and are driven by subscription to human capital theory (a view of the skills, knowledge and experience of individuals in terms of their ability to produce economic value). This may be described as the ‘economization of education’: a shift in educational concerns away from ‘such things as civic participation, protecting human rights, and environmentalism to economic growth and employment’ (Spring, 2015: xiii). Both students and teachers are seen as human capital. For students, human capital education places an emphasis on the cognitive skills needed to succeed in the workplace and the ‘soft skills’, needed to function in the corporate world (Spring, 2015: 2). Accordingly, World Bank investments require ‘justifications on the basis of manpower demands’ (Heyneman, 2003: 317). One of the Bank’s current strategic priorities is the education of girls: although human rights and equity may also play a part, the Bank’s primary concern is that ‘Not Educating Girls Costs Countries Trillions of Dollars’ .

According to the Bank’s logic, its educational aims can best be achieved through a combination of support for the following:

  • cost accounting and quantification (since returns on investment must be carefully measured)
  • competition and market incentives (since it is believed that the ‘invisible hand’ of the market leads to the greatest benefits)
  • the private sector in education and a rolling back of the role of the state (since it is believed that private ownership improves efficiency)

The package of measures is a straightforward reflection of ‘what Western mainstream economists believe’ (Castro, 2009: 474).

Mainstream Western economics is, however, going through something of a rocky patch right now. Human capital theory is ‘useful when prevailing conditions are right’ (Jones, 2007: 248), but prevailing conditions are not right in much of the world (even in the United States), and the theory ‘for the most part ignores the intersections of poverty, equity and education’ (Menashy, 2012). In poorer countries evidence for the positive effects of markets in education is in very short supply, and even in richer countries it is still not conclusive (Verger & Bonal, 2012: 135). An OECD Education Paper (Waslander et al., 2010: 64) found that the effects of choice and competition between schools were at best small, if indeed any effects were found at all. Similarly, the claim that privatization improves efficiency is not sufficiently supported by evidence. Analyses of PISA data would seem to indicate that, ‘all else being equal (especially when controlling for the socio-economic status of the students), the type of ownership of the school, whether it is a private or a state school, has only modest effects on student achievement or none at all’ (Verger & Bonal, 2012: 133). Educational privatization as a one-size-fits-all panacea to educational problems has little to recommend it.

There are, then, serious limitations in the Bank’s theoretical approach. Its practical track record is also less than illustrious, even by the Bank’s own reckoning. Many of the Bank’s interventions have proved very ‘costly to developing countries. At the Bank’s insistence countries over-invested in vocational and technical education. Because of the narrow definition of recurrent costs, countries ignored investments in reading materials and in maintaining teacher salaries. Later at the Bank’s insistence, countries invested in thousands of workshops and laboratories that, for the most part, became useless ‘white elephants’ (Heyneman, 2003: 333).

As a bank, the World Bank is naturally interested in the rate of return of investment in that capital, and is therefore concerned with efficiency and efficacy. This raises the question of ‘Effective for what?’ and given that what may be effective for one individual or group may not necessarily be effective for another individual or group, one may wish to add a second question: ‘Effective for whom?’ (Biesta, 2020: 31). Critics of the World Bank, of whom there are many, argue that its policies serve ‘the interests of corporations by keeping down wages for skilled workers, cause global brain migration to the detriment of developing countries, undermine local cultures, and ensure corporate domination by not preparing school graduates who think critically and are democratically oriented’ (Spring, 2015: 56). Lest this sound a bit harsh, we can turn to the Bank’s own commissioned history: ‘The way in which [the Bank’s] ideology has been shaped conforms in significant degree to the interests and conventional wisdom of its principal stockholders [i.e. bankers and economists from wealthy nations]. International competitive bidding, reluctance to accord preferences to local suppliers, emphasis on financing foreign exchange costs, insistence on a predominant use of foreign consultants, attitudes toward public sector industries, assertion of the right to approve project managers – all proclaim the Bank to be a Western capitalist institution’ (Mason & Asher, 1973: 478 – 479).

The teaching of ‘life skills’, the promotion of data-capturing digital technologies and the push to evaluate teachers’ performance are, then, all closely linked to the agenda of the World Bank, and owe their existence in the ELT landscape, in no small part, to the way that the World Bank has shaped educational discourse. There is, however, one other connection between ELT and the World Bank which must be mentioned.

The World Bank’s foreign language instructional goals are directly related to English as a global language. The Bank urges, ‘Policymakers in developing countries …to ensure that young people acquire a language with more than just local use, preferably one used internationally.’ What is this international language? First, the World Bank mentions that schools of higher education around the world are offering courses in English. In addition, the Bank states, ‘People seeking access to international stores of knowledge through the internet require, principally, English language skills.’ (Spring, 2015: 48).

Without the World Bank, then, there might be a lot less English language teaching than there is. I have written this piece to encourage people to think more about the World Bank, its policies and particular instantiations of those policies. You might or might not agree that the Bank is an undemocratic, technocratic, neoliberal institution unfit for the necessities of today’s world (Klees, 2017). But whatever you think about the World Bank, you might like to consider the answers to Tony Benn’s ‘five little democratic questions’ (quoted in Sardowski, 2020: 17):

  • What power has it got?
  • Where did it get this power from?
  • In whose interests does it exercise this power?
  • To whom is it accountable?
  • How can we get rid of it?

References

Bau, N. and Das, J. (2017). The Misallocation of Pay and Productivity in the Public Sector : Evidence from the Labor Market for Teachers. Policy Research Working Paper; No. 8050. World Bank, Washington, DC. Retrieved [18 May 2020] from https://openknowledge.worldbank.org/handle/10986/26502

Beech, J. (2009). Who is Strolling Through The Global Garden? International Agencies and Educational Transfer. In Cowen, R. and Kazamias, A. M. (Eds.) Second International Handbook of Comparative Education. Dordrecht: Springer. pp. 341 – 358

Biesta, G. (2020). Educational Research. London: Bloomsbury.

Castro, C. De M., (2009). Can Multilateral Banks Educate The World? In Cowen, R. and Kazamias, A. M. (Eds.) Second International Handbook of Comparative Education. Dordrecht: Springer. pp. 455 – 478

Compton, M. and Weiner, L. (Eds.) (2008). The Global Assault on Teaching, Teachers, and their Unions. New York: Palgrave Macmillan

De Bruyckere, P., Kirschner, P.A. and Hulshof, C. (2020). More Urban Myths about Learning and Education. New York: Routledge.

De Siqueira, A. C. (2012). The 2020 World Bank Education Strategy: Nothing New, or the Same Old Gospel. In Klees, S. J., Samoff, J. and Stromquist, N. P. (Eds.) The World Bank and Education. Rotterdam: Sense Publishers. pp. 69 – 81

Heyneman, S.P. (2003). The history and problems in the making of education policy at the World Bank 1960–2000. International Journal of Educational Development 23 (2003) pp. 315–337. Retrieved [18 May 2020] from https://www.academia.edu/29593153/The_History_and_Problems_in_the_Making_of_Education_Policy_at_the_World_Bank_1960_2000

Jones, P. W. (2007). World Bank Financing of Education. 2nd edition. Abingdon, Oxon.: Routledge.

Klees, S. (2017). A critical analysis of the World Bank’s World Development Report on education. Retrieved [18 May 2020] from: https://www.brettonwoodsproject.org/2017/11/critical-analysis-world-banks-world-development-report-education/

Mason, E. S. & Asher, R. E. (1973). The World Bank since Bretton Woods. Washington, DC: Brookings Institution.

Menashy, F. (2012). Review of Klees, S J., Samoff, J. & Stromquist, N. P. (Eds) (2012). The World Bank and Education: Critiques and Alternatives .Rotterdam: Sense Publishers. Education Review, 15. Retrieved [18 May 2020] from https://www.academia.edu/7672656/Review_of_The_World_Bank_and_Education_Critiques_and_Alternatives

Rizvi, F. & Lingard, B. (2010). Globalizing Education Policy. Abingdon, Oxon.: Routledge.

Sadowski, J. (2020). Too Smart. Cambridge, MA.: MIT Press.

Sahlberg, P. (2016). The global educational reform movement and its impact on schooling. In K. Mundy, A. Green, R. Lingard, & A. Verger (Eds.), The handbook of global policy and policymaking in education. New York, NY: Wiley-Blackwell. pp.128 – 144

Selwyn, N. (2013). Education in a Digital World. New York: Routledge.

Spring, J. (2015). Globalization of Education 2nd Edition. New York: Routledge.

Trilling, B. & C. Fadel (2009). 21st Century Skills. San Francisco: Wiley

Verger, A. & Bonal, X. (2012). ‘All Things Being Equal?’ In Klees, S. J., Samoff, J. and Stromquist, N. P. (Eds.) The World Bank and Education. Rotterdam: Sense Publishers. pp. 69 – 81

Waslander, S., Pater, C. & van der Weide, M. (2010). Markets in Education: An analytical review of empirical research on market mechanisms in education. OECD EDU Working Paper 52.

Weiner, L. (2012). Social Movement Unionism: Teachers Can Lead the Way. Reimagine, 19 (2) Retrieved [18 May 2020] from: https://www.reimaginerpe.org/19-2/weiner-fletcher

Williamson, B., Bayne, S. & Shay, S. (2020). The datafication of teaching in Higher Education: critical issues and perspectives, Teaching in Higher Education, 25:4, 351-365, DOI: 10.1080/13562517.2020.1748811

World Bank. (2013). Life skills : what are they, why do they matter, and how are they taught? (English). Adolescent Girls Initiative (AGI) learning from practice series. Washington DC ; World Bank. Retrieved [18 May 2020] from: http://documents.worldbank.org/curated/en/569931468331784110/Life-skills-what-are-they-why-do-they-matter-and-how-are-they-taught

Colloquium

At the beginning of March, I’ll be going to Cambridge to take part in a Digital Learning Colloquium (for more information about the event, see here ). One of the questions that will be explored is how research might contribute to the development of digital language learning. In this, the first of two posts on the subject, I’ll be taking a broad overview of the current state of play in edtech research.

I try my best to keep up to date with research. Of the main journals, there are Language Learning and Technology, which is open access; CALICO, which offers quite a lot of open access material; and reCALL, which is the most restricted in terms of access of the three. But there is something deeply frustrating about most of this research, and this is what I want to explore in these posts. More often than not, research articles end with a call for more research. And more often than not, I find myself saying ‘Please, no, not more research like this!’

First, though, I would like to turn to a more reader-friendly source of research findings. Systematic reviews are, basically literature reviews which can save people like me from having to plough through endless papers on similar subjects, all of which contain the same (or similar) literature review in the opening sections. If only there were more of them. Others agree with me: the conclusion of one systematic review of learning and teaching with technology in higher education (Lillejord et al., 2018) was that more systematic reviews were needed.

Last year saw the publication of a systematic review of research on artificial intelligence applications in higher education (Zawacki-Richter, et al., 2019) which caught my eye. The first thing that struck me about this review was that ‘out of 2656 initially identified publications for the period between 2007 and 2018, 146 articles were included for final synthesis’. In other words, only just over 5% of the research was considered worthy of inclusion.

The review did not paint a very pretty picture of the current state of AIEd research. As the second part of the title of this review (‘Where are the educators?’) makes clear, the research, taken as a whole, showed a ‘weak connection to theoretical pedagogical perspectives’. This is not entirely surprising. As Bates (2019) has noted: ‘since AI tends to be developed by computer scientists, they tend to use models of learning based on how computers or computer networks work (since of course it will be a computer that has to operate the AI). As a result, such AI applications tend to adopt a very behaviourist model of learning: present / test / feedback.’ More generally, it is clear that technology adoption (and research) is being driven by technology enthusiasts, with insufficient expertise in education. The danger is that edtech developers ‘will simply ‘discover’ new ways to teach poorly and perpetuate erroneous ideas about teaching and learning’ (Lynch, 2017).

This, then, is the first of my checklist of things that, collectively, researchers need to do to improve the value of their work. The rest of this list is drawn from observations mostly, but not exclusively, from the authors of systematic reviews, and mostly come from reviews of general edtech research. In the next blog post, I’ll look more closely at a recent collection of ELT edtech research (Mavridi & Saumell, 2020) to see how it measures up.

1 Make sure your research is adequately informed by educational research outside the field of edtech

Unproblematised behaviourist assumptions about the nature of learning are all too frequent. References to learning styles are still fairly common. The most frequently investigated skill that is considered in the context of edtech is critical thinking (Sosa Neira, et al., 2017), but this is rarely defined and almost never problematized, despite a broad literature that questions the construct.

2 Adopt a sceptical attitude from the outset

Know your history. Decades of technological innovation in education have shown precious little in the way of educational gains and, more than anything else, have taught us that we need to be sceptical from the outset. ‘Enthusiasm and praise that are directed towards ‘virtual education, ‘school 2.0’, ‘e-learning and the like’ (Selwyn, 2014: vii) are indications that the lessons of the past have not been sufficiently absorbed (Levy, 2016: 102). The phrase ‘exciting potential’, for example, should be banned from all edtech research. See, for example, a ‘state-of-the-art analysis of chatbots in education’ (Winkler & Söllner, 2018), which has nothing to conclude but ‘exciting potential’. Potential is fine (indeed, it is perhaps the only thing that research can unambiguously demonstrate – see section 3 below), but can we try to be a little more grown-up about things?

3 Know what you are measuring

Measuring learning outcomes is tricky, to say the least, but it’s understandable that researchers should try to focus on them. Unfortunately, ‘the vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them’ (Lai & Bower, 2019). Metrics such as student grades are hard to interpret, not least because of the large number of variables and the danger of many things being conflated in one score. Equally, or possibly even more, problematic, are self-reporting measures which are rarely robust. It seems that surveys are the most widely used instrument in qualitative research (Sosa Neira, et al., 2017), but these will tell us little or nothing when used for short-term interventions (see point 5 below).

4 Ensure that the sample size is big enough to mean something

In most of the research into digital technology in education that was analysed in a literature review carried out for the Scottish government (ICF Consulting Services Ltd, 2015), there were only ‘small numbers of learners or teachers or schools’.

5 Privilege longitudinal studies over short-term projects

The Scottish government literature review (ICF Consulting Services Ltd, 2015), also noted that ‘most studies that attempt to measure any outcomes focus on short and medium term outcomes’. The fact that the use of a particular technology has some sort of impact over the short or medium term tells us very little of value. Unless there is very good reason to suspect the contrary, we should assume that it is a novelty effect that has been captured (Levy, 2016: 102).

6 Don’t forget the content

The starting point of much edtech research is the technology, but most edtech, whether it’s a flashcard app or a full-blown Moodle course, has content. Research reports rarely give details of this content, assuming perhaps that it’s just fine, and all that’s needed is a little tech to ‘present learners with the ‘right’ content at the ‘right’ time’ (Lynch, 2017). It’s a foolish assumption. Take a random educational app from the Play Store, a random MOOC or whatever, and the chances are you’ll find it’s crap.

7 Avoid anecdotal accounts of technology use in quasi-experiments as the basis of a ‘research article’

Control (i.e technology-free) groups may not always be possible but without them, we’re unlikely to learn much from a single study. What would, however, be extremely useful would be a large, collated collection of such action-research projects, using the same or similar technology, in a variety of settings. There is a marked absence of this kind of work.

8 Enough already of higher education contexts

Researchers typically work in universities where they have captive students who they can carry out research on. But we have a problem here. The systematic review of Lundin et al (2018), for example, found that ‘studies on flipped classrooms are dominated by studies in the higher education sector’ (besides lacking anchors in learning theory or instructional design). With some urgency, primary and secondary contexts need to be investigated in more detail, not just regarding flipped learning.

9 Be critical

Very little edtech research considers the downsides of edtech adoption. Online safety, privacy and data security are hardly peripheral issues, especially with younger learners. Ignoring them won’t make them go away.

More research?

So do we need more research? For me, two things stand out. We might benefit more from, firstly, a different kind of research, and, secondly, more syntheses of the work that has already been done. Although I will probably continue to dip into the pot-pourri of articles published in the main CALL journals, I’m looking forward to a change at the CALICO journal. From September of this year, one issue a year will be thematic, with a lead article written by established researchers which will ‘first discuss in broad terms what has been accomplished in the relevant subfield of CALL. It should then outline which questions have been answered to our satisfaction and what evidence there is to support these conclusions. Finally, this article should pose a “soft” research agenda that can guide researchers interested in pursuing empirical work in this area’. This will be followed by two or three empirical pieces that ‘specifically reflect the research agenda, methodologies, and other suggestions laid out in the lead article’.

But I think I’ll still have a soft spot for some of the other journals that are coyer about their impact factor and that can be freely accessed. How else would I discover (it would be too mean to give the references here) that ‘the effective use of new technologies improves learners’ language learning skills’? Presumably, the ineffective use of new technologies has the opposite effect? Or that ‘the application of modern technology represents a significant advance in contemporary English language teaching methods’?

References

Bates, A. W. (2019). Teaching in a Digital Age Second Edition. Vancouver, B.C.: Tony Bates Associates Ltd. Retrieved from https://pressbooks.bccampus.ca/teachinginadigitalagev2/

ICF Consulting Services Ltd (2015). Literature Review on the Impact of Digital Technology on Learning and Teaching. Edinburgh: The Scottish Government. https://dera.ioe.ac.uk/24843/1/00489224.pdf

Lai, J.W.M. & Bower, M. (2019). How is the use of technology in education evaluated? A systematic review. Computers & Education, 133(1), 27-42. Elsevier Ltd. Retrieved January 14, 2020 from https://www.learntechlib.org/p/207137/

Levy, M. 2016. Researching in language learning and technology. In Farr, F. & Murray, L. (Eds.) The Routledge Handbook of Language Learning and Technology. Abingdon, Oxon.: Routledge. pp.101 – 114

Lillejord S., Børte K., Nesje K. & Ruud E. (2018). Learning and teaching with technology in higher education – a systematic review. Oslo: Knowledge Centre for Education https://www.forskningsradet.no/siteassets/publikasjoner/1254035532334.pdf

Lundin, M., Bergviken Rensfeldt, A., Hillman, T. et al. (2018). Higher education dominance and siloed knowledge: a systematic review of flipped classroom research. International Journal of Educational Technology in Higher Education 15, 20 (2018) doi:10.1186/s41239-018-0101-6

Lynch, J. (2017). How AI Will Destroy Education. Medium, November 13, 2017. https://buzzrobot.com/how-ai-will-destroy-education-20053b7b88a6

Mavridi, S. & Saumell, V. (Eds.) (2020). Digital Innovations and Research in Language Learning. Faversham, Kent: IATEFL

Selwyn, N. (2014). Distrusting Educational Technology. New York: Routledge

Sosa Neira, E. A., Salinas, J. and de Benito Crosetti, B. (2017). Emerging Technologies (ETs) in Education: A Systematic Review of the Literature Published between 2006 and 2016. International Journal of Emerging Technologies in Education, 12 (5). https://online-journals.org/index.php/i-jet/article/view/6939

Winkler, R. & Söllner, M. (2018): Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis. In: Academy of Management Annual Meeting (AOM). Chicago, USA. https://www.alexandria.unisg.ch/254848/1/JML_699.pdf

Zawacki-Richter, O., Bond, M., Marin, V. I. And Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education 2019

In my last post , I asked why it is so easy to believe that technology (in particular, technological innovations) will offer solutions to whatever problems exist in language learning and teaching. A simple, but inadequate, answer is that huge amounts of money have been invested in persuading us. Without wanting to detract from the significance of this, it is clearly not sufficient as an explanation. In an attempt to develop my own understanding, I have been turning more and more to the idea of ‘social imaginaries’. In many ways, this is also an attempt to draw together the various interests that I have had since starting this blog.

The Canadian philosopher, Charles Taylor, describes a ‘social imaginary’ as a ‘common understanding that makes possible common practices and a widely shared sense of legitimacy’ (Taylor, 2004: 23). As a social imaginary develops over time, it ‘begins to define the contours of [people’s] worlds and can eventually come to count as the taken-for-granted shape of things, too obvious to mention’ (Taylor, 2004: 29). It is, however, not just a set of ideas or a shared narrative: it is also a set of social practices that enact those understandings, whilst at the same time modifying or solidifying them. The understandings make the practices possible, and it is the practices that largely carry the understanding (Taylor, 2004: 25). In the process, the language we use is filled with new associations and our familiarity with these associations shapes ‘our perceptions and expectations’ (Worster, 1994, quoted in Moore, 2015: 33). A social imaginary, then, is a complex system that is not technological or economic or social or political or educational, but all of these (Urry, 2016). The image of the patterns of an amorphous mass of moving magma (Castoriadis, 1987), flowing through pre-existing channels, but also, at times, striking out along new paths, may offer a helpful metaphor.

Lava flow Hawaii

Technology, of course, plays a key role in contemporary social imaginaries and the term ‘sociotechnical imaginary’ is increasingly widely used. The understandings of the sociotechnical imaginary typically express visions of social progress and a desirable future that is made possible by advances in science and technology (Jasanoff & Kim, 2015: 4). In education, technology is presented as capable of overcoming human failings and the dark ways of the past, of facilitating a ‘pedagogical utopia of natural, authentic teaching and learning’ (Friesen, forthcoming). As such understandings become more widespread and as the educational practices (platforms, apps, etc.) which both shape and are shaped by them become equally widespread, technology has come to be seen as a ‘solution’ to the ‘problem’ of education (Friesen, forthcoming). We need to be careful, however, that having shaped the technology, it does not comes to shape us (see Cobo, 2019, for a further exploration of this idea).

As a way of beginning to try to understand what is going on in edtech in ELT, which is not so very different from what is taking place in education more generally, I have sketched a number of what I consider key components of the shared understandings and the social practices that are related to them. These are closely interlocking pieces and each of them is itself embedded in much broader understandings. They evolve over time and their history can be traced quite easily. Taken together, they do, I think, help us to understand a little more why technology in ELT seems so seductive.

1 The main purpose of English language teaching is to prepare people for the workplace

There has always been a strong connection between learning an additional living language (such as English) and preparing for the world of work. The first modern language schools, such as the Berlitz schools at the end of the 19th century with their native-speaker teachers and monolingual methods, positioned themselves as primarily vocational, in opposition to the kinds of language teaching taking place in schools and universities, which were more broadly humanistic in their objectives. Throughout the 20th century, and especially as English grew as a global language, the public sector, internationally, grew closer to the methods and objectives of the private schools. The idea that learning English might serve other purposes (e.g. cultural enrichment or personal development) has never entirely gone away, as witnessed by the Council of Europe’s list of objectives (including the promotion of mutual understanding and European co-operation, and the overcoming of prejudice and discrimination) in the Common European Framework, but it is often forgotten.

The clarion calls from industry to better align education with labour markets, present and future, grow louder all the time, often finding expression in claims that ‘education is unfit for purpose.’ It is invariably assumed that this purpose is to train students in the appropriate skills to enhance their ‘human capital’ in an increasingly competitive and global market (Lingard & Gale, 2007). Educational agendas are increasingly set by the world of business (bodies like the OECD or the World Economic Forum, corporations like Google or Microsoft, and national governments which share their priorities (see my earlier post about neo-liberalism and solutionism ).

One way in which this shift is reflected in English language teaching is in the growing emphasis that is placed on ‘21st century skills’ in teaching material. Sometimes called ‘life skills’, they are very clearly concerned with the world of work, rather than the rest of our lives. The World Economic Forum’s 2018 Future of Jobs survey lists the soft skills that are considered important in the near future and they include ‘creativity’, ‘critical thinking’, ‘emotional intelligence’ and ‘leadership’. (The fact that the World Economic Forum is made up of a group of huge international corporations (e.g. J.P. Morgan, HSBC, UBS, Johnson & Johnson) with a very dubious track record of embezzlement, fraud, money-laundering and tax evasion has not resulted in much serious, public questioning of the view of education expounded by the WEF.)

Without exception, the ELT publishers have brought these work / life skills into their courses, and the topic is an extremely popular one in ELT blogs and magazines, and at conferences. Two of the four plenaries at this year’s international IATEFL conference are concerned with these skills. Pearson has a wide range of related products, including ‘a four-level competency-based digital course that provides engaging instruction in the essential work and life skills competencies that adult learners need’. Macmillan ELT made ‘life skills’ the central plank of their marketing campaign and approach to product design, and even won a British Council ELTon (see below) Award for ‘Innovation in teacher resources) in 2015 for their ‘life skills’ marketing campaign. Cambridge University Press has developed a ‘Framework for Life Competencies’ which allows these skills to be assigned numerical values.

The point I am making here is not that these skills do not play an important role in contemporary society, nor that English language learners may not benefit from some training in them. The point, rather, is that the assumption that English language learning is mostly concerned with preparation for the workplace has become so widespread that it becomes difficult to think in another way.

2 Technological innovation is good and necessary

The main reason that soft skills are deemed to be so important is that we live in a rapidly-changing world, where the unsubstantiated claim that 85% (or whatever other figure comes to mind) of current jobs won’t exist 10 years from now is so often repeated that it is taken as fact . Whether or not this is true is perhaps less important to those who make the claim than the present and the future that they like to envisage. The claim is, at least, true-ish enough to resonate widely. Since these jobs will disappear, and new ones will emerge, because of technological innovations, education, too, will need to innovate to keep up.

English language teaching has not been slow to celebrate innovation. There were coursebooks called ‘Cutting Edge’ (1998) and ‘Innovations’ (2005), but more recently the connections between innovation and technology have become much stronger. The title of the recent ‘Language Hub’ (2019) was presumably chosen, in part, to conjure up images of digital whizzkids in fashionable co-working start-up spaces. Technological innovation is explicitly promoted in the Special Interest Groups of IATEFL and TESOL. Despite a singular lack of research that unequivocally demonstrates a positive connection between technology and language learning, the former’s objective is ‘to raise awareness among ELT professionals of the power of learning technologies to assist with language learning’. There is a popular annual conference, called InnovateELT , which has the tagline ‘Be Part of the Solution’, and the first problem that this may be a solution to is that our students need to be ‘ready to take on challenging new careers’.

Last, but by no means least, there are the annual British Council ELTon awards  with a special prize for digital innovation. Among the British Council’s own recent innovations are a range of digitally-delivered resources to develop work / life skills among teens.

Again, my intention (here) is not to criticise any of the things mentioned in the preceding paragraphs. It is merely to point to a particular structure of feeling and the way that is enacted and strengthened through material practices like books, social groups, conferences and other events.

3 Technological innovations are best driven by the private sector

The vast majority of people teaching English language around the world work in state-run primary and secondary schools. They are typically not native-speakers of English, they hold national teaching qualifications and they are frequently qualified to teach other subjects in addition to English (often another language). They may or may not self-identify as teachers of ‘ELT’ or ‘EFL’, often seeing themselves more as ‘school teachers’ or ‘language teachers’. People who self-identify as part of the world of ‘ELT or ‘TEFL’ are more likely to be native speakers and to work in the private sector (including private or semi-private language schools, universities (which, in English-speaking countries, are often indistinguishable from private sector institutions), publishing companies, and freelancers). They are more likely to hold international (TEFL) qualifications or higher degrees, and they are less likely to be involved in the teaching of other languages.

The relationship between these two groups is well illustrated by the practice of training days, where groups of a few hundred state-school teachers participate in workshops organised by publishing companies and delivered by ELT specialists. In this context, state-school teachers are essentially in a client role when they are in contact with the world of ‘ELT’ – as buyers or potential buyers of educational products, training or technology.

Technological innovation is invariably driven by the private sector. This may be in the development of technologies (platforms, apps and so on), in the promotion of technology (through training days and conference sponsorship, for example), or in training for technology (with consultancy companies like ELTjam or The Consultants-E, which offer a wide range of technologically oriented ‘solutions’).

As in education more generally, it is believed that the private sector can be more agile and more efficient than state-run bodies, which continue to decline in importance in educational policy-setting. When state-run bodies are involved in technological innovation in education, it is normal for them to work in partnership with the private sector.

4 Accountability is crucial

Efficacy is vital. It makes no sense to innovate unless the innovations improve something, but for us to know this, we need a way to measure it. In a previous post , I looked at Pearson’s ‘Asking More: the Path to Efficacy’ by CEO John Fallon (who will be stepping down later this year). Efficacy in education, says Fallon, is ‘making a measurable impact on someone’s life through learning’. ‘Measurable’ is the key word, because, as Fallon claims, ‘it is increasingly possible to determine what works and what doesn’t in education, just as in healthcare.’ We need ‘a relentless focus’ on ‘the learning outcomes we deliver’ because it is these outcomes that can be measured in ‘a systematic, evidence-based fashion’. Measurement, of course, is all the easier when education is delivered online, ‘real-time learner data’ can be captured, and the power of analytics can be deployed.

Data is evidence, and it’s as easy to agree on the importance of evidence as it is hard to decide on (1) what it is evidence of, and (2) what kind of data is most valuable. While those questions remain largely unanswered, the data-capturing imperative invades more and more domains of the educational world.

English language teaching is becoming data-obsessed. From language scales, like Pearson’s Global Scale of English to scales of teacher competences, from numerically-oriented formative assessment practices (such as those used on many LMSs) to the reporting of effect sizes in meta-analyses (such as those used by John Hattie and colleagues), datafication in ELT accelerates non-stop.

The scales and frameworks are all problematic in a number of ways (see, for example, this post on ‘The Mismeasure of Language’) but they have undeniably shaped the way that we are able to think. Of course, we need measurable outcomes! If, for the present, there are privacy and security issues, it is to be hoped that technology will find solutions to them, too.

REFERENCES

Castoriadis, C. (1987). The Imaginary Institution of Society. Cambridge: Polity Press.

Cobo, C. (2019). I Accept the Terms and Conditions. Montevideo: International Development Research Centre / Center for Research Ceibal Foundation. https://adaptivelearninginelt.files.wordpress.com/2020/01/41acf-cd84b5_7a6e74f4592c460b8f34d1f69f2d5068.pdf

Friesen, N. (forthcoming) The technological imaginary in education, or: Myth and enlightenment in ‘Personalized Learning’. In M. Stocchetti (Ed.) The Digital Age and its Discontents. University of Helsinki Press. Available at https://www.academia.edu/37960891/The_Technological_Imaginary_in_Education_or_Myth_and_Enlightenment_in_Personalized_Learning_

Jasanoff, S. & Kim, S.-H. (2015). Dreamscapes of Modernity. Chicago: University of Chicago Press.

Lingard, B. & Gale, T. (2007). The emergent structure of feeling: what does it mean for critical educational studies and research?, Critical Studies in Education, 48:1, pp. 1-23

Moore, J. W. (2015). Capitalism in the Web of Life. London: Verso.

Robbins, K. & Webster, F. (1989]. The Technical Fix. Basingstoke: Macmillan Education.

Taylor, C. (2014). Modern Social Imaginaries. Durham, NC: Duke University Press.

Urry, J. (2016). What is the Future? Cambridge: Polity Press.

 

Back in the middle of the last century, the first interactive machines for language teaching appeared. Previously, there had been phonograph discs and wire recorders (Ornstein, 1968: 401), but these had never really taken off. This time, things were different. Buoyed by a belief in the power of technology, along with the need (following the Soviet Union’s successful Sputnik programme) to demonstrate the pre-eminence of the United States’ technological expertise, the interactive teaching machines that were used in programmed instruction promised to revolutionize language learning (Valdman, 1968: 1). From coast to coast, ‘tremors of excitement ran through professional journals and conferences and department meetings’ (Kennedy, 1967: 871). The new technology was driven by hard science, supported and promoted by the one of the most well-known and respected psychologists and public intellectuals of the day (Skinner, 1961).

In classrooms, the machines acted as powerfully effective triggers in generating situational interest (Hidi & Renninger, 2006). Even more exciting than the mechanical teaching machines were the computers that were appearing on the scene. ‘Lick’ Licklider, a pioneer in interactive computing at the Advanced Research Projects Agency in Arlington, Virginia, developed an automated drill routine for learning German by hooking up a computer, two typewriters, an oscilloscope and a light pen (Noble, 1991: 124). Students loved it, and some would ‘go on and on, learning German words until they were forced by scheduling to cease their efforts’. Researchers called the seductive nature of the technology ‘stimulus trapping’, and Licklider hoped that ‘before [the student] gets out from under the control of the computer’s incentives, [they] will learn enough German words’ (Noble, 1991: 125).

With many of the developed economies of the world facing a critical shortage of teachers, ‘an urgent pedagogical emergency’ (Hof, 2018), the new approach was considered to be extremely efficient and could equalise opportunity in schools across the country. It was ‘here to stay: [it] appears destined to make progress that could well go beyond the fondest dreams of its originators […] an entire industry is just coming into being and significant sales and profits should not be too long in coming’ (Kozlowski, 1961: 47).

Unfortunately, however, researchers and entrepreneurs had massively underestimated the significance of novelty effects. The triggered situational interest of the machines did not lead to intrinsic individual motivation. Students quickly tired of, and eventually came to dislike, programmed instruction and the machines that delivered it (McDonald et al.: 2005: 89). What’s more, the machines were expensive and ‘research studies conducted on its effectiveness showed that the differences in achievement did not constantly or substantially favour programmed instruction over conventional instruction (Saettler, 2004: 303). Newer technologies, with better ‘stimulus trapping’, were appearing. Programmed instruction lost its backing and disappeared, leaving as traces only its interest in clearly defined learning objectives, the measurement of learning outcomes and a concern with the efficiency of learning approaches.

Hot on the heels of programmed instruction came the language laboratory. Futuristic in appearance, not entirely unlike the deck of the starship USS Enterprise which launched at around the same time, language labs captured the public imagination and promised to explore the final frontiers of language learning. As with the earlier teaching machines, students were initially enthusiastic. Even today, when language labs are introduced into contexts where they may be perceived as new technology, they can lead to high levels of initial motivation (e.g. Ramganesh & Janaki, 2017).

Given the huge investments into these labs, it’s unfortunate that initial interest waned fast. By 1969, many of these rooms had turned into ‘“electronic graveyards,” sitting empty and unused, or perhaps somewhat glorified study halls to which students grudgingly repair to don headphones, turn down the volume, and prepare the next period’s history or English lesson, unmolested by any member of the foreign language faculty’ (Turner, 1969: 1, quoted in Roby, 2003: 527). ‘Many second language students shudder[ed] at the thought of entering into the bowels of the “language laboratory” to practice and perfect the acoustical aerobics of proper pronunciation skills. Visions of sterile white-walled, windowless rooms, filled with endless bolted-down rows of claustrophobic metal carrels, and overseen by a humorless, lab director, evoke[d] fear in the hearts of even the most stout-hearted prospective second-language learners (Wiley, 1990: 44).

By the turn of this century, language labs had mostly gone, consigned to oblivion by the appearance of yet newer technology: the internet, laptops and smartphones. Education had been on the brink of being transformed through new learning technologies for decades (Laurillard, 2008: 1), but this time it really was different. It wasn’t just one technology that had appeared, but a whole slew of them: ‘artificial intelligence, learning analytics, predictive analytics, adaptive learning software, school management software, learning management systems (LMS), school clouds. No school was without these and other technologies branded as ‘superintelligent’ by the late 2020s’ (Macgilchrist et al., 2019). The hardware, especially phones, was ubiquitous and, therefore, free. Unlike teaching machines and language laboratories, students were used to using the technology and expected to use their devices in their studies.

A barrage of publicity, mostly paid for by the industry, surrounded the new technologies. These would ‘meet the demands of Generation Z’, the new generation of students, now cast as consumers, who ‘were accustomed to personalizing everything’.  AR, VR, interactive whiteboards, digital projectors and so on made it easier to ‘create engaging, interactive experiences’. The ‘New Age’ technologies made learning fun and easy,  ‘bringing enthusiasm among the students, improving student engagement, enriching the teaching process, and bringing liveliness in the classroom’. On top of that, they allowed huge amounts of data to be captured and sold, whilst tracking progress and attendance. In any case, resistance to digital technology, said more than one language teaching expert, was pointless (Styring, 2015).slide

At the same time, technology companies increasingly took on ‘central roles as advisors to national governments and local districts on educational futures’ and public educational institutions came to be ‘regarded by many as dispensable or even harmful’ (Macgilchrist et al., 2019).

But, as it turned out, the students of Generation Z were not as uniformly enthusiastic about the new technology as had been assumed, and resistance to digital, personalized delivery in education was not long in coming. In November 2018, high school students at Brooklyn’s Secondary School for Journalism staged a walkout in protest at their school’s use of Summit Learning, a web-based platform promoting personalized learning developed by Facebook. They complained that the platform resulted in coursework requiring students to spend much of their day in front of a computer screen, that made it easy to cheat by looking up answers online, and that some of their teachers didn’t have the proper training for the curriculum (Leskin, 2018). Besides, their school was in a deplorable state of disrepair, especially the toilets. There were similar protests in Kansas, where students staged sit-ins, supported by their parents, one of whom complained that ‘we’re allowing the computers to teach and the kids all looked like zombies’ before pulling his son out of the school (Bowles, 2019). In Pennsylvania and Connecticut, some schools stopped using Summit Learning altogether, following protests.

But the resistance did not last. Protesters were accused of being nostalgic conservatives and educationalists kept largely quiet, fearful of losing their funding from the Chan Zuckerberg Initiative (Facebook) and other philanthro-capitalists. The provision of training in grit, growth mindset, positive psychology and mindfulness (also promoted by the technology companies) was ramped up, and eventually the disaffected students became more quiescent. Before long, the data-intensive, personalized approach, relying on the tools, services and data storage of particular platforms had become ‘baked in’ to educational systems around the world (Moore, 2018: 211). There was no going back (except for small numbers of ultra-privileged students in a few private institutions).

By the middle of the century (2155), most students, of all ages, studied with interactive screens in the comfort of their homes. Algorithmically-driven content, with personalized, adaptive tests had become the norm, but the technology occasionally went wrong, leading to some frustration. One day, two young children discovered a book in their attic. Made of paper with yellow, crinkly pages, where ‘the words stood still instead of moving the way they were supposed to’. The book recounted the experience of schools in the distant past, where ‘all the kids from the neighbourhood came’, sitting in the same room with a human teacher, studying the same things ‘so they could help one another on the homework and talk about it’. Margie, the younger of the children at 11 years old, was engrossed in the book when she received a nudge from her personalized learning platform to return to her studies. But Margie was reluctant to go back to her fractions. She ‘was thinking about how the kids must have loved it in the old days. She was thinking about the fun they had’ (Asimov, 1951).

References

Asimov, I. 1951. The Fun They Had. Accessed September 20, 2019. http://web1.nbed.nb.ca/sites/ASD-S/1820/J%20Johnston/Isaac%20Asimov%20-%20The%20fun%20they%20had.pdf

Bowles, N. 2019. ‘Silicon Valley Came to Kansas Schools. That Started a Rebellion’ The New York Times, April 21. Accessed September 20, 2019. https://www.nytimes.com/2019/04/21/technology/silicon-valley-kansas-schools.html

Hidi, S. & Renninger, K.A. 2006. ‘The Four-Phase Model of Interest Development’ Educational Psychologist, 41 (2), 111 – 127

Hof, B. 2018. ‘From Harvard via Moscow to West Berlin: educational technology, programmed instruction and the commercialisation of learning after 1957’ History of Education, 47 (4): 445-465

Kennedy, R.H. 1967. ‘Before using Programmed Instruction’ The English Journal, 56 (6), 871 – 873

Kozlowski, T. 1961. ‘Programmed Teaching’ Financial Analysts Journal, 17 (6): 47 – 54

Laurillard, D. 2008. Digital Technologies and their Role in Achieving our Ambitions for Education. London: Institute for Education.

Leskin, P. 2018. ‘Students in Brooklyn protest their school’s use of a Zuckerberg-backed online curriculum that Facebook engineers helped build’ Business Insider, 12.11.18 Accessed 20 September 2019. https://www.businessinsider.de/summit-learning-school-curriculum-funded-by-zuckerberg-faces-backlash-brooklyn-2018-11?r=US&IR=T

McDonald, J. K., Yanchar, S. C. & Osguthorpe, R.T. 2005. ‘Learning from Programmed Instruction: Examining Implications for Modern Instructional Technology’ Educational Technology Research and Development, 53 (2): 84 – 98

Macgilchrist, F., Allert, H. & Bruch, A. 2019. ‚Students and society in the 2020s. Three future ‘histories’ of education and technology’. Learning, Media and Technology, https://www.tandfonline.com/doi/full/10.1080/17439884.2019.1656235 )

Moore, M. 2018. Democracy Hacked. London: Oneworld

Noble, D. D. 1991. The Classroom Arsenal. London: The Falmer Press

Ornstein, J. 1968. ‘Programmed Instruction and Educational Technology in the Language Field: Boon or Failure?’ The Modern Language Journal, 52 (7), 401 – 410

Ramganesh, E. & Janaki, S. 2017. ‘Attitude of College Teachers towards the Utilization of Language Laboratories for Learning English’ Asian Journal of Social Science Studies; Vol. 2 (1): 103 – 109

Roby, W.B. 2003. ‘Technology in the service of foreign language teaching: The case of the language laboratory’ In D. Jonassen (ed.), Handbook of Research on Educational Communications and Technology, 2nd ed.: 523 – 541. Mahwah, NJ.: Lawrence Erlbaum Associates

Saettler, P. 2004. The Evolution of American Educational Technology. Greenwich, Conn.: Information Age Publishing

Skinner, B. F. 1961. ‘Teaching Machines’ Scientific American, 205(5), 90-107

Styring, J. 2015. Engaging Generation Z. Cambridge English webinar 2015 https://www.youtube.com/watch?time_continue=4&v=XCxl4TqgQZA

Valdman, A. 1968. ‘Programmed Instruction versus Guided Learning in Foreign Language Acquisition’ Die Unterrichtspraxis / Teaching German, 1 (2), 1 – 14.

Wiley, P. D. 1990. ‘Language labs for 1990: User-friendly, expandable and affordable’. Media & Methods, 27(1), 44–47)

jenny-holzer-untitled-protect-me-from-what-i-want-text-displayed-in-times-square-nyc-1982

Jenny Holzer, Protect me from what I want

In my last post, I looked at the way that, in the absence of a clear, shared understanding of what ‘personalization’ means, it has come to be used as a slogan for the promoters of edtech. In this post, I want to look a little more closely at the constellation of meanings that are associated with the term, suggest a way of evaluating just how ‘personalized’ an instructional method might be, and look at recent research into ‘personalized learning’.

In English language teaching, ‘personalization’ often carries a rather different meaning than it does in broader educational discourse. Jeremy Harmer (Harmer, 2012: 276) defines it as ‘when students use language to talk about themselves and things which interest them’. Most commonly, this is in the context of ‘freer’ language practice of grammar or vocabulary of the following kind: ‘Complete the sentences so that they are true for you’. It is this meaning that Scott Thornbury refers to first in his entry for ‘Personalization’ in his ‘An A-Z of ELT’ (Thornbury, 2006: 160). He goes on, however, to expand his definition of the term to include humanistic approaches such as Community Language Learning / Counseling learning (CLL), where learners decide the content of a lesson, where they have agency. I imagine that no one would disagree that an approach such as this is more ‘personalized’ than a ‘complete-the-sentences-so-they-are-true-for you’ exercise to practise the present perfect.

Outside of ELT, ‘personalization’ has been used to refer to everything from ‘from customized interfaces to adaptive tutors, from student-centered classrooms to learning management systems’ (Bulger, 2016: 3). The graphic below (from Bulger, 2016: 3) illustrates just how wide the definitional reach of ‘personalization’ is.

TheBulger_pie_chart

As with Thornbury’s entry in his ‘A – Z of ELT’, it seems uncontentious to say that some things are more ‘personalized’ than others.

Given the current and historical problems with defining the term, it’s not surprising that a number of people have attempted to develop frameworks that can help us to get to grips with the thorny question of ‘personalization’. In the context of language teaching / learning, Renée Disick (Disick, 1975: 58) offered the following categorisation:

Disick

In a similar vein, a few years later, Howard Altman (Altman, 1980) suggested that teaching activities can differ in four main ways: the time allocated for learning, the curricular goal, the mode of learning and instructional expectations (personalized goal setting). He then offered eight permutations of these variables (see below, Altman, 1980: 9), although many more are imaginable.

Altman 1980 chart

Altman and Disick were writing, of course, long before our current technology-oriented view of ‘personalization’ became commonplace. The recent classification of technologically-enabled personalized learning systems by Monica Bulger (see below, Bulger, 2016: 6) reflects how times have changed.

5_types_of_personalized_learning_system

Bulger’s classification focusses on the technology more than the learning, but her continuum is very much in keeping with the views of Disick and Altman. Some approaches are more personalized than others.

The extent to which choices are offered determines the degree of individualization in a particular program. (Disick, 1975: 5)

It is important to remember that learner-centered language teaching is not a point, but rather a continuum. (Altman, 1980: 6)

Larry Cuban has also recently begun to use a continuum as a way of understanding the practices of ‘personalization’ that he observes as part of his research. The overall goals of schooling at both ends of the curriculum are not dissimilar: helping ‘children grow into adults who are creative thinkers, help their communities, enter jobs and succeed in careers, and become thoughtful, mindful adults’.

Cubans curriculum

As Cuban and others before him (e.g. Januszewski, 2001: 57) make clear, the two perspectives are not completely independent of each other. Nevertheless, we can see that one end of this continuum is likely to be materials-centred with the other learner-centred (Dickinson, 1987: 57). At one end, teachers (or their LMS replacements) are more likely to be content-providers and enact traditional roles. At the other, teachers’ roles are ‘more like those of coaches or facilitators’ (Cavanagh, 2014). In short, one end of the continuum is personalization for the learner; the other end is personalization by the learner.

It makes little sense, therefore, to talk about personalized learning as being a ‘good’ or a ‘bad’ thing. We might perceive one form of personalized learning to be more personalized than another, but that does not mean it is any ‘better’ or more effective. The only possible approach is to consider and evaluate the different elements of personalization in an attempt to establish, first, from a theoretical point of view whether they are likely to lead to learning gains, and, second, from an evidence-based perspective whether any learning gains are measurable. In recent posts on this blog, I have been attempting to do that with elements such as learning styles , self-pacing and goal-setting.

Unfortunately, but perhaps not surprisingly, none of the elements that we associate with ‘personalization’ will lead to clear, demonstrable learning gains. A report commissioned by the Gates Foundation (Pane et al, 2015) to find evidence of the efficacy of personalized learning did not, despite its subtitle (‘Promising Evidence on Personalized Learning’), manage to come up with any firm and unequivocal evidence (see Riley, 2017). ‘No single element of personalized learning was able to discriminate between the schools with the largest achievement effects and the others in the sample; however, we did identify groups of elements that, when present together, distinguished the success cases from others’, wrote the authors (Pane et al., 2015: 28). Undeterred, another report (Pane et al., 2017) was commissioned: in this the authors were unable to do better than a very hedged conclusion: ‘There is suggestive evidence that greater implementation of PL practices may be related to more positive effects on achievement; however, this finding requires confirmation through further research’ (my emphases). Don’t hold your breath!

In commissioning the reports, the Gates Foundation were probably asking the wrong question. The conceptual elasticity of the term ‘personalization’ makes its operationalization in any empirical study highly problematic. Meaningful comparison of empirical findings would, as David Hartley notes, be hard because ‘it is unlikely that any conceptual consistency would emerge across studies’ (Hartley, 2008: 378). The question of what works is unlikely to provide a useful (in the sense of actionable) response.

In a new white paper out this week, “A blueprint for breakthroughs,” Michael Horn and I argue that simply asking what works stops short of the real question at the heart of a truly personalized system: what works, for which students, in what circumstances? Without this level of specificity and understanding of contextual factors, we’ll be stuck understanding only what works on average despite aspirations to reach each individual student (not to mention mounting evidence that “average” itself is a flawed construct). Moreover, we’ll fail to unearth theories of why certain interventions work in certain circumstances. And without that theoretical underpinning, scaling personalized learning approaches with predictable quality will remain challenging. Otherwise, as more schools embrace personalized learning, at best each school will have to go at it alone and figure out by trial and error what works for each student. Worse still, if we don’t support better research, “personalized” schools could end up looking radically different but yielding similar results to our traditional system. In other words, we risk rushing ahead with promising structural changes inherent to personalized learning—reorganizing space, integrating technology tools, freeing up seat-time—without arming educators with reliable and specific information about how to personalize to their particular students or what to do, for which students, in what circumstances. (Freeland Fisher, 2016)

References

Altman, H.B. 1980. ‘Foreign language teaching: focus on the learner’ in Altman, H.B. & James, C.V. (eds.) 1980. Foreign Language Teaching: Meeting Individual Needs. Oxford: Pergamon Press, pp.1 – 16

Bulger, M. 2016. Personalized Learning: The Conversations We’re Not Having. New York: Data and Society Research Institute. https://www.datasociety.net/pubs/ecl/PersonalizedLearning_primer_2016.pdf

Cavanagh, S. 2014. ‘What Is ‘Personalized Learning’? Educators Seek Clarity’ Education Week http://www.edweek.org/ew/articles/2014/10/22/09pl-overview.h34.html

Dickinson, L. 1987. Self-instruction in Language Learning. Cambridge: Cambridge University Press

Disick, R.S. 1975 Individualizing Language Instruction: Strategies and Methods. New York: Harcourt Brace Jovanovich

Freeland Fisher, J. 2016. ‘The inconvenient truth about personalized learning’ [Blog post] retrieved from http://www.christenseninstitute.org/blog/the-inconvenient-truth-about-personalized-learning/ (May 4, 2016)

Harmer, J. 2012. Essential Teacher Knowledge. Harlow: Pearson Education

Hartley, D. 2008. ‘Education, Markets and the Pedagogy of Personalisation’ British Journal of Educational Studies 56 / 4: 365 – 381

Januszewski, A. 2001. Educational Technology: The Development of a Concept. Englewood, Colorado: Libraries Unlimited

Pane, J. F., Steiner, E. D., Baird, M. D. & Hamilton, L. S. 2015. Continued Progress: Promising Evidence on Personalized Learning. Seattle: Rand Corporation retrieved from http://www.rand.org/pubs/research_reports/RR1365.html

Pane, J.F., Steiner, E. D., Baird, M. D., Hamilton, L. S. & Pane, J.D. 2017. Informing Progress: Insights on Personalized Learning Implementation and Effects. Seattle: Rand Corporation retrieved from https://www.rand.org/pubs/research_reports/RR2042.html

Riley, B. 2017. ‘Personalization vs. How People Learn’ Educational Leadership 74 / 6: 68-73

Thornbury, S. 2006. An A – Z of ELT. Oxford: Macmillan Education

 

 

 

Introduction

In the last post, I looked at issues concerning self-pacing in personalized language learning programmes. This time, I turn to personalized goal-setting. Most definitions of personalized learning, such as that offered by Next Generation Learning Challenges http://nextgenlearning.org/ (a non-profit supported by Educause, the Gates Foundation, the Broad Foundation, the Hewlett Foundation, among others), argue that ‘the default perspective [should be] the student’s—not the curriculum, or the teacher, and that schools need to adjust to accommodate not only students’ academic strengths and weaknesses, but also their interests, and what motivates them to succeed.’ It’s a perspective shared by the United States National Education Technology Plan 2017 https://tech.ed.gov/netp/ , which promotes the idea that learning objectives should vary based on learner needs, and should often be self-initiated. It’s shared by the massively funded Facebook initiative that is developing software that ‘puts students in charge of their lesson plans’, as the New York Times https://www.nytimes.com/2016/08/10/technology/facebook-helps-develop-software-that-puts-students-in-charge-of-their-lesson-plans.html?_r=0 put it. How, precisely, personalized goal-setting can be squared with standardized, high-stakes testing is less than clear. Are they incompatible by any chance?

In language learning, the idea that learners should have some say in what they are learning is not new, going back, at least, to the humanistic turn in the 1970s. Wilga Rivers advocated ‘giving the students opportunity to choose what they want to learn’ (Rivers, 1971: 165). A few years later, Renee Disick argued that the extent to which a learning programme can be called personalized (although she used the term ‘individualized’) depends on the extent to which learners have a say in the choice of learning objectives and the content of learning (Disick, 1975). Coming more up to date, Penny Ur advocated giving learners ‘a measure of freedom to choose how and what to learn’ (Ur, 1996: 233).

The benefits of personalized goal-setting

Personalized goal-setting is closely related to learner autonomy and learner agency. Indeed, it is hard to imagine any meaningful sense of learner autonomy or agency without some control of learning objectives. Without this control, it will be harder for learners to develop an L2 self. This matters because ‘ultimate attainment in second-language learning relies on one’s agency … [it] is crucial at the point where the individuals must not just start memorizing a dozen new words and expressions but have to decide on whether to initiate a long, painful, inexhaustive, and, for some, never-ending process of self-translation. (Pavlenko & Lantolf, 2000: 169 – 170). Put bluntly, if learners ‘have some responsibility for their own learning, they are more likely to be engaged than if they are just doing what the teacher tells them to’ (Harmer, 2012: 90). A degree of autonomy should lead to increased motivation which, in turn, should lead to increased achievement (Dickinson, 1987: 32; Cordova & Lepper, 1996: 726).

Strong evidence for these claims is not easy to provide, not least since autonomy and agency cannot be measured. However, ‘negative evidence clearly shows that a lack of agency can stifle learning by denying learners control over aspects of the language-learning process’ (Vandergriff, 2016: 91). Most language teachers (especially in compulsory education) have witnessed the negative effects that a lack of agency can generate in some students. Irrespective of the extent to which students are allowed to influence learning objectives, the desirability of agency / autonomy appears to be ‘deeply embedded in the professional consciousness of the ELT community’ (Borg and Al-Busaidi, 2012; Benson, 2016: 341). Personalized goal-setting may not, for a host of reasons, be possible in a particular learning / teaching context, but in principle it would seem to be a ‘good thing’.

Goal-setting and technology

The idea that learners might learn more and better if allowed to set their own learning objectives is hardly new, dating back at least one hundred years to the establishment of Montessori’s first Casa dei Bambini. In language teaching, the interest in personalized learning that developed in the 1970s (see my previous post) led to numerous classroom experiments in personalized goal-setting. These did not result in lasting changes, not least because the workload of teachers became ‘overwhelming’ (Disick, 1975: 128).

Closely related was the establishment of ‘self-access centres’. It was clear to anyone, like myself, who was involved in the setting-up and maintenance of a self-access centre, that they cost a lot, in terms of both money and work (Ur, 2012: 236). But there were also nagging questions about how effective they were (Morrison, 2005). Even more problematic was a bigger question: did they actually promote the learner autonomy that was their main goal?

Post-2000, online technology rendered self-access centres redundant: who needs the ‘walled garden’ of a self-access centre when ‘learners are able to connect with multiple resources and communities via the World Wide Web in entirely individual ways’ (Reinders, 2012)? The cost problem of self-access centres was solved by the web. Readily available now were ‘myriad digital devices, software, and learning platforms offering educators a once-unimaginable array of options for tailoring lessons to students’ needs’ (Cavanagh, 2014). Not only that … online technology promised to grant agency, to ‘empower language learners to take charge of their own learning’ and ‘to provide opportunities for learners to develop their L2 voice’ (Vandergriff, 2016: 32). The dream of personalized learning has become inseparable from the affordances of educational technologies.

It is, however, striking just how few online modes of language learning offer any degree of personalized goal-setting. Take a look at some of the big providers – Voxy, Busuu, Duolingo, Rosetta Stone or Babbel, for example – and you will find only the most token nods to personalized learning objectives. Course providers appear to be more interested in claiming their products are personalized (‘You decide what you want to learn and when!’) than in developing a sufficient amount of content to permit personalized goal-setting. We are left with the ELT equivalent of personalized cans of Coke: a marketing tool.

coke_cans

The problems with personalized goal-setting

Would language learning products, such as those mentioned above, be measurably any better if they did facilitate the personalization of learning objectives in a significant way? Would they be able to promote learner autonomy and agency in a way that self-access centres apparently failed to achieve? It’s time to consider the square quotes that I put around ‘good thing’.

Researchers have identified a number of potential problems with goal-setting. I have already mentioned the problem of reconciling personalized goals and standardized testing. In most learning contexts, educational authorities (usually the state) regulate the curriculum and determine assessment practices. It is difficult to see, as Campbell et al. (Campbell et al., 2007: 138) point out, how such regulation ‘could allow individual interpretations of the goals and values of education’. Most assessment systems ‘aim at convergent outcomes and homogeneity’ (Benson, 2016: 345) and this is especially true of online platforms, irrespective of their claims to ‘personalization’. In weak (typically internal) assessment systems, the potential for autonomy is strongest, but these are rare.

In all contexts, it is likely that personalized goal-setting will only lead to learning gains when a number of conditions are met. The goals that are chosen need to be both specific, measurable, challenging and non-conflicting (Ordóñez et al. 2009: 2-3). They need to be realistic: if not, it is unlikely that self-efficacy (a person’s belief about their own capability to achieve or perform to a certain level) will be promoted (Koda-Dallow & Hobbs, 2005), and without self-efficacy, improved performance is also unlikely (Bandura, 1997). The problem is that many learners lack self-efficacy and are poor self-regulators. These things are teachable / learnable, but require time and support. Many learners need help in ‘becoming aware of themselves and their own understandings’ (McMahon & Oliver, 2001: 1304). If they do not get it, the potential advantages of personalized goal-setting will be negated. As learners become better self-regulators, they will want and need to redefine their learning goals: goal-setting should be an iterative process (Hussey & Smith, 2003: 358). Again, support will be needed. In online learning, such support is not common.

A further problem that has been identified is that goal-setting can discourage a focus on non-goal areas (Ordóñez et al. 2009: 2) and can lead to ‘a focus on reaching the goal rather than on acquiring the skills required to reach it’ (Locke & Latham, 2006: 266). We know that much language learning is messy and incidental. Students do not only learn the particular thing that they are studying at the time (the belief that they do was described by Dewey as ‘the greatest of all pedagogical fallacies’). Goal-setting, even when personalized, runs the risk of promoting tunnel-vision.

The incorporation of personalized goal-setting in online language learning programmes is, in so many ways, a far from straightforward matter. Simply tacking it onto existing programmes is unlikely to result in anything positive: it is not an ‘over-the-counter treatment for motivation’ (Ordóñez et al.:2). Course developers will need to look at ‘the complex interplay between goal-setting and organizational contexts’ (Ordóñez et al. 2009: 16). Motivating students is not simply ‘a matter of the teacher deploying the correct strategies […] it is an intensely interactive process’ (Lamb, M. 2017). More generally, developers need to move away from a positivist and linear view of learning as a technical process where teaching interventions (such as the incorporation of goal-setting, the deployment of gamification elements or the use of a particular algorithm) will lead to predictable student outcomes. As Larry Cuban reminds us, ‘no persuasive body of evidence exists yet to confirm that belief (Cuban, 1986: 88). The most recent research into personalized learning has failed to identify any single element of personalization that can be clearly correlated with improved outcomes (Pane et al., 2015: 28).

In previous posts, I considered learning styles and self-pacing, two aspects of personalized learning that are highly problematic. Personalized goal-setting is no less so.

References

Bandura, A. 1997. Self-efficacy: The exercise of control. New York: W.H. Freeman and Company

Benson, P. 2016. ‘Learner Autonomy’ in Hall, G. (ed.) The Routledge Handbook of English Language Teaching. Abingdon: Routledge. pp.339 – 352

Borg, S. & Al-Busaidi, S. 2012. ‘Teachers’ beliefs and practices regarding learner autonomy’ ELT Journal 66 / 3: 283 – 292

Cavanagh, S. 2014. ‘What Is ‘Personalized Learning’? Educators Seek Clarity’ Education Week http://www.edweek.org/ew/articles/2014/10/22/09pl-overview.h34.html

Cordova, D. I. & Lepper, M. R. 1996. ‘Intrinsic Motivation and the Process of Learning: Beneficial Effects of Contextualization, Personalization, and Choice’ Journal of Educational Psychology 88 / 4: 715 -739

Cuban, L. 1986. Teachers and Machines. New York: Teachers College Press

Dickinson, L. 1987. Self-instruction in Language Learning. Cambridge: Cambridge University Press

Disick, R.S. 1975 Individualizing Language Instruction: Strategies and Methods. New York: Harcourt Brace Jovanovich

Harmer, J. 2012. Essential Teacher Knowledge. Harlow: Pearson Education

Hussey, T. & Smith, P. 2003. ‘The Uses of Learning Outcomes’ Teaching in Higher Education 8 / 3: 357 – 368

Lamb, M. 2017 (in press) ‘The motivational dimension of language teaching’ Language Teaching 50 / 3

Locke, E. A. & Latham, G. P. 2006. ‘New Directions in Goal-Setting Theory’ Current Directions in Psychological Science 15 / 5: 265 – 268

McMahon, M. & Oliver, R. (2001). Promoting self-regulated learning in an on-line environment. In C. Montgomerie & J. Viteli (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001 (pp. 1299-1305). Chesapeake, VA: AACE

Morrison, B. 2005. ‘Evaluating learning gain in a self-access learning centre’ Language Teaching Research 9 / 3: 267 – 293

Ordóñez, L. D., Schweitzer, M. E., Galinsky, A. D. & Bazerman, M. H. 2009. Goals Gone Wild: The Systematic Side Effects of Over-Prescribing Goal Setting. Harvard Business School Working Paper 09-083

Pane, J. F., Steiner, E. D., Baird, M. D. & Hamilton, L. S. 2015. Continued Progress: Promising Evidence on Personalized Learning. Seattle: Rand Corporation

Pavlenko, A. & Lantolf, J. P. 2000. ‘Second language learning as participation and the (re)construction of selves’ In J.P. Lantolf (ed.), Sociocultural Theory and Second Language Learning. Oxford: Oxford University Press, pp. 155 – 177

Reinders, H. 2012. ‘The end of self-access? From walled garden to public park’ ELT World Online 4: 1 – 5

Rivers, W. M. 1971. ‘Techniques for Developing Proficiency in the Spoken Language in an Individualized Foreign Language program’ in Altman, H.B. & Politzer, R.L. (eds.) 1971. Individualizing Foreign Language Instruction: Proceedings of the Stanford Conference, May 6 – 8, 1971. Washington, D.C.: Office of Education, U.S. Department of Health, Education, and Welfare. pp. 165 – 169

Ur, P. 1996. A Course in Language Teaching: Practice and Theory. Cambridge: Cambridge University Press

Ur, P. 2012. A Course in English Language Teaching. Cambridge: Cambridge University Press

Vandergriff, I. Second-language Discourse in the Digital World. 2016. Amsterdam: John Benjamins

by Philip Kerr & Andrew Wickham

from IATEFL 2016 Birmingham Conference Selections (ed. Tania Pattison) Faversham, Kent: IATEFL pp. 75 – 78

ELT publishing, international language testing and private language schools are all industries: products are produced, bought and sold for profit. English language teaching (ELT) is not. It is an umbrella term that is used to describe a range of activities, some of which are industries, and some of which (such as English teaching in high schools around the world) might better be described as public services. ELT, like education more generally, is, nevertheless, often referred to as an ‘industry’.

Education in a neoliberal world

The framing of ELT as an industry is both a reflection of how we understand the term and a force that shapes our understanding. Associated with the idea of ‘industry’ is a constellation of other ideas and words (such as efficacy, productivity, privatization, marketization, consumerization, digitalization and globalization) which become a part of ELT once it is framed as an industry. Repeated often enough, ‘ELT as an industry’ can become a metaphor that we think and live by. Those activities that fall under the ELT umbrella, but which are not industries, become associated with the desirability of industrial practices through such discourse.

The shift from education, seen as a public service, to educational managerialism (where education is seen in industrial terms with a focus on efficiency, free market competition, privatization and a view of students as customers) can be traced to the 1980s and 1990s (Gewirtz, 2001). In 1999, under pressure from developed economies, the General Agreement on Trade in Services (GATS) transformed education into a commodity that could be traded like any other in the marketplace (Robertson, 2006). The global industrialisation and privatization of education continues to be promoted by transnational organisations (such as the World Bank and the OECD), well-funded free-market think-tanks (such as the Cato Institute), philanthro-capitalist foundations (such as the Gates Foundation) and educational businesses (such as Pearson) (Ball, 2012).

Efficacy and learning outcomes

Managerialist approaches to education require educational products and services to be measured and compared. In ELT, the most visible manifestation of this requirement is the current ubiquity of learning outcomes. Contemporary coursebooks are full of ‘can-do’ statements, although these are not necessarily of any value to anyone. Examples from one unit of one best-selling course include ‘Now I can understand advice people give about hotels’ and ‘Now I can read an article about unique hotels’ (McCarthy et al. 2014: 74). However, in a world where accountability is paramount, they are deemed indispensable. The problem from a pedagogical perspective is that teaching input does not necessarily equate with learning uptake. Indeed, there is no reason why it should.

Drawing on the Common European Framework of Reference for Languages (CEFR) for inspiration, new performance scales have emerged in recent years. These include the Cambridge English Scale and the Pearson Global Scale of English. Moving away from the broad six categories of the CEFR, such scales permit finer-grained measurement and we now see individual vocabulary and grammar items tagged to levels. Whilst such initiatives undoubtedly support measurements of efficacy, the problem from a pedagogical perspective is that they assume that language learning is linear and incremental, as opposed to complex and jagged.

Given the importance accorded to the measurement of language learning (or what might pass for language learning), it is unsurprising that attention is shifting towards the measurement of what is probably the most important factor impacting on learning: the teaching. Teacher competency scales have been developed by Cambridge Assessment, the British Council and EAQUALS (Evaluation and Accreditation of Quality Language Services), among others.

The backwash effects of the deployment of such scales are yet to be fully experienced, but the likely increase in the perception of both language learning and teacher learning as the synthesis of granularised ‘bits of knowledge’ is cause for concern.

Digital technology

Digital technology may offer advantages to both English language teachers and learners, but its rapid growth in language learning is the result, primarily but not exclusively, of the way it has been promoted by those who stand to gain financially. In education, generally, and in English language teaching, more specifically, advocacy of the privatization of education is always accompanied by advocacy of digitalization. The global market for digital English language learning products was reported to be $2.8 billion in 2015 and is predicted to reach $3.8 billion by 2020 (Ambient Insight, 2016).

In tandem with the increased interest in measuring learning outcomes, there is fierce competition in the market for high-stakes examinations, and these are increasingly digitally delivered and marked. In the face of this competition and in a climate of digital disruption, companies like Pearson and Cambridge English are developing business models of vertical integration where they can provide and sell everything from placement testing, to courseware (either print or delivered through an LMS), teaching, assessment and teacher training. Huge investments are being made in pursuit of such models. Pearson, for example, recently bought GlobalEnglish, Wall Street English, and set up a partnership with Busuu, thus covering all aspects of language learning from resources provision and publishing to off- and online training delivery.

As regards assessment, the most recent adult coursebook from Cambridge University Press (in collaboration with Cambridge English Language Assessment), ‘Empower’ (Doff, et. Al, 2015) sells itself on a combination of course material with integrated, validated assessment.

Besides its potential for scalability (and therefore greater profit margins), the appeal (to some) of platform-delivered English language instruction is that it facilitates assessment that is much finer-grained and actionable in real time. Digitization and testing go hand in hand.

Few English language teachers have been unaffected by the move towards digital. In the state sectors, large-scale digitization initiatives (such as the distribution of laptops for educational purposes, the installation of interactive whiteboards, the move towards blended models of instruction or the move away from printed coursebooks) are becoming commonplace. In the private sectors, online (or partially online) language schools are taking market share from the traditional bricks-and-mortar institutions.

These changes have entailed modifications to the skill-sets that teachers need to have. Two announcements at this conference reflect this shift. First of all, Cambridge English launched their ‘Digital Framework for Teachers’, a matrix of six broad competency areas organised into four levels of proficiency. Secondly, Aqueduto, the Association for Quality Education and Training Online, was launched, setting itself up as an accreditation body for online or blended teacher training courses.

Teachers’ pay and conditions

In the United States, and likely soon in the UK, the move towards privatization is accompanied by an overt attack on teachers’ unions, rights, pay and conditions (Selwyn, 2014). As English language teaching in both public and private sectors is commodified and marketized it is no surprise to find that the drive to bring down costs has a negative impact on teachers worldwide. Gwynt (2015), for example, catalogues cuts in funding, large-scale redundancies, a narrowing of the curriculum, intensified workloads (including the need to comply with ‘quality control measures’), the deskilling of teachers, dilapidated buildings, minimal resources and low morale in an ESOL department in one British further education college. In France, a large-scale study by Wickham, Cagnol, Wright and Oldmeadow (Linguaid, 2015; Wright, 2016) found that EFL teachers in the very competitive private sector typically had multiple employers, limited or no job security, limited sick pay and holiday pay, very little training and low hourly rates that were deteriorating. One of the principle drivers of the pressure on salaries is the rise of online training delivery through Skype and other online platforms, using offshore teachers in low-cost countries such as the Philippines. This type of training represents 15% in value and up to 25% in volume of all language training in the French corporate sector and is developing fast in emerging countries. These examples are illustrative of a broad global trend.

Implications

Given the current climate, teachers will benefit from closer networking with fellow professionals in order, not least, to be aware of the rapidly changing landscape. It is likely that they will need to develop and extend their skill sets (especially their online skills and visibility and their specialised knowledge), to differentiate themselves from competitors and to be able to demonstrate that they are in tune with current demands. More generally, it is important to recognise that current trends have yet to run their full course. Conditions for teachers are likely to deteriorate further before they improve. More than ever before, teachers who want to have any kind of influence on the way that marketization and industrialization are shaping their working lives will need to do so collectively.

References

Ambient Insight. 2016. The 2015-2020 Worldwide Digital English Language Learning Market. http://www.ambientinsight.com/Resources/Documents/AmbientInsight_2015-2020_Worldwide_Digital_English_Market_Sample.pdf

Ball, S. J. 2012. Global Education Inc. Abingdon, Oxon.: Routledge

Doff, A., Thaine, C., Puchta, H., Stranks, J. and P. Lewis-Jones 2015. Empower. Cambridge: Cambridge University Press

Gewirtz, S. 2001. The Managerial School: Post-welfarism and Social Justice in Education. Abingdon, Oxon.: Routledge

Gwynt, W. 2015. ‘The effects of policy changes on ESOL’. Language Issues 26 / 2: 58 – 60

McCarthy, M., McCarten, J. and H. Sandiford 2014. Touchstone 2 Student’s Book Second Edition. Cambridge: Cambridge University Press

Linguaid, 2015. Le Marché de la Formation Langues à l’Heure de la Mondialisation. Guildford: Linguaid

Robertson, S. L. 2006. ‘Globalisation, GATS and trading in education services.’ published by the Centre for Globalisation, Education and Societies, University of Bristol, Bristol BS8 1JA, UK at http://www.bris.ac.uk/education/people/academicStaff/edslr/publications/04slr

Selwyn, N. 2014. Distrusting Educational Technology. New York: Routledge

Wright, R. 2016. ‘My teacher is rich … or not!’ English Teaching Professional 103: 54 – 56

 

 

Ok, let’s be honest here. This post is about teacher training, but ‘development’ sounds more respectful, more humane, more modern. Teacher development (self-initiated, self-evaluated, collaborative and holistic) could be adaptive, but it’s unlikely that anyone will want to spend the money on developing an adaptive teacher development platform any time soon. Teacher training (top-down, pre-determined syllabus and externally evaluated) is another matter. If you’re not too clear about this distinction, see Penny Ur’s article in The Language Teacher.

decoding_adaptive jpgThe main point of adaptive learning tools is to facilitate differentiated instruction. They are, as Pearson’s latest infomercial booklet describes them, ‘educational technologies that can respond to a student’s interactions in real-time by automatically providing the student with individual support’. Differentiation or personalization (or whatever you call it) is, as I’ve written before  , the declared goal of almost everyone in educational power these days. What exactly it is may be open to question (see Michael Feldstein’s excellent article), as may be the question of whether or not it is actually such a desideratum (see, for example, this article ). But, for the sake of argument, let’s agree that it’s mostly better than one-size-fits-all.

Teachers around the world are being encouraged to adopt a differentiated approach with their students, and they are being encouraged to use technology to do so. It is technology that can help create ‘robust personalized learning environments’ (says the White House)  . Differentiation for language learners could be facilitated by ‘social networking systems, podcasts, wikis, blogs, encyclopedias, online dictionaries, webinars, online English courses,’ etc. (see Alexandra Chistyakova’s post on eltdiary ).

But here’s the crux. If we want teachers to adopt a differentiated approach, they really need to have experienced it themselves in their training. An interesting post on edweek  sums this up: If professional development is supposed to lead to better pedagogy that will improve student learning AND we are all in agreement that modeling behaviors is the best way to show people how to do something, THEN why not ensure all professional learning opportunities exhibit the qualities we want classroom teachers to have?

Differentiated teacher development / training is rare. According to the Center for Public Education’s Teaching the Teachers report , almost all teachers participate in ‘professional development’ (PD) throughout the year. However, a majority of those teachers find the PD in which they participate ineffective. Typically, the development is characterised by ‘drive-by’ workshops, one-size-fits-all presentations, ‘been there, done that’ topics, little or no modelling of what is being taught, a focus on rotating fads and a lack of follow-up. This report is not specifically about English language teachers, but it will resonate with many who are working in English language teaching around the world.cindy strickland

The promotion of differentiated teacher development is gaining traction: see here or here , for example, or read Cindy A. Strickland’s ‘Professional Development for Differentiating Instruction’.

Remember, though, that it’s really training, rather than development, that we’re talking about. After all, if one of the objectives is to equip teachers with a skills set that will enable them to become more effective instructors of differentiated learning, this is most definitely ‘training’ (notice the transitivity of the verbs ‘enable’ and ‘equip’!). In this context, a necessary starting point will be some sort of ‘knowledge graph’ (which I’ve written about here ). For language teachers, these already exist, including the European Profiling Grid , the Eaquals Framework for Language Teacher Training and Development, the Cambridge English Teaching Framework and the British Council’s Continuing Professional Development Framework (CPD) for Teachers  . We can expect these to become more refined and more granularised, and a partial move in this direction is the Cambridge English Digital Framework for Teachers  . Once a knowledge graph is in place, the next step will be to tag particular pieces of teacher training content (e.g. webinars, tasks, readings, etc.) to locations in the framework that is being used. It would not be too complicated to engineer dynamic frameworks which could be adapted to individual or institutional needs.cambridge_english_teaching_framework jpg

This process will be facilitated by the fact that teacher training content is already being increasingly granularised. Whether it’s an MA in TESOL or a shorter, more practically oriented course, things are getting more and more bite-sized, with credits being awarded to these short bites, as course providers face stiffer competition and respond to market demands.

Visible classroom home_page_screenshotClassroom practice could also form part of such an adaptive system. One tool that could be deployed would be Visible Classroom , an automated system for providing real-time evaluative feedback for teachers. There is an ‘online dashboard providing teachers with visual information about their teaching for each lesson in real-time. This includes proportion of teacher talk to student talk, number and type of questions, and their talking speed.’ John Hattie, who is behind this project, says that teachers ‘account for about 30% of the variance in student achievement and [are] the largest influence outside of individual student effort.’ Teacher development with a tool like Visible Classroom is ultimately all about measuring teacher performance (against a set of best-practice benchmarks identified by Hattie’s research) in order to improve the learning outcomes of the students.Visible_classroom_panel_image jpg

You may have noticed the direction in which this part of this blog post is going. I began by talking about social networking systems, podcasts, wikis, blogs and so on, and just now I’ve mentioned the summative, credit-bearing possibilities of an adaptive teacher development training programme. It’s a tension that is difficult to resolve. There’s always a paradox in telling anyone that they are going to embark on a self-directed course of professional development. Whoever pays the piper calls the tune and, if an institution decides that it is worth investing significant amounts of money in teacher development, they will want a return for their money. The need for truly personalised teacher development is likely to be overridden by the more pressing need for accountability, which, in turn, typically presupposes pre-determined course outcomes, which can be measured in some way … so that quality (and cost-effectiveness and so on) can be evaluated.

Finally, it’s worth asking if language teaching (any more than language learning) can be broken down into small parts that can be synthesized later into a meaningful and valuable whole. Certainly, there are some aspects of language teaching (such as the ability to use a dashboard on an LMS) which lend themselves to granularisation. But there’s a real danger of losing sight of the forest of teaching if we focus on the individual trees that can be studied and measured.

If you’re going to teach vocabulary, you need to organise it in some way. Almost invariably, this organisation is topical, with words grouped into what are called semantic sets. In coursebooks, the example below (from Rogers, M., Taylore-Knowles, J. & S. Taylor-Knowles. 2010. Open Mind Level 1. London: Macmillan, p.68) is fairly typical.

open mind

Coursebooks are almost always organised in a topical way. The example above comes in a unit (of 10 pages), entitled ‘You have talent!’, which contains two main vocabulary sections. It’s unsurprising to find a section called ‘personality adjectives’ in such a unit. What’s more, such an approach lends itself to the requisite, but largely, spurious ‘can-do’ statement in the self-evaluation section: I can talk about people’s positive qualities. We must have clearly identifiable learning outcomes, after all.

There is, undeniably, a certain intuitive logic in this approach. An alternative might entail a radical overhaul of coursebook architecture – this might not be such a bad thing, but might not go down too well in the markets. How else, after all, could the vocabulary strand of the syllabus be organised?

Well, there are a number of ways in which a vocabulary syllabus could be organised. Including the standard approach described above, here are four possibilities:

1 semantic sets (e.g. bee, butterfly, fly, mosquito, etc.)

2 thematic sets (e.g. ‘pets’: cat, hate, flea, feed, scratch, etc.)

3 unrelated sets

4 sets determined by a group of words’ occurrence in a particular text

Before reading further, you might like to guess what research has to say about the relative effectiveness of these four approaches.

The answer depends, to some extent, on the level of the learner. For advanced learners, it appears to make no, or little, difference (Al-Jabri, 2005, cited by Ellis & Shintani, 2014: 106). But, for the vast majority of English language learners (i.e. those at or below B2 level), the research is clear: the most effective way of organising vocabulary items to be learnt is by grouping them into thematic sets (2) or by mixing words together in a semantically unrelated way (3) – not by teaching sets like ‘personality adjectives’. It is surprising how surprising this finding is to so many teachers and materials writers. It goes back at least to 1988 and West’s article on ‘Catenizing’ in ELTJ, which argued that semantic grouping made little sense from a psycho-linguistic perspective. Since then, a large amount of research has taken place. This is succinctly summarised by Paul Nation (2013: 128) in the following terms: Avoid interference from related words. Words which are similar in form (Laufer, 1989) or meaning (Higa, 1963; Nation, 2000; Tinkham, 1993; Tinkham, 1997; Waring, 1997) are more difficult to learn together than they are to learn separately. For anyone who is interested, the most up-to-date review of this research that I can find is in chapter 11 of Barcroft (2105).

The message is clear. So clear that you have to wonder how it is not getting through to materials designers. Perhaps, coursebooks are different. They regularly eschew research findings for commercial reasons. But vocabulary apps? There is rarely, if ever, any pressure on the content-creation side of vocabulary apps (except those that are tied to coursebooks) to follow the popular misconceptions that characterise so many coursebooks. It wouldn’t be too hard to organise vocabulary into thematic sets (like, for example, the approach in the A2 level of Memrise German that I’m currently using). Is it simply because the developers of so many vocabulary apps just don’t know much about language learning?

References

Barcroft, J. 2015. Lexical Input Processing and Vocabulary Learning. Amsterdam: John Benjamins

Nation, I. S. P. 2013. Learning Vocabulary in Another Language 2nd edition. Cambridge: Cambridge University Press

Ellis, R. & N. Shintani, N. 2014. Exploring Language Pedagogy through Second Language Acquisition Research. Abingdon, Oxon: Routledge

West, M. 1988. ‘Catenizing’ English Language Teaching Journal 6: 147 – 151