Perspectives on language teaching research

I’ve written about the relationship (or, rather, the lack of one) between language teachers and language teaching research before. I’m talking about the kind of research that is primarily of the ‘what-works’ variety, since that is likely to be of most relevance to teachers. It’s the kind of research that asks questions like: can correction be beneficial to language learners? Or: can spaced repetition be helpful in vocabulary acquisition? Whether teachers find this relevant or not, there is ample evidence that the vast majority rarely look at it (Borg, 2009).

See here, for example, for a discussion of calls from academic researchers for more dialogue between researchers and teachers. The desire, on the part of researchers, for teachers to engage more (or even a little) with research, continues to grow, as shown by two examples. The first is the development of TESOLgraphics, which aims to make research ‘easy to read and understand to ESL, EFL, EAP, ESP, ESOL, EAL, TEFL teachers’ by producing infographic summaries. The second is a proposed special issue of the journal ‘System’ devoted to ‘the nexus of research and practice in and for language teacher education’ and hopes to find ways of promoting more teacher engagement with research. Will either of these initiatives have much impact? I doubt it, and to explain why, I need to take you on a little detour.

The map and the territory

Riffing off an ultra-short story by Jorge Luis Borges (‘On Exactitude in Science’, 1946), the corpus linguist Michael Stubbs (2013) wrote a piece entitled ‘Of Exactitude in Linguistics’, which marked his professional retirement. In it, he described a world where

the craft of Descriptive Linguistics attained such Perfection that the Transcription of a single short Conversation covered the floor of an entire University seminar room, and the Transcription of a Representative Sample of a single Text-Type covered the floor area of a small department to a depth of several feet. In the course of time, especially after the development of narrow phonetic transcription with intonational and proxemic annotation, even these extensive Records were found somehow wanting, and with the advent of fully automatic voice-to-orthography transcription, the weight of the resulting Text Collections threatened severe structural damage to University buildings.

As with all humour, there’s more than a grain of truth behind this Borgesian fantasy. These jokes pick up on what is known as the Richardson Effect, named after a British mathematician who noted that the length of the coastline of Great Britain varies according to the size of the units that are used to measure it – the smaller the unit, the longer the coastline. But at what point does increasing exactitude cease to tell us anything of value?

Both Borges and Lewis Fry Richardson almost certainly knew Lewis Carroll’s novel ‘Sylvie and Bruno Concluded’ (1893) which features a map that has the scale of a mile to a mile. This extraordinarily accurate map is, however, never used, since it is too large to spread out. The cost of increasing exactitude is practical usefulness.

The map of language

Language is rather like a coastline when it comes to drilling down in order to capture its features with smaller and smaller units of measurement. Before very long, you are forced into making decisions about the variety of the language and the contexts of use that you are studying. Precisely what kind of English are you measuring? At some point, you get down to the level of idiolect, but idiolects can be broken down further as they vary depending on the contexts of use. The trouble, of course, is that idiolects tell us little that is of value about the much broader ‘language’ that you set out to measure in the first place. The linguistic map obscures the linguistic terrain.

In ultra close-up, we can no longer distinguish one named language from another just by using linguistic criteria (Makoni & Pennycook, 20077:1). Extending this logic further, it makes little sense to even talk about named languages like English, to talk about first or second languages, about native speakers or about language errors. The close-up view requires us to redefine the thing – language – that we set out to define and describe. English is no longer a fixed and largely territorial system owned by native-speakers, but a dynamic, complex, social, deterritorialized practice owned by its users (May, 2013; Meier, 2017; Li Wei, 2018). In this view, both the purpose and the consequence of describing language in this way is to get away from the social injustice of native-speaker norms, of accentism, and linguistic prejudice.

A load of Ballungs

Language is a fuzzy and context-dependent concept. It is ‘too multifaceted to be measured on a single metric without loss of meaning, and must be represented by a matrix of indices or by several different measures depending on which goals and values are at play’ (Tal, 2020). In the philosophy of measurement, concepts like these are known as ‘Ballung’ concepts (Cartwright & Bradburn, 2011). Much of what is studied by researchers into language learning are also ‘Ballung’ concepts. Language proficiency and language acquisition are ‘Ballung’ concepts, too. As are reading and listening skills, mediation, metacognition and motivation. Critical thinking and digital literacies … the list goes on. Research into all these areas is characterised by multiple and ever-more detailed taxonomies, as researchers struggle to define precisely what it is that they are studying. It is in the nature of most academic study that it strives towards exactitude by becoming more and more specialised in its analysis of ‘ever more particular fractions of our world’ (Pardo-Guerra, 2022: 17).

But the perspective on language of Makoni, Pennycook, Li Wei et al is not what we might call the ‘canonical view’, the preferred viewpoint of the majority of people in apprehending the reality of the outside world (Palmer, 1981). Canonical views of language are much less close-up and allow for the unproblematic differentiation of one language from another. Canonical views – whether of social constructs like language or everyday objects like teacups or birds – become canonical because they are more functional for many people for everyday purposes than less familiar perspectives. If you want to know how far it is to walk from A to B along a coastal footpath, the more approximate measure of metres is more useful than one that counts every nook and cranny in microns. Canonical views can, of course, change over time – if the purpose to which they are put changes, too.

Language teaching research

There is a clear preference in academia for quantitative, empirical research where as many variables as possible are controlled. Research into language teaching is no different. It’s not enough to ask, in general terms, about the impact on learning of correction or spaced repetition. ‘What works’ is entirely context-dependent (Al-Hoorie, et al., 2023: 278). Since all languages, language learners and language learning contexts are ‘ultimately different’ (Widdowson, 2023: 397), there’s never any end to the avenues that researchers can explore: it is a ‘self-generating academic area of inquiry’ (ibid.). So we can investigate the impact of correction on the writing (as opposed to the speaking) of a group of Spanish (as opposed to another nationality) university students (as opposed to another age group) in an online setting (as opposed to face-to-face) where the correction is delayed (as opposed to immediate) and delivered by WhatsApp (as opposed to another medium) (see, for example, Murphy et al., 2023). We could carry on playing around with the variables for as long as we like – this kind of research has already been going on for decades.

When it comes to spaced repetition, researchers need to consider the impact of different algorithms (e.g. the length of the spaces) on different kinds of learners (age, level, motivation, self-regulation, etc.) in their acquisition of different kinds of lexical items (frequency, multi-word units, etc.) and how these items are selected and grouped, the nature of this acquisition (e.g. is it for productive use or is it purely recognition?). And so on (see the work of Tatsuya Nakata, for example).

Such attempts to control the variables are a necessary part of scientific enquiry, they are part of the ‘disciplinary agenda’, but they are unlikely to be of much relevance to most teachers. Researchers need precision, but the more they attempt to ‘approximate the complexities of real life, the more unwieldy [their] theories inevitably become’ (Al-Hoorie et al., 2023). Teachers, on the other hand, are typically more interested in canonical views that can lead to general take-aways that can be easily applied in their lessons. It is only secondary research in the form of meta-analyses or literature reviews (of the kind that TESOLgraphics) that can avoid the Richardson Effect and might offer something of help to the average classroom practitioner. But this secondary research, stripped of the contextual variables, can only be fairly vague. It can only really tell us, for example, that some form of written correction or spaced repetition may be helpful to some learners in some contexts some of the time. In need of ‘substantial localization’, it has been argued that the broad-stroke generalisations are often closer to ‘pseudo-applications’ (Al-Hoorie et al., 2023) than anything that is reliably actionable. That is not to say, however, that broad-stroke generalisations are of no value at all.

Finding the right map

Henry Widdowson (e.g. 2023) has declared himself sceptical about the practical relevance of SLA research. Reading journals like ‘Studies in Second Language Acquisition’ or ‘System’, it’s hard not to agree. Attempts to increase the accessibility of research (e.g. open-access or simple summaries) may not have the desired impact since they do not do anything about ‘the tenuous link between research and practice’ (Hwang, 2023). They cannot bridge the ‘gap between two sharply contrasting kinds of knowledge’ (McIntyre, 2006).

There is an alternative: classroom-based action research carried out by teachers. One of the central ideas behind it is that teachers may benefit more from carrying out their own research than from reading someone else’s. Enthusiasm for action research has been around for a long time: it was very fashionable in the 1980s when I trained as a teacher. In the 1990s, there was a series of conferences for English language teachers called ‘Teachers Develop Teachers Research’ (see, for example, Field et al., 1997). Tirelessly promoted by people like Richard Smith, Paula Rebolledo (Smith et al., 2014) and Anne Burns, action research seems to be gaining traction. A recent British Council publication (Burns, 2023) is a fine example of what insights teachers may gain and act on with an exploratory action research approach.

References

Al-Hoorie A. H., Hiver, P., Larsen-Freeman, D. & Lowie, W. (2023) From replication to substantiation: A complexity theory perspective. Language Teaching, 56 (2): pp. 276 – 291

Borg, S. (2009) English language teachers’ conceptions of research. Applied Linguistics, 30 (3): 358 – 88

Burns, A. (Ed.) (2023) Exploratory Action Research in Thai Schools: English teachers identifying problems, taking action and assessing results. Bangkok, Thailand: British Council

Cartwright, N., Bradburn, N. M., & Fuller, J. (2016) A theory of measurement. Working Paper. Centre for Humanities Engaging Science and Society (CHESS), Durham.

Field, J., Graham, A., Griffiths, E. & Head. K. (Eds.) (1997) Teachers Develop Teachers Research 2. Whitstable, Kent: IATEFl

Hwang, H.-B. (2023) Is evidence-based L2 pedagogy achievable? The research–practice dialogue in grammar instruction. The Modern Language Journal, 2023: 1 – 22 https://onlinelibrary.wiley.com/doi/full/10.1111/modl.12864

Li Wei. (2018) Translanguaging as a Practical Theory of Language. Applied Linguistics, 39 (1): 9 – 30

Makoni, S. & Pennycook, A. (Eds.) (2007) Disinventing and Reconstituting Languages. Clevedon: Multilingual Matters

May. S. (Ed.) (2013) The multilingual turn: Implications for SLA, TESOL and Bilingual education. New York: Routledge

McIntyre, D. (2006) Bridging the gap between research and practice. Cambridge Journal of Education 35 (3): 357 – 382

Meier, G. S. (2017) The multilingual turn as a critical movement in education: assumptions, challenges and a need for reflection. Applied Linguistics Review, 8 (1): 131-161

Murphy, B., Mackay J. & Tragant, E. (2023) ‘(Ok I think I was totally wrong: new try!)’: language learning in WhatsApp through the provision of delayed corrective feedback provided during and after task performance’, The Language Learning Journal, DOI: 10.1080/09571736.2023.2223217

Palmer, S.E. et al. (1981) Canonical perspective and the perception of objects. In Longand, J. & Baddeley. A. (Eds.) Attention and Performance IX. Hillsdale, NJ: Erlbaum. pp. 135 – 151

Pardo-Guerra, J. P. (2022) The Quantified Scholar. New York: Columbia University Press

Smith, R., Connelly, T. & Rebolledo, P. (2014). Teacher research as CPD: A project with Chilean secondary school teachers. In D. Hayes (Ed.), Innovations in the continuing professional development of English language teachers (pp. 111–128). The British Council.

Tal, E. “Measurement in Science”, In The Stanford Encyclopedia of Philosophy (Fall 2020 Edition), Edward N. Zalta (Ed.), https://plato.stanford.edu/archives/fall2020/entries/measurement-science/

Widdowson, H. (2023) Webinar on the subject of English and applied linguistics. Language Teaching, 56 (3): 393 – 401

Grit and marshmallows

You have probably heard of the marshmallow experiment, one of the most famous and widely cited studies in social psychology. In the experiments, led by Walter Mischel at Stanford University in 1972, pre-school children were offered a choice between an immediate small reward (such as a marshmallow) or a significantly larger reward if they could wait long enough (a few minutes) to receive it. A series of follow-up studies, beginning in 1988, found that those children who had been able to delay gratification in the original experiments had better educational achievements at school and in college than those who had less self-control.

The idea that character traits like self-control could have an important impact on educational outcomes clearly resonated with many people at the time. The studies inspired further research into what is now called socio-emotional learning, and helped to popularise many educational interventions across the world that sought to teach ‘character and resilience’ in schools. In Britain alone, £5 million was pledged for a programme in 2015 to promote what the government called ‘character work’, an initiative that saw rugby coaches being used to instil the values of respect, teamwork, enjoyment, and discipline in school children.

One person who was massively influenced by the marshmallow experiment (and who, in turn, massively influenced the character-building interventions in schools), was Angela Duckworth (Duckworth et al., 2013), who worked at Stanford between 2014 and 2015. Shortly after her studies into delay of gratification, Duckworth gave a TED talk called ‘Grit: the power of passion and perseverance’ which has now had almost 10 million views. A few years later, her book with the same title (Duckworth, 2016) was published. An instant best-seller, ‘grit’ became a ‘hot topic’ in education, and, according to the editors of a special issue of The Journal for the Psychology of Language Learning (MacIntyre & Khajavy, 2021), ‘interest appears to be rapidly expanding’. Duckworth has argued that self-control and grit are different and unrelated, but a number of studies have contradicted this view (Oxford & Khajafy, 2021), and the relationship between the two is clear in Duckworth’s intellectual and publishing trajectory.

This continued and expanding interest in grit is a little surprising. In a previous (June, 2020) blog post , I looked at the problems with the concept of ‘grit’, drawing on the work of Marcus Credé (2017; 2018) that questioned whether it made sense to talk about ‘grit’ as a unitary construct, noted the difficulty of measuring ‘grit’ and the lack of evidence in support of educational interventions to promote ‘grit’ (despite the millions and millions that have been spent). In a more recent article, Credé and his collaborator, Michael Tynan (Credé & Tynan, 2021), double-down on their criticisms, observing that ‘meta-analytic syntheses of the grit literature have shown that grit is a poor predictor of performance and success in its own right, and that it predicts success in academic and work settings far more poorly than other well-known predictors’. One of these other well-known predictors is the socio-economic status of students’ families. Credé and Tynan remain ‘deeply skeptical of the claim that grit, as a unitary construct formed by combining scores on perseverance and passion, holds much value for researchers focused on SLA—or any other domain’.

In the same journal issue as the Credé and Tynan article, Rebecca Oxford and Gholam Khajavy (2021) sound further notes of caution about work on ‘grit’. They suggest that researchers need to avoid confusing grit with other constructs like self-control – a suggestion that may be hard or impossible to follow if, in fact these constructs are not clearly separable (as Oxford and Khajavy note). They argue, too, that much more attention needs to be paid to socio-economic contexts, that structural barriers to achievement must be given fuller consideration if ‘grit’ is to contribute anything positive to social justice. Whether the other papers in this special edition of the Journal for the Psychology of Language Learning that is devoted to ‘grit’ heed the cautionary advice of Credé and Tynan, Oxford and Khajavy is, I think, open to debate. Perhaps the idea of a whole edition of a journal devoted to ‘grit’ is a problematic starting point. Since there is no shortage of reasons to believe that ‘grit’ isn’t actually a ‘thing’, why take ‘grit’ as a starting point for scholarly enquiry?

It might be instructive to go back to how ‘grit’ became a ‘thing’ in the first place. It’s an approach that the contributors to the special issue of the Journal for the Psychology of Language Learning have not adopted. This brings me back to the marshmallow test. At the time that ‘grit’ was getting going, Alfie Kohn brought out a book called ‘The Myth of the Spoiled Child’ (Kohn, 2014) that included a chapter ‘Why Self-Discipline Is Overrated: A Closer Look at Grit, Marshmallows, and Control from Within’. Kohn argued that educational ideas about ‘grit’ had misrepresented the findings of the marshmallow test and its follow-up studies. He argued that setting was more important than individual self-control, and that deferral of gratification was likely an effect, not a cause of anything. His ideas were supported by some of the original researchers, including Mischel himself. Another, Yuichi Shoda, a co-author of a key paper that linked delay of gratification to SAT scores, has observed that ‘Our paper does not mention anything about interventions or policies’ – many other factors would need to be controlled to validate the causal relationship between self-control and academic achievement (Resnick, 2018).

Interest in recent years in replicating experiments in social psychology has led to confirmation that something was seriously wrong with the follow-ups to the marshmallow experiment. Studies (e.g. Watts et al., 2018) with more representative and larger groups of children have found that correlations between academic achievement and self-control almost vanished when controlled for factors like family background and intelligence. Even if you can teach a child to delay gratification, it won’t necessarily lead to any benefits later on.

Self-control and ‘grit’ may or may not be different things, but one thing they clearly have in common is their correlation with socio-economic differences. It is distinctly possible that attention to ‘grit’, in language learning and in other fields, is a distraction from more pressing concerns. Pity the poor researchers who have hitched themselves to the ‘grit’ bandwagon … As Angela Duckworth has said, research into grit is itself ‘a delay of gratification test’ (Duckworth, 2013). You have to be really passionate about grit and show sustained persistence if you want to keep on publishing on the subject, despite all that we now know. She hopes ‘that as a field we follow through on our intentions to forgo more immediately rewarding temptations to instead do what is best for science in the long-run’. How about forgoing the immediately rewarding temptation of publishing yet more stuff on this topic?

References

Credé, M. (2018) What shall we do about grit? A critical review of what we know and what we don’t know. Educational Researcher, 47 (9), 606-611.

Credé, M. & Tynan, M. C. (2021) Should Language Acquisition Researchers Study “Grit”? A Cautionary Note and Some Suggestions. Journal for the Psychology of Language Learning, 3 (2), 37 – 44

Credé, M., Tynan, M. C. & Harms, P. D. (2017) Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113 (3)

Duckworth, A. L. (2013) Is It Really Self-control: A Critical Analysis of the “Marshmallow Test” Society of Personality and Social Psychology Connections November 10, 2013 https://spsptalks.wordpress.com/2013/11/10/is-it-really-self-control-a-critical-analysis-of-the-marshmallow-test/

Duckworth, A. L., Tsukayama, E. & Kirby, T. A. (2013) Is it really self-control? Examining the predictive power of the delay of gratification response. Personality and Social Psychology Bulletin, 39, 843-855.

Duckworth, A. (2016) Grit: the power of passion and perseverance. New York: Scribner

Kohn, A. (2014) The Myth of the Spoiled Child. Boston: Da Capo Press

MacIntyre, P. & Khajavy, G. H. (2021) Grit in Second Language Learning and Teaching: Introduction to the Special Issue. Journal for the Psychology of Language Learning, 3 (2), 1-6. http://www.jpll.org/index.php/journal/article/view/86

Oxford, R. & Khajafy, G. H. (2021) Exploring Grit: “Grit Linguistics” and Research on Domain-General Grit and L2 Grit. Journal for the Psychology of Language Learning, 3 (2), 7 – 35

Resnick, B. (2018) The “marshmallow test” said patience was a key to success. A new replication tells us s’more. Vox, June 6, 2018. https://www.vox.com/science-and-health/2018/6/6/17413000/marshmallow-test-replication-mischel-psychology

Watts, T.W., Duncan, G.J. & Quan, H. (2018) Revisiting the Marshmallow Test: A Conceptual Replication Investigating Links Between Early Delay of Gratification and Later Outcomes. Psychological Science 29 (7): 1159-1177.

Glossing (extensions)

In the latest issue of ‘Language Teaching’, there’s a ‘state-of-the-art’ article by Frank Boers entitled ‘Glossing and vocabulary learning’. The effect of glosses (‘a brief definition or synonym, either in L1 or L2, which is provided with [a] text’ (Nation, 2013: 238)) on reading comprehension and vocabulary acquisition has been well researched over the years. See Kim et al. (2020) for just one recent meta-analysis.

It’s a subject I have written about before on this blog (see here), when I focussed on Plonsky ad Ziegler (2016), a critical evaluation of a number of CALL meta-analyses, including a few that investigated glosses. Plonsky and Ziegler found that glosses can have a positive effect on language learning, that digital glosses may be more valuable than paper-based ones, and that both L1 and L2 glosses can be beneficial (clearly, the quality / accuracy of the gloss is as important as the language it is written in). Different learners have different preferences. Boers’ article covers similar ground, without, I think, adding any new takeaways. It concludes with a predictable call for further research.

Boers has a short section on the ‘future of glossing’ in which he notes that (1) ‘onscreen reading [is] becoming the default mode’, and (2) that ‘materials developers no longer need to create glosses themselves, but can insert hyperlinks to online resources’. This is not the future, but the present. In my last blog post on glossing (August 2017), I discussed Lingro, a digital dictionary tool that you can have running in the background, allowing you to click on any word on any website and bring up L1 or L2 glosses. My reservation about Lingro was that the quality of the glosses left much to be desired, relying as they did on Wiktionary. Things would be rather different if it used decent content – sourced, for example, from Oxford dictionaries, Robert (for French) or Duden (for German).

And this is where the content for the Google Dictionary for Chrome extension comes from. It’s free, and takes only seconds to install. It allows you to double-click on a word to bring up translations or English definitions. One more click will take you to a more extensive dictionary page. It also allows you to select a phrase or longer passage and bring up translations generated by Google Translate. It allows you to keep track of the items you have looked up, and to download these on a spreadsheet, which can then be converted to flashcards (e.g. Quizlet) if you wish. If you use the Safari browser, a similar tool is already installed. It has similar features to the Google extension, but also offers you the possibility of linking to examples of the targeted word in web sources like Wikipedia.

Boers was thinking of the provision of hyperlinks, but with these browser extensions it is entirely up to the reader of a text to decide how many and which items to look up, what kind of items (single words, phrases or longer passages) they want to look up, how far they want to explore the information available to them, and what they want to do with the information (e.g. store / record it).

It’s extraordinary that a ‘state-of-the-art article’ in an extremely reputable journal should be so out of date. The value of glossing in language learning is in content-focussed reading, and these tools mean that any text on the web can be glossed. I think this means that further research of the kind that Boers means would be a waste of time and effort. The availability of free technology does not, of course, solve all our problems. Learners will continue to benefit from guidance, support and motivation in selecting appropriate texts to read. They will likely benefit from training in optimal ways of using these browser extensions. They may need help in finding a balance between content-focussed reading and content-focussed reading with a language learning payoff.

References

Boers, F. (2022). Glossing and vocabulary learning. Language Teaching, 55 (1), 1 – 23

Kim, H.S., Lee, J.H. & Lee, H. (2020). The relative effects of L1 and L2 glosses on L2 learning: A meta-analysis. Language Teaching Research. December 2020.

Nation, I.S.P. (2013). Learning Vocabulary in Another Language. Cambridge: Cambridge University Press

Plonsky, L. & Ziegler, N. (2016). The CALL–SLA interface: insights from a second-order synthesis. Language Learning & Technology 20 / 2: 17 – 37

Grit: I can do nothing for you, son

from Quartz at Work magazine

 

Grit is on the up. You may have come across articles like ‘How to Be Gritty in the Time of COVID-19’ or ‘Rediscovering the meaning of grit during COVID-19’ . If you still want more, there are new videos from Angela Duckworth herself where we can learn how to find our grit in the face of the pandemic.

Schools and educational authorities love grit. Its simple, upbeat message (‘Yes, you can’) has won over hearts and minds. Back in 2014, the British minister for education announced a £5million plan to encourage teaching ‘character and resilience’ in schools – specifically looking at making Britain’s pupils ‘grittier’. The spending on grit hasn’t stopped since.

The publishers of Duckworth’s book paid a seven-figure sum to acquire the US rights, and sales have proved the wisdom of the investment. Her TED talk has had over 6.5 million views on YouTube, although it’s worth looking at the comments to see why many people have been watching it.

The world of English language teaching, always on the lookout for a new bandwagon to jump onto, is starting to catch up with the wider world of education. Luke Plonsky, an eminent SLA scholar, specialist in meta-analyses and grit enthusiast, has a bibliography of grit studies related to L2 learning, that he deems worthy of consideration. Here’s a summary, by year, of those publications. More details will follow in the next section.

We can expect interest in ‘grit’ to continue growing, and this may be accelerated by the publication this year of Engaging Language Learners in Contemporary Classrooms by Sarah Mercer and Zoltán Dörnyei. In this book, the authors argue that a ‘facilitative mindset’ is required for learner engagement. They enumerate five interrelated principles for developing a ‘facilitative mindset’: promote a sense of competence, foster a growth mindset, promote learners’ sense of ownership and control, develop proactive learners and, develop gritty learners. After a brief discussion of grit, they write: ‘Thankfully, grit can be learnt and developed’ (p.38).

Unfortunately, they don’t provide any evidence at all for this. Unfortunately, too, this oversight is easy to explain. Such evidence as there is does not lend unequivocal support to the claim. Two studies that should have been mentioned in this book are ‘Much ado about grit: A meta-analytic synthesis of the grit literature’ (Credé et al, 2017) and ‘What shall we do about grit? A critical review of what we know and what we don’t know’ (Credé, 2018). The authors found that ‘grit as it is currently measured does not appear to be particularly predictive of success and performance’ (Credé et al, 2017) and that there is no support for the claim that ‘grit is likely to be responsive to interventions’ (Credé, 2018). In the L2 learning context, Teimouri et al (2020) concluded that more research in SLA substantiating the role of grit in L2 contexts was needed before any grit interventions can be recommended.

It has to be said that such results are hardly surprising. If, as Duckworth claims, ‘grit’ is a combination of passion and persistence, how on earth can the passion part of it be susceptible to educational interventions? ‘If there is one thing that cannot be learned, it’s passion. A person can have it and develop it, but learn it? Sadly, not’. (De Bruyckere et al., 2020: 83)

Even Duckworth herself is not convinced. In an interview on a Freakonomics podcast, she states that she hopes it’s something people can learn, but also admits not having enough proof to confirm that they can (Kirschner & Neelen, 2016)!

Is ‘grit’ a thing?

Marc Jones, in a 2016 blog post entitled ‘Gritty Politti: Grit, Growth Mindset and Neoliberal Language Teaching’, writes that ‘Grit is so difficult to define that it takes Duckworth (2016) the best part of a book to describe it adequately’. Yes, ‘grit’ is passion and persistence (or perseverance), but it’s also conscientiousness, practice and hope. Credé et al (2017) found that ‘grit is very strongly correlated with conscientiousness’ (which has already been widely studied in the educational literature). Why lump this together with passion? Another study (Muenks et al., 2017) found that ‘Students’ grit overlapped empirically with their concurrently reported self-control, self-regulation, and engagement. Students’ perseverance of effort (but not their consistency of interests) predicted their later grades, although other self-regulation and engagement variables were stronger predictors of students’ grades than was grit’. Credé (2018) concluded that ‘there appears to be no reason to accept the combination of perseverance and passion for long-term goals into a single grit construct’.

The L2 learning research listed in Plonsky’s bibliography does not offer much in support of ‘grit’, either. Many of the studies identified problems with ‘grit’ as a construct, but, even when accepting it, did not find it to be of much value. Wei et al. (2019) found a positive but weak correlation between grit and English language course grades. Yamashita (2018) found no relationship between learners’ grit and their course grades. Taşpinar & Külekçi (2018) found that students’ grit levels and academic achievement scores did not relate to each other (but still found that ‘grit, perseverance, and tenacity are the essential elements that impact learners’ ability to succeed to be prepared for the demands of today’s world’!).

There are, then, grounds for suspecting that Duckworth and her supporters have fallen foul of the ‘jangle fallacy’ – the erroneous assumption that two identical or almost identical things are different because they are labelled differently. This would also help to explain the lack of empirical support for the notion of ‘grit’. Not only are the numerous variables insufficiently differentiated, but the measures of ‘grit’ (such as Duckworth’s Grit-S measure) do not adequately target some of these variables (e.g. long-term goals, where ‘long-term’ is not defined) (Muenks et al., 2017). In addition, these measures are self-reporting and not, therefore, terribly reliable.

Referring to more general approaches to character education, one report (Gutman & Schoon, 2012) has argued that there is little empirical evidence of a causal relationship between self-concept and educational outcomes. Taking this one step further, Kathryn Ecclestone (Ecclestone, 2012) suggests that at best, the concepts and evidence that serve as the basis of these interventions are inconclusive and fragmented; ‘at worst, [they are] prey to ‘advocacy science’ or, in [their] worst manifestations, to simple entrepreneurship that competes for publicly funded interventions’ (cited in Cabanas & Illouz, 2019: 80).

Criticisms of ‘grit’

Given the lack of supporting research, any practical application of ‘grit’ ideas is premature. Duckworth herself, in an article entitled ‘Don’t Believe the Hype About Grit, Pleads the Scientist Behind the Concept’ (Dahl, 2016), cautions against hasty applications:

[By placing too much emphasis on grit, the danger is] that grit becomes a scapegoat — another reason to blame kids for not doing well, or to say that we don’t have a responsibility as a society to help them. [She worries that some interpretations of her work might make a student’s failure seem like his problem, as if he just didn’t work hard enough.] I think to separate and pit against each other character strengths on the one hand — like grit — and situational opportunities on the other is a false dichotomy […] Kids need to develop character, and they need our support in doing so.

Marc Jones, in the blog mentioned above, writes that ‘to me, grit is simply another tool for attacking the poor and the other’. You won’t win any prizes for guessing which kinds of students are most likely to be the targets of grit interventions. A clue: think of the ‘no-nonsense’ charters in the US and academies in the UK. This is what Kenneth Saltzman has to say:

‘Grit’ is a pedagogy of control that is predicated upon a promise made to poor children that if they learnt the tools of self-control and learnt to endure drudgery, then they can compete with rich children for scarce economic resources. (Saltzman, 2017: 38)

[It] is a behaviourist form of learned self-control targeting poor students of color and has been popularized post-crisis in the wake of educational privatization and defunding as the cure for poverty. [It] is designed to suggest that individual resilience and self-reliance can overcome social violence and unsupportive social contexts in the era of the shredded social state. (Saltzman, 2017: 15)

Grit is misrepresented by proponents as opening a world of individual choices rather than discussed as a mode of educational and social control in the austere world of work defined by fewer and fewer choices as secure public sector work is scaled back, unemployment continuing at high levels. (Saltzman, 2017: 49)

Whilst ‘grit’ is often presented as a way of dealing with structural inequalities in schools, critics see it as more of a problem than a solution: ‘It’s the kids who are most impacted by, rebel against, or criticize the embedded racism and classism of their institutions that are being told to have more grit, that school is hard for everyone’ (EquiTEA, 2018). A widely cited article by Nicholas Tampio (2016) points out that ‘Duckworth celebrates educational models such as Beast at West Point that weed out people who don’t obey orders’. He continues ‘that is a disastrous model for education in a democracy. US schools ought to protect dreamers, inventors, rebels and entrepreneurs – not crush them in the name of grit’.

If you’re interested in reading more critics of grit, the blog ‘Debunked!’ is an excellent source of links.

Measuring grit

Analyses of emotional behaviour have become central to economic analysis and, beginning in the 1990s, there have been constant efforts to create ‘formal instruments of classification of emotional behaviour and the elaboration of the notion of emotional competence’ (Illouz, 2007: 64). The measurement and manipulation of various aspects of ‘emotional intelligence’ have become crucial as ways ‘to control, predict, and boost performance’ (Illouz, 2007: 65). An article in the Journal of Benefit-Cost Analysis (Belfield et al., 2015) makes the economic importance of emotions very clear. Entitled ‘The Economic Value of Social and Emotional Learning’, it examines the economic value of these skills within a benefit-cost analysis (BCA) framework, and finds that the benefits of [social and emotional learning] interventions substantially outweigh the costs.

In recent years, the OECD has commissioned a number of reports on social and emotional learning and, as with everything connected with the OECD, is interested in measuring ‘non-cognitive skills such as perseverance (“grit”), conscientiousness, self-control, trust, attentiveness, self-esteem and self-efficacy, resilience to adversity, openness to experience, empathy, humility, tolerance of diverse opinions and the ability to engage productively in society’ (Kautz et al., 2014: 9). The measurement of personality factors will feature in the OECD’s PISA programme. Elsewhere, Ben Williamson reports that ‘US schools [are] now under pressure—following the introduction of the Every Student Succeeds Act in 2015—to provide measurable evidence of progress on the development of students’ non-academic learning’ (Williamson, 2017).

Grit, which ‘starts and ends with the lone individual as economic actor, worker, and consumer’ (Saltzman, 2017: 50), is a recent addition to the categories of emotional competence, and it should come as no surprise that educational authorities have so wholeheartedly embraced it. It is the claim that something (i.e. ‘grit’) can be taught and developed that leads directly to the desire to measure it. In a world where everything must be accountable, we need to know how effective and cost-effective our grit interventions have been.

The idea of measuring personality constructs like ‘grit’ worries even Angela Duckworth. She writes (Duckworth, 2016):

These days, however, I worry I’ve contributed, inadvertently, to an idea I vigorously oppose: high-stakes character assessment. New federal legislation can be interpreted as encouraging states and schools to incorporate measures of character into their accountability systems. This year, nine California school districts will begin doing this. But we’re nowhere near ready — and perhaps never will be — to use feedback on character as a metric for judging the effectiveness of teachers and schools. We shouldn’t be rewarding or punishing schools for how students perform on these measures.

Diane Ravitch (Ravitch, 2016) makes the point rather more forcefully: ‘The urge to quantify the unmeasurable must be recognized for what it is: stupid; arrogant; harmful; foolish, yet another way to standardize our beings’. But, like it or not, attempts to measure ‘grit’ and ‘grit’ interventions are unlikely to go away any time soon.

‘Grit’ and technology

Whenever there is talk about educational measurement and metrics, we are never far away from the world of edtech. It may not have escaped your notice that the OECD and the US Department of State for Education, enthusiasts for promoting ‘grit’, are also major players in the promotion of the datafication of education. The same holds true for organisations like the World Education Forum, the World Bank and the various philanthro-capitalist foundations to which I have referred so often in this blog. Advocacy of social and emotional learning goes hand in hand with edtech advocacy.

Two fascinating articles by Ben Williamson (2017; 2019) focus on ClassDojo, which, according to company information, reaches more than 10 million children globally every day. The founding directors of ClassDojo, writes Ben Williamson (2017), ‘explicitly describe its purpose as promoting ‘character development’ in schools and it is underpinned by particular psychological concepts from character research. Its website approvingly cites the journalist Paul Tough, author of two books on promoting ‘grit’ and ‘character’ in children, and is informed by character research conducted with the US network of KIPP charter schools (Knowledge is Power Program)’. In a circular process, ClassDojo has also ‘helped distribute and popularise concepts such as growth mindset, grit and mindfulness’ (Williamson, 2019).

The connections between ‘grit’ and edtech are especially visible when we focus on Stanford and Silicon Valley. ClassDojo was born in Palo Alto. Duckworth was a consulting scholar at Stanford 2014 -15, where Carol Dweck is a Professor of Psychology. Dweck is the big name behind growth mindset theory, which, as Sarah Mercer and Zoltán Dörnyei indicate, is closely related to ‘grit’. Dweck is also the co-founder of MindsetWorks, whose ‘Brainology’ product is ‘an online interactive program in which middle school students learn about how the brain works, how to strengthen their own brains, and how to ….’. Stanford is also home to the Stanford Lytics Lab, ‘which has begun applying new data analytics techniques to the measurement of non-cognitive learning factors including perseverance, grit, emotional state, motivation and self-regulation’, as well as the Persuasive Technologies Lab, ‘which focuses on the development of machines designed to influence human beliefs and behaviors across domains including health, business, safety, and education’ (Williamson, 2017). The Professor of Education Emeritus at Stanford is Linda Darling-Hammond, one of the most influential educators in the US. Darling-Hammond is known, among many other things, for collaborating with Pearson to develop the edTPA, ‘a nationally available, performance-based assessment for measuring the effectiveness of teacher candidates’. She is also a strong advocate of social-emotional learning initiatives and extols the virtues of ‘developing grit and a growth mindset’ (Hamadi & Darling-Hammond, 2015).

The funding of grit

Angela Duckworth’s Character Lab (‘Our mission is to advance scientific insights that help kids thrive’) is funded by, among others, the Chan Zuckerberg Initiative, the Bezos Family Foundation and Stanford’s Mindset Scholars Network. Precisely how much money Character Lab has is difficult to ascertain, but the latest grant from the Chan Zuckerberg Initiative was worth $1,912,000 to cover the period 2018 – 2021. Covering the same period, the John Templeton Foundation, donated $3,717,258 , the purpose of the grant being to ‘make character development fast, frictionless, and fruitful’.

In an earlier period (2015 – 2018), the Walton Family Foundation pledged $6.5 million ‘to promote and measure character education, social-emotional learning, and grit’, with part of this sum going to Character Lab and part going to similar research at Harvard Graduate School of Education. Character Lab also received $1,300,000 from the Overdeck Family Foundation for the same period.

It is not, therefore, an overstatement to say that ‘grit’ is massively funded. The funders, by and large, are the same people who have spent huge sums promoting personalized learning through technology (see my blog post Personalized learning: Hydra and the power of ambiguity). Whatever else it might be, ‘grit’ is certainly ‘a commercial tech interest’ (as Ben Williamson put it in a recent tweet).

Postscript

In the 2010 Cohen brothers’ film, ‘True Grit’, the delinquent ‘kid’, Moon, is knifed by his partner, Quincy. Turning to Rooster Cogburn, the man of true grit, Moon begs for help. In response, Cogburn looks at the dying kid and deadpans ‘I can do nothing for you, son’.

References

Belfield, C., Bowden, A., Klapp, A., Levin, H., Shand, R., & Zander, S. (2015). The Economic Value of Social and Emotional Learning. Journal of Benefit-Cost Analysis, 6(3), pp. 508-544. doi:10.1017/bca.2015.55

Cabanas, E. & Illouz, E. (2019). Manufacturing Happy Citizens. Cambridge: Polity Press.

Chaykowski, K. (2017). How ClassDojo Built One Of The Most Popular Classroom Apps By Listening To Teachers. Forbes, 22 May, 2017. https://www.forbes.com/sites/kathleenchaykowski/2017/05/22/how-classdojo-built-one-of-the-most-popular-classroom-apps-by-listening-to-teachers/#ea93d51e5ef5

Credé, M. (2018). What shall we do about grit? A critical review of what we know and what we don’t know. Educational Researcher, 47(9), 606-611.

Credé, M., Tynan, M. C., & Harms, P. D. (2017). Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113(3), 492. doi:10.1037/pspp0000102

Dahl, M. (2016). Don’t Believe the Hype About Grit, Pleads the Scientist Behind the Concept. The Cut, May 9, 2016. https://www.thecut.com/2016/05/dont-believe-the-hype-about-grit-pleads-the-scientist-behind-the-concept.html

De Bruyckere, P., Kirschner, P. A. & Hulshof, C. (2020). More Urban Myths about Learning and Education. Routledge.

Duckworth, A. (2016). Don’t Grade Schools on Grit. New York Times, March 26, 2016 https://www.nytimes.com/2016/03/27/opinion/sunday/dont-grade-schools-on-grit.html?auth=login-google&smid=nytcore-ipad-share&smprod=nytcore-ipad

Ecclestone, K. (2012). From emotional and psychological well-being to character education: Challenging policy discourses of behavioural science and ‘vulnerability’. Research Papers in Education, 27 (4), pp. 463-480

EquiTEA (2018). The Problem with Teaching ‘Grit’. Medium, 11 December 2018. https://medium.com/@eec/the-problem-with-teaching-grit-8b37ce43a87e

Gutman, L. M. & Schoon, I. (2013). The impact of non-cognitive skills on outcomes for young people: Literature review. London: Institute of Education, University of London

Hamedani, M. G. & Darling-Hammond, L. (2015). Social Emotional Learning in High School: How Three Urban High Schools Engage, Educate, and Empower Youth. Stanford Center for Opportunity Policy in Education

Kirschner, P.A. & Neelen, M. (2016). To Grit Or Not To Grit: That’s The Question. 3-Star Learning Experiences, July 5, 2016 https://3starlearningexperiences.wordpress.com/2016/07/05/to-grit-or-not-to-grit-thats-the-question/

Illouz, E. (2007). Cold Intimacies: The making of emotional capitalism. Cambridge: Polity Press

Kautz, T., Heckman, J. J., Diris, R., ter Weel, B & Borghans, L. (2014). Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success. OECD Education Working Papers 110, OECD Publishing.

Mercer, S. & Dörnyei, Z. (2020). Engaging Language Learners in Contemporary Classrooms. Cambridge University Press.

Muenks, K., Wigfield, A., Yang, J. S., & O’Neal, C. R. (2017). How true is grit? Assessing its relations to high school and college students’ personality characteristics, self-regulation, engagement, and achievement. Journal of Educational Psychology, 109, pp. 599–620.

Ravitch, D. (2016). Angela Duckworth, please don’t assess grit. Blog post, 27 March 2016, https://dianeravitch.net/2016/03/27/angela-duckworth-please-dont-assess-grit/

Saltzman, K. J. (2017). Scripted Bodies. Routledge.

Tampio, N. (2016). Teaching ‘grit’ is bad for children, and bad for democracy. Aeon, 2 June: https://aeon.co/ideas/teaching-grit-is-bad-for-children-and-bad-for-democracy

Taşpinar, K., & Külekçi, G. (2018). GRIT: An Essential Ingredient of Success in the EFL Classroom. International Journal of Languages’ Education and Teaching, 6, 208-226.

Teimouri, Y., Plonsky, L., & Tabandeh, F. (2020). L2 Grit: Passion and perseverance for second-language learning. Language Teaching Research.

Wei, H., Gao, K., & Wang, W. (2019). Understanding the relationship between grit and foreign language performance among middle schools students: The roles of foreign language enjoyment and classroom Environment. Frontiers in Psychology, 10, 1508. doi: 10.3389/fpsyg.2019.01508

Williamson, B. (2017). Decoding ClassDojo: psycho-policy, social-emotional learning and persuasive educational technologies. Learning, Media and Technology, 42 (4): pp. 440-453, DOI: 10.1080/17439884.2017.1278020

Williamson, B. (2019). ‘Killer Apps for the Classroom? Developing Critical Perspectives on ClassDojo and the ‘Ed-tech’ Industry. Journal of Professional Learning, 2019 (Semester 2) https://cpl.asn.au/journal/semester-2-2019/killer-apps-for-the-classroom-developing-critical-perspectives-on-classdojo

Yamashita, T. (2018). Grit and second language acquisition: Can passion and perseverance predict performance in Japanese language learning? Unpublished MA thesis, University of Massachusetts, Amherst.

 

The EdTech Imaginary in ELT

In my last post , I asked why it is so easy to believe that technology (in particular, technological innovations) will offer solutions to whatever problems exist in language learning and teaching. A simple, but inadequate, answer is that huge amounts of money have been invested in persuading us. Without wanting to detract from the significance of this, it is clearly not sufficient as an explanation. In an attempt to develop my own understanding, I have been turning more and more to the idea of ‘social imaginaries’. In many ways, this is also an attempt to draw together the various interests that I have had since starting this blog.

The Canadian philosopher, Charles Taylor, describes a ‘social imaginary’ as a ‘common understanding that makes possible common practices and a widely shared sense of legitimacy’ (Taylor, 2004: 23). As a social imaginary develops over time, it ‘begins to define the contours of [people’s] worlds and can eventually come to count as the taken-for-granted shape of things, too obvious to mention’ (Taylor, 2004: 29). It is, however, not just a set of ideas or a shared narrative: it is also a set of social practices that enact those understandings, whilst at the same time modifying or solidifying them. The understandings make the practices possible, and it is the practices that largely carry the understanding (Taylor, 2004: 25). In the process, the language we use is filled with new associations and our familiarity with these associations shapes ‘our perceptions and expectations’ (Worster, 1994, quoted in Moore, 2015: 33). A social imaginary, then, is a complex system that is not technological or economic or social or political or educational, but all of these (Urry, 2016). The image of the patterns of an amorphous mass of moving magma (Castoriadis, 1987), flowing through pre-existing channels, but also, at times, striking out along new paths, may offer a helpful metaphor.

Technology, of course, plays a key role in contemporary social imaginaries and the term ‘sociotechnical imaginary’ is increasingly widely used. The understandings of the sociotechnical imaginary typically express visions of social progress and a desirable future that is made possible by advances in science and technology (Jasanoff & Kim, 2015: 4). In education, technology is presented as capable of overcoming human failings and the dark ways of the past, of facilitating a ‘pedagogical utopia of natural, authentic teaching and learning’ (Friesen, forthcoming). As such understandings become more widespread and as the educational practices (platforms, apps, etc.) which both shape and are shaped by them become equally widespread, technology has come to be seen as a ‘solution’ to the ‘problem’ of education (Friesen, forthcoming). We need to be careful, however, that having shaped the technology, it does not comes to shape us (see Cobo, 2019, for a further exploration of this idea).

As a way of beginning to try to understand what is going on in edtech in ELT, which is not so very different from what is taking place in education more generally, I have sketched a number of what I consider key components of the shared understandings and the social practices that are related to them. These are closely interlocking pieces and each of them is itself embedded in much broader understandings. They evolve over time and their history can be traced quite easily. Taken together, they do, I think, help us to understand a little more why technology in ELT seems so seductive.

1 The main purpose of English language teaching is to prepare people for the workplace

There has always been a strong connection between learning an additional living language (such as English) and preparing for the world of work. The first modern language schools, such as the Berlitz schools at the end of the 19^th century with their native-speaker teachers and monolingual methods, positioned themselves as primarily vocational, in opposition to the kinds of language teaching taking place in schools and universities, which were more broadly humanistic in their objectives. Throughout the 20^th century, and especially as English grew as a global language, the public sector, internationally, grew closer to the methods and objectives of the private schools. The idea that learning English might serve other purposes (e.g. cultural enrichment or personal development) has never entirely gone away, as witnessed by the Council of Europe’s list of objectives (including the promotion of mutual understanding and European co-operation, and the overcoming of prejudice and discrimination) in the Common European Framework, but it is often forgotten.

The clarion calls from industry to better align education with labour markets, present and future, grow louder all the time, often finding expression in claims that ‘education is unfit for purpose.’ It is invariably assumed that this purpose is to train students in the appropriate skills to enhance their ‘human capital’ in an increasingly competitive and global market (Lingard & Gale, 2007). Educational agendas are increasingly set by the world of business (bodies like the OECD or the World Economic Forum, corporations like Google or Microsoft, and national governments which share their priorities (see my earlier post about neo-liberalism and solutionism ).

One way in which this shift is reflected in English language teaching is in the growing emphasis that is placed on ‘21^st century skills’ in teaching material. Sometimes called ‘life skills’, they are very clearly concerned with the world of work, rather than the rest of our lives. The World Economic Forum’s 2018 Future of Jobs survey lists the soft skills that are considered important in the near future and they include ‘creativity’, ‘critical thinking’, ‘emotional intelligence’ and ‘leadership’. (The fact that the World Economic Forum is made up of a group of huge international corporations (e.g. J.P. Morgan, HSBC, UBS, Johnson & Johnson) with a very dubious track record of embezzlement, fraud, money-laundering and tax evasion has not resulted in much serious, public questioning of the view of education expounded by the WEF.)

Without exception, the ELT publishers have brought these work / life skills into their courses, and the topic is an extremely popular one in ELT blogs and magazines, and at conferences. Two of the four plenaries at this year’s international IATEFL conference are concerned with these skills. Pearson has a wide range of related products, including ‘a four-level competency-based digital course that provides engaging instruction in the essential work and life skills competencies that adult learners need’. Macmillan ELT made ‘life skills’ the central plank of their marketing campaign and approach to product design, and even won a British Council ELTon (see below) Award for ‘Innovation in teacher resources) in 2015 for their ‘life skills’ marketing campaign. Cambridge University Press has developed a ‘Framework for Life Competencies’ which allows these skills to be assigned numerical values.

The point I am making here is not that these skills do not play an important role in contemporary society, nor that English language learners may not benefit from some training in them. The point, rather, is that the assumption that English language learning is mostly concerned with preparation for the workplace has become so widespread that it becomes difficult to think in another way.

2 Technological innovation is good and necessary

The main reason that soft skills are deemed to be so important is that we live in a rapidly-changing world, where the unsubstantiated claim that 85% (or whatever other figure comes to mind) of current jobs won’t exist 10 years from now is so often repeated that it is taken as fact . Whether or not this is true is perhaps less important to those who make the claim than the present and the future that they like to envisage. The claim is, at least, true-ish enough to resonate widely. Since these jobs will disappear, and new ones will emerge, because of technological innovations, education, too, will need to innovate to keep up.

English language teaching has not been slow to celebrate innovation. There were coursebooks called ‘Cutting Edge’ (1998) and ‘Innovations’ (2005), but more recently the connections between innovation and technology have become much stronger. The title of the recent ‘Language Hub’ (2019) was presumably chosen, in part, to conjure up images of digital whizzkids in fashionable co-working start-up spaces. Technological innovation is explicitly promoted in the Special Interest Groups of IATEFL and TESOL. Despite a singular lack of research that unequivocally demonstrates a positive connection between technology and language learning, the former’s objective is ‘to raise awareness among ELT professionals of the power of learning technologies to assist with language learning’. There is a popular annual conference, called InnovateELT , which has the tagline ‘Be Part of the Solution’, and the first problem that this may be a solution to is that our students need to be ‘ready to take on challenging new careers’.

Last, but by no means least, there are the annual British Council ELTon awards  with a special prize for digital innovation. Among the British Council’s own recent innovations are a range of digitally-delivered resources to develop work / life skills among teens.

Again, my intention (here) is not to criticise any of the things mentioned in the preceding paragraphs. It is merely to point to a particular structure of feeling and the way that is enacted and strengthened through material practices like books, social groups, conferences and other events.

3 Technological innovations are best driven by the private sector

The vast majority of people teaching English language around the world work in state-run primary and secondary schools. They are typically not native-speakers of English, they hold national teaching qualifications and they are frequently qualified to teach other subjects in addition to English (often another language). They may or may not self-identify as teachers of ‘ELT’ or ‘EFL’, often seeing themselves more as ‘school teachers’ or ‘language teachers’. People who self-identify as part of the world of ‘ELT or ‘TEFL’ are more likely to be native speakers and to work in the private sector (including private or semi-private language schools, universities (which, in English-speaking countries, are often indistinguishable from private sector institutions), publishing companies, and freelancers). They are more likely to hold international (TEFL) qualifications or higher degrees, and they are less likely to be involved in the teaching of other languages.

The relationship between these two groups is well illustrated by the practice of training days, where groups of a few hundred state-school teachers participate in workshops organised by publishing companies and delivered by ELT specialists. In this context, state-school teachers are essentially in a client role when they are in contact with the world of ‘ELT’ – as buyers or potential buyers of educational products, training or technology.

Technological innovation is invariably driven by the private sector. This may be in the development of technologies (platforms, apps and so on), in the promotion of technology (through training days and conference sponsorship, for example), or in training for technology (with consultancy companies like ELTjam or The Consultants-E, which offer a wide range of technologically oriented ‘solutions’).

As in education more generally, it is believed that the private sector can be more agile and more efficient than state-run bodies, which continue to decline in importance in educational policy-setting. When state-run bodies are involved in technological innovation in education, it is normal for them to work in partnership with the private sector.

4 Accountability is crucial

Efficacy is vital. It makes no sense to innovate unless the innovations improve something, but for us to know this, we need a way to measure it. In a previous post , I looked at Pearson’s ‘Asking More: the Path to Efficacy’ by CEO John Fallon (who will be stepping down later this year). Efficacy in education, says Fallon, is ‘making a measurable impact on someone’s life through learning’. ‘Measurable’ is the key word, because, as Fallon claims, ‘it is increasingly possible to determine what works and what doesn’t in education, just as in healthcare.’ We need ‘a relentless focus’ on ‘the learning outcomes we deliver’ because it is these outcomes that can be measured in ‘a systematic, evidence-based fashion’. Measurement, of course, is all the easier when education is delivered online, ‘real-time learner data’ can be captured, and the power of analytics can be deployed.

Data is evidence, and it’s as easy to agree on the importance of evidence as it is hard to decide on (1) what it is evidence of, and (2) what kind of data is most valuable. While those questions remain largely unanswered, the data-capturing imperative invades more and more domains of the educational world.

English language teaching is becoming data-obsessed. From language scales, like Pearson’s Global Scale of English to scales of teacher competences, from numerically-oriented formative assessment practices (such as those used on many LMSs) to the reporting of effect sizes in meta-analyses (such as those used by John Hattie and colleagues), datafication in ELT accelerates non-stop.

The scales and frameworks are all problematic in a number of ways (see, for example, this post on ‘The Mismeasure of Language’) but they have undeniably shaped the way that we are able to think. Of course, we need measurable outcomes! If, for the present, there are privacy and security issues, it is to be hoped that technology will find solutions to them, too.

REFERENCES

Castoriadis, C. (1987). The Imaginary Institution of Society. Cambridge: Polity Press.

Cobo, C. (2019). I Accept the Terms and Conditions. Montevideo: International Development Research Centre / Center for Research Ceibal Foundation. https://adaptivelearninginelt.files.wordpress.com/2020/01/41acf-cd84b5_7a6e74f4592c460b8f34d1f69f2d5068.pdf

Friesen, N. (forthcoming) The technological imaginary in education, or: Myth and enlightenment in ‘Personalized Learning’. In M. Stocchetti (Ed.) The Digital Age and its Discontents. University of Helsinki Press. Available at https://www.academia.edu/37960891/The_Technological_Imaginary_in_Education_or_Myth_and_Enlightenment_in_Personalized_Learning_

Jasanoff, S. & Kim, S.-H. (2015). Dreamscapes of Modernity. Chicago: University of Chicago Press.

Lingard, B. & Gale, T. (2007). The emergent structure of feeling: what does it mean for critical educational studies and research?, Critical Studies in Education, 48:1, pp. 1-23

Moore, J. W. (2015). Capitalism in the Web of Life. London: Verso.

Robbins, K. & Webster, F. (1989]. The Technical Fix. Basingstoke: Macmillan Education.

Taylor, C. (2014). Modern Social Imaginaries. Durham, NC: Duke University Press.

Urry, J. (2016). What is the Future? Cambridge: Polity Press.

 

What we can learn from glosses and research into glosses

I’m a sucker for meta-analyses, those aggregates of multiple studies that generate an effect size, and I am even fonder of meta-meta analyses. I skip over the boring stuff about inclusion criteria and statistical procedures and zoom in on the results and discussion. I’ve pored over Hattie (2009) and, more recently, Dunlosky et al (2013), and quoted both more often than is probably healthy. Hardly surprising, then, that I was eager to read Luke Plonsky and Nicole Ziegler’s ‘The CALL–SLA interface: insights from a second-order synthesis’ (Plonsky & Ziegler, 2016), an analysis of nearly 30 meta-analyses (later whittled down to 14) looking at the impact of technology on L2 learning. The big question they were looking to find an answer to? How effective is computer-assisted language learning compared to face-to-face contexts?

Plonsky & Ziegler

Plonsky and Ziegler found that there are unequivocally ‘positive effects of technology on language learning’. In itself, this doesn’t really tell us anything, simply because there are too many variables. It’s a statistical soundbite, ripe for plucking by anyone with an edtech product to sell. Much more useful is to understand which technologies used in which ways are likely to have a positive effect on learning. It appears from Plonsky and Ziegler’s work that the use of CALL glosses (to develop reading comprehension and vocabulary development) provides the strongest evidence of technology’s positive impact on learning. The finding is reinforced by the fact that this particular technology was the most well-represented research area in the meta-analyses under review.

What we know about glosses

A gloss is ‘a brief definition or synonym, either in L1 or L2, which is provided with [a] text’ (Nation, 2013: 238). They can take many forms (e.g. annotations in the margin or at the foot a printed page), but electronic or CALL glossing is ‘an instant look-up capability – dictionary or linked’ (Taylor, 2006; 2009) which is becoming increasingly standard in on-screen reading. One of the most widely used is probably the translation function in Microsoft Word: here’s the French gloss for the word ‘gloss’.

Language learning tools and programs are making increasing use of glosses. Here are two examples. The first is Lingro , a dictionary tool that learners can have running alongside any webpage: clicking on a word brings up a dictionary entry, and the word can then be exported into a wordlist which can be practised with spaced repetition software. The example here is using the English-English dictionary, but a number of bilingual pairings are available. The second is from Bliu Bliu , a language learning app that I unkindly reviewed here .

So, what did Plonsky and Ziegler discover about glosses? There were two key takeways:

• both L1 and L2 CALL glossing can be beneficial to learners’ vocabulary development (Taylor, 2006, 2009, 2013)
• CALL / electronic glosses lead to more learning gains than paper-based glosses (p.22)

On the surface, this might seem uncontroversial, but if you took a good look at the three examples (above) of online glosses, you’ll be thinking that something is not quite right here. Lingro’s gloss is a fairly full dictionary entry: it contains too much information for the purpose of a gloss. Cognitive Load Theory suggests that ‘new information be provided concisely so as not to overwhelm the learner’ (Khezrlou et al, 2017: 106): working out which definition is relevant here (the appropriate definition is actually the sixth in this list) will overwhelm many learners and interfere with the process of reading … which the gloss is intended to facilitate. In addition, the language of the definitions is more difficult than the defined item. Cognitive load is, therefore, further increased. Lingro needs to use a decent learner’s dictionary (with a limited defining vocabulary), rather than relying on the free Wiktionary.

Nation (2013: 240) cites research which suggests that a gloss is most effective when it provides a ‘core meaning’ which users will have to adapt to what is in the text. This is relatively unproblematic, from a technological perspective, but few glossing tools actually do this. The alternative is to use NLP tools to identify the context-specific meaning: our ability to do this is improving all the time but remains some way short of total accuracy. At the very least, NLP tools are needed to identify part of speech (which will increase the probability of hitting the right meaning). Bliu Bliu gets things completely wrong, confusing the verb and the adjective ‘own’.

Both Lingro and Bliu Bliu fail to meet the first requirement of a gloss: ‘that it should be understood’ (Nation, 2013: 239). Neither is likely to contribute much to the vocabulary development of learners. We will need to modify Plonsky and Ziegler’s conclusions somewhat: they are contingent on the quality of the glosses. This is not, however, something that can be assumed …. as will be clear from even the most cursory look at the language learning tools that are available.

Nation (2013: 447) also cites research that ‘learning is generally better if the meaning is written in the learner’s first language. This is probably because the meaning can be easily understood and the first language meaning already has many rich associations for the learner. Laufer and Shmueli (1997) found that L1 glosses are superior to L2 glosses in both short-term and long-term (five weeks) retention and irrespective of whether the words are learned in lists, sentences or texts’. Not everyone agrees, and a firm conclusion either way is probably not possible: learner variables (especially learner preferences) preclude anything conclusive, which is why I’ve highlighted Nation’s use of the word ‘generally’. If we have a look at Lingro’s bilingual gloss, I think you’ll agree that the monolingual and bilingual glosses are equally unhelpful, equally unlikely to lead to better learning, whether it’s vocabulary acquisition or reading comprehension.

 

The issues I’ve just discussed illustrate the complexity of the ‘glossing’ question, but they only scratch the surface. I’ll dig a little deeper.

1 Glosses are only likely to be of value to learning if they are used selectively. Nation (2013: 242) suggests that ‘it is best to assume that the highest density of glossing should be no more than 5% and preferably around 3% of the running words’. Online glosses make the process of look-up extremely easy. This is an obvious advantage over look-ups in a paper dictionary, but there is a real risk, too, that the ease of online look-up encourages unnecessary look-ups. More clicks do not always lead to more learning. The value of glosses cannot therefore be considered independently of a consideration of the level (i.e. appropriacy) of the text that they are being used with.

2 A further advantage of online glosses is that they can offer a wide range of information, e.g. pronunciation, L1 translation, L2 definition, visuals, example sentences. The review of literature by Khezrlou et al (2017: 107) suggests that ‘multimedia glosses can promote vocabulary learning but uncertainty remains as to whether they also facilitate reading comprehension’. Barcroft (2015), however, warns that pictures may help learners with meaning, but at the cost of retention of word form, and the research of Boers et al did not find evidence to support the use of pictures. Even if we were to accept the proposition that pictures might be helpful, we would need to hold two caveats. First, the amount of multimodal support should not lead to cognitive overload. Second, pictures need to be clear and appropriate: a condition that is rarely met in online learning programs. The quality of multimodal glosses is more important than their inclusion / exclusion.

3 It’s a commonplace to state that learners will learn more if they are actively engaged or involved in the learning, rather than simply (receptively) looking up a gloss. So, it has been suggested that cognitive engagement can be stimulated by turning the glosses into a multiple-choice task, and a fair amount of research has investigated this possibility. Barcroft (2015: 143) reports research that suggests that ‘multiple-choice glosses [are] more effective than single glosses’, but Nation (2013: 246) argues that ‘multiple choice glosses are not strongly supported by research’. Basically, we don’t know and even if we have replication studies to re-assess the benefits of multimodal glosses (as advocated by Boers et al, 2017), it is again likely that learner variables will make it impossible to reach a firm conclusion.

Learning from meta-analyses

Discussion of glosses is not new. Back in the late 19^th century, ‘most of the Reform Movement teachers, took the view that glossing was a sensible technique’ (Howatt, 2004: 191). Sensible, but probably not all that important in the broader scheme of language learning and teaching. Online glosses offer a number of potential advantages, but there is a huge number of variables that need to be considered if the potential is to be realised. In essence, I have been arguing that asking whether online glosses are more effective than print glosses is the wrong question. It’s not a question that can provide us with a useful answer. When you look at the details of the research that has been brought together in the meta-analysis, you simply cannot conclude that there are unequivocally positive effects of technology on language learning, if the most positive effects are to be found in the digital variation of an old sensible technique.

Interesting and useful as Plonsky and Ziegler’s study is, I think it needs to be treated with caution. More generally, we need to be cautious about using meta-analyses and effect sizes. Mura Nava has a useful summary of an article by Adrian Simpson (Simpson, 2017), that looks at inclusion criteria and statistical procedures and warns us that we cannot necessarily assume that the findings of meta-meta-analyses are educationally significant. More directly related to technology and language learning, Boulton’s paper (Boulton, 2016) makes a similar point: ‘Meta-analyses need interpreting with caution: in particular, it is tempting to seize on a single figure as the ultimate answer to the question: Does it work? […] More realistically, we need to look at variation in what works’.

For me, the greatest value in Plonsky and Ziegler’s paper was nothing to do with effect sizes and big answers to big questions. It was the bibliography … and the way it forced me to be rather more critical about meta-analyses.

References

Barcroft, J. 2015. Lexical Input Processing and Vocabulary Learning. Amsterdam: John Benjamins

Boers, F., Warren, P., He, L. & Deconinck, J. 2017. ‘Does adding pictures to glosses enhance vocabulary uptake from reading?’ System 66: 113 – 129

Boulton, A. 2016. ‘Quantifying CALL: significance, effect size and variation’ in S. Papadima-Sophocleus, L. Bradley & S. Thouësny (eds.) CALL Communities and Culture – short papers from Eurocall 2016 pp.55 – 60 http://files.eric.ed.gov/fulltext/ED572012.pdf

Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J. & Willingham, D.T. 2013. ‘Improving Students’ Learning With Effective Learning Techniques’ Psychological Science in the Public Interest 14 / 1: 4 – 58

Hattie, J.A.C. 2009. Visible Learning. Abingdon, Oxon.: Routledge

Howatt, A.P.R. 2004. A History of English Language Teaching 2^nd edition. Oxford: Oxford University Press

Khezrlou, S., Ellis, R. & K. Sadeghi 2017. ‘Effects of computer-assisted glosses on EFL learners’ vocabulary acquisition and reading comprehension in three learning conditions’ System 65: 104 – 116

Laufer, B. & Shmueli, K. 1997. ‘Memorizing new words: Does teaching have anything to do with it?’ RELC Journal 28 / 1: 89 – 108

Nation, I.S.P. 2013. Learning Vocabulary in Another Language. Cambridge: Cambridge University Press

Plonsky, L. & Ziegler, N. 2016. ‘The CALL–SLA interface:  insights from a second-order synthesis’ Language Learning & Technology 20 / 2: 17 – 37

Simpson, A. 2017. ‘The misdirection of public policy: Comparing and combining standardised effect sizes’ Journal of Education Policy, 32 / 4: 450-466

Taylor, A. M. 2006. ‘The effects of CALL versus traditional L1 glosses on L2 reading comprehension’. CALICO Journal, 23, 309–318.

Taylor, A. M. 2009. ‘CALL-based versus paper-based glosses: Is there a difference in reading comprehension?’ CALICO Journal, 23, 147–160.

Taylor, A. M. 2013. CALL versus paper: In which context are L1 glosses more effective? CALICO Journal, 30, 63-8