Posts Tagged ‘research’

A week or so ago, someone in the Macmillan marketing department took it upon themselves to send out this tweet. What grabbed my attention was the claim that it is ‘a well-known fact’ that teaching students a growth mindset makes them perform better academically over time. The easily demonstrable reality (which I’ll come on to) is that this is not a fact. It’s fake news, being used for marketing purposes. The tweet links to a blog post of over a year ago. In it, Chia Suan Chong offers five tips for developing a growth mindset in students: educating students about neuroplasticity, delving deeper into success stories, celebrating challenges and mistakes, encouraging students to go outside their comfort zones, and giving ‘growth-mindset-feedback’. All of which, she suggests, might help our students. Indeed, it might, and, even if it doesn’t, it might be worth a try anyway. Chia doesn’t make any claims beyond the potential of the suggested strategies, so I wonder where the Macmillan Twitter account person got the ‘well-known fact’.

If you google ‘mindset ELT’, you will find webpage after webpage offering tips about how to promote growth mindset in learners. It’s rare for the writers of these pages to claim that the positive effects of mindset interventions are a ‘fact’, but it’s even rarer to come across anyone who suggests that mindset interventions might be an à la mode waste of time and effort. Even in more serious literature (e.g. Mercer, S. & Ryan, S. (2010). A mindset for EFL: learners’ beliefs about the role of natural talent. ELT Journal, 64 (4): 436 – 444), the approach is fundamentally enthusiastic, with no indication that there might be a problem with mindset theory. Given that this enthusiasm is repeated so often, perhaps we should not blame the Macmillan tweeter for falling victim to the illusory truth effect. After all, it appears that 98% of teachers in the US feel that growth mindset approaches should be adopted in schools (Hendrick, 2019).

Chia suggests that we can all have fixed mindsets in certain domains (e.g. I know all about that, there’s nothing more I can learn). One domain where it seems that fixed mindsets are prevalent is mindset theory itself. This post is an attempt to nudge towards more ‘growth’ and, in trying to persuade you to be more sceptical, I will quote as much as possible from Carol Dweck, the founder of mindset theory, and her close associates.

Carol Dweck’s book ‘Mindset: The New Psychology of Success’ appeared in 2006. In it, she argued that people can be placed on a continuum between those who have ‘a fixed mindset–those who believe that abilities are fixed—[and who] are less likely to flourish [and] those with a growth mindset–those who believe that abilities can be developed’ (from the back cover of the updated (2007) version of the book). There was nothing especially new about the idea. It is very close to Bandura’s (1982) theory of self-efficacy, which will be familiar to anyone who has read Zoltán Dörnyei’s more recent work on motivation in language learning. It’s closely related to Carl Roger’s (1969) ideas about self-concept and it’s not a million miles removed, either, from Maslow’s (1943) theory of self-actualization. The work of Rogers and Maslow was at the heart of the ‘humanistic turn’ in ELT in the latter part of the 20th century (see, for example, Early, 1981), so mindset theory is likely to resonate with anyone who was inspired by the humanistic work of people like Moskowitz, Stevick or Rinvolucri. The appeal of mindset theory is easy to see. Besides its novelty value, it resonates emotionally with the values that many teachers share, writes Tom Bennett: it feels right that you don’t criticise the person, but invite them to believe that, through hard work and persistence, you can achieve.

We might even trace interest in the importance of self-belief back to the Stoics (who, incidentally but not coincidentally, are experiencing a revival of interest), but Carol Dweck introduced a more modern flavour to the old wine and packaged it skilfully and accessibly in shiny new bottles. Her book was a runaway bestseller, with sales in the millions, and her TED Talk has now had over 11 million views. It was in education that mindset theory became particularly popular. As a mini-industry it is now worth millions and millions. Just one research project into the efficacy of one mindset product has received 3.5 million dollars in US federal funding.

But, much like other ideas that have done a roaring trade in popular psychology (Howard Gardner’s ‘multiple intelligences theory, for example) which seem to offer simple solutions to complex problems, there was soon pushback. It wasn’t hard for critics to scoff at motivational ‘yes-you-can’ posters in classrooms or accounts of well-meaning but misguided teacher interventions, like this one reported by Carl Hendrick:

One teacher [took] her children out into the pristine snow covering the school playground, she instructed them to walk around, taking note of their footprints. “Look at these paths you’ve been creating,” the teacher said. “In the same way that you’re creating new pathways in the snow, learning creates new pathways in your brain.”

Carol Dweck was sympathetic to the critics. She has described the early reaction to her book as ‘uncontrollable’. She freely admits that she and her colleagues had underestimated the issues around mindset interventions in the classrooms and that such interventions were ‘not yet evidence-based’. She identified two major areas where mindset interventions have gone awry. The first of these is when a teacher teaches the concept of mindsets to students, but does not change other policies and practices in the classroom. The second is that some teachers have focussed too much on praising their learners’ efforts. Teachers have taken mindset recipes and tips, without due consideration. She says:

Teachers have to ask, what exactly is the evidence suggesting? They have to realise it takes deep thought and deep experimentation on their part in the classroom to see how best the concept can be implemented there. This should be a group enterprise, in which they share what worked, what did not work, for whom and when. People need to recognise we are researchers, we have produced a body of evidence that says under these conditions this is what happened. We have not explored all the conditions that are possible. Teacher feedback on what is working and not working is hugely valuable to us to tell us what we have not done and what we need to do.

Critics like Dylan William, Carl Hendrick and Timothy Bates found that it was impossible to replicate Dweck’s findings, and that there were at best weak correlations between growth mindset and academic achievement, and between mindset interventions and academic gains. They were happy to concede that typical mindset interventions would not do any harm, but asked whether the huge amounts of money being spent on mindset would not be better invested elsewhere.

Carol Dweck seems to like the phrase ‘not yet’. She argues, in her TED Talk, that simply using the words ‘not yet’ can build students’ confidence, and her tip is often repeated by others. She also talks about mindset interventions being ‘not yet evidence-based’, which is a way of declaring her confidence that they soon will be. But, with huge financial backing, Dweck and her colleagues have recently been carrying out a lot of research and the results are now coming in. There are a small number of recent investigations that advocates of mindset interventions like to point to. For reasons of space, I’ll refer to two of them.

The first (Outes-Leon, et al., 2020) of these looked at an intervention with children in the first grades in a few hundred Peruvian secondary schools. The intervention consisted of students individually reading a text designed to introduce them to the concept of growth-mindset. This was followed by a group debate about the text, before students had to write individually a reflective letter to a friend/relative describing what they had learned. In total, this amounted to about 90 minutes of activity. Subsequently, teachers made a subjective assessment of the ‘best’ letters and attached these to the classroom wall, along with a growth mindset poster, for the rest of the school year. Teachers were also asked to take a picture of the students alongside the letters and the poster and to share this picture by email.

Academic progress was measured 2 and 14 months after the intervention and compared to a large control group. The short-term (2 months) impact of the intervention was positive for mathematics, but less so for reading comprehension. (Why?) These gains were only visible in regional schools, not at all in metropolitan schools. Similar results were found when looking at the medium-term (14 month) impact. The reasons for this are unclear. It is hypothesized that the lower-achieving students in regional schools might benefit more from the intervention. Smaller class sizes in regional schools might also be a factor. But, of course, many other explanations are possible.

The paper is entitled The Power of Believing You Can Get Smarter. The authors make it clear that they were looking for positive evidence of the intervention and they were supported by mindset advocates (e.g. David Yeager) from the start. It was funded by the World Bank, which is a long-standing advocate of growth mindset interventions. (Rather jumping the gun, the World Bank’s Mindset Team wrote in 2014 that teaching growth mindset is not just another policy fad. It is backed by a burgeoning body of empirical research.) The paper’s authors conclude that ‘the benefits of the intervention were relevant and long-lasting in the Peruvian context’, and they focus strongly on the low costs of the intervention. They acknowledge that the way the tool is introduced (design of the intervention) and the context in which this occurs (i.e., school and teacher characteristics) both matter to understand potential gains. But without understanding the role of the context, we haven’t really learned anything practical that we can take away from the research. Our understanding of the power of believing you can get smarter has not been meaningfully advanced.

The second of these studies (Yeager et al., 2019) took many thousands of lower-achieving American 9th graders from a representative sample of schools. It is a very well-designed and thoroughly reported piece of research. The intervention consisted of two 25-minute online sessions, 20 days apart, which sought to reduce the negative effort beliefs of students (the belief that having to try hard or ask for help means you lack ability), fixed-trait attributions (the attribution that failure stems from low ability) and performance avoidance goals (the goal of never looking stupid). An analysis of academic achievement at the end of the school year indicated clearly that the intervention led to improved performance. These results lead to very clear grounds for optimism about the potential of growth mindset interventions, but the report is careful to avoid overstatement. We have learnt about one particular demographic with one particular intervention, but it would be wrong to generalise beyond that. The researchers had hoped that the intervention would help to compensate for unsupportive school norms, but found that this was not the case. Instead, they found that it was when the peer norm supported the adoption of intellectual challenges that the intervention promoted sustained benefits. Context, as in the Peruvian study, was crucial. The authors write:

We emphasize that not all forms of growth mindset interventions can be expected to increase grades or advanced course-taking, even in the targeted subgroups. New growth mindset interventions that go beyond the module and population tested here will need to be subjected to rigorous development and validation processes.

I think that a reasonable conclusion from reading this research is that it may well be worth experimenting with growth mindset interventions in English language classes, but without any firm expectation of any positive impact. If nothing else, the interventions might provide useful, meaningful practice of the four skills. First, though, it would make sense to read two other pieces of research (Sisk et al., 2018; Burgoyne et al., 2020). Unlike the projects I have just discussed, these were not carried out by researchers with an a priori enthusiasm for growth-mindset interventions. And the results were rather different.

The first of these (Sisk et al., 2018) was a meta-analysis of the literature. It found that there was only a weak correlation between mindset and academic achievement, and only a weak correlation between mindset interventions and academic gains. It did, however, lend support to one of the conclusions of Yeager et al (2019), that such interventions may benefit students who are academically at risk.

The second (Burgoyne et al., 2020) found that the foundations of mind-set theory are not firm and that bold claims about mind-set appear to be overstated. Other constructs such as self-efficacy and need for achievement, [were] found to correlate much more strongly with presumed associates of mind-set.

So, where does this leave us? We are clearly a long way from ‘facts’; mindset interventions are ‘not yet evidence-based’. Carl Hendrick (2019) provides a useful summary:

The truth is we simply haven’t been able to translate the research on the benefits of a growth mindset into any sort of effective, consistent practice that makes an appreciable difference in student academic attainment. In many cases, growth mindset theory has been misrepresented and miscast as simply a means of motivating the unmotivated through pithy slogans and posters. […] Recent evidence would suggest that growth mindset interventions are not the elixir of student learning that many of its proponents claim it to be. The growth mindset appears to be a viable construct in the lab, which, when administered in the classroom via targeted interventions, doesn’t seem to work at scale. It is hard to dispute that having a self-belief in their own capacity for change is a positive attribute for students. Paradoxically, however, that aspiration is not well served by direct interventions that try to instil it.

References

Bandura, Albert (1982). Self-efficacy mechanism in human agency. American Psychologist, 37 (2): pp. 122–147. doi:10.1037/0003-066X.37.2.122.

Burgoyne, A. P., Hambrick, D. Z., & Macnamara, B. N. (2020). How Firm Are the Foundations of Mind-Set Theory? The Claims Appear Stronger Than the Evidence. Psychological Science, 31(3), 258–267. https://doi.org/10.1177/0956797619897588

Early, P. (Ed.) ELT Documents 1113 – Humanistic Approaches: An Empirical View. London: The British Council

Dweck, C. S. (2006). Mindset: The New Psychology of Success. New York: Ballantine Books

Hendrick, C. (2019). The growth mindset problem. Aeon,11 March 2019.

Maslow, A. (1943). A Theory of Human Motivation. Psychological Review, 50: pp. 370-396.

Outes-Leon, I., Sanchez, A. & Vakis, R. (2020). The Power of Believing You Can Get Smarter : The Impact of a Growth-Mindset Intervention on Academic Achievement in Peru (English). Policy Research working paper, no. WPS 9141 Washington, D.C. : World Bank Group. http://documents.worldbank.org/curated/en/212351580740956027/The-Power-of-Believing-You-Can-Get-Smarter-The-Impact-of-a-Growth-Mindset-Intervention-on-Academic-Achievement-in-Peru

Rogers, C. R. (1969). Freedom to Learn: A View of What Education Might Become. Columbus, Ohio: Charles Merill

Sisk, V. F., Burgoyne, A. P., Sun, J., Butler, J. L., Macnamara, B. N. (2018). To what extent and under which circumstances are growth mind-sets important to academic achievement? Two meta-analyses. Psychological Science, 29, 549–571. doi:10.1177/0956797617739704

Yeager, D.S., Hanselman, P., Walton, G.M. et al. (2019). A national experiment reveals where a growth mindset improves achievement. Nature 573, 364–369. https://doi.org/10.1038/s41586-019-1466-y

subtitlesAs both a language learner and a teacher, I have a number of questions about the value of watching subtitled videos for language learning. My interest is in watching extended videos, rather than short clips for classroom use, so I am concerned with incidental, rather than intentional, learning, mostly of vocabulary. My questions include:

  • Is it better to watch a video that is subtitled or unsubtitled?
  • Is it better to watch a video with L1 or L2 subtitles?
  • If a video is watched more than once, what is the best way to start and proceed? In which order (no subtitles, L1 subtitles and L2 subtitles) is it best to watch?

For help, I turned to three recent books about video and language learning: Ben Goldstein and Paul Driver’s Language Learning with Digital Video (CUP, 2015), Kieran Donaghy’s Film in Action (Delta, 2015) and Jamie Keddie’s Bringing Online Video into the Classroom (OUP, 2014). I was surprised to find no advice, but, as I explored more, I discovered that there may be a good reason for these authors’ silence.

There is now a huge literature out there on subtitles and language learning, and I cannot claim to have read it all. But I think I have read enough to understand that I am not going to find clear-cut answers to my questions.

The learning value of subtitles

It has been known for some time that the use of subtitles during extensive viewing of video in another language can help in the acquisition of that language. The main gains are in vocabulary acquisition and the development of listening skills (Montero Perez et al., 2013). This is true of both L1 subtitles (with an L2 audio track), sometimes called interlingual subtitles, (Incalcaterra McLoughlin et al, 2011) and L2 subtitles (with an L2 audio track), sometimes called intralingual subtitles or captions (Vanderplank, 1988). Somewhat more surprisingly, vocabulary gains may also come from what are called reversed subtitles (L2 subtitles and an L1 audio track) (Burczyńska, 2015). Of course, certain conditions apply for subtitled video to be beneficial, and I’ll come on to these. But there is general research agreement (an exception is Karakaş & Sariçoban, 2012) that more learning is likely to take place from watching a subtitled video in a target language than an unsubtitled one.

Opposition to the use of subtitles as a tool for language learning has mostly come from three angles. The first of these, which concerns L1 subtitles, is an antipathy to any use at all of L1. Although such an attitude remains entrenched in some quarters, there is no evidence to support it (Hall & Cook, 2012; Kerr, 2016). Researchers and, increasingly, teachers have moved on.

The second reservation that is sometimes expressed is that learners may not attend to either the audio track or the subtitles if they do not need to. They may, for example, ignore the subtitles in the case of reversed subtitles or ignore the L2 audio track when there are L1 subtitles. This can, of course, happen, but it seems that, on the whole, this is not the case. In an eye-tracking study by Bisson et al (2012), for example, it was found that most people followed the subtitles, irrespective of what kind they were. Unsurprisingly, they followed the subtitles more closely when the audio track was in a language that was less familiar. When conditions are right (see below), reading subtitles becomes a very efficient and partly automatized cognitive activity, which does not prevent people from processing the audio track at the same time (d’Ydewalle & Pavakanun, 1997).

Related to the second reservation is the concern that the two sources of information (audio and subtitles), combined with other information (images and music or sound effects), may be in competition and lead to cognitive overload, impacting negatively on both comprehension and learning. Recent research suggests that this concern is ungrounded (Kruger et al, 2014). L1 subtitles generate less cognitive load than L2 subtitles, but overload is not normally reached and mental resources are still available for learning (Baranowska, 2020). The absence of subtitles generates more cognitive load.

Conditions for learning

Before looking at the differences between L1 and L2 subtitles, it’s a good idea to look at the conditions under which learning is more likely to take place with subtitles. Some of these are obvious, others less so.

First of all, the video material must be of sufficient intrinsic interest to the learner. Secondly, the subtitles must be of a sufficiently high quality. This is not always the case with automatically generated captions, especially if the speech-to-text software struggles with the audio accent. It is also not always the case with professionally produced L1 subtitles, especially when the ‘translations are non-literal and made at the phrase level, making it hard to find connections between the subtitle text and the words in the video’ (Kovacs, 2013, cited by Zabalbeascoa et al., 2015: 112). As a minimum, standard subtitling guidelines, such as those produced for the British Channel 4, should be followed. These limit, for example, the number of characters per line to about 40 and a maximum of two lines.

For reasons that I’ll come on to, learners should be able to switch easily between L1 and L2 subtitles. They are also likely to benefit if reliably accurate glosses or hyperlinks are ‘embedded in the subtitles, making it possible for a learner to simply click for additional verbal, auditory or even pictorial glosses’ (Danan, 2015: 49).

At least as important as considerations of the materials or tools, is a consideration of what the learner brings to the activity (Frumuselu, 2019: 104). Vanderplank (2015) describes these different kinds of considerations as the ‘effects of’ subtitles on a learner and the ‘effects with’ subtitles on learner behaviour.

In order to learn from subtitles, you need to be able to read fast enough to process them. Anyone with a slow reading speed (e.g. some dyslexics) in their own language is going to struggle. Even with L1 subtitles, Vanderplank (2015: 24) estimates that it is only around the age of 10 that children can do this with confidence. Familarity with both the subject matter and with subtitle use will impact on this ability to read subtitles fast enough.

With L2 subtitles, the language proficiency of the learner related to the level of difficulty (especially lexical difficulty) of the subtitles will clearly be of some significance. It is unlikely that L2 subtitles will be of much benefit to beginners (Taylor, 2005). It also suggests that, at lower levels, materials need to be chosen carefully. On the whole, researchers have found that higher proficiency levels correlate with greater learning gains (Pujadas & Muñoz, 2019; Suárez & Gesa, 2019), but one earlier meta-analysis (Montero Perez et al., 2013) did not find that proficiency levels were significant.

Measures of general language proficiency may be too blunt an instrument to help us all of the time. I can learn more from Portuguese than from Arabic subtitles, even though I am a beginner in both languages. The degree of proximity between two languages, especially the script (Winke et al., 2010), is also likely to be significant.

But a wide range of other individual learner differences will also impact on the learning from subtitles. It is known that learners approach subtitles in varied and idiosyncratic ways (Pujolá, 2002), with some using L2 subtitles only as a ‘back-up’ and others relying on them more. Vanderplank (2019) grouped learners into three broad categories: minimal users who were focused throughout on enjoying films as they would in their L1, evolving users who showed marked changes in their viewing behaviour over time, and maximal users who tended to be experienced at using films to enhance their language learning.

Categories like these are only the tip of the iceberg. Sensory preferences, personality types, types of motivation, the impact of subtitles on anxiety levels and metacognitive strategy awareness are all likely to be important. For the last of these, Danan (2015: 47) asks whether learners should be taught ‘techniques to make better use of subtitles and compensate for weaknesses: techniques such as a quick reading of subtitles before listening, confirmation of word recognition or meaning after listening, as well as focus on form for spelling or grammatical accuracy?’

In short, it is, in practice, virtually impossible to determine optimal conditions for learning from subtitles, because we cannot ‘take into account all the psycho-social, cultural and pedagogic parameters’ (Gambier, 2015). With that said, it’s time to take a closer look at the different potential of L1 and L2 subtitles.

L1 vs L2 subtitles

Since all other things are almost never equal, it is not possible to say that one kind of subtitles offers greater potential for learning than another. As regards gains in vocabulary acquisition and listening comprehension, there is no research consensus (Baranowska, 2020: 107). Research does, however, offer us a number of pointers.

Extensive viewing of subtitled video (both L1 and L2) can offer ‘massive quantities of authentic and comprehensible input’ (Vanderplank, 1988: 273). With lower level learners, the input is likely to be more comprehensible with L1 subtitles, and, therefore, more enjoyable and motivating. This makes them often more suitable for what Caimi (2015: 11) calls ‘leisure viewing’. Vocabulary acquisition may be better served with L2 subtitles, because they can help viewers to recognize the words that are being spoken, increase their interaction with the target language, provide further language context, and increase the redundancy of information, thereby enhancing the possibility of this input being stored in long-term memory (Frumuselu et al., 2015). These effects are much more likely with Vanderplank’s (2019) motivated, ‘maximal’ users than with ‘minimal’ users.

There is one further area where L2 subtitles may have the edge over L1. One of the values of extended listening in a target language is the improvement in phonetic retuning (see, for example, Reinisch & Holt, 2013), the ability to adjust the phonetic boundaries in your own language to the boundaries that exist in the target language. Learning how to interpret unusual speech-sounds, learning how to deal with unusual mappings between sounds and words and learning how to deal with the acoustic variations of different speakers of the target language are all important parts of acquiring another language. Research by Mitterer and McQueen (2009) suggests that L2 subtitles help in this process, but L1 subtitles hinder it.

Classroom implications?

The literature on subtitles and language learning echoes with the refrain of ‘more research needed’, but I’m not sure that further research will lead to less ambiguous, practical conclusions. One of my initial questions concerned the optimal order of use of different kinds of subtitles. In most extensive viewing contexts, learners are unlikely to watch something more than twice. If they do (watching a recorded academic lecture, for example), they are likely to be more motivated by a desire to learn from the content than to learn language from the content. L1 subtitles will probably be preferred, and will have the added bonus of facilitating note-taking in the L1. For learners who are more motivated to learn the target language (Vanderplank’s ‘maximal’ users), a sequence of subtitle use, starting with the least cognitively challenging and moving to greater challenge, probably makes sense. Danan (2015: 46) suggests starting with an L1 soundtrack and reversed (L2) subtitles, then moving on to an L2 soundtrack and L2 subtitles, and ending with an L2 soundtrack and no subtitles. I would replace her first stage with an L2 soundtrack and L1 subtitles, but this is based on hunch rather than research.

This sequencing of subtitle use is common practice in language classrooms, but, here, (1) the video clips are usually short, and (2) the aim is often not incidental learning of vocabulary. Typically, the video clip has been selected as a tool for deliberate teaching of language items, so different conditions apply. At least one study has confirmed the value of the common teaching practice of pre-teaching target vocabulary items before viewing (Pujadas & Muñoz, 2019). The drawback is that, by getting learners to focus on particular items, less incidental learning of other language features is likely to take place. Perhaps this doesn’t matter too much. In a short clip of a few minutes, the opportunities for incidental learning are limited, anyway. With short clips and a deliberate learning aim, it seems reasonable to use L2 subtitles for a first viewing, and no subtitles thereafter.

An alternative frequent use of short video clips in classrooms is to use them as a springboard for speaking. In these cases, Baranowska (2020: 113) suggests that teachers may opt for L1 subtitles first, and follow up with L2 subtitles. Of course, with personal viewing devices or in online classes, teachers may want to exploit the possibilities of differentiating the subtitle condition for different learners.

REFERENCES

Baranowska, K. (2020). Learning most with least effort: subtitles and cognitive load. ELT Journal 74 (2): pp.105 – 115

Bisson, M.-J., Van Heuven, W.J.B., Conklin, K. and Tunney, R.J. (2012). Processing of native and foreign language subtitles in films: An eye tracking study. Applied Psycholingistics, 35 (2): pp. 399 – 418

Burczyńska, P. (2015). Reversed Subtitles as a Powerful Didactic Tool in SLA. In Gambier, Y., Caimi, A. & Mariotti, C. (Eds.), Subtitles and Language Learning. Principles, strategies and practical experiences. Bern: Peter Lang (pp. 221 – 244)

Caimi, A. (2015). Introduction. In Gambier, Y., Caimi, A. & Mariotti, C. (Eds.), Subtitles and Language Learning. Principles, strategies and practical experiences. Bern: Peter Lang (pp. 9 – 18)

Danan, M. (2015). Subtitling as a Language Learning Tool: Past Findings, Current Applications, and Future Paths. In Gambier, Y., Caimi, A. & Mariotti, C. (Eds.), Subtitles and Language Learning. Principles, strategies and practical experiences. Bern: Peter Lang (pp. 41 – 61)

d’Ydewalle, G. & Pavakanun, U. (1997). Could Enjoying a Movie Lead to Language Acquisition?. In: Winterhoff-Spurk, P., van der Voort, T.H.A. (Eds.) New Horizons in Media Psychology. VS Verlag für Sozialwissenschaften, Wiesbaden. https://doi.org/10.1007/978-3-663-10899-3_10

Frumuselu, A.D., de Maeyer, S., Donche, V. & Gutierrez Colon Plana, M. (2015). Television series inside the EFL classroom: bridging the gap between teaching and learning informal language through subtitles. Linguistics and Education, 32: pp. 107 – 17

Frumuselu, A. D. (2019). ‘A Friend in Need is a Film Indeed’: Teaching Colloquial Expressions with Subtitled Television Series. In Herrero, C. & Vanderschelden, I. (Eds.) Using Film and Media in the Language Classroom. Bristol: Multimedia Matters. pp.92 – 107

Gambier, Y. (2015). Subtitles and Language Learning (SLL): Theoretical background. In Gambier, Y., Caimi, A. & Mariotti, C. (Eds.), Subtitles and Language Learning. Principles, strategies and practical experiences. Bern: Peter Lang (pp. 63 – 82)

Hall, G. & Cook, G. (2012). Own-language Use in Language Teaching and Learning. Language Learning, 45 (3): pp. 271 – 308

Incalcaterra McLoughlin, L., Biscio, M. & Ní Mhainnín, M. A. (Eds.) (2011). Audiovisual Translation, Subtitles and Subtitling. Theory and Practice. Bern: Peter Lang

Karakaş, A. & Sariçoban, A. (2012). The impact of watching subtitled animated cartoons on incidental vocabulary learning of ELT students. Teaching English with Technology, 12 (4): pp. 3 – 15

Kerr, P. (2016). Questioning ‘English-only’ Classrooms: Own-language Use in ELT. In Hall, G. (Ed.) The Routledge Handbook of English Language Teaching (pp. 513 – 526)

Kruger, J. L., Hefer, E. & Matthew, G. (2014). Attention distribution and cognitive load in a subtitled academic lecture: L1 vs. L2. Journal of Eye Movement Research, 7: pp. 1 – 15

Mitterer, H. & McQueen, J. M. (2009). Foreign Subtitles Help but Native-Language Subtitles Harm Foreign Speech Perception. PLoS ONE 4 (11): e7785.doi:10.1371/journal.pone.0007785

Montero Perez, M., Van Den Noortgate, W., & Desmet, P. (2013). Captioned video for L2 listening and vocabulary learning: A meta-analysis. System, 41, pp. 720–739 doi:10.1016/j.system.2013.07.013

Pujadas, G. & Muñoz, C. (2019). Extensive viewing of captioned and subtitled TV series: a study of L2 vocabulary learning by adolescents, The Language Learning Journal, 47:4, 479-496, DOI: 10.1080/09571736.2019.1616806

Pujolá, J.- T. (2002). CALLing for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL, 14 (2): pp. 235 – 262

Reinisch, E. & Holt, L. L. (2013). Lexically Guided Phonetic Retuning of Foreign-Accented Speech and Its Generalization. Journal of Experimental Psychology: Human Perception and Performance. Advance online publication. doi: 10.1037/a0034409

Suárez, M. & Gesa, F. (2019) Learning vocabulary with the support of sustained exposure to captioned video: do proficiency and aptitude make a difference? The Language Learning Journal, 47:4, 497-517, DOI: 10.1080/09571736.2019.1617768

Taylor, G. (2005). Perceived processing strategies of students watching captioned video. Foreign Language Annals, 38(3), pp. 422-427

Vanderplank, R. (1988). The value of teletext subtitles in language learning. ELT Journal, 42 (4): pp. 272 – 281

Vanderplank, R. (2015). Thirty Years of Research into Captions / Same Language Subtitles and Second / Foreign Language Learning: Distinguishing between ‘Effects of’ Subtitles and ‘Effects with’ Subtitles for Future Research. In Gambier, Y., Caimi, A. & Mariotti, C. (Eds.), Subtitles and Language Learning. Principles, strategies and practical experiences. Bern: Peter Lang (pp. 19 – 40)

Vanderplank, R. (2019). ‘Gist watching can only take you so far’: attitudes, strategies and changes in behaviour in watching films with captions, The Language Learning Journal, 47:4, 407-423, DOI: 10.1080/09571736.2019.1610033

Winke, P., Gass, S. M., & Sydorenko, T. (2010). The Effects of Captioning Videos Used for Foreign Language Listening Activities. Language Learning & Technology, 1 (1): pp. 66 – 87

Zabalbeascoa, P., González-Casillas, S. & Pascual-Herce, R. (2015). In Gambier, Y., Caimi, A. & Mariotti, C. (Eds.), Subtitles and Language Learning. Principles, strategies and practical experiences Bern: Peter Lang (pp. 105–126)

Precarity

Barely liveable hourly wages, no job security because there is no permanent contract (so employment may be terminated at short or no notice), no social security, paid health care or pension, struggling to meet everyday needs, such as food and accommodation … this is the situation for at least one in five workers in the UK and similar figures exist in many countries (e.g. one in six in New Zealand). As Bourdieu (1998: 81ff.) noted, job insecurity is now everywhere.

Many English language teachers, especially those working for private schools or universities operating like private schools, belong to what has been termed the global educational precariat. In addition to language school and university language teachers, there are hundreds of thousands of teachers, mostly American and British, working in English-medium schools ‘international schools’ around the world (Bunnell, 2016). Besides financial insecurity, many of these teachers also suffer from a lack of agency and a marginalisation of their professional identities (Poole, 2019). There’s a very useful article on ‘precarity’ in ELT Journal (Walsh, 2019) that I’d recommend.

Even teachers with reasonable pay and job security are facing attacks on their pay and working conditions. A few weeks ago in Jordan, security forces shut down the teachers’ union and arrested leading members. Teachers union leaders have also been imprisoned recently in Iran and Cambodia. The pages of the website of Education International , a global federation of teachers’ trade unions, catalogue the crises in education and the lives of teachers around the world.

Teacher bashing, in particular attacks on teacher unions, has been relentless. Four years ago, it was reported that teacher bashing had ‘reached unprecedented levels’ in the US (Saltzman, 2017: 39), where there has been a concerted attempt, over many years, to blame teachers for shortcomings in the educational system (see, for example, Kumashiro, 2012). Although it may have been the US that led the way, closely followed by Australia and the UK, attacks on teachers have become a global phenomenon. Mary Compton and Lois Weiner’s book, ‘The Global Assault on Teaching, Teachers and their Unions’ (Compton & Weiner 2008), gives examples from China to South Africa, from Denmark to Mexico, of how teachers’ pay and conditions have been eroded. The reason? Quite simply, it is because teachers have stood in the way of so-called ‘reforms’ (e.g. pay cuts). It is because they have, as they are doing now in times of COVID-19, stood in the way of what governments have wanted to do. In an earlier post, I wrote in more detail about the ways in which the World Bank has spearheaded the drive towards privatized, lower cost education around the world.

COVID-19 has, of course, made matters worse, much worse. As often as not, the pandemic has been used as an excuse to accelerate attacks on teachers that were well under way long before.

Wellbeing

In the circumstances, it is not surprising that teacher wellbeing has recently become a more talked-about topic. Precisely because there is so little of it about.

The publication earlier this year of a book about teacher wellbeing (Mercer & Gregersen, 2020) for language teachers is very timely. The authors acknowledge that real change for wellbeing [must] addresses structural and systemic levels of change and is not just a matter for individual teachers to cope with alone. They acknowledge that teachers should not have to compensate for fundamental flaws in the system as a whole that undermine their wellbeing, and they express concern about the risks associated with discussing teacher wellbeing at the individual level and not acknowledging that the systems in which teachers work may be at fault (Mercer & Gregersen, 2020: 9). But, with these caveats out of the way, the matter is closed, and the whole book is about how individuals can improve their wellbeing. Indeed, the book begins: As you read the title of this chapter, you might have thought how self-seeking or egocentric it sounds: It’s all about me? Our response is, ‘Yes, you!’ Throughout this book, we want you to focus your attention on yourself for a change, without any guilty feelings (Mercer & Gregersen, 2020: 1). Mindfulness techniques, tips for time management, ways of thinking positively and so on – it’s a compendium of self-help advice that may be helpful for language teachers. The real ravages of precarity, the real causes of so much lack of wellbeing, these do not get a mention.

Banksy_-_Grin_Reaper_With_TagPositive psychology

Mercer and Gregersen’s approach is directly inspired by the work of Martin Seligman, often referred to as the founder of ‘positive psychology’ (see, for example, Seligman, 2011; 2018). Positive psychology and Seligman’s ideas about wellbeing are not uncontested (see, for example, Bache & Reardon, 2016; Bache & Scott, 2018). The nub of the critiques is that positive psychology chooses to focus on happiness or wellbeing, rather than, say, justice, solidarity or loyalty. It articulates an underlying individualism and narrow sense of the social (Cabanas & Illouz, 2019: 68) and it is, therefore, not entirely surprising that much of the funding that made the rapid growth of positive psychology possible came from the ultra-conservative and religious institution, the John Templeton Foundation (Cabanas & Illouz, 2019: 20).

Mercer and Gregersen are not unaware of such critiques (see, for example, MacIntyre et al., 2016: 375). They mention the critiques of Barbara Ehrenreich (Ehrenreich, 2009), but, to the best of my knowledge, they have never troubled to respond to them. They have a very clear agenda – the promotion of positive psychology ideas in language teaching / learning contexts – which is made explicit in MacIntyre and Mercer (2014). A slew of articles, books and conference presentations have followed since then, and ‘Teacher Wellbeing’ is one of them. Mission seems to have been achieved.

Positive psychology has not only been criticised for its focus on the individual. Others have focused on its foundational assumptions, including decontextualized and ethnocentric claims; theoretical oversimplifications, tautologies and contradictions; methodological shortcomings; severe replicability problems; exaggerated generalizations; and even its therapeutic efficacy and scientific status (Cabanas & Illous, 2019: 29). Probably the most important of these critics was Richard Lazarus, whose work is certainly familiar to Mercer, Gregersen and their collaborators, since Lazarus’s criticisms are listed in MacIntyre and Mercer (2014) and elsewhere. These include:

  • the over-use of crosssectional research designs
  • a tendency to treat emotion too simplistically as either positive or negative
  • inadequate attention to both differences among individuals within a group as well as the overlap between groups when discussing statistically significant group differences
  • poor quality measurement of emotions.

However, as with the critiques of Ehrenreich, I have yet to find any examples of these authors actually addressing the criticisms. Instead, they prefer to talk about how problems such as those listed above need to be avoided in the future. For example, there is no doubt that the future development of the [positive psychology] approach within SLA can learn from these and other criticisms, write MacIntyre and Mercer (2014:161), and they see the future of positive psychology in language learning / teaching as being fundamentally grounded in science.

Empirical science

Acknowledging, but without actually addressing, past criticisms of the scientific shortcomings of positive psychology, MacIntyre and Mercer (2014: 15) insist that positive psychology is the empirical study of how people thrive and flourish […] it represents a form of “rebirth” for humanistic psychology, but with a stronger emphasis on empirical research. The word ‘empirical’ appears 4 times on this page and another 5 times in the article. In their follow-up book, ‘Positive Psychology in SLA’ (Macintyre et al., 2016), there is a whole section (over a third of the book) entitled ‘Empirical’. In a historical survey of positive psychology in foreign language teaching, written by close collaborators of Mercer, Gregersen and MacIntyre (Dewaele et al.,2019), the same focus on empirical science is chosen, with a description of positive psychology as being underpinned by solid empirical research. The frequency of this word choice is enough to set alarm bells ringing.

A year before the MacIntyre and Mercer article (2014), an article by Brown et al (2013) questioned one of the key empirical foundations of positive psychology, the so-called ‘critical positivity ratio’ (Fredrickson & Losada, 2005). Wikipedia explains this as the ratio of positive to negative emotions which distinguishes “flourishing” people from “languishing” people, and the ratio was 2.9013. A slightly later article (Brown et al, 2014) further debunked the work of Fredrickson, arguing that her work was full of conceptual difficulties and statistical flaws. Wikipedia now describes the ‘critical positivity ratio’ as ‘a largely discredited concept’. In contrast, Mercer and Gregersen (2020: 14) acknowledge that although the exact ratio (3:1) of positivity has been called into question by some, they reassert the value of Fredrickson’s work. They neither cite the criticisms, nor rebut them. In this, they are following a well-established tradition of positive psychology (Rhyff, 2003).

Given growing scepticism about the claims of positive psychology, MacIntyre et al (2016) elected to double-down. Even if empirical evidence for positive psychology was in short supply, it was incumbent on them to provide it. Hence, the section in their book entitled ‘Empirical’. Personally, I would have advised against it. The whole point of positive psychology, as outlined by Seligman, is to promote ‘wellbeing’. But what, exactly, is this? For some, like Mercer and Gregersen (2020: 3), it’s about finding meaning and connection in the world. For others, it’s not a ‘thing’ that needs research to uncover its essential nature, but as a social and cultural construction which is interesting as such, not least for what it can tell us about other social and cultural phenomena (Ereaut & Whiting, 2008). We may agree that it’s ‘a good thing’, but it lacks solidity as a construct. Even Seligman (2011: 15) comes to the conclusion that ‘wellbeing’ is not ‘a real thing. Rather, he says, it is a construct which has several measurable elements, each a real thing, each contributing to well-being, but none defining well-being. This, however, simply raises the question of how much of a ‘thing’ each of these elements are (Dodge et al., 2012). Seligman’s elements (Positive Emotion, Engagement, Relationships, Meaning, and Accomplishment (PERMA)) form the basis of Mercer and Gregersen’s book, but none lend themselves to clear, workable definitions. In the absence of construct validity, empirical research evidence will prove hard to find.

How well does the ‘Empirical’ section of Positive Psychology in SLA (MacIntyre et al., 2016) stand up? I don’t have space here to discuss all 7 chapters. However, I’ve selected the first of these, ‘Positive Psychology Exercises Build Social Capital for Language Learners: Preliminary Evidence’ (Gregersen et al, 2016) because it includes ‘evidence’ in the title and because it was written by two of the book’s editors. The research reported in this chapter involved five volunteer women, aged 20 -23, in an English program at an American university, who took part in a number of positive psychology exercises (PPEs) which entailed laughter, exercise, interaction with animals, listening to music, expressing gratitude and engaging in altruism. The data collected was self-rating questionnaires and some self-reflection discussion. The results indicated that the PPEs led to more positive emotions, with exercise and laughter leading to the greatest gains (but since the order of the PPEs was not randomized, and since the sample size was so small, this doesn’t really tell us anything). Some of the participants doubted the value of some of the PPEs. However, the participants developed better relationships with their partners and this may have led to gains in confidence. The authors conclude that although the present data-set is small, we see preliminary evidence of all three pillars of positive psychology supporting positive outcomes (p.164).

My own view is that this is wishful thinking. The only thing that this study does is to indicate that in this particular context with these particular learners, feeling good about what you are doing may help things along a bit. In addition, this has absolutely nothing to do with ‘social capital’, which the authors seem to have misunderstood. Citing an article by Nawyn et al (2012), they describe ‘social capital’ as emerging friendships that provide learners with positive emotional experiences and intangible resources for language acquisition (Gregersen et al, 2016: 147). But this is a misreading of the Nawyn et al article, which adheres fairly closely to Bourdieu’s notion of social capital as fundamentally about power relations, but extends it beyond purely economic power relations. Given the connections between the lack of teacher wellbeing and precarity, and given Bourdieu’s writings about precarity, the authors’ attempt to bring Bourdieu into their justification of positive psychological experiences, best undertaken at the individual level (Gregersen et al., 2016: 149), is really quite extraordinary. And if this is empirical evidence for anything, I’m a positive psychologist!

Cui bono?

It may be that some of the exercises suggested in Teacher Wellbeing will be of benefit to some, even many, teachers. Maybe. But the claims of empirical science behind this book are questionable, to say the least. More beneficial to teacher wellbeing would almost certainly be strong teacher unions, but these are only mentioned in passing. There is, incidentally, some recent evidence from the U.S. (Han, 2020), that highly unionized districts have higher average teacher quality and improved educational outcomes. But positive psychologists seem unwilling to explore the role that unions might play in teacher wellbeing. It is not, perhaps, coincidental that the chapter in Teacher Wellbeing that deals with teachers in their workplaces contains three recommendations for further reading, and all three are written for managers. The first on the list is called Build It: The Rebel Playbook for World-class Employee Engagement (Elliott & Corey, 2018).

The problems that teachers are facing, exacerbated by COVID-19, are fundamentally systemic and political. Mercer and Gregersen may be aware that there is a risk associated with discussing teacher wellbeing at the individual level and not acknowledging that the systems in which teachers work may be at fault, but it’s a risk they have chosen to take, believing that their self-help ideas are sufficiently valuable to make the risk worthwhile. I agree with a writer on the National Education Association blog, who thinks that self-care is important but argues that it is an insufficient and entirely too passive way to address the problems teachers are encountering today.

There are other ways of conceptualising teacher wellbeing (see, for example, the entries on the Education International website with this tag) and the Mercer / Gregersen book may be viewed as an attempt to ‘claim the field’. To return to Paul Walsh, whose article about precarity I recommended earlier, it is useful to see the current interest in teacher wellbeing in context. He writes: Well-being has entered ELT at a time when teachers have been demanding greater visibility and acceptance of issues such as mental health, poor working conditions, non-native speaker and gender equality. Yet to subsume these issues under a catch-all category does them a disservice. Because as soon as we put these issues under the well-being umbrella, they effectively vanish in a cloud of conceptual mist—and lose their sharp edges.

In this sense, a book like Teacher Wellbeing, although well-meaning, may well contribute to the undermining of the very thing it seeks to promote.

References

Bache, I. & Reardon, L. (2016) The Politics and Policy of Wellbeing: Understanding the Rise and Significance of a New Agenda. Cheltenham: Edward Elgar

Bache, I. and Scott, K. (eds.) (2018). The Politics of Wellbeing: Theory, Policy and Practice. Palgrave Macmillan

Bourdieu, P. (1998). Acts of Resistance: against the new myths of our time. Cambridge: Polity Press

Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2013). The complex dynamics of wishful thinking: The critical positivity ratio. American Psychologist, pp. 68, 801–813. http://dx.doi.org/10.1037/a0032850

Brown, N. J. L., MacDonald, D. A., Samanta, M. P., Friedman, H. L. & Coyne, J. C. (2014). A critical reanalysis of the relationship between genomics and well-being. Proceedings of the National Academy of Sciences of the United States of America, 111, 12705–12709. http://dx.doi.org/10.1073/pnas.1407057111

Bunnell, T. (2016). Teachers in International schools: a global educational ‘precariat’? Globalisation, Societies and Education, 14(4), pp. 543-559

Cabanas, E. & Illouz, E. (2019). Manufacturing Happy Citizens. Cambridge: Polity Press

Compton, M. & Weiner, L. (Eds.) (2008). The Global Assault on Teaching, Teachers and their Unions. Palgrave Macmillan

Dewaele, J. M., Chen, X., Padilla, A. M. & Lake, J. (2019). The Flowering of Positive Psychology in Foreign Language Teaching and Acquisition Research. Frontiers in psychology, 10, 2128. https://doi.org/10.3389/fpsyg.2019.02128

Dodge, R., Daly, A., Huyton, J. & Sanders, L. (2012). The challenge of defining wellbeing. International Journal of Wellbeing, 2(3), pp. 222-235. doi:10.5502/ijw.v2i3.4

Ehrenreich, B. (2009). Bright-Sided: How the relentless promotion of positive thinking has undermined America. New York: Metropolitan Books

Elliott, G. & Corey, D. (2018). Build It: The Rebel Playbook for World-class Employee Engagement. Chichester: Wiley

Ereaut, G. & Whiting, R. (2008). What do we mean by ‘wellbeing’? And why might it matter? Research Report No DCSF-RW073 Department for Children, Schools and Families https://dera.ioe.ac.uk/8572/1/dcsf-rw073%20v2.pdf

Fredrickson, B. L. & Losada, M.F. (2005). Positive affect and the complex dynamics of human flourishing. American Psychology, 60 (7): pp. 678–86. doi:10.1037/0003-066X.60.7.678

Gregersen, T., MacIntyre, P.D. & Meza, M. (2016). Positive Psychology Exercises Build Social Capital for Language Learners: Preliminary Evidence. In MacIntyre, P.D., Gregersen, T. & Mercer, S. (Eds.) Positive Psychology in SLA. Bristol: Multilingual Matters. pp.147 – 167

Han, E. S. (2020). The Myth of Unions’ Overprotection of Bad Teachers: Evidence from the District–Teacher Matched Data on Teacher Turnover. Industrial Relations, 59 (2): pp. 316 – 352

Kumashiro, K. K. (2012). Bad Teacher! How Blaming Teachers Distorts the Bigger Picture. Teachers College Press

Lazarus, R. S. (2003). Target article: Does the positive psychology movement have legs? Psychological Inquiry, 14 (2): pp. 93 – 109

MacIntyre, P.D. & Mercer, S. (2014). Introducing positive psychology to SLA. Studies in Second Language Learning and Teaching, 4 (2): pp. 153 -172

MacIntyre, P.D., Gregersen, T. & Mercer, S. (2016). Conclusion. In MacIntyre, P.D., Gregersen, T. & Mercer, S. (Eds.) Positive Psychology in SLA. Bristol: Multilingual Matters. pp.374 – 379

MacIntyre, P.D., Gregersen, T. & Mercer, S. (Eds.) (2016). Positive Psychology in SLA. Bristol: Multilingual Matters.

Mercer, S. & Gregersen, T. (2020). Teacher Wellbeing. Oxford: OUP

Nawyn, S.J., Gjokai, L., Agbenyiga, D.L. & Grace, B. (2012). Linguistic isolation, social capital, and immigrant belonging. Journal of Contemporary Ethnography 41 (3), pp.255 -282

Poole, A. (2019). International Education Teachers’ Experiences as an Educational Precariat in China. Journal of Research in International Education, 18 (1): pp. 60-76

Ryff, C. D. (2003). Corners of myopia in the positive psychology parade. Psychological Inquiry, 14: pp. 153–159

Saltzman, K. J. 2017. Scripted Bodies. Routledge

Seligman, M. (2011). Flourish – A new understanding of happiness and well-being – and how to achieve them. London: Nicholas Brealey Publishing.

Seligman, M. (2018). PERMA and the building blocks of well-being. The Journal of Positive Psychology, DOI: 10.1080/17439760.2018.1437466

Walsh, P. (2019). Precarity. ELT Journal, 73 (4), pp.459 – 462

Wong, P. T. P., & Roy, S. (2017). Critique of positive psychology and positive interventions. In N. J. L. Brown, T. Lomas, & F. J. Eiroa-Orosa (Eds.) The Routledge International Handbook of Critical Positive Psychology. London: Routledge.

I noted in a recent post about current trends in ELT that mindfulness has been getting a fair amount of attention recently. Here are three recent examples:

  • Pearson recently produced the Pearson Experiences: A Pocket Guide to Mindfulness, written by Amy Malloy. Amy has written also written a series of blog posts for Pearson on the topic and she is a Pearson-sponsored speaker (Why use mindfulness in the classroom?) at the English Australia Ed-Tech SIG Online Symposium this week.
  • Russell Stannard has written two posts for Express Publishing (here and here)
  • Sarah Mercer and Tammy Gregersen’s new book, ‘Teacher Wellbeing’ (OUP, 2020) includes a section in which they recommend mindfulness practices to teachers as a buffer against stress and as a way to promote wellbeing.

The claims

Definitions of mindfulness often vary slightly, but the following from Amy Malloy is typical: ‘mindfulness is about the awareness that comes from consciously focussing on the present moment’. Claims for the value of mindfulness practices also vary slightly. Besides the general improvements to wellbeing suggested by Sarah and Tammy, attention, concentration and resilience are also commonly mentioned.

Amy: [Mindfulness] develops [children’s] brains, which in turn helps them find it easier to calm down and stay calm. … It changes our brains for the better. …. [Mindfulness] helps children concentrate more easily on classroom activities.

Russell: Students going through mindfulness training have increased levels of determination and willpower, they are less likely to give up if they find something difficult … Mindfulness has been shown to improve concentration. Students are able to study for longer periods of time and are more focused … Studies have shown that practicing mindfulness can lead to reduced levels of anxiety and stress.

In addition to the behavioural changes that mindfulness can supposedly bring about, both Amy and Russell refer to neurological changes:

Amy: Studies have shown that the people who regularly practise mindfulness develop the areas of the brain associated with patience, compassion, focus, concentration and emotional regulation.

Russell: At the route of our current understanding of mindfulness is brain plasticity. … in probably the most famous neuroimaging research project, scientists took a group of people and found that by doing a programme of 8 weeks of mindfulness training based around gratitude, they could actually increase the size of the areas of the brain generally associated with happiness.

Supporting evidence

In her pocket guide for Pearson, Amy provides no references to support her claims.

In Russell’s first post, he links to a piece of research which looked at the self-reported psychological impact of a happiness training programme developed by a German cabaret artist and talk show host. The programme wasn’t specifically mindfulness-oriented, so tells us nothing about mindfulness, but it is also highly suspect as a piece of research, not least because one of the co-authors is the cabaret artist himself. His second link is to an article about human attention, a long-studied area of psychology, but this has nothing to do with mindfulness, although Russell implies that there is a connection. His third link is to a very selective review of research into mindfulness, written by two mindfulness enthusiasts. It’s not so much a review of research as a selection of articles which support mindfulness advocacy.

In his second post, Russell links to a review of mindfulness-based interventions (MBIs) in education. Appearing in the ‘Mindfulness’ journal, it is obviously in broad support of MBIs, but its conclusions are hedged: ‘Research on the neurobiology of mindfulness in adults suggests that sustained mindfulness practice can ….’ ‘mindfulness training holds promise for being one such intervention for teachers.’ His second link is to a masterpiece of pseudo-science delivered by Joe Dispenza, author of many titles including ‘Becoming Supernatural: How Common People are Doing the Uncommon’ and ‘Breaking the Habit of Being Yourself’. Russell’s 3rd link is to an interview with Matthieu Ricard, one of the Dalai Lama’s translators. Interestingly, but not in this interview, Ricard is very dismissive of secular mindfulness (‘Buddhist meditation without the Buddhism’). His fourth link is to a video presentation about mindfulness from Diana Winston of UCLA. The presenter doesn’t give citations for the research she mentions (so I can’t follow them up): instead, she plugs her own book.

Sarah and Tammy’s three references are not much better. The first is to a self-help book, called ‘Every Teacher Matters: Inspiring Well-Being through Mindfulness’ by K. Lovewell (2012), whose other work includes ‘The Little Book of Self-Compassion: Learn to be your own Best Friend’. The second (Cresswell, J. D. & Lindsay, E.K. (2014). How does mindfulness training affect health? A mindfulness stress buffering account. Current Directions in Psychological Science 23 (6): pp. 401-407) is more solid, but a little dated now. The third (Garland, E., Gaylord, S.A. & Fredrickson, B. L. (2011). Positive Reappraisal Mediates the Stress-Reductive Effects of Mindfulness: An Upward Spiral Process. Mindfulness 2 (1): pp. 59 – 67) is an interesting piece, but of limited value since there was no control group in the research and it tells us nothing about MBIs per se.

The supporting evidence provided by these writers for the claims they make is thin, to say the least. It is almost as if the truth of the claims is self-evident, and for these writers (all of whom use mindfulness practices themselves) there is clearly a personal authentication. But, not having had an epiphany myself and being somewhat reluctant to roll a raisin around my mouth, concentrating on its texture and flavours, fully focussing on the process of eating it (as recommended by Sarah and Tammy), I will, instead, consciously focus on the present moment of research.

Mindfulness and research

The first thing to know is that there has been a lot of research into mindfulness in recent years. The second thing to know is that much of it is poor quality. Here’s why:

  • There is no universally accepted technical definition of ‘mindfulness’ nor any broad agreement about detailed aspects of the underlying concept to which it refers (Van Dam, N. T. et al. (2018). Mind the Hype: A Critical Evaluation and Prescriptive Agenda for Research on Mindfulness and Meditation. Perspectives on Psychological Science 13: pp. 36 – 61)
  • To date, there are at least nine different psychometric questionnaires, all of which define and measure mindfulness differently (Purser, R.E. (2019). McMindfulness. Repeater Books. p.128)
  • Mindfulness research tends to rely on self-reporting, which is notoriously unreliable.
  • The majority of studies did not utilize randomized control groups (Goyal, M., Singh, S., Sibinga, E.S., et al. (201). Meditation Programs for Psychological Stress and Well-being: A Systematic Review and Meta-analysis. JAMA Intern Med. 2014. doi:10.1001/ jamainternmed.2013.13018).
  • Early meditation studies were mostly cross-sectional studies: that is, they compared data from a group of meditators with data from a control group at one point in time. A cross-sectional study design precludes causal attribution. (Tang, Y., Hölzel, B. & Posner, M. (2015). The neuroscience of mindfulness meditation. Nature Reviews Neuroscience 16, 213–225)
  • Sample sizes tend to be small and there is often no active control group. There are few randomized controlled trials (Dunning, D.L., Griffiths, K., Kuyken, W., Crane, C., Foulkes, L., Parker, J. and Dalgleish, T. (2019), Research Review: The effects of mindfulness‐based interventions on cognition and mental health in children and adolescents – a meta‐analysis of randomized controlled trials. (Journal of Child Psychology and Psychiatry, 60: 244-258. doi:10.1111/jcpp.12980)
  • There is a relatively strong bias towards the publication of positive or significant results (Coronado-Montoya, S., Levis, A.W., Kwakkenbos, L., Steele, R.J., Turner, E.H. & Thombs, B.D. (2016). Reporting of Positive Results in Randomized Controlled Trials of Mindfulness-Based Mental Health Interventions. PLoS ONE 11(4): e0153220. https://doi.org/10.1371/journal.pone.0153220)
  • More recent years have not seen significant improvements in the rigorousness of research (Goldberg SB, Tucker RP, Greene PA, Simpson TL, Kearney DJ, Davidson RJ (2017). Is mindfulness research methodology improving over time? A systematic review. PLoS ONE 12(10): e0187298).

 

The overall quality of the research into mindfulness is so poor that a group of fifteen researchers came together to write a paper entitled ‘Mind the Hype: A Critical Evaluation and Prescriptive Agenda for Research on Mindfulness and Meditation’ (Van Dam, N. T. et al. (2018). Mind the Hype: A Critical Evaluation and Prescriptive Agenda for Research on Mindfulness and Meditation. Perspectives on Psychological Science 13: pp. 36 – 61).

So, the research is problematic and replication is needed, but it does broadly support the claim that mindfulness meditation exerts beneficial effects on physical and mental health, and cognitive performance (Tang, Y., Hölzel, B. & Posner, M. (2015). The neuroscience of mindfulness meditation. Nature Reviews Neuroscience 16, 213–225). The italicized broadly is important here. As one of the leaders of the British Mindfulness in Schools Project (which has trained thousands of teachers in the UK) puts it, ‘research on mindfulness in schools is still in its infancy, particularly in relation to impacts on behaviour, academic performance and physical health. It can best be described as ‘promising’ and ‘worth trying’ (Weare, K. (2018). Evidence for the Impact of Mindfulness on Children and Young People. The Mindfulness in Schools Project). We don’t know what kind of MBIs are most effective, what kind of ‘dosage’ should be administered, what kinds of students it is (and is not) appropriate for, whether instructor training is significant or what cost-benefits it might bring. In short, there is more that we do not know than we know.

One systematic review, for example, found that MBIs had ‘small, positive effects on cognitive and socioemotional processes but these effects were not seen for behavioral or academic outcomes’. What happened to the promises of improved concentration, calmer behaviour and willpower? The review concludes that ‘the evidence from this review urges caution in the widespread adoption of MBIs and encourages rigorous evaluation of the practice should schools choose to implement it’ (Maynard, B. R., Solis, M., Miller, V. & Brendel, K. E. (2017). Mindfulness-based interventions for improving cognition, academic achievement, behavior and socio-emotional functioning of primary and secondary students. A Campbell Systematic Review 2017:5).

What about the claims for neurological change? As a general rule, references to neuroscience by educators should be taken with skepticism. Whilst it appears that ‘mindfulness meditation might cause neuroplastic changes in the structure and function of brain regions involved in regulation of attention, emotion and self-awareness’ (Tang, Y., Hölzel, B. & Posner, M. (2015). The neuroscience of mindfulness meditation. Nature Reviews Neuroscience 16, 213–225), this doesn’t really tell us very much. A complex mental state like mindfulness ‘is likely to be supported by the large-scale brain networks’ (ibid) and insights derived from fMRI scans of particular parts of the brain provide us with, at best, only a trivial understanding of what is going on. Without a clear definition of what mindfulness actually is, it is going to be some time before we unravel the neural mechanisms underpinning it. If, in fact, we ever do. By way of comparison, you might be interested in reading about neuroscientific studies into prayer , which also appears to correlate with enhanced wellbeing.

Rather than leaving things with the research, I’d like to leave you with a few more short mindfulness raisins to chew on.

Mindfulness and money

As Russell says in his blog post, ‘research in science doesn’t come out of a vacuum’. Indeed, it tends to follow the money. It is estimated that mindfulness is now ‘a $4 billion industry’ (Purser, R.E. (2019). McMindfulness. Repeater Books. p.13): ‘More than 100,000 books for sale on Amazon have a variant of ‘mindfulness’ in their title, touting the benefits of Mindful Parenting, Mindful Eating, Mindful Teaching, Mindful Therapy, Mindful Leadership, Mindful Finance, a Mindful Nation, and Mindful Dog Owners, to name just a few. There is also The Mindfulness Coloring Book, a bestselling genre in itself. Besides books, there are workshops, online courses, glossy magazines, documentary films, smartphone apps, bells, cushions, bracelets, beauty products and other paraphernalia, as well as a lucrative and burgeoning conference circuit’.

It is precisely because so much money is at stake that so much research has been funded. More proof is desperately needed, and it is sadly unforthcoming. Meanwhile, in the immortal words of Kayleigh McEnany, ‘science should not stand in the way.’

Minefulness and the individual

Mindfulness may be aptly described as a ‘technology of the self’. Ronald Purser, the author of ‘McMindfulness’, puts it like this: ‘Rather than discussing how attention is monetized and manipulated by corporations such as Google, Facebook, Twitter and Apple, [mindfulness advocates] locate crisis in our minds. It is not the nature of the capitalist system that is inherently problematic; rather, it is the failure of individuals to be mindful and resilient in a precarious and uncertain economy. Then they sell us solutions that make us contented mindful capitalists’.

It is this focus on the individual that makes it so appealing to right-wing foundations (e.g. the Templeton Foundation) that fund the research into mindfulness. For more on this topic, see my post about grit .

Mindfulness and religion

It is striking how often mindfulness advocates, like Amy, feel the need to insist that mindfulness is not a religious practice. Historically, of course, mindfulness comes direct from a Buddhist tradition, but in its present Western incarnation, it is a curious hybrid. Jon Kabat-Zinn  who, more than anyone else, has transformed mindfulness into a marketable commodity, is profoundly ambiguous on the topic. Buddhists, like Matthieu Ricard or David Forbes (author of ‘Mindfulness and its Discontents’, Fernwood Publishing, 2019), have little time for the cultural appropriation of the Pali term ‘Sati’, especially when mindfulness is employed by the American military for training for snipers. Others, like Goldie Hawn, whose MindUP programme sells well in the US, are quite clear about their religious affiliation and their desire to bring Buddhism into schools through the back door.

I personally find it hard to see the banging of Tibetan bowls as anything other than a religious act, but I am less bothered by this than those American school districts who saw MBIs as ‘covert religious indoctrination’ and banned them. Having said that, why not promote more prayer in schools if the ‘neuroscience’ supports it?

Clare is a busy teacher

Image from Andrew Percival , inspired by The Ladybird Book of Mindfulness and similar titles.

The idea of ‘digital natives’ emerged at the turn of the century, was popularized by Marc Prensky (2001), and rapidly caught the public imagination, especially the imagination of technology marketers. Its popularity has dwindled a little since then, but is still widely used. Alternative terms include ‘Generation Y’, ‘Millennials’ and the ‘Net Generation’, definitions of which vary slightly from writer to writer. Two examples of the continued currency of the term ‘digital native’ are a 2019 article on the Pearson Global Scale of English website entitled ‘Teaching digital natives to become more human’ and an article in The Pie News (a trade magazine for ‘professionals in international education’), extolling the virtues of online learning for digital natives in times of Covid-19.

Key to understanding ‘digital natives’, according to users of the term, is their fundamental differences from previous generations. They have grown up immersed in technology, have shorter attention spans, and are adept at multitasking. They ‘are no longer the people our educational system was designed to teach’ (Prensky, 2001), so educational systems must change in order to accommodate their needs.

The problem is that ‘digital natives’ are a myth. Prensky’s ideas were not based on any meaningful research: his observations and conclusions, seductive though they might be, were no more than opinions. Kirschner and De Bruyckere (2017) state the research consensus:

There is no such thing as a digital native who is information-skilled simply because (s)he has never known a world that was not digital. […] One of the alleged abilities of students in this generation, the ability to multitask, does not exist and that designing education that assumes the presence of this ability hinders rather than helps learning.

This is neither new (see Bennett et al., 2008) nor contentious. Almost ten years ago, Thomas (2011:3) reported that ‘some researchers have been asked to remove all trace of the term from academic papers submitted to conferences in order to be seriously considered for inclusion’. There are reasons, he added, to consider some uses of the term nothing more than technoevangelism (Thomas, 2011:4). Perhaps someone should tell Pearson and the Pie News? Then, again, perhaps, they wouldn’t care.

The attribution of particular characteristics to ‘digital natives’ / ‘Generation Y’ / ‘Millennials’ is an application of Generation Theory. This can be traced back to a 1928 paper by Karl Mannheim, called ‘Das Problem der Generationen’ which grew in popularity after being translated into English in the 1950s. According to Jauregui et al (2019), the theory was extensively debated in the 1960s and 1970s, but then disappeared from academic study. The theory was not supported by empirical research, was considered to be overly schematised and too culturally-bound, and led inexorably to essentialised and reductive stereotypes.

But Generation Theory gained a new lease of life in the 1990s, following the publication of ‘Generations’ by William Strauss and Neil Howe. The book was so successful that it spawned a slew of other titles leading up to ‘Millennials Rising’ (Howe & Strauss, 2000). This popularity has continued to the present, with fans including Steve Bannon (Kaiser, 2016) who made an ‘apocalyptical and polemical’ documentary film about the 2007 – 2008 financial crisis entitled ‘Generation Zero’. The work of Strauss and Howe has been dismissed as ‘more popular culture than social science’ (Jauregui et al., 2019: 63) and in much harsher terms in two fascinating articles in Jacobin (Hart, 2018) and Aeon (Onion, 2015). The sub-heading of the latter is ‘generational labels are lazy, useless and just plain wrong’. Although dismissed by scholars as pseudo-science, the popularity of such Generation Theory helps explain why Prensky’s paper about ‘digital natives’ fell on such fertile ground. The saying, often falsely attributed to Mark Twain, that we should ‘never let the truth get in the way of a good story’ comes to mind.

But by the end of the first decade of this century, ‘digital natives’ had become problematic in two ways: not only did the term not stand up to close analysis, but it also no longer referred to the generational cohort that pundits and marketers wanted to talk about.

Around January 2018, use of the term ‘Generation Z’ began to soar, and is currently at its highest point ever in the Google Trends graph. As with ‘digital natives’, the precise birth dates of Generation Z vary from writer to writer. After 2001, according to the Cambridge dictionary; slightly earlier according to other sources. The cut-off point is somewhere between the mid and late 2010s. Other terms for this cohort have been proposed, but ‘Generation Z’ is the most popular.

William Strauss died in 2007 and Neil Howe was in his late 60s when ‘Generation Z’ became a thing, so there was space for others to take up the baton. The most successful have probably been Corey Seemiller and Meghan Grace, who, since 2016, have been churning out a book a year devoted to ‘Generation Z’. In the first of these (Seemiller & Grace, 2016), they were clearly keen to avoid some of the criticisms that had been levelled at Strauss and Howe, and they carried out research. This consisted of 1143 responses to a self-reporting questionnaire by students at US institutions of higher education. The survey also collected information about Kolb’s learning styles and multiple intelligences. With refreshing candour, they admit that the sample is not entirely representative of higher education in the US. And, since it only looked at students in higher education, it told us nothing at all about those who weren’t.

In August 2018, Pearson joined the party, bringing out a report entitled ‘Beyond Millennials: The Next Generation of Learners’. Conducted by the Harris Poll, the survey looked at 2,587 US respondents, aged between 14 and 40. The results were weighted for age, gender, race/ethnicity, marital status, household income, and education, so were rather more representative than the Seemiller & Grace research.

In ELT and educational references to ‘Generation Z’, research, of even the very limited kind mentioned above, is rarely cited. When it is, Seemiller and Grace feature prominently (e.g. Mohr & Mohr, 2017). Alternatively, even less reliable sources are used. In an ELT webinar entitled ‘Engaging Generation Z’, for example, information about the characteristics of ‘Generation Z’ learners is taken from an infographic produced by an American office furniture company.

But putting aside quibbles about the reliability of the information, and the fact that it most commonly[1] refers to Americans (who are not, perhaps, the most representative group in global terms), what do the polls tell us?

Despite claims that Generation Z are significantly different from their Millennial predecessors, the general picture that emerges suggests that differences are more a question of degree than substance. These include:

  • A preference for visual / video information over text
  • A variety of bite-sized, entertaining educational experiences
  • Short attention spans and zero tolerance for delay

All of these were identified in 2008 (Williams et al., 2008) as characteristics of the ‘Google Generation’ (a label which usually seems to span Millennials and Generation Z). There is nothing fundamentally different from Prensky’s description of ‘digital natives’. The Pearson report claimed that ‘Generation Z expects experiences both inside and outside the classroom that are more rewarding, more engaging and less time consuming. Technology is no longer a transformative phenomena for this generation, but rather a normal, integral part of life’. However, there is no clear disjuncture or discontinuity between Generation Z and Millennials, any more than there was between ‘digital natives’ and previous generations (Selwyn, 2009: 375). What has really changed is that the technology has moved on (e.g. YouTube was founded in 2005 and the first iPhone was released in 2007).

TESOL TurkeyThe discourse surrounding ‘Generation Z’ is now steadily finding its way into the world of English language teaching. The 2nd TESOL Turkey International ELT Conference took place last November with ‘Teaching Generation Z: Passing on the baton from K12 to University’ as its theme. A further gloss explained that the theme was ‘in reference to the new digital generation of learners with outstanding multitasking skills; learners who can process and absorb information within mere seconds and yet possess the shortest attention span ever’.

 

A few more examples … Cambridge University Press ran a webinar ELT webinar entitled ‘Engaging Generation Z’ and Macmillan Education has a coursebook series called ‘Exercising English for Generation Z’. EBC, a TEFL training provider, ran a blog post in November last year, ‘Teaching English to generation Z students’. And EFL Magazine had an article, ‘Critical Thinking – The Key Competence For The Z Generation’, in February of this year.

The pedagogical advice that results from this interest in Generation Z seems to boil down to: ‘Accept the digital desires of the learners, use lots of video (i.e. use more technology in the classroom) and encourage multi-tasking’.

No one, I suspect, would suggest that teachers should not make use of topics and technologies that appeal to their learners. But recommendations to change approaches to language teaching, ‘based solely on the supposed demands and needs of a new generation of digital natives must be treated with caution’ (Bennett et al., 2008: 782). It is far from clear that generational differences (even if they really exist) are important enough ‘to be considered during the design of instruction or the use of different educational technologies – at this time, the weight of the evidence is negative’ (Reeves, 2008: 21).

Perhaps, it would be more useful to turn away from surveys of attitudes and towards more fact-based research. Studies in both the US and the UK have found that myopia and other problems with the eyes is rising fast among the Generation Z cohort, and that there is a link with increased screen time, especially with handheld devices. At the same time, Generation Zers are much more likely than their predecessors to be diagnosed with anxiety disorder and depression. While the connection between technology use and mental health is far from clear, it is possible that  ‘the rise of the smartphone and social media have at least something to do with [the rise in mental health issues]’ (Twenge, 2017).

Should we be using more technology in class because learners say they want or need it? If we follow that logic, perhaps we should also be encouraging the consumption of fast food, energy drinks and Ritalin before and after lessons?

[1] Studies have been carried out in other geographical settings, including Europe (e.g. Triple-a-Team AG, 2016) and China (Tang, 2019).

References

Bennett S., Maton K., & Kervin, L. (2008). The ‘digital natives’ debate: a critical review of the evidence. British Jmournal of Educational Technology, 39 (5):pp. 775-786.

Hart, A. (2018). Against Generational Politics. Jacobin, 28 February 2018. https://jacobinmag.com/2018/02/generational-theory-millennials-boomers-age-history

Howe, N. & Strauss, W. (2000). Millennials Rising: The Next Great Generation. New York, NY: Vintage Books.

Jauregui, J., Watsjold, B., Welsh, L., Ilgen, J. S. & Robins, L. (2019). Generational “othering”: The myth of the Millennial learner. Medical Education,54: pp.60–65. https://onlinelibrary.wiley.com/doi/pdf/10.1111/medu.13795

Kaiser, D. (2016). Donald Trump, Stephen Bannon and the Coming Crisis in American National Life. Time, 18 November 2016. https://time.com/4575780/stephen-bannon-fourth-turning/

Kirschner, P.A. & De Bruyckere P. (2017). The myths of the digital native and the multitasker. Teaching and Teacher Education, 67: pp. 135-142. https://www.sciencedirect.com/science/article/pii/S0742051X16306692

Mohr, K. A. J. & Mohr, E. S. (2017). Understanding Generation Z Students to Promote a Contemporary Learning Environment. Journal on Empowering Teacher Excellence, 1 (1), Article 9 DOI: https://doi.org/10.15142/T3M05T

Onion, R. (2015). Against generations. Aeon, 19 May, 2015. https://aeon.co/essays/generational-labels-are-lazy-useless-and-just-plain-wrong

Pearson (2018). Beyond Millennials: The Next Generation of Learners. https://www.pearson.com/content/dam/one-dot-com/one-dot-com/global/Files/news/news-annoucements/2018/The-Next-Generation-of-Learners_final.pdf

Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9: pp. 1- 6

Reeves, T.C. (2008). Do Generational Differences Matter in Instructional Design? Athens, GA: University of Georgia, Department of Educational Psychology and Instructional Technology

Seemiller, C. & and Grace, M. (2016). Generation Z Goes to College. San Francisco: Jossey-Bass

Selwyn, N. (2009). The digital native-myth and reality. Perspectives, 61: pp. 364-379

Strauss W. & Howe, N. (1991). Generations: The History of America’s Future, 1584 to 2069. New York, New York: HarperCollins.

Tang F. (2019). A critical review of research on the work-related attitudes of Generation Z in China. Social Psychology and Society, 10 (2): pp. 19—28. Available at: https://psyjournals.ru/files/106927/sps_2019_n2_Tang.pdf

Thomas, M. (2011). Technology, Education, and the Discourse of the Digital Native: Between Evangelists and Dissenters. In Thomas, M. (ed). (2011). Deconstructing Digital Natives: Young people, technology and the new literacies. London: Routledge. pp. 1- 13)

Triple-a-Team AG. (2016). Generation Z Metastudie über die kommende Generation. Biglen, Switzerland. Available at: http://www.sprachenrat.bremen.de/files/aktivitaeten/Generation_Z_Metastudie.pdf

Twenge, J. M. (2017). iGen. New York: Atria Books

Williams, P., Rowlands, I. & Fieldhouse, M. (2008). The ‘Google Generation’ – myths and realities about young people’s digital information behaviour. In Nicholas, D. & Rowlands, I. (eds.) (2008). Digital Consumers. London: Facet Publishers.

Definition of gritGrit book cover

from Quartz at Work magazine

 

Grit is on the up. You may have come across articles like ‘How to Be Gritty in the Time of COVID-19’ or ‘Rediscovering the meaning of grit during COVID-19’ . If you still want more, there are new videos from Angela Duckworth herself where we can learn how to find our grit in the face of the pandemic.

Schools and educational authorities love grit. Its simple, upbeat message (‘Yes, you can’) has won over hearts and minds. Back in 2014, the British minister for education announced a £5million plan to encourage teaching ‘character and resilience’ in schools – specifically looking at making Britain’s pupils ‘grittier’. The spending on grit hasn’t stopped since.

The publishers of Duckworth’s book paid a seven-figure sum to acquire the US rights, and sales have proved the wisdom of the investment. Her TED talk has had over 6.5 million views on YouTube, although it’s worth looking at the comments to see why many people have been watching it.

Youtube comments

The world of English language teaching, always on the lookout for a new bandwagon to jump onto, is starting to catch up with the wider world of education. Luke Plonsky, an eminent SLA scholar, specialist in meta-analyses and grit enthusiast, has a bibliography of grit studies related to L2 learning, that he deems worthy of consideration. Here’s a summary, by year, of those publications. More details will follow in the next section.

Plonsky biblio

We can expect interest in ‘grit’ to continue growing, and this may be accelerated by the publication this year of Engaging Language Learners in Contemporary Classrooms by Sarah Mercer and Zoltán Dörnyei. In this book, the authors argue that a ‘facilitative mindset’ is required for learner engagement. They enumerate five interrelated principles for developing a ‘facilitative mindset’: promote a sense of competence, foster a growth mindset, promote learners’ sense of ownership and control, develop proactive learners and, develop gritty learners. After a brief discussion of grit, they write: ‘Thankfully, grit can be learnt and developed’ (p.38).

Unfortunately, they don’t provide any evidence at all for this. Unfortunately, too, this oversight is easy to explain. Such evidence as there is does not lend unequivocal support to the claim. Two studies that should have been mentioned in this book are ‘Much ado about grit: A meta-analytic synthesis of the grit literature’ (Credé et al, 2017) and ‘What shall we do about grit? A critical review of what we know and what we don’t know’ (Credé, 2018). The authors found that ‘grit as it is currently measured does not appear to be particularly predictive of success and performance’ (Credé et al, 2017) and that there is no support for the claim that ‘grit is likely to be responsive to interventions’ (Credé, 2018). In the L2 learning context, Teimouri et al (2020) concluded that more research in SLA substantiating the role of grit in L2 contexts was needed before any grit interventions can be recommended.

It has to be said that such results are hardly surprising. If, as Duckworth claims, ‘grit’ is a combination of passion and persistence, how on earth can the passion part of it be susceptible to educational interventions? ‘If there is one thing that cannot be learned, it’s passion. A person can have it and develop it, but learn it? Sadly, not’. (De Bruyckere et al., 2020: 83)

Even Duckworth herself is not convinced. In an interview on a Freakonomics podcast, she states that she hopes it’s something people can learn, but also admits not having enough proof to confirm that they can (Kirschner & Neelen, 2016)!

Is ‘grit’ a thing?

Marc Jones, in a 2016 blog post entitled ‘Gritty Politti: Grit, Growth Mindset and Neoliberal Language Teaching’, writes that ‘Grit is so difficult to define that it takes Duckworth (2016) the best part of a book to describe it adequately’. Yes, ‘grit’ is passion and persistence (or perseverance), but it’s also conscientiousness, practice and hope. Credé et al (2017) found that ‘grit is very strongly correlated with conscientiousness’ (which has already been widely studied in the educational literature). Why lump this together with passion? Another study (Muenks et al., 2017) found that ‘Students’ grit overlapped empirically with their concurrently reported self-control, self-regulation, and engagement. Students’ perseverance of effort (but not their consistency of interests) predicted their later grades, although other self-regulation and engagement variables were stronger predictors of students’ grades than was grit’. Credé (2018) concluded that ‘there appears to be no reason to accept the combination of perseverance and passion for long-term goals into a single grit construct’.

The L2 learning research listed in Plonsky’s bibliography does not offer much in support of ‘grit’, either. Many of the studies identified problems with ‘grit’ as a construct, but, even when accepting it, did not find it to be of much value. Wei et al. (2019) found a positive but weak correlation between grit and English language course grades. Yamashita (2018) found no relationship between learners’ grit and their course grades. Taşpinar & Külekçi (2018) found that students’ grit levels and academic achievement scores did not relate to each other (but still found that ‘grit, perseverance, and tenacity are the essential elements that impact learners’ ability to succeed to be prepared for the demands of today’s world’!).

There are, then, grounds for suspecting that Duckworth and her supporters have fallen foul of the ‘jangle fallacy’ – the erroneous assumption that two identical or almost identical things are different because they are labelled differently. This would also help to explain the lack of empirical support for the notion of ‘grit’. Not only are the numerous variables insufficiently differentiated, but the measures of ‘grit’ (such as Duckworth’s Grit-S measure) do not adequately target some of these variables (e.g. long-term goals, where ‘long-term’ is not defined) (Muenks et al., 2017). In addition, these measures are self-reporting and not, therefore, terribly reliable.

Referring to more general approaches to character education, one report (Gutman & Schoon, 2012) has argued that there is little empirical evidence of a causal relationship between self-concept and educational outcomes. Taking this one step further, Kathryn Ecclestone (Ecclestone, 2012) suggests that at best, the concepts and evidence that serve as the basis of these interventions are inconclusive and fragmented; ‘at worst, [they are] prey to ‘advocacy science’ or, in [their] worst manifestations, to simple entrepreneurship that competes for publicly funded interventions’ (cited in Cabanas & Illouz, 2019: 80).

Criticisms of ‘grit’

Given the lack of supporting research, any practical application of ‘grit’ ideas is premature. Duckworth herself, in an article entitled ‘Don’t Believe the Hype About Grit, Pleads the Scientist Behind the Concept’ (Dahl, 2016), cautions against hasty applications:

[By placing too much emphasis on grit, the danger is] that grit becomes a scapegoat — another reason to blame kids for not doing well, or to say that we don’t have a responsibility as a society to help them. [She worries that some interpretations of her work might make a student’s failure seem like his problem, as if he just didn’t work hard enough.] I think to separate and pit against each other character strengths on the one hand — like grit — and situational opportunities on the other is a false dichotomy […] Kids need to develop character, and they need our support in doing so.

Marc Jones, in the blog mentioned above, writes that ‘to me, grit is simply another tool for attacking the poor and the other’. You won’t win any prizes for guessing which kinds of students are most likely to be the targets of grit interventions. A clue: think of the ‘no-nonsense’ charters in the US and academies in the UK. This is what Kenneth Saltzman has to say:

‘Grit’ is a pedagogy of control that is predicated upon a promise made to poor children that if they learnt the tools of self-control and learnt to endure drudgery, then they can compete with rich children for scarce economic resources. (Saltzman, 2017: 38)

[It] is a behaviourist form of learned self-control targeting poor students of color and has been popularized post-crisis in the wake of educational privatization and defunding as the cure for poverty. [It] is designed to suggest that individual resilience and self-reliance can overcome social violence and unsupportive social contexts in the era of the shredded social state. (Saltzman, 2017: 15)

Grit is misrepresented by proponents as opening a world of individual choices rather than discussed as a mode of educational and social control in the austere world of work defined by fewer and fewer choices as secure public sector work is scaled back, unemployment continuing at high levels. (Saltzman, 2017: 49)

Whilst ‘grit’ is often presented as a way of dealing with structural inequalities in schools, critics see it as more of a problem than a solution: ‘It’s the kids who are most impacted by, rebel against, or criticize the embedded racism and classism of their institutions that are being told to have more grit, that school is hard for everyone’ (EquiTEA, 2018). A widely cited article by Nicholas Tampio (2016) points out that ‘Duckworth celebrates educational models such as Beast at West Point that weed out people who don’t obey orders’. He continues ‘that is a disastrous model for education in a democracy. US schools ought to protect dreamers, inventors, rebels and entrepreneurs – not crush them in the name of grit’.

If you’re interested in reading more critics of grit, the blog ‘Debunked!’ is an excellent source of links.

Measuring grit

Analyses of emotional behaviour have become central to economic analysis and, beginning in the 1990s, there have been constant efforts to create ‘formal instruments of classification of emotional behaviour and the elaboration of the notion of emotional competence’ (Illouz, 2007: 64). The measurement and manipulation of various aspects of ‘emotional intelligence’ have become crucial as ways ‘to control, predict, and boost performance’ (Illouz, 2007: 65). An article in the Journal of Benefit-Cost Analysis (Belfield et al., 2015) makes the economic importance of emotions very clear. Entitled ‘The Economic Value of Social and Emotional Learning’, it examines the economic value of these skills within a benefit-cost analysis (BCA) framework, and finds that the benefits of [social and emotional learning] interventions substantially outweigh the costs.

In recent years, the OECD has commissioned a number of reports on social and emotional learning and, as with everything connected with the OECD, is interested in measuringnon-cognitive skills such as perseverance (“grit”), conscientiousness, self-control, trust, attentiveness, self-esteem and self-efficacy, resilience to adversity, openness to experience, empathy, humility, tolerance of diverse opinions and the ability to engage productively in society’ (Kautz et al., 2014: 9). The measurement of personality factors will feature in the OECD’s PISA programme. Elsewhere, Ben Williamson reports that ‘US schools [are] now under pressure—following the introduction of the Every Student Succeeds Act in 2015—to provide measurable evidence of progress on the development of students’ non-academic learning’ (Williamson, 2017).

Grit, which ‘starts and ends with the lone individual as economic actor, worker, and consumer’ (Saltzman, 2017: 50), is a recent addition to the categories of emotional competence, and it should come as no surprise that educational authorities have so wholeheartedly embraced it. It is the claim that something (i.e. ‘grit’) can be taught and developed that leads directly to the desire to measure it. In a world where everything must be accountable, we need to know how effective and cost-effective our grit interventions have been.

The idea of measuring personality constructs like ‘grit’ worries even Angela Duckworth. She writes (Duckworth, 2016):

These days, however, I worry I’ve contributed, inadvertently, to an idea I vigorously oppose: high-stakes character assessment. New federal legislation can be interpreted as encouraging states and schools to incorporate measures of character into their accountability systems. This year, nine California school districts will begin doing this. But we’re nowhere near ready — and perhaps never will be — to use feedback on character as a metric for judging the effectiveness of teachers and schools. We shouldn’t be rewarding or punishing schools for how students perform on these measures.

Diane Ravitch (Ravitch, 2016) makes the point rather more forcefully: ‘The urge to quantify the unmeasurable must be recognized for what it is: stupid; arrogant; harmful; foolish, yet another way to standardize our beings’. But, like it or not, attempts to measure ‘grit’ and ‘grit’ interventions are unlikely to go away any time soon.

‘Grit’ and technology

Whenever there is talk about educational measurement and metrics, we are never far away from the world of edtech. It may not have escaped your notice that the OECD and the US Department of State for Education, enthusiasts for promoting ‘grit’, are also major players in the promotion of the datafication of education. The same holds true for organisations like the World Education Forum, the World Bank and the various philanthro-capitalist foundations to which I have referred so often in this blog. Advocacy of social and emotional learning goes hand in hand with edtech advocacy.

Two fascinating articles by Ben Williamson (2017; 2019) focus on ClassDojo, which, according to company information, reaches more than 10 million children globally every day. The founding directors of ClassDojo, writes Ben Williamson (2017), ‘explicitly describe its purpose as promoting ‘character development’ in schools and it is underpinned by particular psychological concepts from character research. Its website approvingly cites the journalist Paul Tough, author of two books on promoting ‘grit’ and ‘character’ in children, and is informed by character research conducted with the US network of KIPP charter schools (Knowledge is Power Program)’. In a circular process, ClassDojo has also ‘helped distribute and popularise concepts such as growth mindset, grit and mindfulness’ (Williamson, 2019).

The connections between ‘grit’ and edtech are especially visible when we focus on Stanford and Silicon Valley. ClassDojo was born in Palo Alto. Duckworth was a consulting scholar at Stanford 2014 -15, where Carol Dweck is a Professor of Psychology. Dweck is the big name behind growth mindset theory, which, as Sarah Mercer and Zoltán Dörnyei indicate, is closely related to ‘grit’. Dweck is also the co-founder of MindsetWorks, whose ‘Brainology’ product is ‘an online interactive program in which middle school students learn about how the brain works, how to strengthen their own brains, and how to ….’. Stanford is also home to the Stanford Lytics Lab, ‘which has begun applying new data analytics techniques to the measurement of non-cognitive learning factors including perseverance, grit, emotional state, motivation and self-regulation’, as well as the Persuasive Technologies Lab, ‘which focuses on the development of machines designed to influence human beliefs and behaviors across domains including health, business, safety, and education’ (Williamson, 2017). The Professor of Education Emeritus at Stanford is Linda Darling-Hammond, one of the most influential educators in the US. Darling-Hammond is known, among many other things, for collaborating with Pearson to develop the edTPA, ‘a nationally available, performance-based assessment for measuring the effectiveness of teacher candidates’. She is also a strong advocate of social-emotional learning initiatives and extols the virtues of ‘developing grit and a growth mindset’ (Hamadi & Darling-Hammond, 2015).

The funding of grit

Angela Duckworth’s Character Lab (‘Our mission is to advance scientific insights that help kids thrive’) is funded by, among others, the Chan Zuckerberg Initiative, the Bezos Family Foundation and Stanford’s Mindset Scholars Network. Precisely how much money Character Lab has is difficult to ascertain, but the latest grant from the Chan Zuckerberg Initiative was worth $1,912,000 to cover the period 2018 – 2021. Covering the same period, the John Templeton Foundation, donated $3,717,258 , the purpose of the grant being to ‘make character development fast, frictionless, and fruitful’.

In an earlier period (2015 – 2018), the Walton Family Foundation pledged $6.5 millionto promote and measure character education, social-emotional learning, and grit’, with part of this sum going to Character Lab and part going to similar research at Harvard Graduate School of Education. Character Lab also received $1,300,000 from the Overdeck Family Foundation for the same period.

It is not, therefore, an overstatement to say that ‘grit’ is massively funded. The funders, by and large, are the same people who have spent huge sums promoting personalized learning through technology (see my blog post Personalized learning: Hydra and the power of ambiguity). Whatever else it might be, ‘grit’ is certainly ‘a commercial tech interest’ (as Ben Williamson put it in a recent tweet).

Postscript

In the 2010 Cohen brothers’ film, ‘True Grit’, the delinquent ‘kid’, Moon, is knifed by his partner, Quincy. Turning to Rooster Cogburn, the man of true grit, Moon begs for help. In response, Cogburn looks at the dying kid and deadpans ‘I can do nothing for you, son’.

References

Belfield, C., Bowden, A., Klapp, A., Levin, H., Shand, R., & Zander, S. (2015). The Economic Value of Social and Emotional Learning. Journal of Benefit-Cost Analysis, 6(3), pp. 508-544. doi:10.1017/bca.2015.55

Cabanas, E. & Illouz, E. (2019). Manufacturing Happy Citizens. Cambridge: Polity Press.

Chaykowski, K. (2017). How ClassDojo Built One Of The Most Popular Classroom Apps By Listening To Teachers. Forbes, 22 May, 2017. https://www.forbes.com/sites/kathleenchaykowski/2017/05/22/how-classdojo-built-one-of-the-most-popular-classroom-apps-by-listening-to-teachers/#ea93d51e5ef5

Credé, M. (2018). What shall we do about grit? A critical review of what we know and what we don’t know. Educational Researcher, 47(9), 606-611.

Credé, M., Tynan, M. C., & Harms, P. D. (2017). Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113(3), 492. doi:10.1037/pspp0000102

Dahl, M. (2016). Don’t Believe the Hype About Grit, Pleads the Scientist Behind the Concept. The Cut, May 9, 2016. https://www.thecut.com/2016/05/dont-believe-the-hype-about-grit-pleads-the-scientist-behind-the-concept.html

De Bruyckere, P., Kirschner, P. A. & Hulshof, C. (2020). More Urban Myths about Learning and Education. Routledge.

Duckworth, A. (2016). Don’t Grade Schools on Grit. New York Times, March 26, 2016 https://www.nytimes.com/2016/03/27/opinion/sunday/dont-grade-schools-on-grit.html?auth=login-google&smid=nytcore-ipad-share&smprod=nytcore-ipad

Ecclestone, K. (2012). From emotional and psychological well-being to character education: Challenging policy discourses of behavioural science and ‘vulnerability’. Research Papers in Education, 27 (4), pp. 463-480

EquiTEA (2018). The Problem with Teaching ‘Grit’. Medium, 11 December 2018. https://medium.com/@eec/the-problem-with-teaching-grit-8b37ce43a87e

Gutman, L. M. & Schoon, I. (2013). The impact of non-cognitive skills on outcomes for young people: Literature review. London: Institute of Education, University of London

Hamedani, M. G. & Darling-Hammond, L. (2015). Social Emotional Learning in High School: How Three Urban High Schools Engage, Educate, and Empower Youth. Stanford Center for Opportunity Policy in Education

Kirschner, P.A. & Neelen, M. (2016). To Grit Or Not To Grit: That’s The Question. 3-Star Learning Experiences, July 5, 2016 https://3starlearningexperiences.wordpress.com/2016/07/05/to-grit-or-not-to-grit-thats-the-question/

Illouz, E. (2007). Cold Intimacies: The making of emotional capitalism. Cambridge: Polity Press

Kautz, T., Heckman, J. J., Diris, R., ter Weel, B & Borghans, L. (2014). Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success. OECD Education Working Papers 110, OECD Publishing.

Mercer, S. & Dörnyei, Z. (2020). Engaging Language Learners in Contemporary Classrooms. Cambridge University Press.

Muenks, K., Wigfield, A., Yang, J. S., & O’Neal, C. R. (2017). How true is grit? Assessing its relations to high school and college students’ personality characteristics, self-regulation, engagement, and achievement. Journal of Educational Psychology, 109, pp. 599–620.

Ravitch, D. (2016). Angela Duckworth, please don’t assess grit. Blog post, 27 March 2016, https://dianeravitch.net/2016/03/27/angela-duckworth-please-dont-assess-grit/

Saltzman, K. J. (2017). Scripted Bodies. Routledge.

Tampio, N. (2016). Teaching ‘grit’ is bad for children, and bad for democracy. Aeon, 2 June: https://aeon.co/ideas/teaching-grit-is-bad-for-children-and-bad-for-democracy

Taşpinar, K., & Külekçi, G. (2018). GRIT: An Essential Ingredient of Success in the EFL Classroom. International Journal of Languages’ Education and Teaching, 6, 208-226.

Teimouri, Y., Plonsky, L., & Tabandeh, F. (2020). L2 Grit: Passion and perseverance for second-language learning. Language Teaching Research.

Wei, H., Gao, K., & Wang, W. (2019). Understanding the relationship between grit and foreign language performance among middle schools students: The roles of foreign language enjoyment and classroom Environment. Frontiers in Psychology, 10, 1508. doi: 10.3389/fpsyg.2019.01508

Williamson, B. (2017). Decoding ClassDojo: psycho-policy, social-emotional learning and persuasive educational technologies. Learning, Media and Technology, 42 (4): pp. 440-453, DOI: 10.1080/17439884.2017.1278020

Williamson, B. (2019). ‘Killer Apps for the Classroom? Developing Critical Perspectives on ClassDojo and the ‘Ed-tech’ Industry. Journal of Professional Learning, 2019 (Semester 2) https://cpl.asn.au/journal/semester-2-2019/killer-apps-for-the-classroom-developing-critical-perspectives-on-classdojo

Yamashita, T. (2018). Grit and second language acquisition: Can passion and perseverance predict performance in Japanese language learning? Unpublished MA thesis, University of Massachusetts, Amherst.

 

I’ve long felt that the greatest value of technology in language learning is to facilitate interaction between learners, rather than interaction between learners and software. I can’t claim any originality here. Twenty years ago, Kern and Warschauer (2000) described ‘the changing nature of computer use in language teaching’, away from ‘grammar and vocabulary tutorials, drill and practice programs’, towards computer-mediated communication (CMC). This change has even been described as a paradigm shift (Ciftci & Kocoglu, 2012: 62), although I suspect that the shift has affected approaches to research much more than it has actual practices.

However, there is one application of CMC that is probably at least as widespread in actual practice as it is in the research literature: online peer feedback. Online peer feedback on writing, especially in the development of academic writing skills in higher education, is certainly very common. To a much lesser extent, online peer feedback on speaking (e.g. in audio and video blogs) has also been explored (see, for example, Yeh et al., 2019 and Rodríguez-González & Castañeda, 2018).

Peer feedback

Interest in feedback has spread widely since the publication of Hattie and Timperley’s influential ‘The Power of Feedback’, which argued that ‘feedback is one of the most powerful influences on learning and achievement’ (Hattie & Timperley, 2007: 81). Peer feedback, in particular, has generated much optimism in the general educational literature as a formative practice (Double et al., 2019) because of its potential to:

  • ‘promote a sense of ownership, personal responsibility, and motivation,
  • reduce assessee anxiety and improve acceptance of negative feedback,
  • increase variety and interest, activity and interactivity, identification and bonding, self-confidence, and empathy for others’ (Topping, 1988: 256)
  • improve academic performance (Double et al., 2019).

In the literature on language learning, this enthusiasm is mirrored and peer feedback is generally recommended by both methodologists and researchers (Burkert & Wally, 2013). The reasons given, in addition to those listed above, include the following:

  • it can benefit both the receiver and the giver of feedback (Storch & Aldossary, 2019: 124),
  • it requires the givers of feedback to listen to or read attentively the language of their peers, and, in the process, may provide opportunities for them to make improvements in their own speaking and writing (Alshuraidah & Storch, 2019: 166–167,
  • it can facilitate a move away from a teacher centred classroom, and promote independent learning (and the skill of self-correction) as well as critical thinking (Hyland & Hyland, 2019: 7),
  • the target reader is an important consideration in any piece of writing (it is often specified in formal assessment tasks). Peer feedback may be especially helpful in developing the idea of what audience the writer is writing for (Nation, 2009: 139),
  • many learners are very receptive to peer feedback (Biber et al., 2011: 54),
  • it can reduce a teacher’s workload.

The theoretical arguments in support of peer feedback are supported to some extent by research. A recent meta-analysis found ‘an overall small to medium effect of peer assessment on academic performance’ (Double et al., 2019) in general educational settings. In language learning, ‘recent research has provided generally positive evidence to support the use of peer feedback in L2 writing classes’ (Yu & Lee, 2016: 467). However, ‘firm causal evidence is as yet unavailable’ (Yu & Lee, 2016: 466).

Online peer feedback

Taking peer feedback online would seem to offer a number of advantages over traditional face-to-face oral or written channels. These include:

  • a significant reduction of the logistical burden (Double et al.: 2019) because there are fewer constraints of time and place (Ho, 2015: 1),
  • the possibility (with many platforms) of monitoring students’ interactions more closely (DiGiovanni & Nagaswami, 2001: 268),
  • the encouragement of ‘greater and more equal member participation than face-to-face feedback’ (Yu & Lee, 2016: 469),
  • the possibility of reducing learners’ anxiety (which may be greater in face-to-face settings and / or when an immediate response to feedback is required) (Yeh et al.: 2019: 1).

Given these potential advantages, it is disappointing to find that a meta-analysis of peer assessment in general educational contexts did not find any significant difference between online and offline feedback (Double et al.:2019). Similarly, in language learning contexts, Yu & Lee (2016: 469) report that ‘there is inconclusive evidence about the impact of computer-mediated peer feedback on the quality of peer comments and text revisions’. The rest of this article is an exploration of possible reasons why online peer feedback is not more effective than it is.

The challenges of online peer feedback

Peer feedback is usually of greatest value when it focuses on the content and organization of what has been expressed. Learners, however, have a tendency to focus on formal accuracy, rather than on the communicative success (or otherwise) of their peers’ writing or speaking. Training can go a long way towards remedying this situation (Yu & Lee, 2016: 472 – 473): indeed, ‘the importance of properly training students to provide adequately useful peer comments cannot be over-emphasized’ (Bailey & Cassidy, 2018: 82). In addition, clearly organised rubrics to guide the feedback giver, such as those offered by feedback platforms like Peergrade, may also help to steer feedback in appropriate directions. There are, however, caveats which I will come on to.

A bigger problem occurs when the interaction which takes places when learners are supposedly engaged in peer feedback is completely off-task. In one analysis of students’ online discourse in two writing tasks, ‘meaning negotiation, error correction, and technical actions seldom occurred and […] social talk, task management, and content discussion predominated the chat’ (Liang, 2010: 45). One proposed solution to this is to grade peer comments: ‘reviewers will be more motivated to spend time in their peer review process if they know that their instructors will assess or even grade their comments’ (Choi, 2014: 225). Whilst this may sometimes be an effective strategy, the curtailment of social chat may actually create more problems than it solves, as we will see later.

Other challenges of peer feedback may be even less amenable to solutions. The most common problem concerns learners’ attitudes towards peer feedback: some learners are not receptive to feedback from their peers, preferring feedback from their teachers (Maas, 2017), and some learners may be reluctant to offer peer feedback for fear of giving offence. Attitudinal issues may derive from personal or cultural factors, or a combination of both. Whatever the cause, ‘interpersonal variables play a substantial role in determining the type and quality of peer assessment’ (Double et al., 2019). One proposed solution to this is to anonymise the peer feedback process, since it might be thought that this would lead to greater honesty and fewer concerns about loss of face. Research into this possibility, however, offers only very limited support: two studies out of three found little benefit of anonymity (Double et al., 2019). What is more, as with the curtailment of social chat, the practice must limit the development of the interpersonal relationship, and therefore positive pair / group dynamics (Liang, 2010: 45), that is necessary for effective collaborative work.

Towards solutions?

Online peer feedback is a form of computer-supported collaborative learning (CSCL), and it is to research in this broader field that I will now turn. The claim that CSCL ‘can facilitate group processes and group dynamics in ways that may not be achievable in face-to-face collaboration’ (Dooly, 2007: 64) is not contentious, but, in order for this to happen, a number of ‘motivational or affective perceptions are important preconditions’ (Chen et al., 2018: 801). Collaborative learning presupposes a collaborative pattern of peer interaction, as opposed to expert-novice, dominant- dominant, dominant-passive, or passive-passive patterns (Yu & Lee, 2016: 475).

Simply putting students together into pairs or groups does not guarantee collaboration. Collaboration is less likely to take place when instructional management focusses primarily on cognitive processes, and ‘socio-emotional processes are ignored, neglected or forgotten […] Social interaction is equally important for affiliation, impression formation, building social relationships and, ultimately, the development of a healthy community of learning’ (Kreijns et al., 2003: 336, 348 – 9). This can happen in all contexts, but in online environments, the problem becomes ‘more salient and critical’ (Kreijns et al., 2003: 336). This is why the curtailment of social chat, the grading of peer comments, and the provision of tight rubrics may be problematic.

There is no ‘single learning tool or strategy’ that can be deployed to address the challenges of online peer feedback and CSCL more generally (Chen et al., 2018: 833). In some cases, for personal or cultural reasons, peer feedback may simply not be a sensible option. In others, where effective online peer feedback is a reasonable target, the instructional approach must find ways to train students in the specifics of giving feedback on a peer’s work, to promote mutual support, to show how to work effectively with others, and to develop the language skills needed to do this (assuming that the target language is the language that will be used in the feedback).

So, what can we learn from looking at online peer feedback? I think it’s the same old answer: technology may confer a certain number of potential advantages, but, unfortunately, it cannot provide a ‘solution’ to complex learning issues.

 

Note: Some parts of this article first appeared in Kerr, P. (2020). Giving feedback to language learners. Part of the Cambridge Papers in ELT Series. Cambridge: Cambridge University Press. Available at: https://www.cambridge.org/gb/files/4415/8594/0876/Giving_Feedback_minipaper_ONLINE.pdf

 

References

Alshuraidah, A. and Storch, N. (2019). Investigating a collaborative approach to feedback. ELT Journal, 73 (2), pp. 166–174

Bailey, D. and Cassidy, R. (2018). Online Peer Feedback Tasks: Training for Improved L2 Writing Proficiency, Anxiety Reduction, and Language Learning Strategies. CALL-EJ, 20(2), pp. 70-88

Biber, D., Nekrasova, T., and Horn, B. (2011). The Effectiveness of Feedback for L1-English and L2-Writing Development: A Meta-Analysis, TOEFL iBT RR-11-05. Princeton: Educational Testing Service. Available at: https://www.ets.org/Media/Research/pdf/RR-11-05.pdf

Burkert, A. and Wally, J. (2013). Peer-reviewing in a collaborative teaching and learning environment. In Reitbauer, M., Campbell, N., Mercer, S., Schumm Fauster, J. and Vaupetitsch, R. (Eds.) Feedback Matters. Frankfurt am Main: Peter Lang, pp. 69–85

Chen, J., Wang, M., Kirschner, P.A. and Tsai, C.C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88 (6) (2018), pp. 799-843

Choi, J. (2014). Online Peer Discourse in a Writing Classroom. International Journal of Teaching and Learning in Higher Education, 26 (2): pp. 217 – 231

Ciftci, H. and Kocoglu, Z. (2012). Effects of Peer E-Feedback on Turkish EFL Students’ Writing Performance. Journal of Educational Computing Research, 46 (1), pp. 61 – 84

DiGiovanni, E. and Nagaswami. G. (2001). Online peer review: an alternative to face-to-face? ELT Journal 55 (3), pp. 263 – 272

Dooly, M. (2007). Joining forces: Promoting metalinguistic awareness through computer-supported collaborative learning. Language Awareness, 16 (1), pp. 57-74

Double, K.S., McGrane, J.A. and Hopfenbeck, T.N. (2019). The Impact of Peer Assessment on Academic Performance: A Meta-analysis of Control Group Studies. Educational Psychology Review (2019)

Hattie, J. and Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), pp. 81–112

Ho, M. (2015). The effects of face-to-face and computer-mediated peer review on EFL writers’ comments and revisions. Australasian Journal of Educational Technology, 2015, 31(1)

Hyland K. and Hyland, F. (2019). Contexts and issues in feedback on L2 writing. In Hyland K. & Hyland, F. (Eds.) Feedback in Second Language Writing. Cambridge: Cambridge University Press, pp. 1–22

Kern, R. and Warschauer, M. (2000). Theory and practice of network-based language teaching. In M. Warschauer and R. Kern (eds) Network-Based Language Teaching: Concepts and Practice. New York: Cambridge University Press. pp. 1 – 19

Kreijns, K., Kirschner, P. A. and Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research. Computers in Human Behavior, 19(3), pp. 335-353

Liang, M. (2010). Using Synchronous Online Peer Response Groups in EFL Writing: Revision-Related Discourse. Language Learning and Technology, 14 (1), pp. 45 – 64

Maas, C. (2017). Receptivity to learner-driven feedback. ELT Journal, 71 (2), pp. 127–140

Nation, I. S. P. (2009). Teaching ESL / EFL Reading and Writing. New York: Routledge

Panadero, E. and Alqassab, M. (2019). An empirical review of anonymity effects in peer assessment, peer feedback, peer review, peer evaluation and peer grading. Assessment & Evaluation in Higher Education, 1–26

Rodríguez-González, E. and Castañeda, M. E. (2018). The effects and perceptions of trained peer feedback in L2 speaking: impact on revision and speaking quality, Innovation in Language Learning and Teaching, 12 (2), pp. 120-136, DOI: 10.1080/17501229.2015.1108978

Storch, N. and Aldossary, K. (2019). Peer Feedback: An activity theory perspective on givers’ and receivers’ stances. In Sato, M. and Loewen, S. (Eds.) Evidence-based Second Language Pedagogy. New York: Routledge, pp. 123–144

Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68 (3), pp. 249-276.

Yeh, H.-C., Tseng, S.-S., and Chen, Y.-S. (2019). Using Online Peer Feedback through Blogs to Promote Speaking Performance. Educational Technology & Society, 22 (1), pp. 1–14

Yu, S. and Lee, I. (2016). Peer feedback in second language writing (2005 – 2014). Language Teaching, 49 (4), pp. 461 – 493

From time to time, I have mentioned Programmed Learning (or Programmed Instruction) in this blog (here and here, for example). It felt like time to go into a little more detail about what Programmed Instruction was (and is) and why I think it’s important to know about it.

A brief description

The basic idea behind Programmed Instruction was that subject matter could be broken down into very small parts, which could be organised into an optimal path for presentation to students. Students worked, at their own speed, through a series of micro-tasks, building their mastery of each nugget of learning that was presented, not progressing from one to the next until they had demonstrated they could respond accurately to the previous task.

There were two main types of Programmed Instruction: linear programming and branching programming. In the former, every student would follow the same path, the same sequence of frames. This could be used in classrooms for whole-class instruction and I tracked down a book (illustrated below) called ‘Programmed English Course Student’s Book 1’ (Hill, 1966), which was an attempt to transfer the ideas behind Programmed Instruction to a zero-tech, class environment. This is very similar in approach to the material I had to use when working at an Inlingua school in the 1980s.

Programmed English Course

Comparatives strip

An example of how self-paced programming worked is illustrated here, with a section on comparatives.

With branching programming, ‘extra frames (or branches) are provided for students who do not get the correct answer’ (Kay et al., 1968: 19). This was only suitable for self-study, but it was clearly preferable, as it allowed for self-pacing and some personalization. The material could be presented in books (which meant that students had to flick back and forth in their books) or with special ‘teaching machines’, but the latter were preferred.

In the words of an early enthusiast, Programmed Instruction was essentially ‘a device to control a student’s behaviour and help him to learn without the supervision of a teacher’ (Kay et al.,1968: 58). The approach was inspired by the work of Skinner and it was first used as part of a university course in behavioural psychology taught by Skinner at Harvard University in 1957. It moved into secondary schools for teaching mathematics in 1959 (Saettler, 2004: 297).

Enthusiasm and uptake

The parallels between current enthusiasm for the power of digital technology to transform education and the excitement about Programmed Instruction and teaching machines in the 1960s are very striking (McDonald et al., 2005: 90). In 1967, it was reported that ‘we are today on the verge of what promises to be a revolution in education’ (Goodman, 1967: 3) and that ‘tremors of excitement ran through professional journals and conferences and department meetings from coast to coast’ (Kennedy, 1967: 871). The following year, another commentator referred to the way that the field of education had been stirred ‘with an almost Messianic promise of a breakthrough’ (Ornstein, 1968: 401). Programmed instruction was also seen as an exciting business opportunity: ‘an entire industry is just coming into being and significant sales and profits should not be too long in coming’, wrote one hopeful financial analyst as early as 1961 (Kozlowski, 1967: 47).

The new technology seemed to offer a solution to the ‘problems of education’. Media reports in 1963 in Germany, for example, discussed a shortage of teachers, large classes and inadequate learning progress … ‘an ‘urgent pedagogical emergency’ that traditional teaching methods could not resolve’ (Hof, 2018). Individualised learning, through Programmed Instruction, would equalise educational opportunity and if you weren’t part of it, you would be left behind. In the US, two billion dollars were spent on educational technology by the government in the decade following the passing of the National Defense Education Act, and this was added to by grants from private foundations. As a result, ‘the production of teaching machines began to flourish, accompanied by the marketing of numerous ‘teaching units’ stamped into punch cards as well as less expensive didactic programme books and index cards. The market grew dramatically in a short time’ (Hof, 2018).

In the field of language learning, however, enthusiasm was more muted. In the year in which he completed his doctoral studies[1], the eminent linguist, Bernard Spolsky noted that ‘little use is actually being made of the new technique’ (Spolsky, 1966). A year later, a survey of over 600 foreign language teachers at US colleges and universities reported that only about 10% of them had programmed materials in their departments (Valdman, 1968: 1). In most of these cases, the materials ‘were being tried out on an experimental basis under the direction of their developers’. And two years after that, it was reported that ‘programming has not yet been used to any very great extent in language teaching, so there is no substantial body of experience from which to draw detailed, water-tight conclusions’ (Howatt, 1969: 164).

By the early 1970s, Programmed Instruction was already beginning to seem like yesterday’s technology, even though the principles behind it are still very much alive today (Thornbury (2017) refers to Duolingo as ‘Programmed Instruction’). It would be nice to think that language teachers of the day were more sceptical than, for example, their counterparts teaching mathematics. It would be nice to think that, like Spolsky, they had taken on board Chomsky’s (1959) demolition of Skinner. But the widespread popularity of Audiolingual methods suggests otherwise. Audiolingualism, based essentially on the same Skinnerian principles as Programmed Instruction, needed less outlay on technology. The machines (a slide projector and a record or tape player) were cheaper than the teaching machines, could be used for other purposes and did not become obsolete so quickly. The method also lent itself more readily to established school systems (i.e. whole-class teaching) and the skills sets of teachers of the day. Significantly, too, there was relatively little investment in Programmed Instruction for language teaching (compared to, say, mathematics), since this was a smallish and more localized market. There was no global market for English language learning as there is today.

Lessons to be learned

1 Shaping attitudes

It was not hard to persuade some educational authorities of the value of Programmed Instruction. As discussed above, it offered a solution to the problem of ‘the chronic shortage of adequately trained and competent teachers at all levels in our schools, colleges and universities’, wrote Goodman (1967: 3), who added, there is growing realisation of the need to give special individual attention to handicapped children and to those apparently or actually retarded’. The new teaching machines ‘could simulate the human teacher and carry out at least some of his functions quite efficiently’ (Goodman, 1967: 4). This wasn’t quite the same thing as saying that the machines could replace teachers, although some might have hoped for this. The official line was more often that the machines could ‘be used as devices, actively co-operating with the human teacher as adaptive systems and not just merely as aids’ (Goodman, 1967: 37). But this more nuanced message did not always get through, and ‘the Press soon stated that robots would replace teachers and conjured up pictures of classrooms of students with little iron men in front of them’ (Kay et al., 1968: 161).

For teachers, though, it was one thing to be told that the machines would free their time to perform more meaningful tasks, but harder to believe when this was accompanied by a ‘rhetoric of the instructional inadequacies of the teacher’ (McDonald, et al., 2005: 88). Many teachers felt threatened. They ‘reacted against the ‘unfeeling machine’ as a poor substitute for the warm, responsive environment provided by a real, live teacher. Others have seemed to take it more personally, viewing the advent of programmed instruction as the end of their professional career as teachers. To these, even the mention of programmed instruction produces a momentary look of panic followed by the appearance of determination to stave off the ominous onslaught somehow’ (Tucker, 1972: 63).

Some of those who were pushing for Programmed Instruction had a bigger agenda, with their sights set firmly on broader school reform made possible through technology (Hof, 2018). Individualised learning and Programmed Instruction were not just ends in themselves: they were ways of facilitating bigger changes. The trouble was that teachers were necessary for Programmed Instruction to work. On the practical level, it became apparent that a blend of teaching machines and classroom teaching was more effective than the machines alone (Saettler, 2004: 299). But the teachers’ attitudes were crucial: a research study involving over 6000 students of Spanish showed that ‘the more enthusiastic the teacher was about programmed instruction, the better the work the students did, even though they worked independently’ (Saettler, 2004: 299). In other researched cases, too, ‘teacher attitudes proved to be a critical factor in the success of programmed instruction’ (Saettler, 2004: 301).

2 Returns on investment

Pricing a hyped edtech product is a delicate matter. Vendors need to see a relatively quick return on their investment, before a newer technology knocks them out of the market. Developments in computing were fast in the late 1960s, and the first commercially successful personal computer, the Altair 8800, appeared in 1974. But too high a price carried obvious risks. In 1967, the cheapest teaching machine in the UK, the Tutorpack (from Packham Research Ltd), cost £7 12s (equivalent to about £126 today), but machines like these were disparagingly referred to as ‘page-turners’ (Higgins, 1983: 4). A higher-end linear programming machine cost twice this amount. Branching programme machines cost a lot more. The Mark II AutoTutor (from USI Great Britain Limited), for example, cost £31 per month (equivalent to £558), with eight reels of programmes thrown in (Goodman, 1967: 26). A lower-end branching machine, the Grundytutor, could be bought for £ 230 (worth about £4140 today).

Teaching machines (from Goodman)AutoTutor Mk II (from Goodman)

This was serious money, and any institution splashing out on teaching machines needed to be confident that they would be well used for a long period of time (Nordberg, 1965). The programmes (the software) were specific to individual machines and the content could not be updated easily. At the same time, other technological developments (cine projectors, tape recorders, record players) were arriving in classrooms, and schools found themselves having to pay for technical assistance and maintenance. The average teacher was ‘unable to avail himself fully of existing aids because, to put it bluntly, he is expected to teach for too many hours a day and simply has not the time, with all the administrative chores he is expected to perform, either to maintain equipment, to experiment with it, let alone keeping up with developments in his own and wider fields. The advent of teaching machines which can free the teacher to fulfil his role as an educator will intensify and not diminish the problem’ (Goodman, 1967: 44). Teaching machines, in short, were ‘oversold and underused’ (Cuban, 2001).

3 Research and theory

Looking back twenty years later, B. F. Skinner conceded that ‘the machines were crude, [and] the programs were untested’ (Skinner, 1986: 105). The documentary record suggests that the second part of this statement is not entirely true. Herrick (1966: 695) reported that ‘an overwhelming amount of research time has been invested in attempts to determine the relative merits of programmed instruction when compared to ‘traditional’ or ‘conventional’ methods of instruction. The results have been almost equally overwhelming in showing no significant differences’. In 1968, Kay et al (1968: 96) noted that ‘there has been a definite effort to examine programmed instruction’. A later meta-analysis of research in secondary education (Kulik et al.: 1982) confirmed that ‘Programmed Instruction did not typically raise student achievement […] nor did it make students feel more positively about the subjects they were studying’.

It was not, therefore, the case that research was not being done. It was that many people were preferring not to look at it. The same holds true for theoretical critiques. In relation to language learning, Spolsky (1966) referred to Chomsky’s (1959) rebuttal of Skinner’s arguments, adding that ‘there should be no need to rehearse these inadequacies, but as some psychologists and even applied linguists appear to ignore their existence it might be as well to remind readers of a few’. Programmed Instruction might have had a limited role to play in language learning, but vendors’ claims went further than that and some people believed them: ‘Rather than addressing themselves to limited and carefully specified FL tasks – for example the teaching of spelling, the teaching of grammatical concepts, training in pronunciation, the acquisition of limited proficiency within a restricted number of vocabulary items and grammatical features – most programmers aimed at self-sufficient courses designed to lead to near-native speaking proficiency’ (Valdman, 1968: 2).

4 Content

When learning is conceptualised as purely the acquisition of knowledge, technological optimists tend to believe that machines can convey it more effectively and more efficiently than teachers (Hof, 2018). The corollary of this is the belief that, if you get the materials right (plus the order in which they are presented and appropriate feedback), you can ‘to a great extent control and engineer the quality and quantity of learning’ (Post, 1972: 14). Learning, in other words, becomes an engineering problem, and technology is its solution.

One of the problems was that technology vendors were, first and foremost, technology specialists. Content was almost an afterthought. Materials writers needed to be familiar with the technology and, if not, they were unlikely to be employed. Writers needed to believe in the potential of the technology, so those familiar with current theory and research would clearly not fit in. The result was unsurprising. Kennedy (1967: 872) reported that ‘there are hundreds of programs now available. Many more will be published in the next few years. Watch for them. Examine them critically. They are not all of high quality’. He was being polite.

5 Motivation

As is usually the case with new technologies, there was a positive novelty effect with Programmed Instruction. And, as is always the case, the novelty effect wears off: ‘students quickly tired of, and eventually came to dislike, programmed instruction’ (McDonald et al.: 89). It could not really have been otherwise: ‘human learning and intrinsic motivation are optimized when persons experience a sense of autonomy, competence, and relatedness in their activity. Self-determination theorists have also studied factors that tend to occlude healthy functioning and motivation, including, among others, controlling environments, rewards contingent on task performance, the lack of secure connection and care by teachers, and situations that do not promote curiosity and challenge’ (McDonald et al.: 93). The demotivating experience of using these machines was particularly acute with younger and ‘less able’ students, as was noted at the time (Valdman, 1968: 9).

The unlearned lessons

I hope that you’ll now understand why I think the history of Programmed Instruction is so relevant to us today. In the words of my favourite Yogi-ism, it’s like deja vu all over again. I have quoted repeatedly from the article by McDonald et al (2005) and I would highly recommend it – available here. Hopefully, too, Audrey Watters’ forthcoming book, ‘Teaching Machines’, will appear before too long, and she will, no doubt, have much more of interest to say on this topic.

References

Chomsky N. 1959. ‘Review of Skinner’s Verbal Behavior’. Language, 35:26–58.

Cuban, L. 2001. Oversold & Underused: Computers in the Classroom. (Cambridge, MA: Harvard University Press)

Goodman, R. 1967. Programmed Learning and Teaching Machines 3rd edition. (London: English Universities Press)

Herrick, M. 1966. ‘Programmed Instruction: A critical appraisal’ The American Biology Teacher, 28 (9), 695 -698

Higgins, J. 1983. ‘Can computers teach?’ CALICO Journal, 1 (2)

Hill, L. A. 1966. Programmed English Course Student’s Book 1. (Oxford: Oxford University Press)

Hof, B. 2018. ‘From Harvard via Moscow to West Berlin: educational technology, programmed instruction and the commercialisation of learning after 1957’ History of Education, 47:4, 445-465

Howatt, A. P. R. 1969. Programmed Learning and the Language Teacher. (London: Longmans)

Kay, H., Dodd, B. & Sime, M. 1968. Teaching Machines and Programmed Instruction. (Harmondsworth: Penguin)

Kennedy, R.H. 1967. ‘Before using Programmed Instruction’ The English Journal, 56 (6), 871 – 873

Kozlowski, T. 1961. ‘Programmed Teaching’ Financial Analysts Journal, 17 / 6, 47 – 54

Kulik, C.-L., Schwalb, B. & Kulik, J. 1982. ‘Programmed Instruction in Secondary Education: A Meta-analysis of Evaluation Findings’ Journal of Educational Research, 75: 133 – 138

McDonald, J. K., Yanchar, S. C. & Osguthorpe, R.T. 2005. ‘Learning from Programmed Instruction: Examining Implications for Modern Instructional Technology’ Educational Technology Research and Development, 53 / 2, 84 – 98

Nordberg, R. B. 1965. Teaching machines-six dangers and one advantage. In J. S. Roucek (Ed.), Programmed teaching: A symposium on automation in education (pp. 1–8). (New York: Philosophical Library)

Ornstein, J. 1968. ‘Programmed Instruction and Educational Technology in the Language Field: Boon or Failure?’ The Modern Language Journal, 52 / 7, 401 – 410

Post, D. 1972. ‘Up the programmer: How to stop PI from boring learners and strangling results’. Educational Technology, 12(8), 14–1

Saettler, P. 2004. The Evolution of American Educational Technology. (Greenwich, Conn.: Information Age Publishing)

Skinner, B. F. 1986. ‘Programmed Instruction Revisited’ The Phi Delta Kappan, 68 (2), 103 – 110

Spolsky, B. 1966. ‘A psycholinguistic critique of programmed foreign language instruction’ International Review of Applied Linguistics in Language Teaching, Volume 4, Issue 1-4: 119–130

Thornbury, S. 2017. Scott Thornbury’s 30 Language Teaching Methods. (Cambridge: Cambridge University Press)

Tucker, C. 1972. ‘Programmed Dictation: An Example of the P.I. Process in the Classroom’. TESOL Quarterly, 6(1), 61-70

Valdman, A. 1968. ‘Programmed Instruction versus Guided Learning in Foreign Language Acquisition’ Die Unterrichtspraxis / Teaching German, 1 (2), 1 – 14

 

 

 

[1] Spolsky’ doctoral thesis for the University of Montreal was entitled ‘The psycholinguistic basis of programmed foreign language instruction’.

 

 

 

 

 

Book_coverIn my last post, I looked at shortcomings in edtech research, mostly from outside the world of ELT. I made a series of recommendations of ways in which such research could become more useful. In this post, I look at two very recent collections of ELT edtech research. The first of these is Digital Innovations and Research in Language Learning, edited by Mavridi and Saumell, and published this February by the Learning Technologies SIG of IATEFL. I’ll refer to it here as DIRLL. It’s available free to IATEFL LT SIG members, and can be bought for $10.97 as an ebook on Amazon (US). The second is the most recent edition (February 2020) of the Language Learning & Technology journal, which is open access and available here. I’ll refer to it here as LLTJ.

In both of these collections, the focus is not on ‘technology per se, but rather issues related to language learning and language teaching, and how they are affected or enhanced by the use of digital technologies’. However, they are very different kinds of publication. Nobody involved in the production of DIRLL got paid in any way (to the best of my knowledge) and, in keeping with its provenance from a teachers’ association, has ‘a focus on the practitioner as teacher-researcher’. Almost all of the contributing authors are university-based, but they are typically involved more in language teaching than in research. With one exception (a grant from the EU), their work was unfunded.

The triannual LLTJ is funded by two American universities and published by the University of Hawaii Press. The editors and associate editors are well-known scholars in their fields. The journal’s impact factor is high, close to the impact factor of the paywalled reCALL (published by the University of Cambridge), which is the highest-ranking journal in the field of CALL. The contributing authors are all university-based, many with a string of published articles (in prestige journals), chapters or books behind them. At least six of the studies were funded by national grant-awarding bodies.

I should begin by making clear that there was much in both collections that I found interesting. However, it was not usually the research itself that I found informative, but the literature review that preceded it. Two of the chapters in DIRLL were not really research, anyway. One was the development of a template for evaluating ICT-mediated tasks in CLIL, another was an advocacy of comics as a resource for language teaching. Both of these were new, useful and interesting to me. LLTJ included a valuable literature review of research into VR in FL learning (but no actual new research). With some exceptions in both collections, though, I felt that I would have been better off curtailing my reading after the reviews. Admittedly, there wouldn’t be much in the way of literature reviews if there were no previous research to report …

It was no surprise to see the learners who were the subjects of this research were overwhelmingly university students. In fact, only one article (about a high-school project in Israel, reported in DIRLL) was not about university students. The research areas focused on reflected this bias towards tertiary contexts: online academic reading skills, academic writing, online reflective practices in teacher training programmes, etc.

In a couple of cases, the selection of experimental subjects seemed plain bizarre. Why, if you want to find out about the extent to which Moodle use can help EAP students become better academic readers (in DIRLL), would you investigate this with a small volunteer cohort of postgraduate students of linguistics, with previous experience of using Moodle and experience of teaching? Is a less representative sample imaginable? Why, if you want to investigate the learning potential of the English File Pronunciation app (reported in LLTJ), which is clearly most appropriate for A1 – B1 levels, would you do this with a group of C1-level undergraduates following a course in phonetics as part of an English Studies programme?

More problematic, in my view, was the small sample size in many of the research projects. The Israeli virtual high school project (DIRLL), previously referred to, started out with only 11 students, but 7 dropped out, primarily, it seems, because of institutional incompetence: ‘the project was probably doomed […] to failure from the start’, according to the author. Interesting as this was as an account of how not to set up a project of this kind, it is simply impossible to draw any conclusions from 4 students about the potential of a VLE for ‘interaction, focus and self-paced learning’. The questionnaire investigating experience of and attitudes towards VR (in DIRLL) was completed by only 7 (out of 36 possible) students and 7 (out of 70+ possible) teachers. As the author acknowledges, ‘no great claims can be made’, but then goes on to note the generally ‘positive attitudes to VR’. Perhaps those who did not volunteer had different attitudes? We will never know. The study of motivational videos in tertiary education (DIRLL) started off with 15 subjects, but 5 did not complete the necessary tasks. The research into L1 use in videoconferencing (LLTJ) started off with 10 experimental subjects, all with the same L1 and similar cultural backgrounds, but there was no data available from 4 of them (because they never switched into L1). The author claims that the paper demonstrates ‘how L1 is used by language learners in videoconferencing as a social semiotic resource to support social presence’ – something which, after reading the literature review, we already knew. But the paper also demonstrates quite clearly how L1 is not used by language learners in videoconferencing as a social semiotic resource to support social presence. In all these cases, it is the participants who did not complete or the potential participants who did not want to take part that have the greatest interest for me.

Unsurprisingly, the LLTJ articles had larger sample sizes than those in DIRLL, but in both collections the length of the research was limited. The production of one motivational video (DIRLL) does not really allow us to draw any conclusions about the development of students’ critical thinking skills. Two four-week interventions do not really seem long enough to me to discover anything about learner autonomy and Moodle (DIRLL). An experiment looking at different feedback modes needs more than two written assignments to reach any conclusions about student preferences (LLTJ).

More research might well be needed to compensate for the short-term projects with small sample sizes, but I’m not convinced that this is always the case. Lacking sufficient information about the content of the technologically-mediated tools being used, I was often unable to reach any conclusions. A gamified Twitter environment was developed in one project (DIRLL), using principles derived from contemporary literature on gamification. The authors concluded that the game design ‘failed to generate interaction among students’, but without knowing a lot more about the specific details of the activity, it is impossible to say whether the problem was the principles or the particular instantiation of those principles. Another project, looking at the development of pronunciation materials for online learning (LLTJ), came to the conclusion that online pronunciation training was helpful – better than none at all. Claims are then made about the value of the method used (called ‘innovative Cued Pronunciation Readings’), but this is not compared to any other method / materials, and only a very small selection of these materials are illustrated. Basically, the reader of this research has no choice but to take things on trust. The study looking at the use of Alexa to help listening comprehension and speaking fluency (LLTJ) cannot really tell us anything about IPAs unless we know more about the particular way that Alexa is being used. Here, it seems that the students were using Alexa in an interactive storytelling exercise, but so little information is given about the exercise itself that I didn’t actually learn anything at all. The author’s own conclusion is that the results, such as they are, need to be treated with caution. Nevertheless, he adds ‘the current study illustrates that IPAs may have some value to foreign language learners’.

This brings me onto my final gripe. To be told that IPAs like Alexa may have some value to foreign language learners is to be told something that I already know. This wasn’t the only time this happened during my reading of these collections. I appreciate that research cannot always tell us something new and interesting, but a little more often would be nice. I ‘learnt’ that goal-setting plays an important role in motivation and that gamification can boost short-term motivation. I ‘learnt’ that reflective journals can take a long time for teachers to look at, and that reflective video journals are also very time-consuming. I ‘learnt’ that peer feedback can be very useful. I ‘learnt’ from two papers that intercultural difficulties may be exacerbated by online communication. I ‘learnt’ that text-to-speech software is pretty good these days. I ‘learnt’ that multimodal literacy can, most frequently, be divided up into visual and auditory forms.

With the exception of a piece about online safety issues (DIRLL), I did not once encounter anything which hinted that there may be problems in using technology. No mention of the use to which student data might be put. No mention of the costs involved (except for the observation that many students would not be happy to spend money on the English File Pronunciation app) or the cost-effectiveness of digital ‘solutions’. No consideration of the institutional (or other) pressures (or the reasons behind them) that may be applied to encourage teachers to ‘leverage’ edtech. No suggestion that a zero-tech option might actually be preferable. In both collections, the language used is invariably positive, or, at least, technology is associated with positive things: uncovering the possibilities, promoting autonomy, etc. Even if the focus of these publications is not on technology per se (although I think this claim doesn’t really stand up to close examination), it’s a little disingenuous to claim (as LLTJ does) that the interest is in how language learning and language teaching is ‘affected or enhanced by the use of digital technologies’. The reality is that the overwhelming interest is in potential enhancements, not potential negative effects.

I have deliberately not mentioned any names in referring to the articles I have discussed. I would, though, like to take my hat off to the editors of DIRLL, Sophia Mavridi and Vicky Saumell, for attempting to do something a little different. I think that Alicia Artusi and Graham Stanley’s article (DIRLL) about CPD for ‘remote’ teachers was very good and should interest the huge number of teachers working online. Chryssa Themelis and Julie-Ann Sime have kindled my interest in the potential of comics as a learning resource (DIRLL). Yu-Ju Lan’s article about VR (LLTJ) is surely the most up-to-date, go-to article on this topic. There were other pieces, or parts of pieces, that I liked, too. But, to me, it’s clear that ‘more research is needed’ … much less than (1) better and more critical research, and (2) more digestible summaries of research.

Colloquium

At the beginning of March, I’ll be going to Cambridge to take part in a Digital Learning Colloquium (for more information about the event, see here ). One of the questions that will be explored is how research might contribute to the development of digital language learning. In this, the first of two posts on the subject, I’ll be taking a broad overview of the current state of play in edtech research.

I try my best to keep up to date with research. Of the main journals, there are Language Learning and Technology, which is open access; CALICO, which offers quite a lot of open access material; and reCALL, which is the most restricted in terms of access of the three. But there is something deeply frustrating about most of this research, and this is what I want to explore in these posts. More often than not, research articles end with a call for more research. And more often than not, I find myself saying ‘Please, no, not more research like this!’

First, though, I would like to turn to a more reader-friendly source of research findings. Systematic reviews are, basically literature reviews which can save people like me from having to plough through endless papers on similar subjects, all of which contain the same (or similar) literature review in the opening sections. If only there were more of them. Others agree with me: the conclusion of one systematic review of learning and teaching with technology in higher education (Lillejord et al., 2018) was that more systematic reviews were needed.

Last year saw the publication of a systematic review of research on artificial intelligence applications in higher education (Zawacki-Richter, et al., 2019) which caught my eye. The first thing that struck me about this review was that ‘out of 2656 initially identified publications for the period between 2007 and 2018, 146 articles were included for final synthesis’. In other words, only just over 5% of the research was considered worthy of inclusion.

The review did not paint a very pretty picture of the current state of AIEd research. As the second part of the title of this review (‘Where are the educators?’) makes clear, the research, taken as a whole, showed a ‘weak connection to theoretical pedagogical perspectives’. This is not entirely surprising. As Bates (2019) has noted: ‘since AI tends to be developed by computer scientists, they tend to use models of learning based on how computers or computer networks work (since of course it will be a computer that has to operate the AI). As a result, such AI applications tend to adopt a very behaviourist model of learning: present / test / feedback.’ More generally, it is clear that technology adoption (and research) is being driven by technology enthusiasts, with insufficient expertise in education. The danger is that edtech developers ‘will simply ‘discover’ new ways to teach poorly and perpetuate erroneous ideas about teaching and learning’ (Lynch, 2017).

This, then, is the first of my checklist of things that, collectively, researchers need to do to improve the value of their work. The rest of this list is drawn from observations mostly, but not exclusively, from the authors of systematic reviews, and mostly come from reviews of general edtech research. In the next blog post, I’ll look more closely at a recent collection of ELT edtech research (Mavridi & Saumell, 2020) to see how it measures up.

1 Make sure your research is adequately informed by educational research outside the field of edtech

Unproblematised behaviourist assumptions about the nature of learning are all too frequent. References to learning styles are still fairly common. The most frequently investigated skill that is considered in the context of edtech is critical thinking (Sosa Neira, et al., 2017), but this is rarely defined and almost never problematized, despite a broad literature that questions the construct.

2 Adopt a sceptical attitude from the outset

Know your history. Decades of technological innovation in education have shown precious little in the way of educational gains and, more than anything else, have taught us that we need to be sceptical from the outset. ‘Enthusiasm and praise that are directed towards ‘virtual education, ‘school 2.0’, ‘e-learning and the like’ (Selwyn, 2014: vii) are indications that the lessons of the past have not been sufficiently absorbed (Levy, 2016: 102). The phrase ‘exciting potential’, for example, should be banned from all edtech research. See, for example, a ‘state-of-the-art analysis of chatbots in education’ (Winkler & Söllner, 2018), which has nothing to conclude but ‘exciting potential’. Potential is fine (indeed, it is perhaps the only thing that research can unambiguously demonstrate – see section 3 below), but can we try to be a little more grown-up about things?

3 Know what you are measuring

Measuring learning outcomes is tricky, to say the least, but it’s understandable that researchers should try to focus on them. Unfortunately, ‘the vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them’ (Lai & Bower, 2019). Metrics such as student grades are hard to interpret, not least because of the large number of variables and the danger of many things being conflated in one score. Equally, or possibly even more, problematic, are self-reporting measures which are rarely robust. It seems that surveys are the most widely used instrument in qualitative research (Sosa Neira, et al., 2017), but these will tell us little or nothing when used for short-term interventions (see point 5 below).

4 Ensure that the sample size is big enough to mean something

In most of the research into digital technology in education that was analysed in a literature review carried out for the Scottish government (ICF Consulting Services Ltd, 2015), there were only ‘small numbers of learners or teachers or schools’.

5 Privilege longitudinal studies over short-term projects

The Scottish government literature review (ICF Consulting Services Ltd, 2015), also noted that ‘most studies that attempt to measure any outcomes focus on short and medium term outcomes’. The fact that the use of a particular technology has some sort of impact over the short or medium term tells us very little of value. Unless there is very good reason to suspect the contrary, we should assume that it is a novelty effect that has been captured (Levy, 2016: 102).

6 Don’t forget the content

The starting point of much edtech research is the technology, but most edtech, whether it’s a flashcard app or a full-blown Moodle course, has content. Research reports rarely give details of this content, assuming perhaps that it’s just fine, and all that’s needed is a little tech to ‘present learners with the ‘right’ content at the ‘right’ time’ (Lynch, 2017). It’s a foolish assumption. Take a random educational app from the Play Store, a random MOOC or whatever, and the chances are you’ll find it’s crap.

7 Avoid anecdotal accounts of technology use in quasi-experiments as the basis of a ‘research article’

Control (i.e technology-free) groups may not always be possible but without them, we’re unlikely to learn much from a single study. What would, however, be extremely useful would be a large, collated collection of such action-research projects, using the same or similar technology, in a variety of settings. There is a marked absence of this kind of work.

8 Enough already of higher education contexts

Researchers typically work in universities where they have captive students who they can carry out research on. But we have a problem here. The systematic review of Lundin et al (2018), for example, found that ‘studies on flipped classrooms are dominated by studies in the higher education sector’ (besides lacking anchors in learning theory or instructional design). With some urgency, primary and secondary contexts need to be investigated in more detail, not just regarding flipped learning.

9 Be critical

Very little edtech research considers the downsides of edtech adoption. Online safety, privacy and data security are hardly peripheral issues, especially with younger learners. Ignoring them won’t make them go away.

More research?

So do we need more research? For me, two things stand out. We might benefit more from, firstly, a different kind of research, and, secondly, more syntheses of the work that has already been done. Although I will probably continue to dip into the pot-pourri of articles published in the main CALL journals, I’m looking forward to a change at the CALICO journal. From September of this year, one issue a year will be thematic, with a lead article written by established researchers which will ‘first discuss in broad terms what has been accomplished in the relevant subfield of CALL. It should then outline which questions have been answered to our satisfaction and what evidence there is to support these conclusions. Finally, this article should pose a “soft” research agenda that can guide researchers interested in pursuing empirical work in this area’. This will be followed by two or three empirical pieces that ‘specifically reflect the research agenda, methodologies, and other suggestions laid out in the lead article’.

But I think I’ll still have a soft spot for some of the other journals that are coyer about their impact factor and that can be freely accessed. How else would I discover (it would be too mean to give the references here) that ‘the effective use of new technologies improves learners’ language learning skills’? Presumably, the ineffective use of new technologies has the opposite effect? Or that ‘the application of modern technology represents a significant advance in contemporary English language teaching methods’?

References

Bates, A. W. (2019). Teaching in a Digital Age Second Edition. Vancouver, B.C.: Tony Bates Associates Ltd. Retrieved from https://pressbooks.bccampus.ca/teachinginadigitalagev2/

ICF Consulting Services Ltd (2015). Literature Review on the Impact of Digital Technology on Learning and Teaching. Edinburgh: The Scottish Government. https://dera.ioe.ac.uk/24843/1/00489224.pdf

Lai, J.W.M. & Bower, M. (2019). How is the use of technology in education evaluated? A systematic review. Computers & Education, 133(1), 27-42. Elsevier Ltd. Retrieved January 14, 2020 from https://www.learntechlib.org/p/207137/

Levy, M. 2016. Researching in language learning and technology. In Farr, F. & Murray, L. (Eds.) The Routledge Handbook of Language Learning and Technology. Abingdon, Oxon.: Routledge. pp.101 – 114

Lillejord S., Børte K., Nesje K. & Ruud E. (2018). Learning and teaching with technology in higher education – a systematic review. Oslo: Knowledge Centre for Education https://www.forskningsradet.no/siteassets/publikasjoner/1254035532334.pdf

Lundin, M., Bergviken Rensfeldt, A., Hillman, T. et al. (2018). Higher education dominance and siloed knowledge: a systematic review of flipped classroom research. International Journal of Educational Technology in Higher Education 15, 20 (2018) doi:10.1186/s41239-018-0101-6

Lynch, J. (2017). How AI Will Destroy Education. Medium, November 13, 2017. https://buzzrobot.com/how-ai-will-destroy-education-20053b7b88a6

Mavridi, S. & Saumell, V. (Eds.) (2020). Digital Innovations and Research in Language Learning. Faversham, Kent: IATEFL

Selwyn, N. (2014). Distrusting Educational Technology. New York: Routledge

Sosa Neira, E. A., Salinas, J. and de Benito Crosetti, B. (2017). Emerging Technologies (ETs) in Education: A Systematic Review of the Literature Published between 2006 and 2016. International Journal of Emerging Technologies in Education, 12 (5). https://online-journals.org/index.php/i-jet/article/view/6939

Winkler, R. & Söllner, M. (2018): Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis. In: Academy of Management Annual Meeting (AOM). Chicago, USA. https://www.alexandria.unisg.ch/254848/1/JML_699.pdf

Zawacki-Richter, O., Bond, M., Marin, V. I. And Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education 2019