Generative AI and ELT Materials

I must begin by apologizing for my last, flippant blog post where I spun a tale about using generative AI to produce ‘teacher development content’ and getting rid of teacher trainers. I had just watched a webinar by Giuseppe Tomasello of edugo.ai, ‘Harnessing Generative AI to Supercharge Language Education’, and it felt as if much of the script of this webinar had been generated by ChatGPT: ‘write a relentlessly positive product pitch for a language teaching platform in the style of a typical edtech vendor’.

In the webinar, Tomasello talked about using GPT-3 to generate texts for language learners. Yes, it’s doable, the results are factually and linguistically accurate, but dull in the extreme (see Steve Smith’s experiment with some texts for French learners). André Hedlund summarises: ‘limited by a rigid structure … no big differences in style or linguistic devices. They follow a recipe, a formula.’ Much like many coursebook / courseware texts, in other words.

More interesting texts can be generated with GPT-3 when the prompts are crafted in careful detail, but this requires creative human imagination. Crafting such prompts requires practice, trial and error, and it requires knowledge of the intended readers.

AI can be used to generate dull texts at a certain level (B1, B2, C1, etc.), but reliability is a problem. So many variables (starting with familiarity with the subject matter and the learner’s first language) impact on levels of reading difficulty that automation can never satisfactorily resolve this challenge.

However, the interesting question is not whether we can quickly generate courseware-like texts using GPT-3, but whether we should even want to. What is the point of using such texts? Giuseppe Tomasello’s demonstration made it clear that the way these texts should be exploited is by using (automatically generating) a series of comprehension questions and a list of key words which can also be (automatically) converted into a variety of practice tasks (flashcards, gapfills, etc.). Like courseware texts, then, these texts are language objects (TALOs), to be tested for comprehension and mined for items to be deliberately learnt. They are not, in any meaningful way, texts as vehicles of information (TAVIs) – see here for more about TALOs and TAVIs.

Comprehension testing is a standard feature of language teaching, but there are good reasons to believe that it will do little, if anything, to improve a learner’s reading skills (see, for example, Swan & Walter, 2017; Grabe, W. & Yamashita, J., 2022) or language competence. It has nothing to do with the kind of extensive reading (see, for example, Leather & Uden, 2021) that is known to be such a powerful force in language acquisition. This use of text is all about deliberate language study. We are talking here about a synthetic approach to language learning.

The problem, of course, is that deliberate language study is far from uncontested. Synthetic approaches wrongly assume that ‘the explicit teaching of declarative knowledge (knowledge about the L2) will lead to the procedural knowledge learners need in order to successfully use the L2 for communicative purpose’ (see Geoff Jordan on this topic). Even if it were true that some explicit teaching was of value, it’s widely agreed that explicit teaching should not form the core of a language learning syllabus. The edugo.ai product, however, is basically all about explicit teaching.

Judging from the edugo.ai presentation on YouTube, the platform offers a wide range of true / false, multiple choice questions, gapfills, dictations and so on, all of which can be gamified. The ‘methodology’ is called ‘Flip and Flop the Classroom’. In this approach, learners do some self-study (presumably of some pre-selected discrete language item(s), practise it in the synchronous lesson, and then have more self-study where this language is reviewed. In the ‘flop’ section, the learner’s spoken contribution to the live lesson is recorded, transcribed and analysed by the system which identifies aspects of the learner’s language which can be improved through personalized practice.

The focus is squarely on accuracy, and the importance of accuracy is underlined by the presenter’s observation that Communicative Language Teaching does not focus enough on accuracy. Accuracy is also to the fore in another task type, strangely called ‘chunking’ (see below). Apparently, this will lead to fluency: ‘The goal of this template is that they can repeat it a lot of times and reach fluency at the end’.

The YouTube presentation is called ‘How to structure your language course using popular pedagogical approaches’ and suggests that you can mix’n’match bits from ‘Grammar Translation’, ‘Direct Method’ and ‘CLT’. Sorry, Giuseppe, you can mix’n’match all you like, but you can’t be methodologically agnostic. This is a synthetic approach all the way down the line. As such, it’s not going to supercharge language education. It’s basically more of what we already have too much of.

Let’s assume, though, for a moment that what we really want is a kind of supercharged, personalizable, quickly generated combination of vocabulary flashcards and ‘English Grammar in Use’ loaded with TALOs. How does this product stand up? We’ll need to consider two related questions: (1) its reliability, and (2) how time-saving it actually is.

As far as I can judge from the YouTube presentation, reliability is not exactly optimal. There’s the automated key word identifier that identified ‘Denise’, ‘the’ and ‘we’ as ‘key words’ in an extract of learner-produced text (‘Hello, my name is Denise and I founded my makeup company in 2021. We produce skin care products using 100% natural ingredients.’). There’s the automated multiple choice / translation generator which tests your understanding of ‘Denise’ (see below), and there’s the voice recognition software which transcribed ‘it cost’ as ‘they cost’.

In the more recent ‘webinar’ (i.e commercial presentation) that has not yet been uploaded to YouTube, participants identified a number of other bloopers. In short, reliability is an issue. This shouldn’t surprise anyone. Automation of some of these tasks is extremely difficult (see my post about the automated generation of vocabulary learning materials). Perhaps impossible … but how much error is acceptable?

edugo.ai does not sell content: they sell a platform for content-generation, course-creation and selling. Putative clients are institutions wanting to produce and sell learning content of the synthetic kind. The problem with a lack of reliability, any lack of reliability, is that you immediately need skilled staff to work with the platform, to check for error, to edit, to improve on the inevitable limitations of the AI (starting, perhaps, with the dull texts it has generated). It is disingenuous to suggest that anyone can do this without substantial training / supervision. Generative AI only offers a time-saving route, which does not sacrifice reliability, if a skilled and experienced writer is working with it.

edugo.ai is a young start-up that raised $345K in first round funding in September of last year. The various technologies they are using are expensive, and a lot more funding will be needed to make the improvements and additions (such as spaced repetition) that are necessary. In both presentations, there was lots of talk that the platform will be able to do this and will be able to do that. For the moment, though, nothing has been proved, and my suspicion is that some of the problems they are trying to solve do not have technological solutions. First of all, they’ll need a better understanding of what these problems are, and, for that, there has to be a coherent and credible theory of second language acquisition. There are all sorts of good uses that GPT-3 / AI could be put to in language teaching. I doubt this is one of them.

To wrap up, here’s a little question. What are the chances that edugo.ai’s claims that the product will lead to ‘+50% student engagement’ and ‘5X Faster creating language courses’ were also generated by GPT-3?

References

Grabe, W. & Yamashita, J. (2022) Reading in a Second Language 2^nd edition. New York: Cambridge University Press

Leather, S. & Uden, J. (2021) Extensive Reading. New York: Routledge

Swan, M. & Walter, C. (2017) Misunderstanding comprehension. ELT Journal, 71 (2): 228 – 236

Coaching questions

I came across this course description the other day. The course motto is ‘Honour the Learner’.

Discussions and assigned work covered in the curriculum includes multiple intelligence, positive deviance, brain science during stressful situations, how people learn, PBL, Failing Forward, Bloom’s Taxonomy, de-escalation, cognitive load and courageous conversations, with the interwoven golden threads of leadership theory and emotional intelligence.

I love the idea of positive deviance and the interwoven golden threads!

The description is of a coaching course for police professionals in Ontario. ‘The intense, week-long program follows a robust agenda that embraces a modified PBL-approach rather than a traditional, lecture-based format’. Participants write a personal mission statement and, in the process, they have ‘the opportunity to reflect on their own contributions and commitment to the effective, efficient and values-based delivery of policing’. As opposed to violence-based, for example.

The course began in 2017 and has been considered a success. But what sort of real impact has it had? And how has it adapted to going online? Is there anything people involved in language teaching can learn about coaching from the Ontario Police approach?

‘Coach’ (as in ‘life coach’) is, of course, a slightly tricky word. There are people who think it reflects an important reality in our lives, and others who struggle to take the word seriously. The former will write blog posts or give talks about coaching, the latter probably won’t read them.

The cause of coaching is not really helped by the lack of any broadly recognised certification. By people who think they can charge more just by claiming they are ‘coaches’. In the language teaching world, as elsewhere, some coaches are attempting to set up little trademarked enclaves, sprinkled with acronyms, pyramids and lightbulb illustrations, in order to differentiate just anyone who claims to be a coach from ‘proper coaches’ with certificates.

If you want to be certified, it’s not always easy to choose from the possibilities out there. I have recently read ‘Neurolanguage Coaching: Brain Friendly Language Learning’ by Rachel Paling. I’m normally very suspicious of anything with a ‘brain-friendly’ label. Worse still, I collocate ‘neuro’ more strongly with ‘bollox’ than with ‘language’. So it was an intriguing read. Without wanting to give too much away, I can tell you that it’s all to do with motivation (the limbic system, no less), being non-judgemental of the coachee, and breaking down language into manageable chunks: ‘from present tenses to future, to conditional etc.’ The key, continues the author, ‘is to start with grammar that gets the learner speaking the fastest. In English this would necessarily be the verbs ‘to be’, ‘to have’ and the impersonal ‘there is’ and ‘there are’, and the formulation of questions and negatives of these. Then it would be a step-by-step building the language: introducing present continuous as the real present and the present simple as the facts and habits tense’. (Paling, 2017: 83) And, hey pesto, it’s as simple as that, when you’ve mastered the necessary skills. To get a firmer understanding of this trademarked approach to language coaching, you’d probably have to follow one of the many courses that are certified by Efficient Language Coaching® (online, prices on request).

The International Language Coaching Association (https://internationallanguagecoaching.com/) would seem to be a competitor to Neurolanguage Coaching®. They, too, run courses: $450 for the Foundation Course, but the price includes ‘12 month membership in the pioneering ILCA community’. Rather a lot for 4 live sessions and 4 supplementary study videos. I suppose if you were really keen, you could do both. But times are tight, and instead of splashing out, I invested in ‘Coaching for Language Learning’ by Emmanuelle Betham, another self-published book (only $18.73 on Kindle). The author’s skills, according to LinkedIn, include, besides life coaching, Neuro Linguistic Programming, Mindfulness, and Clean Language. Again, I was intrigued.

As with ‘Neurolanguage Coaching’, there were quite a few slogans in ‘Coaching for Language Learning (CFLL)’, not a lot of awareness of SLA research (the work of Krashen seems to be the limit of the reading informing the CFLL approach), and a crude stereotyping of teachers and trainers as ‘informative’, and coaches as ‘transformative’. But, unlike Rachel Paling, Emmanuelle Betham seems to think that grammar instruction (even when delivered through coaching questions) is less helpful. Instead, learners need to learn to think in English, and the best way of doing that is by having mindful conversations with their coach. Nevertheless, there is a section on ‘coaching grammar rules’. Some standard teaching activities are also recommended. Running dictations can be used. The English language is not melodious like Roman languages. If you’re struggling to make sense of all this, you’re not alone. CFLL, you see, is ‘a new paradigm that needs to be appreciated in practice, as defined in its context, and which cannot be comprehended within the wisdom of previous hypotheses’ (Betham, 2018 – 2020: 45). What’s more, CFLL is primarily interested in ‘what works’: the concepts in the book ‘are not to be agreed or disagreed with, they are just examples of visualizations that have worked well for some learners in practice before’ (Betham, 34). You see, ‘truth is relative […] the rationale for our new paradigm, CFLL rests on the assumption that we are free to interpret and construct our own truth’ (Betham, 35). It’s all heady stuff.

Coming down to ground, the most useful thing I’ve read about language teaching and coaching is ‘From English Teacher to Learner Coach’ by Daniel Barber and Duncan Foord (2014). If coaching is ultimately about helping people to become more autonomous … in combination with education, it’s all about learner autonomy. At least, that was the message I took from this book. It’s very cheap, and it also comes in a ‘Student’s Book’ version, which is very handy if you want your students to try out a pile of suggestions for becoming more autonomous learners. I thought the suggestions were good and plentiful, but it all seemed to be more about learner autonomy than about language coaching. Perhaps it’s not unreasonable to claim they are the same thing?

But could AI do away with real-life coaches altogether? I’m very interested in the idea of coaching bots. How easy / hard would it be to fool people that they were chatting with a real-life coach, rather than an algorithm? It wouldn’t be too hard to load up a corpus of coaching conversational strategies, hedges and questions and automate a linkage between key words produced by the coachee (e.g. stress, frustration, work, COVID, resisting arrest) and a range of conversation prompts. Computers are getting better and better at doing empathy. How long before a coachbot passes the Turing test? Maybe this is what the Ontario Police are toying with?

There are, of course, plenty of coaching apps out there. Things like HabitBull, Coach.me, Symbifly, Mindsail … Mostly for sport, health and getting rich. They’re cheaper than paying for a coach, but the bonding experience is a bit different. Would they work with language learners or teachers? Somehow, I doubt it, but you never know. Do gamification and coaching fit together?

I’m not sure what I was hoping to learn from my exploration of coaching. I’m not sure what questions I was looking for answers to. Perhaps I needed a coach to guide me? But I have learned from Emmanuelle Betham that learning is a seed and I am a gardener. Or something like that.

Barber, D. & Foord, D. (2014) From English Teacher to Learner Coach.

Betham, E. (2018 – 2020) An Introduction to Coaching for Language Learning.

Paling, R. (2017) Neurolanguage Coaching: Brain Friendly Language Learning.

Personalized learning at IATEFL

It’s international ELT conference season again, with TESOL Chicago having just come to a close and IATEFL Brighton soon to start. I decided to take a look at how the subject of personalized learning will be covered at the second of these. Taking the conference programme , I trawled through looking for references to my topic.

My first question was: how do conference presenters feel about personalised learning? One way of finding out is by looking at the adjectives that are found in close proximity. This is what you get.

The overall enthusiasm is even clearer when the contexts are looked at more closely. Here are a few examples:

• inspiring assessment, personalising learning
• personalised training can contribute to professionalism and […] spark ideas for teacher trainers
• a personalised educational experience that ultimately improves learner outcomes
• personalised teacher development: is it achievable?

Particularly striking is the complete absence of anything that suggests that personalized learning might not be a ‘good thing’. The assumption throughout is that personalized learning is desirable and the only question that is asked is how it can be achieved. Unfortunately (and however much we might like to believe that it is a ‘good thing’), there is a serious lack of research evidence which demonstrates that this is the case. I have written about this here and here and here . For a useful summary of the current situation, see Benjamin Riley’s article where he writes that ‘it seems wise to ask what evidence we presently have that personalized learning works. Answer: Virtually none. One remarkable aspect of the personalized-learning craze is how quickly the concept has spread despite the almost total absence of rigorous research in support of it, at least thus far.’

Given that personalized learning can mean so many things and given the fact that people do not have space to define their terms in their conference abstracts, it is interesting to see what other aspects of language learning / teaching it is associated with. The four main areas are as follows (in alphabetical order):

• assessment (especially formative assessment) / learning outcomes
• continuous professional development
• learner autonomy
• technology / blended learning

The IATEFL TD SIG would appear to be one of the main promoters of personalized learning (or personalized teacher development) with a one-day pre-conference event entitled ‘Personalised teacher development – is it achievable?’ and a ‘showcase’ forum entitled ‘Forum on Effective & personalised: the holy grail of CPD’. Amusingly (but coincidentally, I suppose), the forum takes place in the ‘Cambridge room’ (see below).

I can understand why the SIG organisers may have chosen this focus. It’s something of a hot topic, and getting hotter. For example:

• Cambridge University Press has identified personalization as one of the ‘six key principles of effective teacher development programmes’ and is offering tailor-made teacher development programmes for institutions.
• NILE and Macmillan recently launched a partnership whose brief is to ‘curate personalised professional development with an appropriate mix of ‘formal’ and ‘informal’ learning delivered online, blended and face to face’.
• Pearson has developed the Pearson’s Teacher Development Interactive (TDI) – ‘an interactive online course to train and certify teachers to deliver effective instruction in English as a foreign language […] You can complete each module on your own time, at your own pace from anywhere you have access to the internet.’

These examples do not, of course, provide any explanation for why personalized learning is a hot topic, but the answer to that is simple. Money. Billions and billions, and if you want a breakdown, have a look at the appendix of Monica Bulger’s report, ‘Personalized Learning: The Conversations We’re Not Having’ . Starting with Microsoft and the Gates Foundation plus Facebook and the Chan / Zuckerberg Foundation, dozens of venture philanthropists have thrown unimaginable sums of money at the idea of personalized learning. They have backed up their cash with powerful lobbying and their message has got through. Consent has been successfully manufactured.

One of the most significant players in this field is Pearson, who have long been one of the most visible promoters of personalized learning (see the screen capture). At IATEFL, two of the ten conference abstracts which include the word ‘personalized’ are directly sponsored by Pearson. Pearson actually have ten presentations they have directly sponsored or are very closely associated with. Many of these do not refer to personalized learning in the abstract, but would presumably do so in the presentations themselves. There is, for example, a report on a professional development programme in Brazil using TDI (see above). There are two talks about the GSE, described as a tool ‘used to provide a personalised view of students’ language’. The marketing intent is clear: Pearson is to be associated with personalized learning (which is, in turn, associated with a variety of tech tools) – they even have a VP of data analytics, data science and personalized learning.

But the direct funding of the message is probably less important these days than the reinforcement, by those with no vested interests, of the set of beliefs, the ideology, which underpin the selling of personalized learning products. According to this script, personalized learning can promote creativity, empowerment, inclusiveness and preparedness for the real world of work. It sets itself up in opposition to lockstep and factory models of education, and sets learners free as consumers in a world of educational choice. It is a message with which it is hard for many of us to disagree.

It is also a marvellous example of propaganda, of the way that consent is manufactured. (If you haven’t read it yet, it’s probably time to read Herman and Chomsky’s ‘Manufacturing Consent: The Political Economy of the Mass Media’.) An excellent account of the way that consent for personalized learning has been manufactured can be found at Benjamin Doxtdator’s blog .

So, a hot topic it is, and its multiple inclusion in the conference programme will no doubt be welcomed by those who are selling ‘personalized’ products. It must be very satisfying to see how normalised the term has become, how it’s no longer necessary to spend too much on promoting the idea, how it’s so associated with technology, (formative) assessment, autonomy and teacher development … since others are doing it for you.

Privacy

I mentioned the issue of privacy very briefly in Part 9 of the ‘Guide’, and it seems appropriate to take a more detailed look.

Adaptive learning needs big data. Without the big data, there is nothing for the algorithms to work on, and the bigger the data set, the better the software can work. Adaptive language learning will be delivered via a platform, and the data that is generated by the language learner’s interaction with the English language program on the platform is likely to be only one, very small, part of the data that the system will store and analyse. Full adaptivity requires a psychometric profile for each student.

It would make sense, then, to aggregate as much data as possible in one place. Besides the practical value in massively combining different data sources (in order to enhance the usefulness of the personalized learning pathways), such a move would possibly save educational authorities substantial amounts of money and allow educational technology companies to mine the rich goldmine of student data, along with the standardised platform specifications, to design their products.

And so it has come to pass. The Gates Foundation (yes, them again) provided most of the $100 million funding. A division of Murdoch’s News Corp built the infrastructure. Once everything was ready, a non-profit organization called inBloom was set up to run the thing. The inBloom platform is open source and the database was initially free, although this will change. Preliminary agreements were made with 7 US districts and involved millions of children. The data includes ‘students’ names, birthdates, addresses, social security numbers, grades, test scores, disability status, attendance, and other confidential information’ (Ravitch, D. ‘Reign of Error’ NY: Knopf, 2013, p. 235-236). Under federal law, this information can be ‘shared’ with private companies selling educational technology and services.

The edtech world rejoiced. ‘This is going to be a huge win for us’, said one educational software provider; ‘it’s a godsend for us,’ said another. Others are not so happy. If the technology actually works, if it can radically transform education and ‘produce game-changing outcomes’ (as its proponents claim so often), the price to be paid might just conceivably be worth paying. But the price is high and the research is not there yet. The price is privacy.

The problem is simple. InBloom itself acknowledges that it ‘cannot guarantee the security of the information stored… or that the information will not be intercepted when it is being transmitted.’ Experience has already shown us that organisations as diverse as the CIA or the British health service cannot protect their data. Hackers like a good challenge. So do businesses.

The anti-privatization (and, by extension, the anti-adaptivity) lobby in the US has found an issue which is resonating with electors (and parents). These dissenting voices are led by Class Size Matters, and their voice is being heard. Of the original partners of inBloom, only one is now left. The others have all pulled out, mostly because of concerns about privacy, although the remaining partner, New York, involves personal data on 2.7 million students, which can be shared without any parental notification or consent.

This might seem like a victory for the anti-privatization / anti-adaptivity lobby, but it is likely to be only temporary. There are plenty of other companies that have their eyes on the data-mining opportunities that will be coming their way, and Obama’s ‘Race to the Top’ program means that the inBloom controversy will be only a temporary setback. ‘The reality is that it’s going to be done. It’s not going to be a little part. It’s going to be a big part. And it’s going to be put in place partly because it’s going to be less expensive than doing professional development,’ says Eva Baker of the Center for the Study of Evaluation at UCLA.

It is in this light that the debate about adaptive learning becomes hugely significant. Class Size Matters, the odd academic like Neil Selwyn or the occasional blogger like myself will not be able to reverse a trend with seemingly unstoppable momentum. But we are, collectively, in a position to influence the way these changes will take place.

If you want to find out more, check out the inBloom and Class Size Matters links. And you might like to read more from the news reports which I have used for information in this post. Of these, the second was originally published by Scientific American (owned by Macmillan, one of the leading players in ELT adaptive learning). The third and fourth are from Education Week, which is funded in part by the Gates Foundation.

http://www.reuters.com/article/2013/03/03/us-education-database-idUSBRE92204W20130303

http://www.salon.com/2013/08/01/big_data_puts_teachers_out_of_work_partner/

http://www.edweek.org/ew/articles/2014/01/08/15inbloom_ep.h33.html

http://blogs.edweek.org/edweek/marketplacek12/2013/12/new_york_battle_over_inBloom_data_privacy_heading_to_court.html