Posts Tagged ‘eltjam’


Posted: December 1, 2016 in ed tech
Tags: , , , , , , , ,


Here’s a conversation I had recently with Cleverbot.


User (me): Hi Cleverbot. What are your views on the future of chatbots in language learning?

Cleverbot: Are you talking to yourself?

User: No, I’m asking you a question.

Cleverbot: Then what’s the question again? I forgot.

User: What are your views on the future of chatbots in language learning?

Cleverbot: Don’t you want me to finish my story?

User: No, I want you to answer my question.

Cleverbot: What is your question?

User: Do you think there is a future for chatbots in language education?

Cleverbot: Sure, if you are talking about the moon.

aliceI also tried to have a conversation with Alice, the avatar from EFL Classroom Bot, listed by Larry Ferlazzo as one of ‘the best online chatbots for practicing English’. I didn’t get any more sense out of her than out of Cleverbot.

Chatbots, apparently, are the next big thing. Again. David Mattin, head of trends and insights at, writes (in the September 2016 issue of ‘Business Life’) that ‘the chatbot revolution is coming’ and that chatbots are a step towards the dream of an interface between user and technology that is so intuitive that the interface ‘simply fades away’. Chatbots have been around for some time. Remember Clippy – the Microsoft Office bot in the late 1990s – which you had to disable in order to stop yourself punching your computer screen? Since then, bots have become ubiquitous. There have been problems, such as Microsoft’s Tay bot that had to be taken down after sixteen hours earlier this year, when, after interacting with other Twitter users, it developed into an abusive Nazi. But chatbots aren’t going away and you’ve probably interacted with one to book a taxi, order food or attempt to talk to your bank. In September this year, the Guardian described them as ‘the talk of the town’ and ‘hot property in Silicon Valley’.

The real interest in chatbots is not, however, in the ‘exciting interface’ possibilities (both user interface and user experience remain pretty crude), but in the way that they are leaner, sit comfortably with the things we actually do on a phone and the fact that they offer a way of cutting out the high fees that developers have to pay to app stores . After so many start-up failures, chatbots offer a glimmer of financial hope to developers.

It’s no surprise, of course, to find the world of English language teaching beginning to sit up and take notice of this technology. A 2012 article by Ben Lehtinen in PeerSpectives enthuses about the possibilities in English language learning and reports the positive feedback of the author’s own students. ELTJam, so often so quick off the mark, developed an ELT Bot over the course of a hackathon weekend in March this year. Disappointingly, it wasn’t really a bot – more a case of humans pretending to be a bot pretending to be humans – but it probably served its exploratory purpose. duolingoAnd a few months ago Duolingo began incorporating bots. These are currently only available for French, Spanish and German learners in the iPhone app, so I haven’t been able to try it out and evaluate it. According to an infomercial in TechCrunch, ‘to make talking to the bots a bit more compelling, the company tried to give its different bots a bit of personality. There’s Chef Robert, Renee the Driver and Officer Ada, for example. They will react differently to your answers (and correct you as necessary), but for the most part, the idea here is to mimic a real conversation. These bots also allow for a degree of flexibility in your answers that most language-learning software simply isn’t designed for. There are plenty of ways to greet somebody, for example, but most services will often only accept a single answer. When you’re totally stumped for words, though, Duolingo offers a ‘help my reply’ button with a few suggested answers.’ In the last twelve months or so, Duolingo has considerably improved its ability to recognize multiple correct ways of expressing a particular idea, and its ability to recognise alternative answers to its translation tasks. However, I’m highly sceptical about its ability to mimic a real conversation any better than Cleverbot or Alice the EFL Bot, or its ability to provide systematically useful corrections.

My reasons lie in the current limitations of AI and NLP (Natural Language Processing). In a nutshell, we simply don’t know how to build a machine that can truly understand human language. Limited exchanges in restricted domains can be done pretty well (such as the early chatbot that did a good job of simulating an encounter with an evasive therapist, or, more recently ordering a taco and having a meaningless, but flirty conversation with a bot), but despite recent advances in semantic computing, we’re a long way from anything that can mimic a real conversation. As Audrey Watters puts it, we’re not even close.

When it comes to identifying language errors made by language learners, we’re not really much better off. Apps like Grammarly are not bad at identifying grammatical errors (but not good enough to be reliable), but pretty hopeless at dealing with lexical appropriacy. Much more reliable feedback to learners can be offered when the software is trained on particular topics and text types. Write & Improve does this with a relatively small selection of Cambridge English examination tasks, but a free conversation ….? Forget it.

So, how might chatbots be incorporated into language teaching / learning? A blog post from December 2015 entitled AI-powered chatbots and the future of language learning suggests one plausible possibility. Using an existing messenger service, such as WhatsApp or Telegram, an adaptive chatbot would send tasks (such as participation in a conversation thread with a predetermined topic, register, etc., or pronunciation practice or translation exercises) to a learner, provide feedback and record the work for later recycling. At the same time, the bot could send out reminders of work that needs to be done or administrative tasks that must be completed.

Kat Robb has written a very practical article about using instant messaging in English language classrooms. Her ideas are interesting (although I find the idea of students in a F2F classroom messaging each other slightly bizarre) and it’s easy to imagine ways in which her activities might be augmented with chatbot interventions. The Write & Improve app, mentioned above, could deploy a chatbot interface to give feedback instead of the flat (and, in my opinion, perfectly adequate) pop-up boxes currently in use. Come to think of it, more or less any digital language learning tool could be pimped up with a bot. Countless revisions can be envisioned.

But the overwhelming question is: would it be worth it? Bots are not likely, any time soon, to revolutionise language learning. What they might just do, however, is help to further reduce language teaching to a series of ‘mechanical and scripted gestures’. More certain is that a lot of money will be thrown down the post-truth edtech drain. Then, in the not too distant future, this latest piece of edtech will fall into the trough of disillusionment, to be replaced by the latest latest thing.



Photo 01-07-2015 16 23 47Flovoco is a vocabulary app developed by ELTjam. This review was written by Mike Harrison and first appeared on his blog. After the review, I’ve added a response by Jo Sayers (of ELTjam). Many thanks to Mike for allowing me to repost his review, and thanks to Jo for allowing me to repost his comment.

I first became aware of this mobile application around July 2014, when ELTjam first posted about their product development. There was a fair amount of heat – the Edtech-meets-ELT specialists Nick Robinson, Laurie Harrison, Tim Gifford, and newbie at the time Jo Sayers, pitched us their product-in-waiting. A year or so later, I saw a presentation at IATEFL in Manchester where Jo talked about reviewing educational and ELT apps. And so to this review of an ELTjam ELT app.

This review follows a model presented by Jo, reviewing the app across four categories: pedagogy and methodology; instructional design; user experience; cost and access.

The version of the app I’m reviewing is 1.0 in the Apple AppStore, and I’m working on an iPhone 5S.

Note – the app is only available in Spanish and English at the moment.

Initial impressions

But first, it is almost impossible to comment on an app without some ‘at first glance’ context, so here it is. I initially thought that ELTjam’s marketing was a little audacious – creating a landing page for an app that didn’t exist – but following conversations recently I now realise this is fairly common practice, and a good way to generate leads for email lists and such. I do still balk a little at the audacity of fanfare around the app (more on that to come under ‘pedagogy and methodology’). On opening the app for the first time, I was impressed by how slick it looked and felt (and more on that under ‘user experience’!) – I have to hold my hands up and say that I have yet to be blown away by an educational app, whether designed for ELT or more general educational fields. Let’s see whether this will change.

untitledhimPedagogy and methodology


The overarching principle behind Flovoco is that words are important. And that in order to get better at using a language, the best thing to do is bump up what you know about the most common words in that language. So far , so good. Flovoco aims to help learners learn more about words by presenting them with a number of activities focusing on different information about a given word – its meaning, pronunciation, how it might be used in a phrase, and then again with words deemed to be confusing. These different areas are named as Translation, Listening, Usage, and Confused Words – and they are presented as ‘levels’, Translation being the first (easiest?) level and Confused Words being the fourth level (and most difficult of the four?). This all seems fairly logical.

But then it all starts to go a bit wrong. In the Translation activity, words are presented with possible matches that aren’t even the same part of speech. Alright, wrong answers are clearly marked with a red light and wrong buzzer sound – but this is behaviourism at best. At worst this may actually confuse learners. Not to mention that this only works by looking at a single meaning of a word – so there won’t be many colloquial meanings and translations included here.

Photo 30-06-2015 18 32 10 eggsThe Listening level does the same thing. Words that are presented alongside each other are completely different. Sometimes phrases are presented, but the audio only comprises of one word in the phrase. For example, you might see ‘pay for something’ but only hear ‘play’. There may be less potential for confusion among words here, but having such a mismatch between the audio and text may be more problematic.

The next two levels are essentially the same – gap fills featuring the words that are being studied. Level three, Usage, again often presents possible answers that are completely different parts of speech. Confused Words is more of the same, but this time the words presented in the three answer slots are potentially easily confused – this seems to be focusing mainly on spelling and/or pronunciation.

Overall, pedagogically Flovoco has a noble aim – looking at the different things it’s useful to know about a word – but in practice it’s a bit confused. Methodologically, the structure of feedback is very behaviouristic. Didn’t we leave Skinner behind a while back?

Instructional design


Admittedly I’m getting my head around the jargon of app development here, but I can’t see this. There isn’t any adaptivity (is that a word) built in to the app. You just work your way up the levels, aiming to get all 500 words into the Your Word Collection at the top level. Nothing adapts down if I keep getting a word wrong – there’s no help other than hoping that I’ll see the red ‘wrong’ button highlight enough times to learn. If I’m racing through the words, there isn’t any extension to what can be done with the words. Maybe there is space for some kind of freer practice, perhaps based around the community of language learners that ELTjam probably hope to cultivate with the app.

Photo 01-07-2015 16 43 09 word collectionPhoto 22-04-2015 08 01 36 372Photo 01-07-2015 16 42 56 profilePhoto 01-07-2015 16 43 05 usage

User experience


It looks pretty. It’s fairly easy to tap-and-go in getting started working with the app. There aren’t too many (any?) instructions, so it does rather rely on learners using the app to work things out by trial and error. The user Profile screen is relatively clean and looks straigtforward. But I haven’t worked out exactly how the Daily Word Goal works (maybe because I don’t use the app regularly) and navigating the Profile screens is a little laggy.

There is a playful feel to the app, from the Flovoco logo font to the coloured circles and stars representing the different levels. If they wanted to make learning words look like a game, that’s the impression you get.

Cost and access

Free. Spanish and English only. iOS only (the website says an Android version is in the works).

Technical requirements (iOS):

Category: Education

Updated: Feb 25, 2015

Version: 1.1.4

Size: 21.1 MB

Compatibility: Requires iOS 7.1 or later. Compatible with iPhone, iPad, and iPod touch. This app is optimized for iPhone 5.

Not yet available on Google Play.

You can get the iOS version from here:

Overall score:


This app certainly got people talking when ELTjam first posted about it, and there is the intent to do something different in the world of ELT apps. But from what I see so far, putting it into practice leaves a fair amount to be desired. Further thought needs to be made about plausible distractors, the focus on the word level meaning may need to be expanded to make this truly different, and sorting out some kind of support/challenge for more and less able learners is something I think is quite key.

Flovoco a go? There is some potential for this app to be really good. But sorry Flovoco, at the moment it’s a no from me.

The response from Jo Sayers:

Hi Mike,

Firstly, thanks very much for writing a review; we really appreciate the feedback and reviews like this will help us as we build Flovoco to a state where both learners and industry professionals can clearly see the value it delivers. This is still our v1.0, as you point out, and so there are many things that we had to do in a way that was good enough to ship it, but necessarily not as finalised as we’d want it to be in an ideal world. The hope was that we would test some key assumptions with this version and learn things that will help us to build it out in a direction that sees it add real value to learners.

The review matches well with a lot of what we have already identified as the strengths and weaknesses of the app and we’re really happy that there are enough positive things in this early version that it already gets 2.5 out of 5. There are a few comments in the review that I wanted to respond to though:

The behaviourist approach. Our intention with the app is not to offer a complete language learning solution, rather to offer a way of quickly acquiring key vocabulary that will act as a foundation for other language development. We feel that lexical acquisition is an area that lends itself well to a more systematic (behaviourist?) approach, and also helps to achieve the flow state that we were aiming for.

Multiple senses. We made a decision to focus only on the primary senses of the words in this version to avoid the complexity of having to demonstrate what is a second or other sense of a word that they’ve encountered before. What you’re actually pushing through the levels in the app is a single sense of a word, rather than a word itself.

The distractors. The distractors for levels 1 and 2 were chosen automatically by an algorithm. I’m not sure I agree that they should all be the same part of speech. This would actually have been very straightforward for us to achieve programmatically, but we felt that it wasn’t any more pedagogically sound than mixing parts of speech. In level 2, the algorithm choosing the distractors selected words with the same initial letter and as close to the same total number of letters as possible, but as the distractors are chosen only from the 500 words in the initial pool this reduced their effectiveness. We have a plan for the next version which should make the selection much more effective. What we wanted to test here, ultimately, was that the algorithm approach worked and didn’t interfere with the learner’s ability to complete the activities.

Levels 3 and 4 being the same. Level 3 focuses on collocation by keeping the target word in the sentence (in bold) and removing a collocate; the distractors are, where possible, similar words that don’t collocate with the target word. Level 4 initially aimed to focus on derived forms of the target word and in this case the target itself is removed and other words which are similar to it act as distractors. As a lot of the words at A1 don’t have strong and obvious collocates and many don’t have derived and inflected forms this was more challenging than it would be at higher levels with fewer functional words and lower frequency content.

Instructional design. Yes, there is no adaptivity. But if you get words wrong they move to the level below, and if you get them right they move up to the next level. In terms of the feedback, the word list on the results page gives a bit of additional information about the words that you have seen. We have plans to incorporate in some community aspects to the product too. However, our plan is not to build that up within ELTjam. As I have discussed in this post for a product website, we have made a shift away from offering this directly to consumers, and now see this as an opportunity to work with publishers and offer the product to their learners through partnerships with them. The B2C version acts as a way of gaining market validation.

Daily word goal. Yes, there are bugs in this which affect the calendar and the lag you mentioned. A new version was built out last month and we should be able to ship the updated version soon.

I hope that helps to put a few of the things you noted into context. It’s also worth pointing out that during the first two months of the app being out and promoted (we are no longer promoting this version as we have learned the things we hoped to) the average session length was over 9 minutes; way more than average in most industries and even more than the average for music apps, according to this data. We also had around 8% of sessions that were over 30 minutes long. So there were a core group of people who were incredibly motivated to use the app. And given that very little is done to incentivise returning to the app (no push notifications, leaderboards, 2 player games etc.) we were really pleased to see that around 8% of learners came back for an 8th visit. So, while there are definitely things to improve, the core offering resonates with learners in what is clearly a very busy and highly competitive marketplace.

Thanks again for the review, it would be really interesting to see what your thoughts are on future iterations. Looking forward to chatting it over next time we meet.

Jo is an Israeli start-up which, in its own words, ‘is an innovative new learning solution that helps you learn a language from the open web’. Its platform ‘uses big-data paired with spaced repetition to help users bootstrap their way to fluency’. You can read more of this kind of adspeak at the blog  or the Wikipedia entry  which seems to have been written by someone from the company.

How does it work? First of all, state the language you want to study (currently there are 10 available) and the language you already speak (currently there are 18 available). Then, there are three possible starting points: insert a word which you want to study, click on a word in any web text or click on a word in one of the suggested reading texts. This then brings up a bilingual dictionary entry which, depending on the word, will offer a number of parts of speech and a number of translated word senses. Click on the appropriate part of speech and the appropriate word sense, and the item will be added to your personal word list. Once you have a handful of words in your word list, you can begin practising these words. Here there are two options. The first is a spaced repetition flashcard system. It presents the target word and 8 different translations in your own language, and you have to click on the correct option. Like most flashcard apps, spaced repetition software determines when and how often you will be re-presented with the item.

The second option is to read an authentic web text which contains one or more of your target items. The company calls this ‘digital language immersion, a method of employing a virtual learning environment to simulate the language learning environment’. The app ‘relies on a number of applied linguistics principles, including the Natural Approach and Krashen’s Input Hypothesis’, according to the Wikipedia entry. Apparently, the more you use the app, the more it knows about you as a learner, and the better able it is to select texts that are appropriate for you. As you read these texts, of course, you can click on more words and add them to your word list.

I tried out, logging on as a French speaker wanting to learn English, and clicking on words as the fancy took me. I soon had a selection of texts to read. Users are offered a topic menu which consisted of the following: arts, business, education, entertainment, food, weird, beginners, green, health, living, news, politics, psychology, religion, science, sports, style. The sources are varied and not at all bad – Christian Science Monitor, The Grauniad, Huffington Post, Time, for example –and there are many very recent articles. Some texts were interesting; others seemed very niche. I began clicking on more words that I thought would be interesting to explore and here my problems began.

I quickly discovered that the system could only deal with single words, so phrasal verbs were off limits. One text I looked at had the phrasal verb ‘ripping off’, and although I could get translations for ‘ripping’ and ‘off’, this was obviously not terribly helpful. Learners who don’t know the phrasal verb ‘ripped off’ do not necessarily know that it is a phrasal verb, so the translations offered for the two parts of the verb are worse than unhelpful; they are actually misleading. Proper nouns were also a problem, although some of the more common ones were recognised. But the system failed to recognise many proper nouns for what they were, and offered me translations of homonymous nouns. new_word_added_'ripping_off' With some words (e.g. ‘stablemate’), the dictionary offered only one translation (in this case, the literal translation), but not the translation (the much more common idiomatic one) that was needed in the context in which I came across the word. With others (e.g. ‘pertain’), I was offered a list of translations which included the one that was appropriate in the context, but, unfortunately, this is the French word ‘porter’, which has so many possible meanings that, if you genuinely didn’t know the word, you would be none the wiser.

Once you’ve clicked on an appropriate part of speech and translation (if you can find one), the dictionary look-up function offers both photos and example sentences. Here again there were problems. I’d clicked on the verb ‘pan’ which I’d encountered in the context of a critic panning a book they’d read. I was able to select an appropriate translation, but when I got to the photos, I was offered only multiple pictures of frying pans. There were no example sentences for my meaning of ‘pan’: instead, I was offered multiple sentences about cooking pans, and one about Peter Pan. In other cases, the example sentences were either unhelpful (e.g. the example for ‘deal’ was ‘I deal with that’) or bizarre (e.g. the example sentence for ‘deemed’ was ‘The boy deemed that he cheated in the examination’). For some words, there were no example sentences at all.

Primed in this way, I was intrigued to see how the system would deal with the phrase ‘heaving bosoms’ which came up in one text. ‘Heaving bosoms’ is an interesting case. It’s a strong collocation, and, statistically, ‘heaving bosoms’ plural are much more frequent than ‘a heaving bosom’ singular. ‘Heaving’, as an adjective, only really collocates with ‘bosoms’. You don’t find ‘heaving’ collocating with any of the synonyms for ‘bosoms’. The phrase is also heavily connoted, strongly associated with romance novels, and often used with humorous intent. Finally, there is also a problem of usage with ‘bosom’ / ‘bosoms’: men or women, one or two – all in all, it’s a tricky word. was no help at all. There was no dictionary entry for an adjectival ‘heaving’, and the translations for the verb ‘heave’ were amusing, but less than appropriate. As for ‘bosom’, there were appropriate translations (‘sein’ and ‘poitrine’), but absolutely no help with how the word is actually used. Example sentences, which are clearly not tagged to the translation which has been chosen, included ‘Or whether he shall die in the bosom of his family or neglected and despised in a foreign land’ and ‘Can a man take fire in his bosom, and his clothes not be burned?’ has a number of problems. First off, its software hinges on a dictionary (it’s a Babylon dictionary) which can only deal with single words, is incomplete, and does not deal with collocation, connotation, style or register. As such, it can only be of limited value for receptive use, and of no value whatsoever for productive use. Secondly, the web corpus that it is using simply isn’t big enough. Thirdly, it doesn’t seem to have any Natural Language Processing tool which could enable it to deal with meanings in context. It can’t disambiguate words automatically. Such software does now exist, and desperately needs it.

Unfortunately, there are other problems, too. The flashcard practice is very repetitive and soon becomes boring. With eight translations to choose from, you have to scroll down the page to see them all. But there’s a timer mechanism, and I frequently timed out before being able to select the correct translation (partly because words are presented with no context, so you have to remember the meaning which you clicked in an earlier study session). The texts do not seem to be graded for level. There is no indication of word frequency or word sense frequency. There is just one gamification element (a score card), but there is no indication of how scores are achieved. Last, but certainly not least, the system is buggy. My word list disappeared into the cloud earlier today, and has not been seen since.

I think it’s a pity that is not better. The idea behind it is good – even if the references to Krashen are a little unfortunate. The company says that they have raised $800,000 in funding, but with their freemium model they’ll be desperately needing more, and they’ve gone to market too soon. One reviewer, Language Surfer,  wrote a withering review of’s Arabic program (‘it will do more harm than good to the Arabic student’), and Brendan Wightman, commenting at eltjam,  called it ‘dull as dish water, […] still very crude, limited and replete with multiple flaws’. But, at least, it’s free.

Personalization is one of the key leitmotifs in current educational discourse. The message is clear: personalization is good, one-size-fits-all is bad. ‘How to personalize learning and how to differentiate instruction for diverse classrooms are two of the great educational challenges of the 21st century,’ write Trilling and Fadel, leading lights in the Partnership for 21st Century Skills (P21)[1]. Barack Obama has repeatedly sung the praises of, and the need for, personalized learning and his policies are fleshed out by his Secretary of State, Arne Duncan, in speeches and on the White House blog: ‘President Obama described the promise of personalized learning when he launched the ConnectED initiative last June. Technology is a powerful tool that helps create robust personalized learning environments.’ In the UK, personalized learning has been government mantra for over 10 years. The EU, UNESCO, OECD, the Gates Foundation – everyone, it seems, is singing the same tune.

Personalization, we might all agree, is a good thing. How could it be otherwise? No one these days is going to promote depersonalization or impersonalization in education. What exactly it means, however, is less clear. According to a UNESCO Policy Brief[2], the term was first used in the context of education in the 1970s by Victor Garcìa Hoz, a senior Spanish educationalist and member of Opus Dei at the University of Madrid. This UNESCO document then points out that ‘unfortunately, up to this date there is no single definition of this concept’.

In ELT, the term has been used in a very wide variety of ways. These range from the far-reaching ideas of people like Gertrude Moskowitz, who advocated a fundamentally learner-centred form of instruction, to the much more banal practice of getting students to produce a few personalized examples of an item of grammar they have just studied. See Scott Thornbury’s A-Z blog for an interesting discussion of personalization in ELT.

As with education in general, and ELT in particular, ‘personalization’ is also bandied around the adaptive learning table. Duolingo advertises itself as the opposite of one-size-fits-all, and as an online equivalent of the ‘personalized education you can get from a small classroom teacher or private tutor’. Babbel offers a ‘personalized review manager’ and Rosetta Stone’s Classroom online solution allows educational institutions ‘to shift their language program away from a ‘one-size-fits-all-curriculum’ to a more individualized approach’. As far as I can tell, the personalization in these examples is extremely restricted. The language syllabus is fixed and although users can take different routes up the ‘skills tree’ or ‘knowledge graph’, they are totally confined by the pre-determination of those trees and graphs. This is no more personalized learning than asking students to make five true sentences using the present perfect. Arguably, it is even less!

This is not, in any case, the kind of personalization that Obama, the Gates Foundation, Knewton, et al have in mind when they conflate adaptive learning with personalization. Their definition is much broader and summarised in the US National Education Technology Plan of 2010: ‘Personalized learning means instruction is paced to learning needs, tailored to learning preferences, and tailored to the specific interests of different learners. In an environment that is fully personalized, the learning objectives and content as well as the method and pace may all vary (so personalization encompasses differentiation and individualization).’ What drives this is the big data generated by the students’ interactions with the technology (see ‘Part 4: big data and analytics’ of ‘The Guide’ on this blog).

What remains unclear is exactly how this might work in English language learning. Adaptive software can only personalize to the extent that the content of an English language learning programme allows it to do so. It may be true that each student using adaptive software ‘gets a more personalised experience no matter whose content the student is consuming’, as Knewton’s David Liu puts it. But the potential for any really meaningful personalization depends crucially on the nature and extent of this content, along with the possibility of variable learning outcomes. For this reason, we are not likely to see any truly personalized large-scale adaptive learning programs for English any time soon.

Nevertheless, technology is now central to personalized language learning. A good learning platform, which allows learners to connect to ‘social networking systems, podcasts, wikis, blogs, encyclopedias, online dictionaries, webinars, online English courses, various apps’, etc (see Alexandra Chistyakova’s eltdiary), means that personalization could be more easily achieved.

For the time being, at least, adaptive learning systems would seem to work best for ‘those things that can be easily digitized and tested like math problems and reading passages’ writes Barbara Bray . Or low level vocabulary and grammar McNuggets, we might add. Ideal for, say, ‘English Grammar in Use’. But meaningfully personalized language learning?


‘Personalized learning’ sounds very progressive, a utopian educational horizon, and it sounds like it ought to be the future of ELT (as Cleve Miller argues). It also sounds like a pretty good slogan on which to hitch the adaptive bandwagon. But somehow, just somehow, I suspect that when it comes to adaptive learning we’re more likely to see more testing, more data collection and more depersonalization.

[1] Trilling, B. & Fadel, C. 2009 21st Century Skills (San Francisco: Wiley) p.33

[2] Personalized learning: a new ICT­enabled education approach, UNESCO Institute for Information Technologies in Education, Policy Brief March 2012


Let’s take a look at the business of adaptive learning from a publisher’s perspective. Not an ELT publisher, but someone a few rungs higher on the ladder with strategic responsibilities. You might not know a great deal about ELT. It is, after all, only one of a number of divisions you are responsible for, and not an especially profitable one at that. You will, however, know a lot about creative destruction, the process by which one industry is replaced by another. The decline and demise of printed magazines, newspapers and books, of book reviewers and traditional booksellers, and their replacement by digital products will be such a part of your professional knowledge that they hardly need to be mentioned. Graphs such as the one below from PricewaterhouseCoopers (PwC) will be terribly familiar. You will also be aware that the gales of creative destruction in publishing are blowing harder than ever before.


In fact, you probably owe your job to your ability to talk convincingly about creative destruction and how to profit from it. Whatever your particular strategy for the future might be, you will have noted the actions of others. You will have evaluated advice, such as the following, from Publishing Perspectives

  • Do not delay in taking action when there are clear signals of a decline in market value.
  • Trade your low-profit print assets (even though some may have limited digital value) for core product that has a higher likelihood of success and can be exploited digitally.
  • Look for an orderly transition from print to digital product which enables a company to reinvent itself.

You will be looking to invest in technology, and prioritizing the acquisition of technological expertise (through partnerships or the purchase of start-ups) over the development of traditional ELT products. Your company will be restructured, and possibly renamed, to facilitate the necessary changes.

You will also know that big data and analytics have already transformed other industries. And you will know that educational publishing is moving towards a winner-take-all business market, where ‘the best performers are able to capture a very large share of the rewards, and the remaining competitors are left with very little’ (Investopedia). Erik Brynjolfsson and Andrew McAfee’s new book, The Second Machine Age (New York: Norton, 2014), argues that ‘each time a market becomes more digital, winner-take-all economics become a little more compelling …Digitization creates winner-take-all markets because [there are] enormous economies of scale, giving the market leader a huge cost advantage and room to beat the price of any competitor while still making a good profit’ (pp.153-155).

the second machine age

It is in this light that we need to understand the way that companies like Pearson and Macmillan are banking everything on a digital future. Laurie Harrison’s excellent blog post at eltjam  summarises the Pearson position: ‘the world’s biggest education publisher is spending £150m on a total restructure which involves an immediate move to digital learning, a focus on emerging markets, and a transformation from publisher to education services provider. If the English language learning market is worth $4billion a year, then Pearson still only have a very small chunk of it. And if you’re a company as successful and ambitious as Pearson, that just isn’t good enough – so a change of direction is needed. In order to deliver this change, the company have recently announced their new senior management team.’

Adaptive learning fits the new business paradigm perfectly. If the hype is to be believed, adaptive learning will be a game-changer. ‘The shifting of education from analog to digital is a one-time event in the history of the human race. At scale, it will have as big an effect on the world as indoor plumbing or electricity,’ writes Jose Ferreira of Knewton. ‘True disruption,’ he says elsewhere, ‘happens when entrepreneurs aim big and go after a giant problem, a problem that, if solved, would usher in an era of large-scale transformation across industries and nations. … Education is the last of the information industries to move online,’ he goes on. ‘When it breaks, it breaks fast. And that’s going to happen in the next five years. All the education content will go online in the next 10 years. And textbooks will go away. … Ultimately, all learning materials will be digital and they will all be adaptive.’

Ferreira clearly knows all about creative disruption. He also knows about winner-take-all markets. ‘The question is who is going to power [the] platform,’ he writes. ‘It’s probably going to be one or two companies’. He states his ambition for Knewton very clearly: ‘Knewton’s goal is to be like Amazon Web Services for education’. ‘It’s pretty clear to us,’ he writes, ‘that there’s going to be one dominant data platform for education, the way there’s one dominant data platform for search, social media, etailing. But in education, it’s going to be even more winner-take-all; there will be a number of companies that make up the platform, like Wintel. People might make a perverse choice to use Bing for search because they don’t like Google. But no one’s going to make the choice to subject their kid to the second-best adaptive learning platform, if that means there’s a 23% structural disadvantage. The data platform industries tend to have a winner-take-all dynamic. You take that and multiply it by a very, very high-stakes product and you get an even more winner-take-all dynamic.’

What is at stake in this winner-take-all market? Over to Jose Ferreira one more time: ‘The industry is massive. It’s so massive that virtually nobody I’ve met truly grasps how big it is. It’s beyond their frame of reference. The total amount of money (both public and private) spent annually exceeds all spending, both online and offline, of every other information industry combined: that is, all media, entertainment, games, news, software, Internet and mobile media, e-tailing, etc.’

But, still, a few questions continue to nag away at me. If all of this is so certain, why does Jose Ferreira feel the need to talk about it so much? If all of this is so certain, why don’t all the ELT publishers jump on the bandwagon? What sort of track record does economic forecasting have, anyway?

Adaptive learning is likely to impact on the lives of language teachers very soon. In my work as a writer of education materials, it has already dramatically impacted on mine. This impact has affected the kinds of things I am asked to write, the way in which I write them and my relationship with the editors and publishers I am writing for. I am as dismissive as Steve Jobs[1] was of the idea that technology can radically transform education, but in the short term it can radically disrupt it. Change is not necessarily progress.

Teachers and teacher trainers need to be very alert to what is going on if they don’t want to wake up one morning and find themselves out of work, or in a very different kind of job. The claims for adaptive language learning need to be considered in the bright light of particular, local contexts. Teachers and teacher trainers can even take a lesson from the proponents of adaptive learning who rail against the educational approach of one-size-fits-all. One size, whether it’s face-to-face with a print coursebook or whether it’s a blended adaptive program, will never fit all. We need to be very skeptical of the publishers and software providers who claim in a TED-style, almost evangelical way that they are doing the right thing for students, our society, or our world. There is a real risk that adaptive learning may be leading simply to ‘a more standardised, minimalist product targeted for a mass market, [that] will further ‘box in’ and ‘dumb down’ education’ (Selwyn, Education and Technology 2011, p.101).

There is nothing wrong, per se, with adaptive learning. It could be put to some good uses, but how likely is this? In order to understand how it may impact on our working lives, we need to be better informed. A historical perspective is often a good place to start and Larry Cuban’s Teachers and Machines: The Classroom Use of Technology since 1920 (New York: Teachers College Press, 1986) is still well worth reading.


To get a good picture of where big data and analytics are now and where they are heading, Mayer-Schonberger & Cukier’s Big Data (London: John Murray, 2013) is informative and entertaining reading. If you are ‘an executive looking to integrate analytics in your decision making or a manager seeking to generate better conversations with the quants in your organisation’, I’d recommend Keeping up with the Quants by Thomas H. Davenport and Jinho Kim (Harvard Business School, 2013). Or you could just read ‘The Economist’ for this kind of thing.

If you want to follow up the connections between educational technology and neo-liberalism, the books by Stephen Ball (Global Education Inc., Abingdon, Oxon: Routledge, 2012), Neil Selwyn (Education and Technology, London: Continuum, 2011; Education in a Digital World, New York: Routledge, 2013; Distrusting Educational Technology, New York: Routledge, 2013), Diane Ravitch (Reign of Error, New York: Knopf, 2013) and Joel Spring (Education Networks, New York: Routledge, 2012; The Great American Education-Industrial Complex with Anthony G. Picciano, Routledge, 2013) are all good reads. And keep a look out for anything new from these writers.

Finally, to keep up to date with recent developments, the eltjam blog is a good one to follow, as is Richard Whiteside’s! page

I’ll be continuing to post things here from time to time! Thanks for following me so far.

[1] Jobs, however, did set his sights ‘on the $8 billion a year textbook industry, which he saw as ‘ripe for digital destruction’. His first instinct seems to have been to relieve kids from having to carry around heavy backpacks crammed with textbooks: ‘The iPad would solve that,’ he said, ever practical’ (Fullan, Stratosphere 2013, p.61).

Adaptive learning is a product to be sold. How?

1 Individualised learning

In the vast majority of contexts, language teaching is tied to a ‘one-size-fits-all’ model. This is manifested in institutional and national syllabuses which provide lists of structures and / or competences that all students must master within a given period of time. It is usually actualized in the use of coursebooks, often designed for ‘global markets’. Reaction against this model has been common currency for some time, and has led to a range of suggestions for alternative approaches (such as DOGME), none of which have really caught on. The advocates of adaptive learning programs have tapped into this zeitgeist and promise ‘truly personalized learning’. Atomico, a venture capital company that focuses on consumer technologies, and a major investor in Knewton, describes the promise of adaptive learning in the following terms: ‘Imagine lessons that adapt on-the-fly to the way in which an individual learns, and powerful predictive analytics that help teachers differentiate instruction and understand what each student needs to work on and why[1].’

This is a seductive message and is often framed in such a way that disagreement seems impossible. A post on one well-respected blog, eltjam, which focuses on educational technology in language learning, argued the case for adaptive learning very strongly in July 2013: ‘Adaptive Learning is a methodology that is geared towards creating a learning experience that is unique to each individual learner through the intervention of computer software. Rather than viewing learners as a homogenous collective with more or less identical preferences, abilities, contexts and objectives who are shepherded through a glossy textbook with static activities/topics, AL attempts to tap into the rich meta-data that is constantly being generated by learners (and disregarded by educators) during the learning process. Rather than pushing a course book at a class full of learners and hoping that it will (somehow) miraculously appeal to them all in a compelling, salubrious way, AL demonstrates that the content of a particular course would be more beneficial if it were dynamic and interactive. When there are as many responses, ideas, personalities and abilities as there are learners in the room, why wouldn’t you ensure that the content was able to map itself to them, rather than the other way around?[2]

Indeed. But it all depends on what, precisely, the content is – a point I will return to in a later post. For the time being, it is worth noting the prominence that this message is given in the promotional discourse. It is a message that is primarily directed at teachers. It is more than a little disingenuous, however, because teachers are not the primary targets of the promotional discourse, for the simple reason that they are not the ones with purchasing power. The slogan on the homepage of the Knewton website shows clearly who the real audience is: ‘Every education leader needs an adaptive learning infrastructure’[3].

2 Learning outcomes and testing

Education leaders, who are more likely these days to come from the world of business and finance than the world of education, are currently very focused on two closely interrelated topics: the need for greater productivity and accountability, and the role of technology. They generally share the assumption of other leaders in the World Economic Forum that ICT is the key to the former and ‘the key to a better tomorrow’ (Spring, Education Networks, 2012, p.52). ‘We’re at an important transition point,’ said Arne Duncan, the U.S. Secretary of Education in 2010, ‘we’re getting ready to move from a predominantly print-based classroom to a digital learning environment’ (quoted by Spring, 2012, p.58). Later in the speech, which was delivered at the time as the release of the new National Education Technology Plan, Duncan said ‘just as technology has increased productivity in the business world, it is an essential tool to help boost educational productivity’. The plan outlines how this increased productivity could be achieved: we must start ‘with being clear about the learning outcomes we expect from the investments we make’ (Office of Educational Technology, Transforming American Education: Learning Powered by Technology, U.S. Department of Education, 2010). The greater part of the plan is devoted to discussion of learning outcomes and assessment of them.

Learning outcomes (and their assessment) are also at the heart of ‘Asking More: the Path to Efficacy’ (Barber and Rizvi (eds), Asking More: the Path to Efficacy Pearson, 2013), Pearson’s blueprint for the future of education. According to John Fallon, the CEO of Pearson, ‘our focus should unfalteringly be on honing and improving the learning outcomes we deliver’ (Barber and Rizvi, 2013, p.3). ‘High quality learning’ is associated with ‘a relentless focus on outcomes’ (ibid, p.3) and words like ‘measuring / measurable’, ‘data’ and ‘investment’ are almost as salient as ‘outcomes’. A ‘sister’ publication, edited by the same team, is entitled ‘The Incomplete Guide to Delivering Learning Outcomes’ (Barber and Rizvi (eds), Pearson, 2013) and explores further Pearson’s ambition to ‘become the world’s leading education company’ and to ‘deliver learning outcomes’.

It is no surprise that words like ‘outcomes’, ‘data’ and ‘measure’ feature equally prominently in the language of adaptive software companies like Knewton (see, for example, the quotation from Jose Ferreira, CEO of Knewton, in an earlier post). Adaptive software is premised on the establishment and measurement of clearly defined learning outcomes. If measurable learning outcomes are what you’re after, it’s hard to imagine a better path to follow than adaptive software. If your priorities include standards and assessment, it is again hard to imagine an easier path to follow than adaptive software, which was used in testing long before its introduction into instruction. As David Kuntz, VP of research at Knewton and, before that, a pioneer of algorithms in the design of tests, points out, ‘when a student takes a course powered by Knewton, we are continuously evaluating their performance, what others have done with that material before, and what [they] know’[4]. Knewton’s claim that every education leader needs an adaptive learning infrastructure has a powerful internal logic.

3 New business models

‘Adapt or die’ (a phrase originally coined by the last prime minister of apartheid South Africa) is a piece of advice that is often given these days to both educational institutions and publishers. British universities must adapt or die, according to Michael Barber, author of ‘An Avalanche is Coming[5]’ (a report commissioned by the British Institute for Public Policy Research), Chief Education Advisor to Pearson, and editor of the Pearson ‘Efficacy’ document (see above). ELT publishers ‘must change or die’, reported the eltjam blog[6], and it is a message that is frequently repeated elsewhere. The move towards adaptive learning is seen increasingly often as one of the necessary adaptations for both these sectors.

The problems facing universities in countries like the U.K. are acute. Basically, as the introduction to ‘An Avalanche is Coming’ puts it, ‘the traditional university is being unbundled’. There are a number of reasons for this including the rising cost of higher education provision, greater global competition for the same students, funding squeezes from central governments, and competition from new educational providers (such as MOOCs). Unsurprisingly, universities (supported by national governments) have turned to technology, especially online course delivery, as an answer to their problems. There are two main reasons for this. Firstly, universities have attempted to reduce operating costs by looking for increases in scale (through mergers, transnational partnerships, international branch campuses and so on). Mega-universities are growing, and there are thirty-three in Asia alone (Selwyn Education in a Digital World New York: Routledge 2013, p.6). Universities like the Turkish Anadolu University, with over one million students, are no longer exceptional in terms of scale. In this world, online educational provision is a key element. Secondly, and not to put too fine a point on it, online instruction is cheaper (Spring, Education Networks 2012, p.2).

All other things being equal, why would any language department of an institute of higher education not choose an online environment with an adaptive element? Adaptive learning, for the time being at any rate, may be seen as ‘the much needed key to the “Iron Triangle” that poses a conundrum to HE providers; cost, access and quality. Any attempt to improve any one of those conditions impacts negatively on the others. If you want to increase access to a course you run the risk of escalating costs and jeopardising quality, and so on.[7]

Meanwhile, ELT publishers have been hit by rampant pirating of their materials, spiraling development costs of their flagship products and the growth of open educational resources. An excellent blog post by David Wiley[8] explains why adaptive learning services are a heaven-sent opportunity for publishers to modify their business model. ‘While the broad availability of free content and open educational resources have trained internet users to expect content to be free, many people are still willing to pay for services. Adaptive learning systems exploit this willingness by deeply intermingling content and services so that you cannot access one with using the other. Naturally, because an adaptive learning service is comprised of content plus adaptive services, it will be more expensive than static content used to be. And because it is a service, you cannot simply purchase it like you used to buy a textbook. An adaptive learning service is something you subscribe to, like Netflix. […] In short, why is it in a content company’s interest to enable you to own anything? Put simply, it is not. When you own a copy, the publisher completely loses control over it. When you subscribe to content through a digital service (like an adaptive learning service), the publisher achieves complete and perfect control over you and your use of their content.’

Although the initial development costs of building a suitable learning platform with adaptive capabilities are high, publishers will subsequently be able to produce and modify content (i.e. learning materials) much more efficiently. Since content will be mashed up and delivered in many different ways, author royalties will be cut or eliminated. Production and distribution costs will be much lower, and sales and marketing efforts can be directed more efficiently towards the most significant customers. The days of ELT sales reps trying unsuccessfully to get an interview with the director of studies of a small language school or university department are becoming a thing of the past. As with the universities, scale will be everything.

[2] (last accessed 13 January 2014)

[3] (last accessed 13 January 2014)

[4] MIT Technology Review, November 26, 2012 (last accessed 13 January 2014)

[7] Tim Gifford Taking it Personally: Adaptive Learning July 9, 2013 (last accessed January 13, 2014)

[8] David Wiley, Buying our Way into Bondage: the risks of adaptive learning services March 20,2013 (last accessed January 13, 2014)