Adaptive learning in ELT: the current state of play

About two and a half years ago when I started writing this blog, there was a lot of hype around adaptive learning and the big data which might drive it. Two and a half years are a long time in technology. A look at Google Trends suggests that interest in adaptive learning has been pretty static for the last couple of years. It’s interesting to note that 3 of the 7 lettered points on this graph are Knewton-related media events (including the most recent, A, which is Knewton’s latest deal with Hachette) and 2 of them concern McGraw-Hill. It would be interesting to know whether these companies follow both parts of Simon Cowell’s dictum of ‘Create the hype, but don’t ever believe it’.

A look at the Hype Cycle (see here for Wikipedia’s entry on the topic and for criticism of the hype of Hype Cycles) of the IT research and advisory firm, Gartner, indicates that both big data and adaptive learning have now slid into the ‘trough of disillusionment’, which means that the market has started to mature, becoming more realistic about how useful the technologies can be for organizations.

A few years ago, the Gates Foundation, one of the leading cheerleaders and financial promoters of adaptive learning, launched its Adaptive Learning Market Acceleration Program (ALMAP) to ‘advance evidence-based understanding of how adaptive learning technologies could improve opportunities for low-income adults to learn and to complete postsecondary credentials’. It’s striking that the program’s aims referred to how such technologies could lead to learning gains, not whether they would. Now, though, with the publication of a report commissioned by the Gates Foundation to analyze the data coming out of the ALMAP Program, things are looking less rosy. The report is inconclusive. There is no firm evidence that adaptive learning systems are leading to better course grades or course completion. ‘The ultimate goal – better student outcomes at lower cost – remains elusive’, the report concludes. Rahim Rajan, a senior program office for Gates, is clear: ‘There is no magical silver bullet here.’

The same conclusion is being reached elsewhere. A report for the National Education Policy Center (in Boulder, Colorado) concludes: Personalized Instruction, in all its many forms, does not seem to be the transformational technology that is needed, however. After more than 30 years, Personalized Instruction is still producing incremental change. The outcomes of large-scale studies and meta-analyses, to the extent they tell us anything useful at all, show mixed results ranging from modest impacts to no impact. Additionally, one must remember that the modest impacts we see in these meta-analyses are coming from blended instruction, which raises the cost of education rather than reducing it (Enyedy, 2014: 15 -see reference at the foot of this post). In the same vein, a recent academic study by Meg Coffin Murray and Jorge Pérez (2015, ‘Informing and Performing: A Study Comparing Adaptive Learning to Traditional Learning’) found that ‘adaptive learning systems have negligible impact on learning outcomes’.

In the latest educational technology plan from the U.S. Department of Education (‘Future Ready Learning: Reimagining the Role of Technology in Education’, 2016) the only mentions of the word ‘adaptive’ are in the context of testing. And the latest OECD report on ‘Students, Computers and Learning: Making the Connection’ (2015), finds, more generally, that information and communication technologies, when they are used in the classroom, have, at best, a mixed impact on student performance.

There is, however, too much money at stake for the earlier hype to disappear completely. Sponsored cheerleading for adaptive systems continues to find its way into blogs and national magazines and newspapers. EdSurge, for example, recently published a report called ‘Decoding Adaptive’ (2016), sponsored by Pearson, that continues to wave the flag. Enthusiastic anecdotes take the place of evidence, but, for all that, it’s a useful read.

In the world of ELT, there are plenty of sales people who want new products which they can call ‘adaptive’ (and gamified, too, please). But it’s striking that three years after I started following the hype, such products are rather thin on the ground. Pearson was the first of the big names in ELT to do a deal with Knewton, and invested heavily in the company. Their relationship remains close. But, to the best of my knowledge, the only truly adaptive ELT product that Pearson offers is the PTE test.

Macmillan signed a contract with Knewton in May 2013 ‘to provide personalized grammar and vocabulary lessons, exam reviews, and supplementary materials for each student’. In December of that year, they talked up their new ‘big tree online learning platform’: ‘Look out for the Big Tree logo over the coming year for more information as to how we are using our partnership with Knewton to move forward in the Language Learning division and create content that is tailored to students’ needs and reactive to their progress.’ I’ve been looking out, but it’s all gone rather quiet on the adaptive / platform front.

In September 2013, it was the turn of Cambridge to sign a deal with Knewton ‘to create personalized learning experiences in its industry-leading ELT digital products for students worldwide’. This year saw the launch of a major new CUP series, ‘Empower’. It has an online workbook with personalized extra practice, but there’s nothing (yet) that anyone would call adaptive. More recently, Cambridge has launched the online version of the 2^nd edition of Touchstone. Nothing adaptive there, either.

Earlier this year, Cambridge published The Cambridge Guide to Blended Learning for Language Teaching, edited by Mike McCarthy. It contains a chapter by M.O.Z. San Pedro and R. Baker on ‘Adaptive Learning’. It’s an enthusiastic account of the potential of adaptive learning, but it doesn’t contain a single reference to language learning or ELT!

So, what’s going on? Skepticism is becoming the order of the day. The early hype of people like Knewton’s Jose Ferreira is now understood for what it was. Companies like Macmillan got their fingers badly burnt when they barked up the wrong tree with their ‘Big Tree’ platform.

Noel Enyedy captures a more contemporary understanding when he writes: Personalized Instruction is based on the metaphor of personal desktop computers—the technology of the 80s and 90s. Today’s technology is not just personal but mobile, social, and networked. The flexibility and social nature of how technology infuses other aspects of our lives is not captured by the model of Personalized Instruction, which focuses on the isolated individual’s personal path to a fixed end-point. To truly harness the power of modern technology, we need a new vision for educational technology (Enyedy, 2014: 16).

Adaptive solutions aren’t going away, but there is now a much better understanding of what sorts of problems might have adaptive solutions. Testing is certainly one. As the educational technology plan from the U.S. Department of Education (‘Future Ready Learning: Re-imagining the Role of Technology in Education’, 2016) puts it: Computer adaptive testing, which uses algorithms to adjust the difficulty of questions throughout an assessment on the basis of a student’s responses, has facilitated the ability of assessments to estimate accurately what students know and can do across the curriculum in a shorter testing session than would otherwise be necessary. In ELT, Pearson and EF have adaptive tests that have been well researched and designed.

Vocabulary apps which deploy adaptive technology continue to become more sophisticated, although empirical research is lacking. Automated writing tutors with adaptive corrective feedback are also developing fast, and I’ll be writing a post about these soon. Similarly, as speech recognition software improves, we can expect to see better and better automated adaptive pronunciation tutors. But going beyond such applications, there are bigger questions to ask, and answers to these will impact on whatever direction adaptive technologies take. Large platforms (LMSs), with or without adaptive software, are already beginning to look rather dated. Will they be replaced by integrated apps, or are apps themselves going to be replaced by bots (currently riding high in the Hype Cycle)? In language learning and teaching, the future of bots is likely to be shaped by developments in natural language processing (another topic about which I’ll be blogging soon). Nobody really has a clue where the next two and a half years will take us (if anywhere), but it’s becoming increasingly likely that adaptive learning will be only one very small part of it.

 

Enyedy, N. 2014. Personalized Instruction: New Interest, Old Rhetoric, Limited Results, and the Need for a New Direction for Computer-Mediated Learning. Boulder, CO: National Education Policy Center. Retrieved 17.07.16 from http://nepc.colorado.edu/publication/personalized-instruction

Part 10: learn more

Adaptive learning is likely to impact on the lives of language teachers very soon. In my work as a writer of education materials, it has already dramatically impacted on mine. This impact has affected the kinds of things I am asked to write, the way in which I write them and my relationship with the editors and publishers I am writing for. I am as dismissive as Steve Jobs[1] was of the idea that technology can radically transform education, but in the short term it can radically disrupt it. Change is not necessarily progress.

Teachers and teacher trainers need to be very alert to what is going on if they don’t want to wake up one morning and find themselves out of work, or in a very different kind of job. The claims for adaptive language learning need to be considered in the bright light of particular, local contexts. Teachers and teacher trainers can even take a lesson from the proponents of adaptive learning who rail against the educational approach of one-size-fits-all. One size, whether it’s face-to-face with a print coursebook or whether it’s a blended adaptive program, will never fit all. We need to be very skeptical of the publishers and software providers who claim in a TED-style, almost evangelical way that they are doing the right thing for students, our society, or our world. There is a real risk that adaptive learning may be leading simply to ‘a more standardised, minimalist product targeted for a mass market, [that] will further ‘box in’ and ‘dumb down’ education’ (Selwyn, Education and Technology 2011, p.101).

There is nothing wrong, per se, with adaptive learning. It could be put to some good uses, but how likely is this? In order to understand how it may impact on our working lives, we need to be better informed. A historical perspective is often a good place to start and Larry Cuban’s Teachers and Machines: The Classroom Use of Technology since 1920 (New York: Teachers College Press, 1986) is still well worth reading.

To get a good picture of where big data and analytics are now and where they are heading, Mayer-Schonberger & Cukier’s Big Data (London: John Murray, 2013) is informative and entertaining reading. If you are ‘an executive looking to integrate analytics in your decision making or a manager seeking to generate better conversations with the quants in your organisation’, I’d recommend Keeping up with the Quants by Thomas H. Davenport and Jinho Kim (Harvard Business School, 2013). Or you could just read ‘The Economist’ for this kind of thing.

If you want to follow up the connections between educational technology and neo-liberalism, the books by Stephen Ball (Global Education Inc., Abingdon, Oxon: Routledge, 2012), Neil Selwyn (Education and Technology, London: Continuum, 2011; Education in a Digital World, New York: Routledge, 2013; Distrusting Educational Technology, New York: Routledge, 2013), Diane Ravitch (Reign of Error, New York: Knopf, 2013) and Joel Spring (Education Networks, New York: Routledge, 2012; The Great American Education-Industrial Complex with Anthony G. Picciano, Routledge, 2013) are all good reads. And keep a look out for anything new from these writers.

Finally, to keep up to date with recent developments, the eltjam blog http://www.eltjam.com/ is a good one to follow, as is Richard Whiteside’s Scoop.it! page http://www.scoop.it/t/elt-publishing-by-richard-whiteside

I’ll be continuing to post things here from time to time! Thanks for following me so far.

---

[1] Jobs, however, did set his sights ‘on the $8 billion a year textbook industry, which he saw as ‘ripe for digital destruction’. His first instinct seems to have been to relieve kids from having to carry around heavy backpacks crammed with textbooks: ‘The iPad would solve that,’ he said, ever practical’ (Fullan, Stratosphere 2013, p.61).

Part 9: neo-liberalism and solutionism

The drive towards adaptive learning is being fuelled less by individual learners or teachers than it is by commercial interests, large educational institutions and even larger agencies, including national governments. How one feels about adaptive learning is likely to be shaped by one’s beliefs about how education should be managed.

Huge amounts of money are at stake. Education is ‘a global marketplace that is estimated conservatively to be worth in excess of $5 trillion per annum’ (Selwyn, Distrusting Educational Technology 2013, p.2). With an eye on this pot, in one year, 2012, ‘venture capital funds, private equity investors and transnational corporations like Pearson poured over $1.1 billion into education technology companies’[1] Knewton, just one of a number of adaptive learning companies, managed to raise $54 million before it signed multi-million dollar contracts with ELT publishers like Macmillan and Cambridge University Press. In ELT, some publishing companies are preferring to sit back and wait to see what happens. Most, however, have their sights firmly set on the earnings potential and are fully aware that late-starters may never be able to catch up with the pace-setters.

The nexus of vested interests that is driving the move towards adaptive learning is both tight and complicated. Fuller accounts of this can be found in Stephen Ball’s ‘Education Inc.’ (2012) and Joel Spring’s ‘Education Networks’ (2012) but for this post I hope that a few examples will suffice.

Leading the way is the Bill and Melinda Gates Foundation, the world’s largest private foundation with endowments of almost $40 billion. One of its activities is the ‘Adaptive Learning Market Acceleration Program’ which seeks to promote adaptive learning and claims that the adaptive learning loop can defeat the iron triangle of costs, quality and access (referred to in The Selling Points of Adaptive Learning, above). It is worth noting that this foundation has also funded Teach Plus, an organisation that has been lobbying US ‘state legislatures to eliminate protection of senior teachers during layoffs’ (Spring, 2012, p.51). It also supports the Foundation for Excellence in Education, ‘a major advocacy group for expanding online instruction by changing state laws’ (ibid., p.51). The chairman of this foundation is Jeb Bush, brother of ex-president Bush, who took the message of his foundation’s ‘Digital Learning Now!’ program on the road in 2011. The message, reports Spring (ibid. p.63) was simple: ‘the economic crises provided an opportunity to reduce school budgets by replacing teachers with online courses.’ The Foundation for Excellence in Education is also supported by the Walton Foundation (the Walmart family) and iQity, a company whose website makes clear its reasons for supporting Jeb Bush’s lobbying. ‘The iQity e-Learning Platform is the most complete solution available for the electronic search and delivery of curriculum, courses, and other learning objects. Delivering over one million courses each year, the iQity Platform is a proven success for students, teachers, school administrators, and district offices; as well as state, regional, and national education officials across the country.[2]’

Another supporter of the Foundation for Excellence in Education is the Pearson Foundation, the philanthropic arm of Pearson. The Pearson Foundation, in its turn, is supported by the Gates Foundation. In 2011, the Pearson Foundation received funding from the Gates Foundation to create 24 online courses, four of which would be distributed free and the others sold by Pearson the publishers (Spring, 2012, p.66).

The campaign to promote online adaptive learning is massively funded and extremely well-articulated. It receives support from transnational agencies such as the World Bank, WTO and OECD, and its arguments are firmly rooted in the discourse ‘of international management consultancies and education businesses’ (Ball, 2012, p.11-12). It is in this context that observers like Neil Selwyn connect the growing use of digital technologies in education to the corporatisation and globalisation of education and neo-liberal ideology.

Adaptive learning also holds rich promise for those who can profit from the huge amount of data it will generate. Jose Fereira, CEO of Knewton, acknowledges that adaptive learning has ‘the capacity to produce a tremendous amount of data, more than maybe any other industry’[3]. He continues ‘Big data is going to impact education in a big way. It is inevitable. It has already begun. If you’re part of an education organization, you need to have a vision for how you will take advantage of big data. Wait too long and you’ll wake up to find that your competitors (and the instructors that use them) have left you behind with new capabilities and insights that seem almost magical.’ Rather paradoxically, he then concludes that ‘we must all commit to the principle that the data ultimately belong to the students and the schools’. It is not easy to understand how such data can be both the property of individuals and, at the same time, be used by educational organizations to gain competitive advantage.

The existence and exploitation of this data may also raise concerns about privacy. In the same way that many people do not fully understand the extent or purpose of ‘dataveillance’ by cookies when they are browsing the internet, students cannot be expected to fully grasp the extent or potential commercial use of the data that they generate when engaged in adaptive learning programs.

Selwyn (Distrusting Educational Technology 2013, p.59-60) highlights a further problem connected with the arrival of big data. ‘Dataveillance’, he writes, also ‘functions to decrease the influence of ‘human’ experience and judgement, with it no longer seeming to matter what a teacher may personally know about a student in the face of his or her ‘dashboard’ profile and aggregated tally of positive and negative ‘events’. As such, there would seem to be little room for ‘professional’ expertise or interpersonal emotion when faced with such data. In these terms, institutional technologies could be said to be both dehumanizing and deprofessionalizing the relationships between people in an education context – be they students, teachers, administrators or managers.’

Adaptive learning in online and blended programs may well offer a number of advantages, but these will need to be weighed against the replacement or deskilling of teachers, and the growing control of big business over educational processes and content. Does adaptive learning increase the risk of transforming language teaching into a digital diploma mill (Noble, Digital Diploma Mills: The automation of higher education 2002)?

Solutionism

Evgeney Morozov’s 2013 best-seller, ‘To Save Everything, Click Here’, takes issue with our current preoccupation with finding technological solutions to complex and contentious problems. If adaptive learning is being presented as a solution, what is the problem that it is the solution of? In Morosov’s analysis, it is not an educational problem. ‘Digital technologies might be a perfect solution to some problems,’ he writes, ‘but those problems don’t include education – not if by education we mean the development of the skills to think critically about any given issue’ (Morosov, 2013, p.8). Only if we conceive of education as the transmission of bits of information (and in the case of language education as the transmission of bits of linguistic information), could adaptive learning be seen as some sort of solution to an educational problem. The push towards adaptive learning in ELT can be seen, in Morosov’s terms, as reaching ‘for the answer before the questions have been fully asked’ (ibid., p.6).

The world of education has been particularly susceptible to the dreams of a ‘technical fix’. Its history, writes Neil Selwyn, ‘has been characterised by attempts to use the ‘power’ of technology in order to solve problems that are non-technological in nature. […] This faith in the technical fix is pervasive and relentless – especially in the minds of the key interests and opinion formers of this digital age. As the co-founder of the influential Wired magazine reasoned more recently, ‘tools and technology drive us. Even if a problem has been caused by technology, the answer will always be more technology’ (Selwyn, Education in a Digital World 2013, p.36).

Morosov cautions against solutionism in all fields of human activity, pointing out that, by the time a problem is ‘solved’, it becomes something else entirely. Anyone involved in language teaching would be well-advised to identify and prioritise the problems that matter to them before jumping to the conclusion that adaptive learning is the ‘solution’. Like other technologies, it might, just possibly, ‘reproduce, perpetuate, strengthen and deepen existing patterns of social relations and structures – albeit in different forms and guises. In this respect, then, it is perhaps best to approach educational technology as a ‘problem changer’ rather than a ‘problem solver’ (Selwyn, Education in a Digital World 2013, p.21).

---

[1] Philip McRae Rebirth of the Teaching Machine through the Seduction of Data Analytics: This time it’s personal April 14, 2013 http://philmcrae.com/2/post/2013/04/rebirth-of-the-teaching-maching-through-the-seduction-of-data-analytics-this-time-its-personal1.html (last accessed 13 January 2014)

[2] http://www.iq-ity.com/ (last accessed 13 January, 2014)

[3] http://www.knewton.com/blog/ceo-jose-ferreira/big-data-in-education/ (last accessed 13 January, 2104)

Part 8: theory, research and practice

One could be forgiven for thinking that there are no problems associated with adaptive learning in ELT. Type the term into a search engine and you’ll mostly come up with enthusiasm or sales talk. There are, however, a number of reasons to be deeply skeptical about the whole business. In the post after this, I will be considering the political background.

1. Learning theory

Jose Fereira, the CEO of Knewton, spoke, in an interview with Digital Journal[1] in October 2009, about getting down to the ‘granular level’ of learning. He was referencing, in an original turn of phrase, the commonly held belief that learning is centrally concerned with ‘gaining knowledge’, knowledge that can be broken down into very small parts that can be put together again. In this sense, the adaptive learning machine is very similar to the ‘teaching machine’ of B.F. Skinner, the psychologist who believed that learning was a complex process of stimulus and response. But how many applied linguists would agree, firstly, that language can be broken down into atomised parts (rather than viewed as a complex, dynamic system), and, secondly, that these atomised parts can be synthesized in a learning program to reform a complex whole? Human cognitive and linguistic development simply does not work that way, despite the strongly-held contrary views of ‘folk’ theories of learning (Selwyn Education and Technology 2011, p.3).

Furthermore, even if an adaptive system delivers language content in personalized and interesting ways, it is still premised on a view of learning where content is delivered and learners receive it. The actual learning program is not personalized in any meaningful way: it is only the way that it is delivered that responds to the algorithms. This is, again, a view of learning which few educationalists (as opposed to educational leaders) would share. Is language learning ‘simply a technical business of well managed information processing’ or is it ‘a continuing process of ‘participation’ (Selwyn, Education and Technology 2011, p.4)?

Finally, adaptive learning is also premised on the idea that learners have particular learning styles, that these can be identified by the analytics (even if they are not given labels), and that actionable insights can be gained from this analysis (i.e. the software can decide on the most appropriate style of content delivery for an individual learner). Although the idea that teaching programs can be modified to cater to individual learning styles continues to have some currency among language teachers (e.g. those who espouse Neuro-Linguistic Programming or Multiple Intelligences Theory), it is not an idea that has much currency in the research community.

It might be the case that adaptive learning programs will work with some, or even many, learners, but it would be wise to carry out more research (see the section on Research below) before making grand claims about its efficacy. If adaptive learning can be shown to be more effective than other forms of language learning, it will be either because our current theories of language learning are all wrong, or because the learning takes place despite the theory, (and not because of it).

2. Practical problems

However good technological innovations may sound, they can only be as good, in practice, as the way they are implemented. Language laboratories and interactive whiteboards both sounded like very good ideas at the time, but they both fell out of favour long before they were technologically superseded. The reasons are many, but one of the most important is that classroom teachers did not understand sufficiently the potential of these technologies or, more basically, how to use them. Given the much more radical changes that seem to be implied by the adoption of adaptive learning, we would be wise to be cautious. The following is a short, selected list of questions that have not yet been answered.

• Language teachers often struggle with mixed ability classes. If adaptive programs (as part of a blended program) allow students to progress at their own speed, the range of abilities in face-to-face lessons may be even more marked. How will teachers cope with this? Teacher – student ratios are unlikely to improve!
• Who will pay for the training that teachers will need to implement effective blended learning and when will this take place?
• How will teachers respond to a technology that will be perceived by some as a threat to their jobs and their professionalism and as part of a growing trend towards the accommodation of commercial interests (see the next post)?
• How will students respond to online (adaptive) learning when it becomes the norm, rather than something ‘different’?

3 Research

Technological innovations in education are rarely, if ever, driven by solidly grounded research, but they are invariably accompanied by grand claims about their potential. Motion pictures, radio, television and early computers were all seen, in their time, as wonder technologies that would revolutionize education (Cuban, Teachers and Machines: The Classroom Use of Technology since 1920 1986). Early research seemed to support the claims, but the passage of time has demonstrated all too clearly the precise opposite. The arrival on the scene of e-learning in general, and adaptive learning in particular, has also been accompanied by much cheer-leading and claims of research support.

Examples of such claims of research support for adaptive learning in higher education in the US and Australia include an increase in pass rates of between 7 and 18%, a decrease of between 14 and 47% in student drop-outs, and an acceleration of 25% in the time needed to complete courses[2]. However, research of this kind needs to be taken with a liberal pinch of salt. First of all, the research has usually been commissioned, and sometimes carried out, by those with vested commercial interests in positive results. Secondly, the design of the research study usually guarantees positive results. Finally, the results cannot be interpreted to have any significance beyond their immediate local context. There is no reason to expect that what happened in a particular study into adaptive learning in, say, the University of Arizona would be replicated in, say, the Universities of Amman, Astana or anywhere else. Very often, when this research is reported, the subject of the students’ study is not even mentioned, as if this were of no significance.

The lack of serious research into the effectiveness of adaptive learning does not lead us to the conclusion that it is ineffective. It is simply too soon to say, and if the examples of motion pictures, radio and television are any guide, it will be a long time before we have any good evidence. By that time, it is reasonable to assume, adaptive learning will be a very different beast from what it is today. Given the recency of this kind of learning, the lack of research is not surprising. For online learning in general, a meta-analysis commissioned by the US Department of Education (Means et al, Evaluation of Evidence-Based Practice in Online Learning 2009, p.9) found that there were only a small number of rigorous published studies, and that it was not possible to attribute any gains in learning outcomes to online or blended learning modes. As the authors of this report were aware, there are too many variables (social, cultural and economic) to compare in any direct way the efficacy of one kind of learning with another. This is as true of attempts to compare adaptive online learning with face-to-face instruction as it is with comparisons of different methodological approaches in purely face-to-face teaching. There is, however, an irony in the fact that advocates of adaptive learning (whose interest in analytics leads them to prioritise correlational relationships over causal ones) should choose to make claims about the causal relationship between learning outcomes and adaptive learning.

Perhaps, as Selwyn (Education and Technology 2011, p.87) suggests, attempts to discover the relative learning advantages of adaptive learning are simply asking the wrong question, not least as there cannot be a single straightforward answer. Perhaps a more useful critique would be to look at the contexts in which the claims for adaptive learning are made, and by whom. Selwyn also suggests that useful insights may be gained from taking a historical perspective. It is worth noting that the technicist claims for adaptive learning (that ‘it works’ or that it is ‘effective’) are essentially the same as those that have been made for other education technologies. They take a universalising position and ignore local contexts, forgetting that ‘pedagogical approach is bound up with a web of cultural assumption’ (Wiske, ‘A new culture of teaching for the 21^st century’ in Gordon, D.T. (ed.) The Digital Classroom: How Technology is Changing the Way we teach and Learn 2000, p.72). Adaptive learning might just possibly be different from other technologies, but history advises us to be cautious.

---

[1] http://www.digitaljournal.com/article/279940?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+frc+(First+Round+Capital+News)

[2] These figures are quoted in Learning to Adapt: A Case for Accelerating Adaptive Learning in Higher Education, a booklet produced in March 2013 by Education Growth Advisors, an education consultancy firm. Their research is available at http://edgrowthadvisors.com/research/

Part 7: The next 4 to 5 years of adaptive learning in ELT – 10 predictions

Given what we know, it is possible to make some predictions about what the next generation of adult ELT materials will be like when they emerge a few years from now. Making predictions is always a hazardous game, but there are a number of reasonable certainties that can be identified, based on the statements and claims of the major publishers and software providers.

1 Major publishers will move gradually away from traditional coursebooks (whether in print or ebook format) towards the delivery of learning content on learning platforms. At its most limited, this will be in the form of workbook-style material with an adaptive element. At its most developed, this will be in the form of courses that can be delivered entirely without traditional coursebooks. These will allow teachers or institutions to decide the extent to which they wish to blend online and face-to-face instruction.

2 The adaptive elements of these courses will focus primarily or exclusively on discrete item grammar, vocabulary, functional language and phonology, since these lend themselves most readily to the software. These courses will be targeted mainly at lower level (B1 and below) learners.

3 The methodological approach of these courses will be significantly influenced by the expectations of the markets where they are predicted to be most popular and most profitable: South and Central America, the Arabian Gulf and Asia.

4 These courses will permit multiple modifications to suit local requirements. They will also allow additional content to be uploaded.

5 Assessment will play an important role in the design of all these courses. Things like discrete item grammar, vocabulary, functional language and phonology, which lend themselves most readily to assessment, will be prioritized over language skills, which are harder to assess.

6 The discrete items of language that are presented will be tagged to level descriptors, using scales like the Common European Framework or English Profile.

7 Language skills work will be included, but only in the more sophisticated (and better-funded) projects will these components be closely tied to the adaptive software.

8 Because of technological differences between different parts of the world, adaptive courses will co-exist with closely related, more traditional print (or ebook) courses.

9 Training for teachers (especially concerning blended learning) will become an increasingly important part of the package sold by the major publishers.

10 These courses will be more than ever driven by the publishers’ perceptions of what the market wants. There will be a concomitant decrease in the extent to which individual authors, or author teams, influence the material.

Part 6: the selling points of adaptive learning

Adaptive learning is a product to be sold. How?

1 Individualised learning

In the vast majority of contexts, language teaching is tied to a ‘one-size-fits-all’ model. This is manifested in institutional and national syllabuses which provide lists of structures and / or competences that all students must master within a given period of time. It is usually actualized in the use of coursebooks, often designed for ‘global markets’. Reaction against this model has been common currency for some time, and has led to a range of suggestions for alternative approaches (such as DOGME), none of which have really caught on. The advocates of adaptive learning programs have tapped into this zeitgeist and promise ‘truly personalized learning’. Atomico, a venture capital company that focuses on consumer technologies, and a major investor in Knewton, describes the promise of adaptive learning in the following terms: ‘Imagine lessons that adapt on-the-fly to the way in which an individual learns, and powerful predictive analytics that help teachers differentiate instruction and understand what each student needs to work on and why[1].’

This is a seductive message and is often framed in such a way that disagreement seems impossible. A post on one well-respected blog, eltjam, which focuses on educational technology in language learning, argued the case for adaptive learning very strongly in July 2013: ‘Adaptive Learning is a methodology that is geared towards creating a learning experience that is unique to each individual learner through the intervention of computer software. Rather than viewing learners as a homogenous collective with more or less identical preferences, abilities, contexts and objectives who are shepherded through a glossy textbook with static activities/topics, AL attempts to tap into the rich meta-data that is constantly being generated by learners (and disregarded by educators) during the learning process. Rather than pushing a course book at a class full of learners and hoping that it will (somehow) miraculously appeal to them all in a compelling, salubrious way, AL demonstrates that the content of a particular course would be more beneficial if it were dynamic and interactive. When there are as many responses, ideas, personalities and abilities as there are learners in the room, why wouldn’t you ensure that the content was able to map itself to them, rather than the other way around?[2]’

Indeed. But it all depends on what, precisely, the content is – a point I will return to in a later post. For the time being, it is worth noting the prominence that this message is given in the promotional discourse. It is a message that is primarily directed at teachers. It is more than a little disingenuous, however, because teachers are not the primary targets of the promotional discourse, for the simple reason that they are not the ones with purchasing power. The slogan on the homepage of the Knewton website shows clearly who the real audience is: ‘Every education leader needs an adaptive learning infrastructure’[3].

2 Learning outcomes and testing

Education leaders, who are more likely these days to come from the world of business and finance than the world of education, are currently very focused on two closely interrelated topics: the need for greater productivity and accountability, and the role of technology. They generally share the assumption of other leaders in the World Economic Forum that ICT is the key to the former and ‘the key to a better tomorrow’ (Spring, Education Networks, 2012, p.52). ‘We’re at an important transition point,’ said Arne Duncan, the U.S. Secretary of Education in 2010, ‘we’re getting ready to move from a predominantly print-based classroom to a digital learning environment’ (quoted by Spring, 2012, p.58). Later in the speech, which was delivered at the time as the release of the new National Education Technology Plan, Duncan said ‘just as technology has increased productivity in the business world, it is an essential tool to help boost educational productivity’. The plan outlines how this increased productivity could be achieved: we must start ‘with being clear about the learning outcomes we expect from the investments we make’ (Office of Educational Technology, Transforming American Education: Learning Powered by Technology, U.S. Department of Education, 2010). The greater part of the plan is devoted to discussion of learning outcomes and assessment of them.

Learning outcomes (and their assessment) are also at the heart of ‘Asking More: the Path to Efficacy’ (Barber and Rizvi (eds), Asking More: the Path to Efficacy Pearson, 2013), Pearson’s blueprint for the future of education. According to John Fallon, the CEO of Pearson, ‘our focus should unfalteringly be on honing and improving the learning outcomes we deliver’ (Barber and Rizvi, 2013, p.3). ‘High quality learning’ is associated with ‘a relentless focus on outcomes’ (ibid, p.3) and words like ‘measuring / measurable’, ‘data’ and ‘investment’ are almost as salient as ‘outcomes’. A ‘sister’ publication, edited by the same team, is entitled ‘The Incomplete Guide to Delivering Learning Outcomes’ (Barber and Rizvi (eds), Pearson, 2013) and explores further Pearson’s ambition to ‘become the world’s leading education company’ and to ‘deliver learning outcomes’.

It is no surprise that words like ‘outcomes’, ‘data’ and ‘measure’ feature equally prominently in the language of adaptive software companies like Knewton (see, for example, the quotation from Jose Ferreira, CEO of Knewton, in an earlier post). Adaptive software is premised on the establishment and measurement of clearly defined learning outcomes. If measurable learning outcomes are what you’re after, it’s hard to imagine a better path to follow than adaptive software. If your priorities include standards and assessment, it is again hard to imagine an easier path to follow than adaptive software, which was used in testing long before its introduction into instruction. As David Kuntz, VP of research at Knewton and, before that, a pioneer of algorithms in the design of tests, points out, ‘when a student takes a course powered by Knewton, we are continuously evaluating their performance, what others have done with that material before, and what [they] know’[4]. Knewton’s claim that every education leader needs an adaptive learning infrastructure has a powerful internal logic.

3 New business models

‘Adapt or die’ (a phrase originally coined by the last prime minister of apartheid South Africa) is a piece of advice that is often given these days to both educational institutions and publishers. British universities must adapt or die, according to Michael Barber, author of ‘An Avalanche is Coming[5]’ (a report commissioned by the British Institute for Public Policy Research), Chief Education Advisor to Pearson, and editor of the Pearson ‘Efficacy’ document (see above). ELT publishers ‘must change or die’, reported the eltjam blog[6], and it is a message that is frequently repeated elsewhere. The move towards adaptive learning is seen increasingly often as one of the necessary adaptations for both these sectors.

The problems facing universities in countries like the U.K. are acute. Basically, as the introduction to ‘An Avalanche is Coming’ puts it, ‘the traditional university is being unbundled’. There are a number of reasons for this including the rising cost of higher education provision, greater global competition for the same students, funding squeezes from central governments, and competition from new educational providers (such as MOOCs). Unsurprisingly, universities (supported by national governments) have turned to technology, especially online course delivery, as an answer to their problems. There are two main reasons for this. Firstly, universities have attempted to reduce operating costs by looking for increases in scale (through mergers, transnational partnerships, international branch campuses and so on). Mega-universities are growing, and there are thirty-three in Asia alone (Selwyn Education in a Digital World New York: Routledge 2013, p.6). Universities like the Turkish Anadolu University, with over one million students, are no longer exceptional in terms of scale. In this world, online educational provision is a key element. Secondly, and not to put too fine a point on it, online instruction is cheaper (Spring, Education Networks 2012, p.2).

All other things being equal, why would any language department of an institute of higher education not choose an online environment with an adaptive element? Adaptive learning, for the time being at any rate, may be seen as ‘the much needed key to the “Iron Triangle” that poses a conundrum to HE providers; cost, access and quality. Any attempt to improve any one of those conditions impacts negatively on the others. If you want to increase access to a course you run the risk of escalating costs and jeopardising quality, and so on.[7]’

Meanwhile, ELT publishers have been hit by rampant pirating of their materials, spiraling development costs of their flagship products and the growth of open educational resources. An excellent blog post by David Wiley[8] explains why adaptive learning services are a heaven-sent opportunity for publishers to modify their business model. ‘While the broad availability of free content and open educational resources have trained internet users to expect content to be free, many people are still willing to pay for services. Adaptive learning systems exploit this willingness by deeply intermingling content and services so that you cannot access one with using the other. Naturally, because an adaptive learning service is comprised of content plus adaptive services, it will be more expensive than static content used to be. And because it is a service, you cannot simply purchase it like you used to buy a textbook. An adaptive learning service is something you subscribe to, like Netflix. […] In short, why is it in a content company’s interest to enable you to own anything? Put simply, it is not. When you own a copy, the publisher completely loses control over it. When you subscribe to content through a digital service (like an adaptive learning service), the publisher achieves complete and perfect control over you and your use of their content.’

Although the initial development costs of building a suitable learning platform with adaptive capabilities are high, publishers will subsequently be able to produce and modify content (i.e. learning materials) much more efficiently. Since content will be mashed up and delivered in many different ways, author royalties will be cut or eliminated. Production and distribution costs will be much lower, and sales and marketing efforts can be directed more efficiently towards the most significant customers. The days of ELT sales reps trying unsuccessfully to get an interview with the director of studies of a small language school or university department are becoming a thing of the past. As with the universities, scale will be everything.

---

[1]http://www.atomico.com/news/item/our-investment-in-knewton (last accessed 13 January 2014)

[2]http://www.eltjam.com/adaptive-learning/ (last accessed 13 January 2014)

[3] http://www.knewton.com/ (last accessed 13 January 2014)

[4] MIT Technology Review, November 26, 2012 http://www.technologyreview.com/news/506366/questions-surround-software-that-adapts-to-students/ (last accessed 13 January 2014)

[5] Published March 11, 2013 http://www.ippr.org/publication/55/10432/an-avalanche-is-coming-higher-education-and-the-revolution-ahead (last accessed 1 January 2014)

[6] Eltjam August 28, 2013 http://www.eltjam.com/elts-collective-consciousness-what-can-we-learn-from-the-questions-we-ask/ (last accessed 13 January 2014)

[7] Tim Gifford Taking it Personally: Adaptive Learning July 9, 2013 http://www.eltjam.com/adaptive-learning/ (last accessed January 13, 2014)

[8] David Wiley, Buying our Way into Bondage: the risks of adaptive learning services March 20,2013 http://opencontent.org/blog/archives/2754 (last accessed January 13, 2014)

Part 5: platforms and more complex adaptive systems

For some years now, universities and other educational institutions around the world have been using online learning platforms, also known as Learning Management Systems (LMSs) or Virtual Learning Environments (VLEs).Well-known versions of these include Blackboard  and Moodle. The latter is used by over 50% of higher education establishments in the UK (Dudeney & Hockly, How to Teach English with Technology Harlow, Essex: Pearson, 2007, p.53). These platforms allow course content – lectures, videos, activities, etc. – to be stored and delivered, and they allow institutions to modify courses to fit their needs. In addition, they usually have inbuilt mechanisms for assessment, tracking of learners, course administration and communication (email, chat, blogs, etc.). While these platforms can be used for courses that are delivered exclusively online, more commonly they are used to manage blended-learning courses (i.e. a mixture of online and face-to-face teaching). The platforms make the running of such courses relatively easy, as they bring together under one roof everything that the institution or teacher needs: ‘tools that have been designed to work together and have the same design ethos, both pedagogically and visually’ (Sharma & Barrett, Blended Learning Oxford: Macmillan, 2007, p.108).

The major ELT publishers all have their own LMSs, sometimes developed by themselves, sometimes developed in partnership with specialist companies. One of the most familiar, because it has been around for a long time, is the Macmillan English Campus. Campus offers both ready-made courses and a mix-and-match option drawing on the thousands of resources available (for grammar, vocabulary, pronunciation and language skills development). Other content can also be uploaded. The platform also offers automatic marking and mark recording, ready-made tests and messaging options.

In the last few years, the situation has changed rapidly. In May 2013, Knewton, the world’s leading adaptive learning technology provider, announced a partnership with Macmillan ‘to build next-generation English Language Learning and Teaching materials’. In September 2013, it was the turn of Cambridge University Press to sign their partnership with Knewton ‘to create personalized learning experiences in [their] industry-leading ELT digital products’. In both cases, Knewton’s adaptive learning technology will be integrated into the publisher’s learning platforms. Pearson, which is also in partnership with Knewton (but not for ELT products), has invested heavily in its MyLab products.

Exactly what will emerge from these new business partnerships and from the continuously evolving technology remains to be seen. The general picture is, however, clearer. We will see an increasing convergence of technologies (administrative systems, educational platforms, communication technologies, big data analytics and adaptive learning) into integrated systems. This will happen first in in-company training departments, universities and colleges of higher education. It is clear already that the ELT divisions of companies like Pearson and Macmillan are beginning to move away from their reliance on printed textbooks for adult learners. This was made graphically clear at the 2013 IATEFL conference in Liverpool when the Pearson exhibition stand had absolutely no books on it (although Pearson now acknowledge this was a ‘mistake). In my next post, I will make a number of more specific predictions about what is coming.

Part 4: big data and analytics

In order to understand more complex models of adaptive learning, it is necessary to take a temporary step sideways away from the world of language learning. Businesses have long used analytics – the analysis of data to find meaningful patterns – in insurance, banking and marketing. With the exponential growth in computer processing power and memory capacity, businesses now have access to volumes of data of almost unimaginable size. This is known as ‘big data’ and has been described as ‘a revolution that will transform how we live, work and think’ (Mayer-Schönberger & Cukier, ‘Big Data’, 2013). Frequently cited examples of the potential of big data are the success of Amazon to analyze and predict buying patterns and the use of big data analysis in Barack Obama’s 2012 presidential re-election. Business commentators are all singing the same song on the subject. This will be looked at again in later posts. For the time being, it is enough to be aware of the main message. ‘The high-performing organisation of the future will be one that places great value on data and analytical exploration’ (The Economist Intelligence Unit, ‘In Search of Insight and Foresight: Getting more out of big data’ 2013, p.15). ‘Almost no sphere of business activity will remain untouched by this movement,’ (McAfee & Brynjolfsson, ‘Big Data: The Management Revolution’, Harvard Business Review (October 2012), p. 65).

With the growing bonds between business and education (another topic which will be explored later), it is unsurprising that language learning / teaching materials are rapidly going down the big data route. In comparison to what is now being developed for ELT, the data that is analyzed in the adaptive learning models I have described in an earlier post is very limited, and the algorithms used to shape the content are very simple.

The volume and variety of data and the speed of processing are now of an altogether different order. Jose Ferreira, CEO of Knewton, one of the biggest players in adaptive learning in ELT, spells out the kind of data that can be tapped[1]:

At Knewton, we divide educational data into five types: one pertaining to student identity and onboarding, and four student activity-based data sets that have the potential to improve learning outcomes. They’re listed below in order of how difficult they are to attain:

1) Identity Data: Who are you? Are you allowed to use this application? What admin rights do you have? What district are you in? How about demographic info?

2) User Interaction Data: User interaction data includes engagement metrics, click rate, page views, bounce rate, etc. These metrics have long been the cornerstone of internet optimization for consumer web companies, which use them to improve user experience and retention. This is the easiest to collect of the data sets that affect student outcomes. Everyone who creates an online app can and should get this for themselves.

3) Inferred Content Data: How well does a piece of content “perform” across a group, or for any one subgroup, of students? What measurable student proficiency gains result when a certain type of student interacts with a certain piece of content? How well does a question actually assess what it intends to? Efficacy data on instructional materials isn’t easy to generate — it requires algorithmically normed assessment items. However it’s possible now for even small companies to “norm” small quantities of items. (Years ago, before we developed more sophisticated methods of norming items at scale, Knewton did so using Amazon’s “Mechanical Turk” service.)

4) System-Wide Data: Rosters, grades, disciplinary records, and attendance information are all examples of system-wide data. Assuming you have permission (e.g. you’re a teacher or principal), this information is easy to acquire locally for a class or school. But it isn’t very helpful at small scale because there is so little of it on a per-student basis. At very large scale it becomes more useful, and inferences that may help inform system-wide recommendations can be teased out.

5) Inferred Student Data: Exactly what concepts does a student know, at exactly what percentile of proficiency? Was an incorrect answer due to a lack of proficiency, or forgetfulness, or distraction, or a poorly worded question, or something else altogether? What is the probability that a student will pass next week’s quiz, and what can she do right this moment to increase it?

Software of this kind keeps complex personal profiles, with millions of variables per student, on as many students as necessary. The more student profiles (and therefore students) that can be compared, the more useful the data is. Big players in this field, such as Knewton, are aiming for student numbers in the tens to hundreds of millions. Once data volume of this order is achieved, the ‘analytics’, or the algorithms that convert data into ‘actionable insights’ (J. Spring, ‘Education Networks’ (New York: Routledge, 2012), p.55) become much more reliable.

---

[1] http://www.knewton.com/blog/knewton/from-jose/2013/07/18/big-data-in-education/ (last accessed 9 December 2013)

Part 3: gamification

An integral part of adaptive learning programs, both the simple models already described and the much more complex systems that are currently under development, is an element of gamification. The term refers to the incorporation of points, levels (analogous to the levels in a typical computer game) and badges into the learning experience. In Duolingo, for example, users have a certain number of ‘lives’ that they can afford to lose without failing an exercise. In addition, they can compare their performance with that of other users, and they can win ‘lingots’, a kind of in-game currency which allows them to ‘buy’ lost ‘lives or to compensate for a day of inactivity.

Gamification and adaptive learning go together like hand in glove because of the data that is generated by the adaptive software (see the next post: Big data, analytics and adaptive learning). The whole thing is premised on comparing the performance of different students, so score cards and leader boards and so on are hardly surprising.

The idea behind this, in case it needs pointing out, is that it can make learning fun and, so, students will be more motivated to do the work, which seems more like play. It is a much hyped idea in education: eltjam referred to the ‘snowballing sexiness’ of the term. In an ELT context, most references to gamification are very positive. See, for example, eltjam’s blog post on the subject or Graham Stanley’s conference presentation on the subject. An excellent infographic summary of and advertisement for the benefits of gamification can be found at the Knewton website.

Not everyone, however, is so positive. Gamification has been described by some writers and researchers as the ‘pointsification’ of everything – the reductionist process of regarding all actions with points and increased personal scores (see, for example, Neil Selwyn, 2013, Distrusting Educational Technology, p.101). The motivation it may generate is clearly extrinsic, and this may not be a good long-term bet. Adults (myself included) get bored of gamification elements very quickly. For both adults and younger learners, once you’ve figured out how to play the system and get extra points (and there’s always a way of finding shortcuts to do this), interest can wane quickly. And once gamification becomes a standard feature of educational experiences (and not just English language learning), its novelty value will disappear.

Part 2: simple models of adaptive learning

‘Adaptive learning’ can mean slightly different things to different people. According to one provider of adaptive learning software (Smart Sparrow https://www.smartsparrow.com/adaptive-elearning), it is ‘an online learning and teaching medium that uses an Intelligent Tutoring System to adapt online learning to the student’s level of knowledge. Adaptive eLearning provides students with customised educational content and the unique feedback that they need, when they need it.’ Essentially, it is software that analyzes the work that a student is doing online, and tailors further learning tasks to the individual learner’s needs (as analyzed by the software).

A relatively simple example of adaptive language learning is Duolingo, a free online service that currently offers seven languages, including English (www.duolingo.com/ ), with over 10 million users in November 2013. Learners progress through a series of translation, dictation and multiple choice exercises that are organised into a ‘skill tree’ of vocabulary and grammar areas. Because translation plays such a central role, the program is only suitable for speakers of one of the languages on offer in combination with one of the other languages on offer. Duolingo’s own blog describes the approach in the following terms: ‘Every time you finish a Duolingo lesson, translation, test, or practice session, you provide valuable data about what you know and what you’re struggling with. Our system uses this info to plan future lessons and select translation tasks specifically for your skills and needs. Similar to how an online store uses your previous purchases to customize your shopping experience, Duolingo uses your learning history to customize your learning experience’ (http://blog.duolingo.com/post/41960192602/duolingos-data-driven-approach-to-education).

Example of a ‘skill tree’ from http://www.duolingo.com

For anyone with a background in communicative language teaching, the experience can be slightly surreal. Examples of sentences that need to be translated include: The dog eats the bird, the boy has a cow, and the fly is eating bread. The system allows you to compete and communicate with other learners, and to win points and rewards (see ‘Gamification’ next post).

Duolingo describes its crowd-sourced, free, adaptive approach as ‘pretty unique’, but uniquely unique it is not. It is essentially a kind of memory trainer, and there are a number available on the market. One of the most well-known is Cerego’s cloud-based iKnow!, which describes itself as a ‘memory management platform’. Particularly strong in Japan, corporate and individual customers pay a monthly subscription to access its English, Chinese and Japanese language programs. A free trial of some of the products is available at http://iknow.jp/  and I experimented with their ‘Erudite English’ program. This presented a series of words which included ‘defalcate’, ‘fleer’ and ‘kvetch’ through English-only definitions, followed by multiple choice and dictated gap-fill exercises. As with Duolingo, there seemed to be no obvious principle behind the choice of items, and example sentences included things like ‘Michael arrogates a slice of carrot cake, unbeknownst to his sister,’ or ‘She found a place in which to posit the flowerpot.’ Based on a user’s performance, Cerego’s algorithms decide which items will be presented, and select the frequency and timing of opportunities for review. The program can be accessed through ordinary computers, as well as iPhone and Android apps. The platform has been designed in such a way as to allow other content to be imported, and then presented and practised in a similar way.

In a similar vein, the Rosetta Stone software also uses spaced repetition to teach grammar and vocabulary. It describes its adaptive learning as ‘Adaptive Recall™’ According to their website, this provides review activities for each lesson ‘at intervals that are determined by your performance in that review. Exceed the program’s expectations for you and the review gets pushed out further. Fall short and you’ll see it sooner. The program gives you a likely date and automatically notifies you when it’s time to take the review again’. Rosetta Stone has won numerous awards and claims that over 20,000 educational institutions around the world have formed partnerships with them. These include the US military, the University of Barcelona and Harrogate Grammar school in the UK (http://www.rosettastone.co.uk/faq ).

Slightly more sophisticated than the memory-trainers described above is the GRE (the Graduate Record Examinations, a test for admission into many graduate schools in the US) online preparation program that is produced by Barron’s (www.barronstestprep.com//gre ). Although this is not an English language course, it provides a useful example of how simple adaptive learning programs can be taken a few steps further. At the time of writing, it is possible to do a free trial, and this gives a good taste of adaptive learning. Barron’s highlights the way that their software delivers individualized study programs: it is not, they say, a case of ‘one size fits all’. After entering the intended test date, the intended number of hours of study, and a simple self-evaluation of different reasoning skills, a diagnostic test completes the information required to set up a personalized ‘prep plan’. This determines the lessons you will be given. As you progress through the course, the ‘prep plan’ adapts to the work that you do, comparing your performance to other students who have taken the course. Measuring your progress and modifying your ‘skill profile’, the order of the lessons and the selection of the 1000+ practice questions can change.