Archive for the ‘platforms’ Category

About two and a half years ago when I started writing this blog, there was a lot of hype around adaptive learning and the big data which might drive it. Two and a half years are a long time in technology. A look at Google Trends suggests that interest in adaptive learning has been pretty static for the last couple of years. It’s interesting to note that 3 of the 7 lettered points on this graph are Knewton-related media events (including the most recent, A, which is Knewton’s latest deal with Hachette) and 2 of them concern McGraw-Hill. It would be interesting to know whether these companies follow both parts of Simon Cowell’s dictum of ‘Create the hype, but don’t ever believe it’.

Google_trends

A look at the Hype Cycle (see here for Wikipedia’s entry on the topic and for criticism of the hype of Hype Cycles) of the IT research and advisory firm, Gartner, indicates that both big data and adaptive learning have now slid into the ‘trough of disillusionment’, which means that the market has started to mature, becoming more realistic about how useful the technologies can be for organizations.

A few years ago, the Gates Foundation, one of the leading cheerleaders and financial promoters of adaptive learning, launched its Adaptive Learning Market Acceleration Program (ALMAP) to ‘advance evidence-based understanding of how adaptive learning technologies could improve opportunities for low-income adults to learn and to complete postsecondary credentials’. It’s striking that the program’s aims referred to how such technologies could lead to learning gains, not whether they would. Now, though, with the publication of a report commissioned by the Gates Foundation to analyze the data coming out of the ALMAP Program, things are looking less rosy. The report is inconclusive. There is no firm evidence that adaptive learning systems are leading to better course grades or course completion. ‘The ultimate goal – better student outcomes at lower cost – remains elusive’, the report concludes. Rahim Rajan, a senior program office for Gates, is clear: ‘There is no magical silver bullet here.’

The same conclusion is being reached elsewhere. A report for the National Education Policy Center (in Boulder, Colorado) concludes: Personalized Instruction, in all its many forms, does not seem to be the transformational technology that is needed, however. After more than 30 years, Personalized Instruction is still producing incremental change. The outcomes of large-scale studies and meta-analyses, to the extent they tell us anything useful at all, show mixed results ranging from modest impacts to no impact. Additionally, one must remember that the modest impacts we see in these meta-analyses are coming from blended instruction, which raises the cost of education rather than reducing it (Enyedy, 2014: 15 -see reference at the foot of this post). In the same vein, a recent academic study by Meg Coffin Murray and Jorge Pérez (2015, ‘Informing and Performing: A Study Comparing Adaptive Learning to Traditional Learning’) found that ‘adaptive learning systems have negligible impact on learning outcomes’.

future-ready-learning-reimagining-the-role-of-technology-in-education-1-638In the latest educational technology plan from the U.S. Department of Education (‘Future Ready Learning: Reimagining the Role of Technology in Education’, 2016) the only mentions of the word ‘adaptive’ are in the context of testing. And the latest OECD report on ‘Students, Computers and Learning: Making the Connection’ (2015), finds, more generally, that information and communication technologies, when they are used in the classroom, have, at best, a mixed impact on student performance.

There is, however, too much money at stake for the earlier hype to disappear completely. Sponsored cheerleading for adaptive systems continues to find its way into blogs and national magazines and newspapers. EdSurge, for example, recently published a report called ‘Decoding Adaptive’ (2016), sponsored by Pearson, that continues to wave the flag. Enthusiastic anecdotes take the place of evidence, but, for all that, it’s a useful read.

In the world of ELT, there are plenty of sales people who want new products which they can call ‘adaptive’ (and gamified, too, please). But it’s striking that three years after I started following the hype, such products are rather thin on the ground. Pearson was the first of the big names in ELT to do a deal with Knewton, and invested heavily in the company. Their relationship remains close. But, to the best of my knowledge, the only truly adaptive ELT product that Pearson offers is the PTE test.

Macmillan signed a contract with Knewton in May 2013 ‘to provide personalized grammar and vocabulary lessons, exam reviews, and supplementary materials for each student’. In December of that year, they talked up their new ‘big tree online learning platform’: ‘Look out for the Big Tree logo over the coming year for more information as to how we are using our partnership with Knewton to move forward in the Language Learning division and create content that is tailored to students’ needs and reactive to their progress.’ I’ve been looking out, but it’s all gone rather quiet on the adaptive / platform front.

In September 2013, it was the turn of Cambridge to sign a deal with Knewton ‘to create personalized learning experiences in its industry-leading ELT digital products for students worldwide’. This year saw the launch of a major new CUP series, ‘Empower’. It has an online workbook with personalized extra practice, but there’s nothing (yet) that anyone would call adaptive. More recently, Cambridge has launched the online version of the 2nd edition of Touchstone. Nothing adaptive there, either.

Earlier this year, Cambridge published The Cambridge Guide to Blended Learning for Language Teaching, edited by Mike McCarthy. It contains a chapter by M.O.Z. San Pedro and R. Baker on ‘Adaptive Learning’. It’s an enthusiastic account of the potential of adaptive learning, but it doesn’t contain a single reference to language learning or ELT!

So, what’s going on? Skepticism is becoming the order of the day. The early hype of people like Knewton’s Jose Ferreira is now understood for what it was. Companies like Macmillan got their fingers badly burnt when they barked up the wrong tree with their ‘Big Tree’ platform.

Noel Enyedy captures a more contemporary understanding when he writes: Personalized Instruction is based on the metaphor of personal desktop computers—the technology of the 80s and 90s. Today’s technology is not just personal but mobile, social, and networked. The flexibility and social nature of how technology infuses other aspects of our lives is not captured by the model of Personalized Instruction, which focuses on the isolated individual’s personal path to a fixed end-point. To truly harness the power of modern technology, we need a new vision for educational technology (Enyedy, 2014: 16).

Adaptive solutions aren’t going away, but there is now a much better understanding of what sorts of problems might have adaptive solutions. Testing is certainly one. As the educational technology plan from the U.S. Department of Education (‘Future Ready Learning: Re-imagining the Role of Technology in Education’, 2016) puts it: Computer adaptive testing, which uses algorithms to adjust the difficulty of questions throughout an assessment on the basis of a student’s responses, has facilitated the ability of assessments to estimate accurately what students know and can do across the curriculum in a shorter testing session than would otherwise be necessary. In ELT, Pearson and EF have adaptive tests that have been well researched and designed.

Vocabulary apps which deploy adaptive technology continue to become more sophisticated, although empirical research is lacking. Automated writing tutors with adaptive corrective feedback are also developing fast, and I’ll be writing a post about these soon. Similarly, as speech recognition software improves, we can expect to see better and better automated adaptive pronunciation tutors. But going beyond such applications, there are bigger questions to ask, and answers to these will impact on whatever direction adaptive technologies take. Large platforms (LMSs), with or without adaptive software, are already beginning to look rather dated. Will they be replaced by integrated apps, or are apps themselves going to be replaced by bots (currently riding high in the Hype Cycle)? In language learning and teaching, the future of bots is likely to be shaped by developments in natural language processing (another topic about which I’ll be blogging soon). Nobody really has a clue where the next two and a half years will take us (if anywhere), but it’s becoming increasingly likely that adaptive learning will be only one very small part of it.

 

Enyedy, N. 2014. Personalized Instruction: New Interest, Old Rhetoric, Limited Results, and the Need for a New Direction for Computer-Mediated Learning. Boulder, CO: National Education Policy Center. Retrieved 17.07.16 from http://nepc.colorado.edu/publication/personalized-instruction

Voxy is another language learning platform that likes to tout itself as ‘the future of language learning’. It has over 2.5 million users and claims to be the No. 1 education iTunes app in 23 countries. Pearson is a major investor and has a seat on the Voxy board. Unsurprisingly, it boasts ‘a new sophisticated and patented adaptive learning technology, […] a dynamic feedback loop which results in lessons and courses that calibrate to the learner. These improvements are fundamental to what makes Voxy unique as lessons become even more personalized.’

Voxy

Voxy uses an integrated web / mobile / SMS platform to deliver its learning programme, which is based around authentic, up-to-date texts. I spent a morning as an advanced learner of English exploring what it had to offer. In what I did, everything was in English, but I imagine this is not the case for lower-level learners. Voxy was originally launched for speakers of Spanish and Portuguese.

As far as I could tell, there is very little that is (what I would call) adaptive. There is, no doubt, adaptive software at work in the vocabulary revision exercises, but it’s hard to see this operating. Before starting, users are asked about their level and what they want to ‘accomplish with English’. The six possible answers are ‘advance my career’, ‘enjoy English media’, ‘pass my English tests’, ‘travel abroad’. ‘day-to-day tasks’ and ‘social and lifestyle’. I was next asked about my interests, and the possible answers here were sports, celebrities and entertainment, business, technology, health and politics. Having answered these questions, my personalized course was ready.

I was offered a deal of $20 a month, with a free trial. This gave me access to the main course, a faily rudimentary grammar guide, a list of words I had ‘studied’, a proficiency test (reading, listening and TOEFL-style M/C grammar) and 13 hours with a ‘live’ tutor.

I decided that I couldn’t pretend to be a real learner and hook up with a tutor. Users can choose a tutor from a menu where the tutors are photographed (obligatory smile). They are young graduates and some, but not all, are described as having ‘Certification: Teaching English’, whatever this means. There are also tutor statements, one of which reads ‘I love that both teaching and studying foreign languages are abound with opportunities to experience international differences and similarities on a personal level’ (sic).

I concentrated on the main course which offered 18 lessons related to each of my declared interests. These were based on authentic texts from sources like Financial Times and New York Daily News. These were generally interesting and up-to-date. In some cases, the article was only 24 hours old.

The usual procedure was to (1) read the text, (2) tap on highlighted words, which would bring up dictionary definitions and a recording of the word, (3) listen to a recording of the text (read very slowly – far too slowly for anyone with an advanced level), (4) answer 2 -4 multiple choice questions, (5) be shown short gapped extracts from the text alongside 4 or 5 boxes, which, when you click on them gave a recording of different words, one of which was the correct answer to the highlighted gap in the text, and (6) do a word – definition matching task (the words from stage 5).

According to Wikipedia, Voxy is based on the principles of task-based language teaching. Jane Willis might beg to differ. What I saw was closer to those pre-1970s textbooks where texts were followed by glossaries. Voxy is technologically advanced, but methodologically, it is positively antediluvian.

A further problem concerns task design. Perhaps because the tasks that accompany the texts have to be produced very quickly (if the texts are really to be hot off the press), there were errors that no experienced materials writer would make, and no experienced ELT editor would fail to spot. The sorts of problems that I identifed included the following:

  • No clear rationale in the selection of vocabulary items; no apparent awareness of word difficulty or frequency.
  • No clear rationale in the selection of multiple choice items.
  • Many M/C vocabulary questions can be answered without understanding the word (simply by using the memory).
  • Vocabulary definition matching tasks often contain language in the definitions which is more complex than the target item.
  • The vocabulary definition matching tasks can mostly be done simply by eliminating the distractors (which have been plucked out of thin air, and have not previously appeared).
  • The definitions in these matching tasks often do not use the same grammar as the target item (e.g. an infinitive in the definition has to be matched to a participle target word).
  • Errors (e.g. ‘The brain reacts more strongly to rejection in real life that online rejection’ (sic) in one M/C item).

I could go on. The material has clearly not been written by experienced writers, it has not been properly edited or trialled. The texts may be interesting, but that’s the only positive that I can offer for the main part of the course that I looked at.

My greatest disappointment concerns the poor use that the technology has been put to. Contrary to Voxy’s claims, this is not a new way to learn a language, it’s not particularly fun and it’s hard to believe that it could be effective. Perhaps my, admittedly limited, experience with Voxy’s product was unrepresentative. Using authentic materials is a good idea, but this needs to be combined with decent social networking possibilities, a much more sophisticated use of adaptive technology, proper investment in item-writers and editors, and more. The future of language learning? Probably not.

voxy_2