Posts Tagged ‘flash cards’

About two and a half years ago when I started writing this blog, there was a lot of hype around adaptive learning and the big data which might drive it. Two and a half years are a long time in technology. A look at Google Trends suggests that interest in adaptive learning has been pretty static for the last couple of years. It’s interesting to note that 3 of the 7 lettered points on this graph are Knewton-related media events (including the most recent, A, which is Knewton’s latest deal with Hachette) and 2 of them concern McGraw-Hill. It would be interesting to know whether these companies follow both parts of Simon Cowell’s dictum of ‘Create the hype, but don’t ever believe it’.

Google_trends

A look at the Hype Cycle (see here for Wikipedia’s entry on the topic and for criticism of the hype of Hype Cycles) of the IT research and advisory firm, Gartner, indicates that both big data and adaptive learning have now slid into the ‘trough of disillusionment’, which means that the market has started to mature, becoming more realistic about how useful the technologies can be for organizations.

A few years ago, the Gates Foundation, one of the leading cheerleaders and financial promoters of adaptive learning, launched its Adaptive Learning Market Acceleration Program (ALMAP) to ‘advance evidence-based understanding of how adaptive learning technologies could improve opportunities for low-income adults to learn and to complete postsecondary credentials’. It’s striking that the program’s aims referred to how such technologies could lead to learning gains, not whether they would. Now, though, with the publication of a report commissioned by the Gates Foundation to analyze the data coming out of the ALMAP Program, things are looking less rosy. The report is inconclusive. There is no firm evidence that adaptive learning systems are leading to better course grades or course completion. ‘The ultimate goal – better student outcomes at lower cost – remains elusive’, the report concludes. Rahim Rajan, a senior program office for Gates, is clear: ‘There is no magical silver bullet here.’

The same conclusion is being reached elsewhere. A report for the National Education Policy Center (in Boulder, Colorado) concludes: Personalized Instruction, in all its many forms, does not seem to be the transformational technology that is needed, however. After more than 30 years, Personalized Instruction is still producing incremental change. The outcomes of large-scale studies and meta-analyses, to the extent they tell us anything useful at all, show mixed results ranging from modest impacts to no impact. Additionally, one must remember that the modest impacts we see in these meta-analyses are coming from blended instruction, which raises the cost of education rather than reducing it (Enyedy, 2014: 15 -see reference at the foot of this post). In the same vein, a recent academic study by Meg Coffin Murray and Jorge Pérez (2015, ‘Informing and Performing: A Study Comparing Adaptive Learning to Traditional Learning’) found that ‘adaptive learning systems have negligible impact on learning outcomes’.

future-ready-learning-reimagining-the-role-of-technology-in-education-1-638In the latest educational technology plan from the U.S. Department of Education (‘Future Ready Learning: Reimagining the Role of Technology in Education’, 2016) the only mentions of the word ‘adaptive’ are in the context of testing. And the latest OECD report on ‘Students, Computers and Learning: Making the Connection’ (2015), finds, more generally, that information and communication technologies, when they are used in the classroom, have, at best, a mixed impact on student performance.

There is, however, too much money at stake for the earlier hype to disappear completely. Sponsored cheerleading for adaptive systems continues to find its way into blogs and national magazines and newspapers. EdSurge, for example, recently published a report called ‘Decoding Adaptive’ (2016), sponsored by Pearson, that continues to wave the flag. Enthusiastic anecdotes take the place of evidence, but, for all that, it’s a useful read.

In the world of ELT, there are plenty of sales people who want new products which they can call ‘adaptive’ (and gamified, too, please). But it’s striking that three years after I started following the hype, such products are rather thin on the ground. Pearson was the first of the big names in ELT to do a deal with Knewton, and invested heavily in the company. Their relationship remains close. But, to the best of my knowledge, the only truly adaptive ELT product that Pearson offers is the PTE test.

Macmillan signed a contract with Knewton in May 2013 ‘to provide personalized grammar and vocabulary lessons, exam reviews, and supplementary materials for each student’. In December of that year, they talked up their new ‘big tree online learning platform’: ‘Look out for the Big Tree logo over the coming year for more information as to how we are using our partnership with Knewton to move forward in the Language Learning division and create content that is tailored to students’ needs and reactive to their progress.’ I’ve been looking out, but it’s all gone rather quiet on the adaptive / platform front.

In September 2013, it was the turn of Cambridge to sign a deal with Knewton ‘to create personalized learning experiences in its industry-leading ELT digital products for students worldwide’. This year saw the launch of a major new CUP series, ‘Empower’. It has an online workbook with personalized extra practice, but there’s nothing (yet) that anyone would call adaptive. More recently, Cambridge has launched the online version of the 2nd edition of Touchstone. Nothing adaptive there, either.

Earlier this year, Cambridge published The Cambridge Guide to Blended Learning for Language Teaching, edited by Mike McCarthy. It contains a chapter by M.O.Z. San Pedro and R. Baker on ‘Adaptive Learning’. It’s an enthusiastic account of the potential of adaptive learning, but it doesn’t contain a single reference to language learning or ELT!

So, what’s going on? Skepticism is becoming the order of the day. The early hype of people like Knewton’s Jose Ferreira is now understood for what it was. Companies like Macmillan got their fingers badly burnt when they barked up the wrong tree with their ‘Big Tree’ platform.

Noel Enyedy captures a more contemporary understanding when he writes: Personalized Instruction is based on the metaphor of personal desktop computers—the technology of the 80s and 90s. Today’s technology is not just personal but mobile, social, and networked. The flexibility and social nature of how technology infuses other aspects of our lives is not captured by the model of Personalized Instruction, which focuses on the isolated individual’s personal path to a fixed end-point. To truly harness the power of modern technology, we need a new vision for educational technology (Enyedy, 2014: 16).

Adaptive solutions aren’t going away, but there is now a much better understanding of what sorts of problems might have adaptive solutions. Testing is certainly one. As the educational technology plan from the U.S. Department of Education (‘Future Ready Learning: Re-imagining the Role of Technology in Education’, 2016) puts it: Computer adaptive testing, which uses algorithms to adjust the difficulty of questions throughout an assessment on the basis of a student’s responses, has facilitated the ability of assessments to estimate accurately what students know and can do across the curriculum in a shorter testing session than would otherwise be necessary. In ELT, Pearson and EF have adaptive tests that have been well researched and designed.

Vocabulary apps which deploy adaptive technology continue to become more sophisticated, although empirical research is lacking. Automated writing tutors with adaptive corrective feedback are also developing fast, and I’ll be writing a post about these soon. Similarly, as speech recognition software improves, we can expect to see better and better automated adaptive pronunciation tutors. But going beyond such applications, there are bigger questions to ask, and answers to these will impact on whatever direction adaptive technologies take. Large platforms (LMSs), with or without adaptive software, are already beginning to look rather dated. Will they be replaced by integrated apps, or are apps themselves going to be replaced by bots (currently riding high in the Hype Cycle)? In language learning and teaching, the future of bots is likely to be shaped by developments in natural language processing (another topic about which I’ll be blogging soon). Nobody really has a clue where the next two and a half years will take us (if anywhere), but it’s becoming increasingly likely that adaptive learning will be only one very small part of it.

 

Enyedy, N. 2014. Personalized Instruction: New Interest, Old Rhetoric, Limited Results, and the Need for a New Direction for Computer-Mediated Learning. Boulder, CO: National Education Policy Center. Retrieved 17.07.16 from http://nepc.colorado.edu/publication/personalized-instruction

Advertisements

I have been putting in a lot of time studying German vocabulary with Memrise lately, but this is not a review of the Memrise app. For that, I recommend you read Marek Kiczkowiak’s second post on this app. Like me, he’s largely positive, although I am less enthusiastic about Memrise’s USP, the use of mnemonics. It’s not that mnemonics don’t work – there’s a lot of evidence that they do: it’s just that there is little or no evidence that they’re worth the investment of time.

Time … as I say, I have been putting in the hours. Every day, for over a month, averaging a couple of hours a day, it’s enough to get me very near the top of the leader board (which I keep a very close eye on) and it means that I am doing more work than 99% of other users. And, yes, my German is improving.

Putting in the time is the sine qua non of any language learning and a well-designed app must motivate users to do this. Relevant content will be crucial, as will satisfactory design, both visual and interactive. But here I’d like to focus on the two other key elements: task design / variety and gamification.

Memrise offers a limited range of task types: presentation cards (with word, phrase or sentence with translation and audio recording), multiple choice (target item with four choices), unscrambling letters or words, and dictation (see below).

Screenshot_2016-05-24-08-10-42Screenshot_2016-05-24-08-10-57Screenshot_2016-05-24-08-11-24Screenshot_2016-05-24-08-11-45Screenshot_2016-05-24-08-12-51Screenshot_2016-05-24-08-13-44

As Marek writes, it does get a bit repetitive after a while (although less so than thumbing through a pack of cardboard flashcards). The real problem, though, is that there are only so many things an app designer can do with standard flashcards, if they are to contribute to learning. True, there could be a few more game-like tasks (as with Quizlet), races against the clock as you pop word balloons or something of the sort, but, while these might, just might, help with motivation, these games rarely, if ever, contribute much to learning.

What’s more, you’ll get fed up with the games sooner or later if you’re putting in serious study hours. Even if Memrise were to double the number of activity types, I’d have got bored with them by now, in the same way I got bored with the Quizlet games. Bear in mind, too, that I’ve only done a month: I have at least another two months to go before I finish the level I’m working on. There’s another issue with ‘fun’ activities / games which I’ll come on to later.

The options for task variety in vocabulary / memory apps are therefore limited. Let’s look at gamification. Memrise has leader boards (weekly, monthly, ‘all time’), streak badges, daily goals, email reminders and (in the laptop and premium versions) a variety of graphs that allow you to analyse your study patterns. Your degree of mastery of learning items is represented by a growing flower that grows leaves, flowers and withers. None of this is especially original or different from similar apps.

Screenshot_2016-05-24-19-17-14The trouble with all of this is that it can only work for a certain time and, for some people, never. There’s always going to be someone like me who can put in a couple of hours a day more than you can. Or someone, in my case, like ‘Nguyenduyha’, who must be doing about four hours a day, and who, I know, is out of my league. I can’t compete and the realisation slowly dawns that my life would be immeasurably sadder if I tried to.

Having said that, I have tried to compete and the way to do so is by putting in the time on the ‘speed review’. This is the closest that Memrise comes to a game. One hundred items are flashed up with four multiple choices and these are against the clock. The quicker you are, the more points you get, and if you’re too slow, or you make a mistake, you lose a life. That’s how you gain lots of points with Memrise. The problem is that, at best, this task only promotes receptive knowledge of the items, which is not what I need by this stage. At worst, it serves no useful learning function at all because I have learnt ways of doing this well which do not really involve me processing meaning at all. As Marek says in his post (in reference to Quizlet), ‘I had the feeling that sometimes I was paying more attention to ‘winning’ the game and scoring points, rather than to the words on the screen.’ In my case, it is not just a feeling: it’s an absolute certainty.

desktop_dashboard

Sadly, the gamification is working against me. The more time I spend on the U-Bahn doing Memrise, the less time I spend reading the free German-language newspapers, the less time I spend eavesdropping on conversations. Two hours a day is all I have time for for my German study, and Memrise is eating it all up. I know that there are other, and better, ways of learning. In order to do what I know I should be doing, I need to ignore the gamification. For those, more reasonable, students, who can regularly do their fifteen minutes a day, day in – day out, the points and leader boards serve no real function at all.

Cheating at gamification, or gaming the system, is common in app-land. A few years ago, Memrise had to take down their leader board when they realised that cheating was taking place. There’s an inexorable logic to this: gamification is an attempt to motivate by rewarding through points, rather than the reward coming from the learning experience. The logic of the game overtakes itself. Is ‘Nguyenduyha’ cheating, or do they simply have nothing else to do all day? Am I cheating by finding time to do pointless ‘speed reviews’ that earn me lots of points?

For users like myself, then, gamification design needs to be a delicate balancing act. For others, it may be largely an irrelevance. I’ve been working recently on a general model of vocabulary app design that looks at two very different kinds of user. On the one hand, there are the self-motivated learners like myself or the millions of other who have chosen to use self-study apps. On the other, there are the millions of students in schools and colleges, studying English among other subjects, some of whom are now being told to use the vocabulary apps that are beginning to appear packaged with their coursebooks (or other learning material). We’ve never found entirely satisfactory ways of making these students do their homework, and the fact that this homework is now digital will change nothing (except, perhaps, in the very, very short term). The incorporation of games and gamification is unlikely to change much either: there will always be something more interesting and motivating (and unconnected with language learning) elsewhere.

Teachers and college principals may like the idea of gamification (without having really experienced it themselves) for their students. But more important for most of them is likely to be the teacher dashboard: the means by which they can check that their students are putting the time in. Likewise, they will see the utility of automated email reminders that a student is not working hard enough to meet their learning objectives, more and more regular tests that contribute to overall course evaluation, comparisons with college, regional or national benchmarks. Technology won’t solve the motivation issue, but it does offer efficient means of control.

If you’re going to teach vocabulary, you need to organise it in some way. Almost invariably, this organisation is topical, with words grouped into what are called semantic sets. In coursebooks, the example below (from Rogers, M., Taylore-Knowles, J. & S. Taylor-Knowles. 2010. Open Mind Level 1. London: Macmillan, p.68) is fairly typical.

open mind

Coursebooks are almost always organised in a topical way. The example above comes in a unit (of 10 pages), entitled ‘You have talent!’, which contains two main vocabulary sections. It’s unsurprising to find a section called ‘personality adjectives’ in such a unit. What’s more, such an approach lends itself to the requisite, but largely, spurious ‘can-do’ statement in the self-evaluation section: I can talk about people’s positive qualities. We must have clearly identifiable learning outcomes, after all.

There is, undeniably, a certain intuitive logic in this approach. An alternative might entail a radical overhaul of coursebook architecture – this might not be such a bad thing, but might not go down too well in the markets. How else, after all, could the vocabulary strand of the syllabus be organised?

Well, there are a number of ways in which a vocabulary syllabus could be organised. Including the standard approach described above, here are four possibilities:

1 semantic sets (e.g. bee, butterfly, fly, mosquito, etc.)

2 thematic sets (e.g. ‘pets’: cat, hate, flea, feed, scratch, etc.)

3 unrelated sets

4 sets determined by a group of words’ occurrence in a particular text

Before reading further, you might like to guess what research has to say about the relative effectiveness of these four approaches.

The answer depends, to some extent, on the level of the learner. For advanced learners, it appears to make no, or little, difference (Al-Jabri, 2005, cited by Ellis & Shintani, 2014: 106). But, for the vast majority of English language learners (i.e. those at or below B2 level), the research is clear: the most effective way of organising vocabulary items to be learnt is by grouping them into thematic sets (2) or by mixing words together in a semantically unrelated way (3) – not by teaching sets like ‘personality adjectives’. It is surprising how surprising this finding is to so many teachers and materials writers. It goes back at least to 1988 and West’s article on ‘Catenizing’ in ELTJ, which argued that semantic grouping made little sense from a psycho-linguistic perspective. Since then, a large amount of research has taken place. This is succinctly summarised by Paul Nation (2013: 128) in the following terms: Avoid interference from related words. Words which are similar in form (Laufer, 1989) or meaning (Higa, 1963; Nation, 2000; Tinkham, 1993; Tinkham, 1997; Waring, 1997) are more difficult to learn together than they are to learn separately. For anyone who is interested, the most up-to-date review of this research that I can find is in chapter 11 of Barcroft (2105).

The message is clear. So clear that you have to wonder how it is not getting through to materials designers. Perhaps, coursebooks are different. They regularly eschew research findings for commercial reasons. But vocabulary apps? There is rarely, if ever, any pressure on the content-creation side of vocabulary apps (except those that are tied to coursebooks) to follow the popular misconceptions that characterise so many coursebooks. It wouldn’t be too hard to organise vocabulary into thematic sets (like, for example, the approach in the A2 level of Memrise German that I’m currently using). Is it simply because the developers of so many vocabulary apps just don’t know much about language learning?

References

Barcroft, J. 2015. Lexical Input Processing and Vocabulary Learning. Amsterdam: John Benjamins

Nation, I. S. P. 2013. Learning Vocabulary in Another Language 2nd edition. Cambridge: Cambridge University Press

Ellis, R. & N. Shintani, N. 2014. Exploring Language Pedagogy through Second Language Acquisition Research. Abingdon, Oxon: Routledge

West, M. 1988. ‘Catenizing’ English Language Teaching Journal 6: 147 – 151

Having spent a lot of time recently looking at vocabulary apps, I decided to put together a Christmas wish list of the features of my ideal vocabulary app. The list is not exhaustive and I’ve given more attention to some features than others. What (apart from testing) have I missed out?

1             Spaced repetition

Since the point of a vocabulary app is to help learners memorise vocabulary items, it is hard to imagine a decent system that does not incorporate spaced repetition. Spaced repetition algorithms offer one well-researched way of improving the brain’s ‘forgetting curve’. These algorithms come in different shapes and sizes, and I am not technically competent to judge which is the most efficient. However, as Peter Ellis Jones, the developer of a flashcard system called CardFlash, points out, efficiency is only one half of the rote memorisation problem. If you are not motivated to learn, the cleverness of the algorithm is moot. Fundamentally, learning software needs to be fun, rewarding, and give a solid sense of progression.

2             Quantity, balance and timing of new and ‘old’ items

A spaced repetition algorithm determines the optimum interval between repetitions, but further algorithms will be needed to determine when and with what frequency new items will be added to the deck. Once a system knows how many items a learner needs to learn and the time in which they have to do it, it is possible to determine the timing and frequency of the presentation of new items. But the system cannot know in advance how well an individual learner will learn the items (for any individual, some items will be more readily learnable than others) nor the extent to which learners will live up to their own positive expectations of time spent on-app. As most users of flashcard systems know, it is easy to fall behind, feel swamped and, ultimately, give up. An intelligent system needs to be able to respond to individual variables in order to ensure that the learning load is realistic.

3             Task variety

A standard flashcard system which simply asks learners to indicate whether they ‘know’ a target item before they flip over the card rapidly becomes extremely boring. A system which tests this knowledge soon becomes equally dull. There needs to be a variety of ways in which learners interact with an app, both for reasons of motivation and learning efficiency. It may be the case that, for an individual user, certain task types lead to more rapid gains in learning. An intelligent, adaptive system should be able to capture this information and modify the selection of task types.

Most younger learners and some adult learners will respond well to the inclusion of games within the range of task types. Examples of such games include the puzzles developed by Oliver Rose in his Phrase Maze app to accompany Quizlet practice.Phrase Maze 1Phrase Maze 2

4             Generative use

Memory researchers have long known about the ‘Generation Effect’ (see for example this piece of research from the Journal of Verbal Learning and Learning Behavior, 1978). Items are better learnt when the learner has to generate, in some (even small) way, the target item, rather than simply reading it. In vocabulary learning, this could be, for example, typing in the target word or, more simply, inserting some missing letters. Systems which incorporate task types that require generative use are likely to result in greater learning gains than simple, static flashcards with target items on one side and definitions or translations on the other.

5             Receptive and productive practice

The most basic digital flashcard systems require learners to understand a target item, or to generate it from a definition or translation prompt. Valuable as this may be, it won’t help learners much to use these items productively, since these systems focus exclusively on meaning. In order to do this, information must be provided about collocation, colligation, register, etc and these aspects of word knowledge will need to be focused on within the range of task types. At the same time, most vocabulary apps that I have seen focus primarily on the written word. Although any good system will offer an audio recording of the target item, and many will offer the learner the option of recording themselves, learners are invariably asked to type in their answers, rather than say them. For the latter, speech recognition technology will be needed. Ideally, too, an intelligent system will compare learner recordings with the audio models and provide feedback in such a way that the learner is guided towards a closer reproduction of the model.

6             Scaffolding and feedback

feebuMost flashcard systems are basically low-stakes, practice self-testing. Research (see, for example, Dunlosky et al’s metastudy ‘Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology’) suggests that, as a learning strategy, practice testing has high utility – indeed, of higher utility than other strategies like keyword mnemonics or highlighting. However, an element of tutoring is likely to enhance practice testing, and, for this, scaffolding and feedback will be needed. If, for example, a learner is unable to produce a correct answer, they will probably benefit from being guided towards it through hints, in the same way as a teacher would elicit in a classroom. Likewise, feedback on why an answer is wrong (as opposed to simply being told that you are wrong), followed by encouragement to try again, is likely to enhance learning. Such feedback might, for example, point out that there is perhaps a spelling problem in the learner’s attempted answer, that the attempted answer is in the wrong part of speech, or that it is semantically close to the correct answer but does not collocate with other words in the text. The incorporation of intelligent feedback of this kind will require a number of NLP tools, since it will never be possible for a human item-writer to anticipate all the possible incorrect answers. A current example of intelligent feedback of this kind can be found in the Oxford English Vocabulary Trainer app.

7             Content

At the very least, a decent vocabulary app will need good definitions and translations (how many different languages?), and these will need to be tagged to the senses of the target items. These will need to be supplemented with all the other information that you find in a good learner’s dictionary: syntactic patterns, collocations, cognates, an indication of frequency, etc. The only way of getting this kind of high-quality content is by paying to license it from a company with expertise in lexicography. It doesn’t come cheap.

There will also need to be example sentences, both to illustrate meaning / use and for deployment in tasks. Dictionary databases can provide some of these, but they cannot be relied on as a source. This is because the example sentences in dictionaries have been selected and edited to accompany the other information provided in the dictionary, and not as items in practice exercises, which have rather different requirements. Once more, the solution doesn’t come cheap: experienced item writers will be needed.

Dictionaries describe and illustrate how words are typically used. But examples of typical usage tend to be as dull as they are forgettable. Learning is likely to be enhanced if examples are cognitively salient: weird examples with odd collocations, for example. Another thing for the item writers to think about.

A further challenge for an app which is not level-specific is that both the definitions and example sentences need to be level-specific. An A1 / A2 learner will need the kind of content that is found in, say, the Oxford Essential dictionary; B2 learners and above will need content from, say, the OALD.

8             Artwork and design

My wordbook2It’s easy enough to find artwork or photos of concrete nouns, but try to find or commission a pair of pictures that differentiate, for example, the adjectives ‘wild’ and ‘dangerous’ … What kind of pictures might illustrate simple verbs like ‘learn’ or ‘remember’? Will such illustrations be clear enough when squeezed into a part of a phone screen? Animations or very short video clips might provide a solution in some cases, but these are more expensive to produce and video files are much heavier.

With a few notable exceptions, such as the British Councils’s MyWordBook 2, design in vocabulary apps has been largely forgotten.

9             Importable and personalisable lists

Many learners will want to use a vocabulary app in association with other course material (e.g. coursebooks). Teachers, however, will inevitably want to edit these lists, deleting some items, adding others. Learners will want to do the same. This is a huge headache for app designers. If new items are going to be added to word lists, how will the definitions, example sentences and illustrations be generated? Will the database contain audio recordings of these words? How will these items be added to the practice tasks (if these include task types that go beyond simple double-sided flashcards)? NLP tools are not yet good enough to trawl a large corpus in order to select (and possibly edit) sentences that illustrate the right meaning and which are appropriate for interactive practice exercises. We can personalise the speed of learning and even the types of learning tasks, so long as the target language is predetermined. But as soon as we allow for personalisation of content, we run into difficulties.

10          Gamification

Maintaining motivation to use a vocabulary app is not easy. Gamification may help. Measuring progress against objectives will be a start. Stars and badges and leaderboards may help some users. Rewards may help others. But gamification features need to be built into the heart of the system, into the design and selection of tasks, rather than simply tacked on as an afterthought. They need to be trialled and tweaked, so analytics will be needed.

11          Teacher support

Although the use of vocabulary flashcards is beginning to catch on with English language teachers, teachers need help with ways to incorporate them in the work they do with their students. What can teachers do in class to encourage use of the app? In what ways does app use require teachers to change their approach to vocabulary work in the classroom? Reporting functions can help teachers know about the progress their students are making and provide very detailed information about words that are causing problems. But, as anyone involved in platform-based course materials knows, teachers need a lot of help.

12          And, of course, …

Apps need to be usable with different operating systems. Ideally, they should be (partially) usable offline. Loading times need to be short. They need to be easy and intuitive to use.

It’s unlikely that I’ll be seeing a vocabulary app with all of these features any time soon. Or, possibly, ever. The cost of developing something that could do all this would be extremely high, and there is no indication that there is a market that would be ready to pay the sort of prices that would be needed to cover the costs of development and turn a profit. We need to bear in mind, too, the fact that vocabulary apps can only ever assist in the initial acquisition of vocabulary: apps alone can’t solve the vocabulary learning problem (despite the silly claims of some app developers). The need for meaningful communicative use, extensive reading and listening, will not go away because a learner has been using an app. So, how far can we go in developing better and better vocabulary apps before users decide that a cheap / free app, with all its shortcomings, is actually good enough?

I posted a follow up to this post in October 2016.

VocApp – a review

Posted: October 28, 2015 in apps
Tags: , , ,

Go to an app store and you’ll find a number of unrelated products called VocApp. One of them, from a Polish-based outfit, has the https://vocapp.com/ url. From over 30 products in the catalogue, I selected the free ‘Top 1000 English Words’: this is, after all, the showcase app which will show you how fast and easy you can learn with us (sic). VocApp Founder, Marcin Młodzki, writes that learning languages and mobile devices are my two greatest passions. Unfortunately there wasn’t any language app on the market which satisfied me in 100% (or even in 70%…). Anki, Babel, DuoLingo, Memorize, Quizlet – each of them has some serious disadvantages. So I decided to create my own app. Prof. Ewa Lajer-Burchardt of Harvard University says it’s undoubtedly one of the best flashcard applications for learning foreign languages on the educational market. This is presumably the eminent Ewa Lajer-Burcharth, a Polish art historian and author of Necklines: The Art of Jacques-Louis David After the Terror. So, how does the app stand up? Will users raise their understanding up to 83%? I was impatient to find out.common english wordsIt’s a flashcard system with spaced repetition. This particular app has target items and audio recordings on one side of the flashcard, definitions in English, along with illustrations, on the other. It is, the makers say, multisensory. Users are then given two self-evaluation options.

ab

And that, I’m afraid, is about all there is to say. Apart, that is, from the content. Many of the definitions have been culled from Wiktionary, not perhaps the best source of definitions for A1 / A2 learners. Others appear to have been made up in-house. Here is an opportunity to raise your own understanding by up to 83%. Look at the VocApp definitions below and see if you can guess what the target word is (answers below[i]).

1 a piece of a whole

2 a) a kind of box b) a formal word for a situation

3 something people do every day e.g. from 10 o’clock to 4 o’clock to get money

4 a group of people who deal with politics and who give new rules

5 when we are born our life begins, when we die our life comes to an end.

6 an object

7 a) where the cars drive b) a method of doing something

8 The place where we live, not only the Earth, everything which exists; ‘world’ is a general world

9 a location of something

10 a) 24 hours b) when the sun is up, not night

Sorry, Marcin. I’m afraid your app didn’t satisfy me in 100% (or even in 70%…).

[i] Answers: 1 part 2 case 3 work 4 government 5 life 6 thing 7 way 8 world 9 place 10 day

MosaLingua  (with the obligatory capital letter in the middle) is a vocabulary app, available for iOS and Android. There are packages for a number of languages and English variations include general English, business English, vocabulary for TOEFL and vocabulary for TOEIC. The company follows the freemium model, with free ‘Lite’ versions and fuller content selling for €4.99. I tried the ‘Lite’ general English app, opting for French as my first language. Since the app is translation-based, you need to have one of the language pairings that are on offer (the other languages are currently Italian, Spanish, Portuguese and German).Mosalingua

The app I looked at is basically a phrase book with spaced repetition. Even though this particular app was general English, it appeared to be geared towards the casual business traveller. It uses the same algorithm as Anki, and users are taken through a sequence of (1) listening to an audio recording of the target item (word or phrase) along with the possibility of comparing a recording of yourself with the recording provided, (2) standard bilingual flashcard practice, (3) a practice stage where you are given the word or phrase in your own language and you have to unscramble words or letters to form the equivalent in English, and (4) a self-evaluation stage where users select from one of four options (“review”, “hard”, “good”, “perfect”) where the choice made will influence the re-presentation of the item within the spaced repetition.

In addition to these words and phrases, there are a number of dialogues where you (1) listen to the dialogue (‘without worrying about understanding everything’), (2) are re-exposed to the dialogue with English subtitles, (3) see it again with subtitles in your own language, (4) practise it with standard flashcards.

The developers seem to be proud of their Mosa Learning Method®: they’ve registered this as a trademark. At its heart is spaced repetition. This is supplemented by what they refer to as ‘Active Recall’, the notion that things are better memorised if the learner has to make some sort of cognitive effort, however minimal, in recalling the target items. The principle is, at least to me, unquestionable, but the realisation (unjumbling words or letters) becomes rather repetitive and, ultimately, tedious. Then, there is what they call ‘metacognition’. Again, this is informed by research, even if the realisation (self-evaluation of learning difficulty into four levels) is extremely limited. Then there is the Pareto principle  – the 80-20 rule. I couldn’t understand the explanation of what this has to do with the trademarked method. Here’s the MosaLingua explanation  – figure it out for yourself:

Did you know that the 100 most common words in English account for half of the written corpus?

Evidently, you shouldn’t quit after learning only 100 words. Instead, you should concentrate on the most frequently used words and you’ll make spectacular progress. What’s more, globish (global English) has shown that it’s possible to express yourself using only 1500 well-chosen words (which would take less than 3 months with only 10 minutes per day with MosaLingua). Once you’ve acquired this base, MosaLingua proposes specialized vocabulary suited to your needs (the application has over 3000 words).

Finally, there’s some stuff about motivation and learner psychology. This boils down to That’s why we offer free learning help via email, presenting the Web’s best resources, as well as tips through bonus material or the learning community on the MosaLingua blog. We’ll give you all the tools you need to develop your own personalized learning method that is adapted to your needs. Some of these tips are not at all bad, but there’s precious little in the way of gamification or other forms of easy motivation.

In short, it’s all reasonably respectable, despite the predilection for sciency language in the marketing blurb. But what really differentiates this product from Anki, as the founder, Samuel Michelot, points out is the content. Mosalingua has lists of vocabulary and phrases that were created by professors. The word ‘professors’ set my alarm bells ringing, and I wasn’t overly reassured when all I could find out about these ‘professors’ was the information about the MosaLingua team .professors

Despite what some people  claim, content is, actually, rather important when it comes to language learning. I’ll leave you with some examples of MosaLingua content (one dialogue and a selection of words / phrases organised by level) and you can make up your own mind.

Dialogue

Hi there, have a seat. What seems to be the problem?

I haven’t been feeling well since this morning. I have a very bad headache and I feel sick.

Do you feel tired? Have you had cold sweats?

Yes, I’m very tired and have had cold sweats. I have been feeling like that since this morning.

Have you been out in the sun?

Yes, this morning I was at the beach with my friends for a couple hours.

OK, it’s nothing serious. It’s just a bad case of sunstroke. You must drink lots of water and rest. I’ll prescribe you something for the headache and some after sun lotion.

Great, thank you, doctor. Bye.

You’re welcome. Bye.

Level 1: could you help me, I would like a …, I need to …, I don’t know, it’s okay, I (don’t) agree, do you speak English, to drink, to sleep, bank, I’m going to call the police

Level 2: I’m French, cheers, can you please repeat that, excuse me how can I get to …, map, turn left, corner, far (from), distance, thief, can you tell me where I can find …

Level 3: what does … mean, I’m learning English, excuse my English, famous, there, here, until, block, from, to turn, street corner, bar, nightclub, I have to be at the airport tomorrow morning

Level 4: OK, I’m thirty (years old), I love this country, how do you say …, what is it, it’s a bit like …, it’s a sort of …, it’s as small / big as …, is it far, where are we, where are we going, welcome, thanks but I can’t, how long have you been here, is this your first trip to England, take care, district / neighbourhood, in front (of)

Level 5: of course, can I ask you a question, you speak very well, I can’t find the way, David this is Julia, we meet at last, I would love to, where do you want to go, maybe another day, I’ll miss you, leave me alone, don’t touch me, what’s you email

Level 6: I’m here on a business trip, I came with some friends, where are the nightclubs, I feel like going to a bar, I can pick you up at your house, let’s go to see a movie, we had a lot of fun, come again, thanks for the invitation

Photo 01-07-2015 16 23 47Flovoco is a vocabulary app developed by ELTjam. This review was written by Mike Harrison and first appeared on his blog. After the review, I’ve added a response by Jo Sayers (of ELTjam). Many thanks to Mike for allowing me to repost his review, and thanks to Jo for allowing me to repost his comment.

I first became aware of this mobile application around July 2014, when ELTjam first posted about their product development. There was a fair amount of heat – the Edtech-meets-ELT specialists Nick Robinson, Laurie Harrison, Tim Gifford, and newbie at the time Jo Sayers, pitched us their product-in-waiting. A year or so later, I saw a presentation at IATEFL in Manchester where Jo talked about reviewing educational and ELT apps. And so to this review of an ELTjam ELT app.

This review follows a model presented by Jo, reviewing the app across four categories: pedagogy and methodology; instructional design; user experience; cost and access.

The version of the app I’m reviewing is 1.0 in the Apple AppStore, and I’m working on an iPhone 5S.

Note – the app is only available in Spanish and English at the moment.

Initial impressions

But first, it is almost impossible to comment on an app without some ‘at first glance’ context, so here it is. I initially thought that ELTjam’s marketing was a little audacious – creating a landing page for an app that didn’t exist – but following conversations recently I now realise this is fairly common practice, and a good way to generate leads for email lists and such. I do still balk a little at the audacity of fanfare around the app (more on that to come under ‘pedagogy and methodology’). On opening the app for the first time, I was impressed by how slick it looked and felt (and more on that under ‘user experience’!) – I have to hold my hands up and say that I have yet to be blown away by an educational app, whether designed for ELT or more general educational fields. Let’s see whether this will change.

untitledhimPedagogy and methodology

★★☆☆☆

The overarching principle behind Flovoco is that words are important. And that in order to get better at using a language, the best thing to do is bump up what you know about the most common words in that language. So far , so good. Flovoco aims to help learners learn more about words by presenting them with a number of activities focusing on different information about a given word – its meaning, pronunciation, how it might be used in a phrase, and then again with words deemed to be confusing. These different areas are named as Translation, Listening, Usage, and Confused Words – and they are presented as ‘levels’, Translation being the first (easiest?) level and Confused Words being the fourth level (and most difficult of the four?). This all seems fairly logical.

But then it all starts to go a bit wrong. In the Translation activity, words are presented with possible matches that aren’t even the same part of speech. Alright, wrong answers are clearly marked with a red light and wrong buzzer sound – but this is behaviourism at best. At worst this may actually confuse learners. Not to mention that this only works by looking at a single meaning of a word – so there won’t be many colloquial meanings and translations included here.

Photo 30-06-2015 18 32 10 eggsThe Listening level does the same thing. Words that are presented alongside each other are completely different. Sometimes phrases are presented, but the audio only comprises of one word in the phrase. For example, you might see ‘pay for something’ but only hear ‘play’. There may be less potential for confusion among words here, but having such a mismatch between the audio and text may be more problematic.

The next two levels are essentially the same – gap fills featuring the words that are being studied. Level three, Usage, again often presents possible answers that are completely different parts of speech. Confused Words is more of the same, but this time the words presented in the three answer slots are potentially easily confused – this seems to be focusing mainly on spelling and/or pronunciation.

Overall, pedagogically Flovoco has a noble aim – looking at the different things it’s useful to know about a word – but in practice it’s a bit confused. Methodologically, the structure of feedback is very behaviouristic. Didn’t we leave Skinner behind a while back?

Instructional design

★★☆☆☆

Admittedly I’m getting my head around the jargon of app development here, but I can’t see this. There isn’t any adaptivity (is that a word) built in to the app. You just work your way up the levels, aiming to get all 500 words into the Your Word Collection at the top level. Nothing adapts down if I keep getting a word wrong – there’s no help other than hoping that I’ll see the red ‘wrong’ button highlight enough times to learn. If I’m racing through the words, there isn’t any extension to what can be done with the words. Maybe there is space for some kind of freer practice, perhaps based around the community of language learners that ELTjam probably hope to cultivate with the app.

Photo 01-07-2015 16 43 09 word collectionPhoto 22-04-2015 08 01 36 372Photo 01-07-2015 16 42 56 profilePhoto 01-07-2015 16 43 05 usage

User experience

★★★☆☆

It looks pretty. It’s fairly easy to tap-and-go in getting started working with the app. There aren’t too many (any?) instructions, so it does rather rely on learners using the app to work things out by trial and error. The user Profile screen is relatively clean and looks straigtforward. But I haven’t worked out exactly how the Daily Word Goal works (maybe because I don’t use the app regularly) and navigating the Profile screens is a little laggy.

There is a playful feel to the app, from the Flovoco logo font to the coloured circles and stars representing the different levels. If they wanted to make learning words look like a game, that’s the impression you get.

Cost and access

Free. Spanish and English only. iOS only (the website says an Android version is in the works).

Technical requirements (iOS):

Category: Education

Updated: Feb 25, 2015

Version: 1.1.4

Size: 21.1 MB

Compatibility: Requires iOS 7.1 or later. Compatible with iPhone, iPad, and iPod touch. This app is optimized for iPhone 5.

Not yet available on Google Play.

You can get the iOS version from here: https://itunes.apple.com/us/app/flovoco/id915793649?mt=8

Overall score:

★★½☆☆

This app certainly got people talking when ELTjam first posted about it, and there is the intent to do something different in the world of ELT apps. But from what I see so far, putting it into practice leaves a fair amount to be desired. Further thought needs to be made about plausible distractors, the focus on the word level meaning may need to be expanded to make this truly different, and sorting out some kind of support/challenge for more and less able learners is something I think is quite key.

Flovoco a go? There is some potential for this app to be really good. But sorry Flovoco, at the moment it’s a no from me.

The response from Jo Sayers:

Hi Mike,

Firstly, thanks very much for writing a review; we really appreciate the feedback and reviews like this will help us as we build Flovoco to a state where both learners and industry professionals can clearly see the value it delivers. This is still our v1.0, as you point out, and so there are many things that we had to do in a way that was good enough to ship it, but necessarily not as finalised as we’d want it to be in an ideal world. The hope was that we would test some key assumptions with this version and learn things that will help us to build it out in a direction that sees it add real value to learners.

The review matches well with a lot of what we have already identified as the strengths and weaknesses of the app and we’re really happy that there are enough positive things in this early version that it already gets 2.5 out of 5. There are a few comments in the review that I wanted to respond to though:

The behaviourist approach. Our intention with the app is not to offer a complete language learning solution, rather to offer a way of quickly acquiring key vocabulary that will act as a foundation for other language development. We feel that lexical acquisition is an area that lends itself well to a more systematic (behaviourist?) approach, and also helps to achieve the flow state that we were aiming for.

Multiple senses. We made a decision to focus only on the primary senses of the words in this version to avoid the complexity of having to demonstrate what is a second or other sense of a word that they’ve encountered before. What you’re actually pushing through the levels in the app is a single sense of a word, rather than a word itself.

The distractors. The distractors for levels 1 and 2 were chosen automatically by an algorithm. I’m not sure I agree that they should all be the same part of speech. This would actually have been very straightforward for us to achieve programmatically, but we felt that it wasn’t any more pedagogically sound than mixing parts of speech. In level 2, the algorithm choosing the distractors selected words with the same initial letter and as close to the same total number of letters as possible, but as the distractors are chosen only from the 500 words in the initial pool this reduced their effectiveness. We have a plan for the next version which should make the selection much more effective. What we wanted to test here, ultimately, was that the algorithm approach worked and didn’t interfere with the learner’s ability to complete the activities.

Levels 3 and 4 being the same. Level 3 focuses on collocation by keeping the target word in the sentence (in bold) and removing a collocate; the distractors are, where possible, similar words that don’t collocate with the target word. Level 4 initially aimed to focus on derived forms of the target word and in this case the target itself is removed and other words which are similar to it act as distractors. As a lot of the words at A1 don’t have strong and obvious collocates and many don’t have derived and inflected forms this was more challenging than it would be at higher levels with fewer functional words and lower frequency content.

Instructional design. Yes, there is no adaptivity. But if you get words wrong they move to the level below, and if you get them right they move up to the next level. In terms of the feedback, the word list on the results page gives a bit of additional information about the words that you have seen. We have plans to incorporate in some community aspects to the product too. However, our plan is not to build that up within ELTjam. As I have discussed in this post for a product website, we have made a shift away from offering this directly to consumers, and now see this as an opportunity to work with publishers and offer the product to their learners through partnerships with them. The B2C version acts as a way of gaining market validation.

Daily word goal. Yes, there are bugs in this which affect the calendar and the lag you mentioned. A new version was built out last month and we should be able to ship the updated version soon.

I hope that helps to put a few of the things you noted into context. It’s also worth pointing out that during the first two months of the app being out and promoted (we are no longer promoting this version as we have learned the things we hoped to) the average session length was over 9 minutes; way more than average in most industries and even more than the average for music apps, according to this data. We also had around 8% of sessions that were over 30 minutes long. So there were a core group of people who were incredibly motivated to use the app. And given that very little is done to incentivise returning to the app (no push notifications, leaderboards, 2 player games etc.) we were really pleased to see that around 8% of learners came back for an 8th visit. So, while there are definitely things to improve, the core offering resonates with learners in what is clearly a very busy and highly competitive marketplace.

Thanks again for the review, it would be really interesting to see what your thoughts are on future iterations. Looking forward to chatting it over next time we meet.

Cheers,
Jo

FluentU, busuu, Bliu Bliu … what is it with all the ‘u’s? Hong-Kong based FluentU used to be called FluentFlix, but they changed their name a while back. The service for English learners is relatively new. Before that, they focused on Chinese, where the competition is much less fierce.

At the core of FluentU is a collection of short YouTube videos, which are sorted into 6 levels and grouped into 7 topic categories. The videos are accompanied by transcriptions. As learners watch a video, they can click on any word in the transcript. This will temporarily freeze the video and show a pop-up which offers a definition of the word, information about part of speech, a couple of examples of this word in other sentences, and more example sentences of the word from other videos that are linked on FluentU. These can, in turn, be clicked on to bring up a video collage of these sentences. Learners can click on an ‘Add to Vocab’ button, which will add the word to personalised vocabulary lists. These are later studied through spaced repetition.

FluentU describes its approach in the following terms: FluentU selects the best authentic video content from the web, and provides the scaffolding and support necessary to bring that authentic content within reach for your students. It seems appropriate, therefore, to look first at the nature of that content. At the moment, there appear to be just under 1,000 clips which are allocated to levels as follows:

Newbie 123 Intermediate 294 Advanced 111
Elementary 138 Upper Int 274 Native 40

It has to be assumed that the amount of content will continue to grow, but, for the time being, it’s not unreasonable to say that there isn’t a lot there. I looked at the Upper Intermediate level where the shortest was 32 seconds long, the longest 4 minutes 34 seconds, but most were between 1 and 2 minutes. That means that there is the equivalent of about 400 minutes (say, 7 hours) for this level.

The actual amount that anyone would want to watch / study can be seen to be significantly less when the topics are considered. These break down as follows:

Arts & entertainment 105 Everyday life 60 Science & tech 17
Business 34 Health & lifestyle 28
Culture 29 Politics & society 6

The screenshots below give an idea of the videos on offer:

menu1menu2

I may be a little difficult, but there wasn’t much here that appealed. Forget the movie trailers for crap movies, for a start. Forget the low level business stuff, too. ‘The History of New Year’s Resolutions’ looked promising, but turned out to be a Wikipedia style piece. FluentU certainly doesn’t have the eye for interesting, original video content of someone like Jamie Keddie or Kieran Donaghy.

But, perhaps, the underwhelming content is of less importance than what you do with it. After all, if you’re really interested in content, you can just go to YouTube and struggle through the transcriptions on your own. The transcripts can be downloaded as pdfs, which, strangely are marked with a FluentU copyright notice.copyright FluentU doesn’t need to own the copyright of the videos, because they just provide links, but claiming copyright for someone else’s script seemed questionable to me. Anyway, the only real reason to be on this site is to learn some vocabulary. How well does it perform?

fluentu1

Level is self-selected. It wasn’t entirely clear how videos had been allocated to level, but I didn’t find any major discrepancies between FluentU’s allocation and my own, intuitive grading of the content. Clicking on words in the transcript, the look-up / dictionary function wasn’t too bad, compared to some competing products I have looked at. The system could deal with some chunks and phrases (e.g. at your service, figure out) and the definitions were appropriate to the way these had been used in context. The accuracy was far from consistent, though. Some definitions were harder than the word they were explaining (e.g. telephone = an instrument used to call someone) and some were plain silly (e.g. the definition of I is me).

have_been_definitionSome chunks were not recognised, so definitions were amusingly wonky. Come out, get through and have been were all wrong. For the phrase talk her into it, the program didn’t recognise the phrasal verb, and offered me communicate using speech for talk, and to the condition, state or form of for into.

For many words, there are pictures to help you with the meaning, but you wonder about some of them, e.g. the picture of someone clutching a suitcase to illustrate the meaning of of, or a woman holding up a finger and thumb to illustrate the meaning of what (as a pronoun).what_definition

The example sentences don’t seem to be graded in any way and are not always useful. The example sentences for of, for example, are The pages of the book are ripped, the lemurs of Madagascar and what time of day are you free. Since the definition is given as belonging to, there seems to be a problem with, at least, the last of these examples!

With the example sentence that link you to other video examples of this word being used, I found that it took a long time to load … and it really wasn’t worth waiting for.

After a catalogue of problems like this, you might wonder how I can say that this function wasn’t too bad, but I’ve seen a lot worse. It was, at least, mostly accurate.

Moving away from the ‘Watch’ options, I explored the ‘Learn’ section. Bearing in mind that I had described myself as ‘Upper Intermediate’, I was surprised to be offered the following words for study: Good morning, may, help, think, so. This then took me to the following screen:great job

I was getting increasingly confused. After watching another video, I could practise some of the words I had highlighted, but, again, I wasn’t sure quite what was going on. There was a task that asked me to ‘pick the correct translation’, but this was, in fact a multiple choice dictation task.translation task

Next, I was asked to study the meaning of the word in, followed by an unhelpful gap-fill task:gap fill

Confused? I was. I decided to look for something a little more straightforward, and clicked on a menu of vocabulary flash cards that I could import. These included sets based on copyright material from both CUP and OUP, and I wondered what these publishers might think of their property being used in this way.flashcards

FluentU claims  that it is based on the following principles:

  1. Individualized scaffolding: FluentU makes language learning easy by teaching new words with vocabulary students already know.
  2. Mastery Learning: FluentU sets students up for success by making sure they master the basics before moving on to more advanced topics.
  3. Gamification: FluentU incorporates the latest game design mechanics to make learning fun and engaging.
  4. Personalization: Each student’s FluentU experience is unlike anyone else’s. Video clips, examples, and quizzes are picked to match their vocabulary and interests.

The ‘individualized scaffolding’ is no more than common sense, dressed up in sciency-sounding language. The reference to ‘Mastery Learning’ is opaque, to say the least, with some confusion between language features and topic. The gamification is rudimentary, and the personalization is pretty limited. It doesn’t come cheap, either.

price table

Lingua.ly is an Israeli start-up which, in its own words, ‘is an innovative new learning solution that helps you learn a language from the open web’. Its platform ‘uses big-data paired with spaced repetition to help users bootstrap their way to fluency’. You can read more of this kind of adspeak at the Lingua.ly blog  or the Wikipedia entry  which seems to have been written by someone from the company.

How does it work? First of all, state the language you want to study (currently there are 10 available) and the language you already speak (currently there are 18 available). Then, there are three possible starting points: insert a word which you want to study, click on a word in any web text or click on a word in one of the suggested reading texts. This then brings up a bilingual dictionary entry which, depending on the word, will offer a number of parts of speech and a number of translated word senses. Click on the appropriate part of speech and the appropriate word sense, and the item will be added to your personal word list. Once you have a handful of words in your word list, you can begin practising these words. Here there are two options. The first is a spaced repetition flashcard system. It presents the target word and 8 different translations in your own language, and you have to click on the correct option. Like most flashcard apps, spaced repetition software determines when and how often you will be re-presented with the item.

The second option is to read an authentic web text which contains one or more of your target items. The company calls this ‘digital language immersion, a method of employing a virtual learning environment to simulate the language learning environment’. The app ‘relies on a number of applied linguistics principles, including the Natural Approach and Krashen’s Input Hypothesis’, according to the Wikipedia entry. Apparently, the more you use the app, the more it knows about you as a learner, and the better able it is to select texts that are appropriate for you. As you read these texts, of course, you can click on more words and add them to your word list.

I tried out Lingua.ly, logging on as a French speaker wanting to learn English, and clicking on words as the fancy took me. I soon had a selection of texts to read. Users are offered a topic menu which consisted of the following: arts, business, education, entertainment, food, weird, beginners, green, health, living, news, politics, psychology, religion, science, sports, style. The sources are varied and not at all bad – Christian Science Monitor, The Grauniad, Huffington Post, Time, for example –and there are many very recent articles. Some texts were interesting; others seemed very niche. I began clicking on more words that I thought would be interesting to explore and here my problems began.

I quickly discovered that the system could only deal with single words, so phrasal verbs were off limits. One text I looked at had the phrasal verb ‘ripping off’, and although I could get translations for ‘ripping’ and ‘off’, this was obviously not terribly helpful. Learners who don’t know the phrasal verb ‘ripped off’ do not necessarily know that it is a phrasal verb, so the translations offered for the two parts of the verb are worse than unhelpful; they are actually misleading. Proper nouns were also a problem, although some of the more common ones were recognised. But the system failed to recognise many proper nouns for what they were, and offered me translations of homonymous nouns. new_word_added_'ripping_off' With some words (e.g. ‘stablemate’), the dictionary offered only one translation (in this case, the literal translation), but not the translation (the much more common idiomatic one) that was needed in the context in which I came across the word. With others (e.g. ‘pertain’), I was offered a list of translations which included the one that was appropriate in the context, but, unfortunately, this is the French word ‘porter’, which has so many possible meanings that, if you genuinely didn’t know the word, you would be none the wiser.

Once you’ve clicked on an appropriate part of speech and translation (if you can find one), the dictionary look-up function offers both photos and example sentences. Here again there were problems. I’d clicked on the verb ‘pan’ which I’d encountered in the context of a critic panning a book they’d read. I was able to select an appropriate translation, but when I got to the photos, I was offered only multiple pictures of frying pans. There were no example sentences for my meaning of ‘pan’: instead, I was offered multiple sentences about cooking pans, and one about Peter Pan. In other cases, the example sentences were either unhelpful (e.g. the example for ‘deal’ was ‘I deal with that’) or bizarre (e.g. the example sentence for ‘deemed’ was ‘The boy deemed that he cheated in the examination’). For some words, there were no example sentences at all.

Primed in this way, I was intrigued to see how the system would deal with the phrase ‘heaving bosoms’ which came up in one text. ‘Heaving bosoms’ is an interesting case. It’s a strong collocation, and, statistically, ‘heaving bosoms’ plural are much more frequent than ‘a heaving bosom’ singular. ‘Heaving’, as an adjective, only really collocates with ‘bosoms’. You don’t find ‘heaving’ collocating with any of the synonyms for ‘bosoms’. The phrase is also heavily connoted, strongly associated with romance novels, and often used with humorous intent. Finally, there is also a problem of usage with ‘bosom’ / ‘bosoms’: men or women, one or two – all in all, it’s a tricky word.

Lingua.ly was no help at all. There was no dictionary entry for an adjectival ‘heaving’, and the translations for the verb ‘heave’ were amusing, but less than appropriate. As for ‘bosom’, there were appropriate translations (‘sein’ and ‘poitrine’), but absolutely no help with how the word is actually used. Example sentences, which are clearly not tagged to the translation which has been chosen, included ‘Or whether he shall die in the bosom of his family or neglected and despised in a foreign land’ and ‘Can a man take fire in his bosom, and his clothes not be burned?’

Lingua.ly has a number of problems. First off, its software hinges on a dictionary (it’s a Babylon dictionary) which can only deal with single words, is incomplete, and does not deal with collocation, connotation, style or register. As such, it can only be of limited value for receptive use, and of no value whatsoever for productive use. Secondly, the web corpus that it is using simply isn’t big enough. Thirdly, it doesn’t seem to have any Natural Language Processing tool which could enable it to deal with meanings in context. It can’t disambiguate words automatically. Such software does now exist, and Lingua.ly desperately needs it.

Unfortunately, there are other problems, too. The flashcard practice is very repetitive and soon becomes boring. With eight translations to choose from, you have to scroll down the page to see them all. But there’s a timer mechanism, and I frequently timed out before being able to select the correct translation (partly because words are presented with no context, so you have to remember the meaning which you clicked in an earlier study session). The texts do not seem to be graded for level. There is no indication of word frequency or word sense frequency. There is just one gamification element (a score card), but there is no indication of how scores are achieved. Last, but certainly not least, the system is buggy. My word list disappeared into the cloud earlier today, and has not been seen since.

I think it’s a pity that Lingua.ly is not better. The idea behind it is good – even if the references to Krashen are a little unfortunate. The company says that they have raised $800,000 in funding, but with their freemium model they’ll be desperately needing more, and they’ve gone to market too soon. One reviewer, Language Surfer,  wrote a withering review of Lingua.ly’s Arabic program (‘it will do more harm than good to the Arabic student’), and Brendan Wightman, commenting at eltjam,  called it ‘dull as dish water, […] still very crude, limited and replete with multiple flaws’. But, at least, it’s free.

I suggested in my last post that vocabulary flashcard systems can have a useful role to play in blended learning contexts. However, for their potential to be exploited, teachers will need to devote classroom time to the things that the apps, on their own, cannot do. This post looks in some detail at what teachers can do.

Spaced repetition may be important to long-term memorization of new vocabulary items, but it will not be enough on its own. Memory researchers refer to three techniques that will improve speed of retention and long-term recall. The first of these is called the ‘generation effect’ – the use of even a little cognitive effort in generating the answer in flashcard practice. A simple example is provided by Brown, Roediger and McDaniel[1]: simply asking a subject to fill in a word’s missing letters resulted in better memory of the word. […] For a pair like foot-shoe, those who studied the pair intact had lower subsequent recall than those who studied the pair from a clue as obvious as foot-s _ _ e. In vocabulary learning, there is much that learners need to know beyond the meaning or translation equivalent: pronunciation, collocation, and associated grammatical patterns, for example. A focus on these aspects of word knowledge will all deepen that knowledge, but can enhance memorization at the same time.

The second of these techniques is called ‘elaboration’ – the process of giving new material meaning by expressing it in your own words and connecting it with what you already know. The more you can explain about the way your new learning relates to your prior knowledge, the stronger your grasp of the new learning will be, and the more connections you create that will help you remember it later[2]. Explaining the meaning or rules of use of a target vocabulary item to a fellow student, or explaining how this word has significance in your life outside the classroom are simple examples of elaboration. Whilst elaboration is important in any kind of memorization, it is probably especially important in vocabulary learning. If the mental lexicon is a network of associations (and we don’t really have a better way of describing it right now!), the fostering of multiple associations or connections will be a vital part of building up this lexicon: When students are asked to manipulate words, relate them to other words and to their own experiences, and then to justify their choices, these word associations are reinforced[3].

The third of these is getting the right kind of feedback. Feedback on flashcard software is typically of the right / wrong variety. At some point, this is obviously necessary, but it has its limitations. First of all, it is usually immediate, and research[4] suggests that a slight delay in getting feedback aids recall. With immediate feedback, learners can easily come to over-rely on it. Secondly, intelligent, scaffolded feedback (e.g. with hints and cues, rather than simple provision of the correct answer) contributes to the ‘generation effect’ (see above). Thirdly, positive feedback (e.g. where a learner sees that she can accurately and appropriately use new items, especially in new contexts) will enhance both learning and motivation. Flashcard software almost invariably presents and practises vocabulary in one context only, and rarely requires learners to produce the language in a communicative context.

The practical classroom suggestions that follow are all attempts to address the issues raised above. This is not in any way a complete list, and I have prioritized, in the ‘Practice Activities’ section, those tasks that offer more than simple re-exposure (for example, activities such as ‘Hangman’, word quizzes, word squares, definition games, and so on). But I hope that it will be a useful starting point.

Preparation activities

  • Put students into pairs and give them a few minutes (at any moment in a lesson, but this is often done at the start) to test each other on the words they are studying.
  • On a regular basis, allocate some classroom time for students to edit / improve their flashcards. This is best done in pairs. Tasks that you could set include: (1) students find example sentences to add to their cards; (2) students find more memorable / amusing example sentences to add to their cards; (3) students research and find useful phrases which include their target items, and add these to their cards; (4) students research and find common collocations of their target words and add these to their cards; (5) students research and find pictures (from an online image search) which they can use to replace their own-language translations; (6) students research, find and add to their cards other parts of speech; (7) students find recordings (via online dictionaries) of their target items and add them to their cards; (8) students record themselves saying the target items and add these to their cards; (9) students gap (or anagrammatize) some of the letters on the English sides of their cards; (10) students compare cards, discuss which are more memorable, and edit their own if they think this is useful
  • The ultimate hope is that learners will become more autonomous in their vocabulary learning. To this end, I’d thoroughly endorse Daniel Barber’s suggestion in a comment on my previous post: get the class to use and review the various wordcard apps and feed back to their classmates, i.e. to discover for themselves the relative merits of digital vs. hand-written / Anki vs. Quizlet and decide for themselves what’s best.

Practice activities

  • Ask students to flip through their flashcard set and make a list of the words that they are finding hardest to remember. They should do this with a partner and, together, should come up with a list of twelve or more words. Ask the pairs to put their words into groups. Initially, it will probably be best to suggest the kinds of groupings they could use. For example: (1) words they think they would probably need to use in their first week in an English-speaking country vs. words they think they are unlikely to need in their first week in an English-speaking country, (2) words they like (for whatever reason) vs. words they dislike; (3) words they can associate with good things vs. words which they can associate with bad things. When students are familiar with this activity type, they can choose their own categories. Once students have completed the task with their partner, they should change partners and exchange ideas. All of this can be done orally.
  • Ask students to flip through their flashcard set and make a list of the words that they are finding hardest to remember. They should do this with a partner and, together, should come up with a list of twelve or more words. Tell them to write these words in a circle on a sheet of paper. word_circle Tell the students to choose, at random, one word in their circle. Next, they must find another word in the circle which they can associate in some way with the first word that they chose. They must explain this association to their partner. They must then find another word which they can associate with their second word. Again they must explain the association. They should continue in this way until they have connected all the words in their circle. Once students have completed the task with their partner, they should change partners and exchange ideas. All of this can be done orally.
  • Using the same kind of circle of words (as in the activity above), students again work with a partner. Starting with any word, they must find and explain an association with another word. Next, beginning with the word they first chose, they must find and explain an association with another word from the circle. They continue in this way until they have found connections between their first word and all the other words in the circle. Once students have completed the task with their partner, they should change partners and exchange ideas. All of this can be done orally.
  • Ask the students to flick through their coursebooks and find four or five images that they find interesting or attractive. Tell them to note the page numbers. straightforward-upperintermediate-sb-1-638 Then, ask the students to flip through their flashcard set and make a list of the words that they are finding hardest to remember. They should do this with a partner and, together, should come up with a list of twelve or more words. The students should then find an association between each of the words on their list and one of the pictures they have selected. They discuss their ideas with their partner, before comparing their ideas with a new partner.
  • Using the pictures and word lists (as in the activity above), students should select one picture, without telling their partner which picture they have selected. They should then look at the word list and choose four words from this list which they can associate with that picture. They then tell their four words to their partner, whose task is to guess which picture the other student was thinking of.
  • Ask students to flip through their flashcard set and make a list of the words that they are finding hardest to remember. Individually, they should then write a series of sentences which contain these words: the sentences can contain one, two, or more of their target words. Half of the sentences should contain true personal information; the other half should contain false personal information. Students then work with a partner, read their sentences aloud, and the partner must decide which sentences are true and which are false.
  • Ask students to flip through their flashcard set and make a list of the words that they are finding hardest to remember. They should do this with a partner and, together, should come up with a list of twelve or more words. Still in pairs, they should prepare a short story which contains at least seven of the items in their list. After preparing their story, they should rehearse it before exchanging stories with another student / pair of students.
  • There’s a fun question-and-answer game, ‘Any Which Way Matching’, from Alex Case, which can be used with any set of vocabulary. It can be found here:
  • Play a class game which recycles the vocabulary that students are having difficulty remembering. You can find the rules for one game, ‘Words in sentences’, which can be used with any set of vocabulary here:

[1] Brown, P.C., Roediger, H.L. & McDaniel, M. A. Make It Stick (Cambridge, Mass.: Belknap Press, 2014) p.32

[2] ibid p.5

[3] Sökmen, A.J. (1997) ‘Current trends in teaching second language vocabulary,’ in Schmitt, N. & McCarthy, M. (eds.) Vocabulary: Description, Acquisition and Pedagogy (Cambridge: CUP, 1997) pp.241-242

[4] Brown, P.C., Roediger, H.L. & McDaniel, M. A. Make It Stick (Cambridge, Mass.: Belknap Press, 2014)  pp.39 – 40