Posts Tagged ‘flipped classroom’

Flipped learning undoubtedly has much potential and now, when F2F teaching is not always possible, the case for exploring what it might offer seems greater still. For a variety of reasons (not the least of which are motivational issues), it may not always be possible to flip the classroom, but, if and when it is, how and what should be flipped?

In the most well-known flipped approaches, such as the Khan Academy, students watch instructional videos in their own time, before coming to class where they can work together on practical problems, applying the knowledge they have gained from the instructional video. The flipped part of the learning does not need to be a video (Bergmann et al., 2013), but, in practice, it usually is. But whether video or something else, one of the big questions for me is what, precisely, does it make sense to flip?

In a recently published Cambridge Paper in ELT that I wrote on Flipped Learning, I noted that it is not uncommon for grammar instruction to be flipped. Al-Harbi & Alshumaimeri (2016), for example, describe a Saudi secondary school where the teacher selected a number of grammar areas from the coursebook and then identified instructional videos from YouTube that addressed these areas. Buitrago & Díaz (2018) describe a Colombian university where students were required to watch instructional videos about grammar, some of which were selected from YouTube and others created by members of staff.

To understand better what learners might be doing in their flipped time, I decided to take a look at a selection of YouTube grammar videos. I focussed on one area of grammar only (‘bored’ vs ‘boring’) and from the huge selection available, I prioritised those that were the most popular. Here’s what I found. After a brief commentary on each of the 10 videos, I wrap up with a few observations.

mmmEnglish 1245K views 8.33 minutes

mmmEnglish

Early on, Emma says ‘These endings are called suffixes and when we add them to the end of a verb, they transform our verb into an adjective, but you need to know how to use each of these types of adjectives and we’re gonna do that right now’. This gives a good taste of what follows. We learn that –ing adjectives refer to ‘the characteristics of a person, a thing, or a situation’ while –ed adjectives refer to an ‘emotion or a feeling’. Bearing in mind that this area of grammar is listed as A2+ (in Pearson’s GSE), explanations of this kind in English may be tricky for many learners. The language grading in explanations like ‘If you say that someone or something is boring, they or it makes you feel bored. Do the thing or the person that is boring is what makes you feel bored. It bores you. OK, there’s our verb’ needs a little attention! On and on goes Emma, until after almost five minutes she reads out a few sentences and students have to decide if the correct adjective has been used. Over a million people have watched this.

Learn English with Let’s Talk 452K views 8.52 minutes

Lets Talk

Rashna explains: ‘First, let’s begin by understanding what are adjectives’. My heart sinks. ‘So ‘pretty’ is doing the job of describing or bringing about a quality of the noun ‘girl’, so ‘pretty’ becomes my adjective. So when you’re confused and don’t know how to spot the adjectives, ask the question ‘what kind’. All right. So, if I say I live in a big city, and if I ask what kind of a city, it’s big, so ‘big’ is an adjective that is describing the noun ‘city’. All right. So remember, adjectives are nothing but just words that describe a noun that tell you more about it or bring about some quality.’ Over a quarter of the way through and we haven’t yet got on to –ed and –ing. I recommend watching all the way through to the end just to admire the whiteboard work. You might enjoy the comments, too (e.g. ‘Thanks very much. This lesson was confused me so much.’) Coming up for half a million views.

Alejo Lopera Inglés 428K views 4.07 minutes

Alejo

The only English here is in the example sentences, with Spanish being used for the rest. The explanation hinges on ‘pienso’ (think) for –ing and ‘sentimiento’ (feeling) for –ed, which only kind of works. Alejo takes us through a few examples using a combination of talking-head video and background slides. His delivery is engaging and using Spanish makes things clearer than English only.

English Lessons with Adam 357K views 5.27 minutes

Adam

Standing in front of the whiteboard, Adam says that his video is especially useful for beginners. He rambles on for over 5 minutes in language which is far more complicated than the language he is explaining. Here’s a flavour: Now, what does it mean to be bored and what does it mean to be boring? When we talk about “bored”, we’re describing a feeling. Okay? When we talk about “interested”, we’re describing a feeling. So all of the “ed” adjectives are actually feelings, and you can only use them to talk about people and sometimes animals. Why? Because things, like chairs, or tables, or whatever, they don’t have feelings. […]”I am worried”, now people don’t realize that “worried” can have “worrying” as another adjective. “The situation is worrying” means the situation is making me feel worried. Okay? Maybe the whole global political situation, whatever. Now, hopefully none of you are confused by this lesson because I’m trying to make it not confusing. Okay? Everybody okay with that? […] Now, I just want to point out one other thing: Don’t confuse feeling adjectives with “ed” with actual feelings. Okay? If somebody is loved, does he feel loved? Maybe yes, maybe no. We’re not talking about that person’s feelings.

Crown Academy of English 270K views 26.57 minutes

Crown academy

Using screen capture and voiceover software, the script is mostly read aloud from the screen. There is no attempt to make either the script or the delivery interesting. The approach is as traditional as can be: it focuses first on form, with no shying away from grammatical jargon, and eventually moves on to meaning. And then, if you’re still awake, there’s a discrimination exercise. After over 25 minutes of death-by-Powerpoint, the lesson comes, mercifully, to an end.

 

Learn English with Rebecca 274K views 3.30 minutes

Rebecca

From the same stable as Adam’s video, this is more controlled than his ramble, and with slightly better language grading, but is still hard to follow, in part because no examples are given in written form. As with Adam, Rebecca bangs on about how important it is to get this grammar right, because ‘if you make a mistake you could be saying something very unpleasant about yourself’. It’s hard to tell what level it’s intended for.

Francisco Ochoa Inglés Fácil 64K views 11.02 minutes

Pacho

Switching between Spanish and English, Pacho rattles non-stop through 6 discrimination sentences, taking the difference between feelings (which take the Spanish ‘estar’) and states (which take the Spanish ‘ser’) as his key explanatory tool. This doesn’t quite work, but following his breakneck delivery is more of a problem. The only thing he doesn’t translate are the commas in his examples. I challenge you not to feel confused / confusing by the time he gets to the third sentence. Even Pacho seems to be struggling. Words like ‘hence’ and tenses like past perfect continuous don’t help his 11 minute monologue. I loved the way that he says at the end that the only way to learn this stuff is by applying the language in the way he has just done.

BBC Learning English 48K views 0.56 minutes

BBC_Learning_English

In under a minute, Sam from BBC Learning English achieves much greater clarity than anyone else I watched, helped by a carefully planned script, very controlled language and a split screen showing the key points as she makes them. Towards the end, she rattles through 5 more –ed / -ing pairs rather too quickly. It’s a shame, I thought, that she (or the producers) felt the need to reference the old trope about how boring grammar lessons are.

Shaw English Online 46K views 8.49 minutes

Shaw English Online

The explanation is mercifully brief and the language of Fanny, the presenter, is well controlled. We could do without the exhortations to listen carefully, etc, ‘because this is very important’, but you can’t have everything. A lot of examples are given, before the explanations are repeated. The repetitions don’t help as Fanny resorts to more complicated language than the language she is explaining (e.g. ‘But when you say the teacher was boring, you are describing the teacher, OK, the teacher made the students feel bored, because he or she was boring’). After nearly 4 minutes of presentation, there are some practice discrimination tasks, but Fanny’s relentless commentary gets seriously in the way. The lesson is rounded off with a few minutes of repeat-after-me pronunciation practice.

Mad English TV 24K views 6.59 minutes

Mad_English

In a surreal opening, the presenter talks about the three different states of H2O, before explaining that people, too, can have different states. Eventually, we get to the idea that ‘boring’ is an accusation, ‘bored’ is a state: ‘If you go up to your teacher and say ‘you’re boring’, that’s an insult’. The language grading is all over the place, as is the explanation itself. As a general rule, the longer the explanation, the less clear it is. At 7 minutes, this video is no exception to the rule. When we get to a mini-test (a useful feature that not all other videos have), the choice is ‘My cat is _______’. To know the answer, you need to know if you’re making an accusation about the cat. Got it?

Flipped learning and grammar

Although grammar instruction might seem a strong candidate for a flipped treatment, videoed explanations are clearly not the way to do it. Many coursebooks have perfectly adequate guided discoveries of this and other standard grammar points. Newer courses on platforms have interactive guided discoveries (and often also offer a more traditional deductive route) that will also do the trick much better than videoed explanations. Would learners not be better off doing something else altogether with their time? Initial vocabulary study, listening, reading, writing, almost anything in fact, is a more appropriate target for flipping than grammar, when approached in this way. Video is not the solution to a problem: on the evidence here, it makes the problem worse.

The popularity of grammar videos

It’s very hard to watch this stuff and not scoff, but there’s no denying the immense popularity of videos like these. Much as I find it difficult to believe, people must be learning something (or think they are learning something) from watching them. Otherwise, they presumably wouldn’t consume them to such an extent. Perhaps, these videos conform to expectations about what English lessons should be like? Perhaps viewers subscribe to a belief in ‘no pain, no gain’? Perhaps they simply don’t know where to find something that would help them more? Or perhaps they have been told to watch by their flipping teachers?

Emma has had 1.25 million views. Advertising earnings from 1 million YouTube views are generally reckoned to be between $600-$7000, but are likely to be at the higher end of this scale if (1) people watch the video through to the end (which is probably the case here), and (2) viewers interact with the video through likes and comment (for this video Emma has received 2353 comments). Earnings are also higher when you have more subscribers to your channel. Emma can count on 3.25 million subscribers and Rachna of Let’s Talk has 4.77 million subscribers. By way of contrast, Russell Stannard’s Teacher Training Videos has 40,000 subscribers. There’s gold in them thar hills.

Grammar videos and the world of ELT

Free grammar videos, along with self-study apps like Duolingo, are a huge and thriving sector of ELT. They rarely, if ever, feature in research, conference presentations or the broader discourse of ELT, a world, it seems, much more oriented to products you have to pay for.

References

Al-Harbi, S.S., & Alshumaimeri, Y.A. (2016). The flipped classroom impact in grammar class on EFL Saudi secondary school students’ performances and attitudes. English Language Teaching, 9(10): 60–80. Available at: https://files.eric.ed.gov/fulltext/EJ1113506.pdf

Bergmann, J., Overmeyer, J., & Wilie, B. (2013). The flipped class: myth vs. reality. The Daily Riff, July 9, 2013. Available at: http://www.thedailyriff.com/articles/the-flipped-class-conversation-689.php

Buitrago, C. R., & Díaz, J. (2018). Flipping your writing lessons: Optimizing your time in your EFL writing classroom. In Mehring, J., & Leis, A. (Eds.), Innovations in Flipping the Language Classroom. Singapore: Springer, 69–91.

Google search resultsUnconditional calls for language teachers to incorporate digital technology into their teaching are common. The reasons that are given are many and typically include the fact that (1) our students are ‘digital natives’ and expect technology to be integrated into their learning, (2) and digital technology is ubiquitous and has so many affordances for learning. Writing on the topic is almost invariably enthusiastic and the general conclusion is that the integration of technology is necessary and essential. Here’s a fairly typical example: digital technology is ‘an essential multisensory extension to the textbook’ (Torben Schmidt and Thomas Strasser in Surkamp & Viebrock, 2018: 221).

 

Teachers who are reluctant or fail to embrace technology are often ‘characterised as technophobic, or too traditional in their teaching style, or reluctant to adopt change’ (Watson, 2001: 253). (It’s those pesky teachers again.)

Claims for the importance of digital technology are often backed up by vague references to research. Michael Carrier, for example, in his introductory chapter to ‘Digital Language Learning and Teaching’ (Carrier et al. 2017: 3) writes that ‘research results […] seem to show conclusively that the use of educational technology adds certain degrees of richness to the learning and teaching process […] at the very least, digital learning seems to provide enhanced motivation for learners’.

Unfortunately, this is simply not true. Neither in language learning / teaching, nor in education more generally, is there any clear evidence of the necessary benefits of introducing educational technology. In the broader context, the ‘PISA analysis of the impact of Information Communication Technology (ICT) on reading, mathematics, and science (OECD, 2015: 3) in countries heavily invested in educational technology showed mixed effects and “no appreciable improvements”’ (Herodotou et al., 2019). Educational technology can or might  ‘add certain degrees of richness’ or ‘provide enhanced motivation’, but that is not the same as saying that it does or will. The shift from can to will, a piece of modal legerdemain used to advocate for educational technology, is neatly illustrated in a quote from the MIT’s Office of Digital Learning, whose remit is to improve learning and teaching across the university via digital learning: ‘Digital Learning technologies can enable students to grasp concepts more quickly [etc….] Digital technologies will enable this in new and better ways and create possibilities beyond the limits of our current imagination’ (quoted by Carrier, 2017: 1).

Before moving on, here’s another example. The introduction to Li Li’s ‘New Technologies and Language Learning’ (Li, 2017: x) states, with a cautious can, that one of the objectives of the book is ‘to provide examples of how technologies can be used in assisting language education’. In the next paragraph, however, caution is thrown to the wind and we are told, unequivocally, that ‘technology is beneficial for language learning’.

Pedagogy before technology

Examples of gratuitous technology use are not hard to find. Mark Warschauer (who, as the founding director of the Digital Learning Lab at the University of California, Irvine, could be fairly described as an edtech enthusiast) describes one example: ‘I remember observing a beginners’ French class a number of years ago, the teacher bragged about how engaged the learners were in creating multimedia in French. However, the students were spending most of their time and energy talking with each other in English about how to make PowerPoints, when, as beginning learners, they really needed to be spending time hearing as much French as possible’ (quoted in the Guardian, May 2014).

As a result, no doubt, of having similar experiences, it seems that many people are becoming a little more circumspect in their enthusiasm for edtech. In the same Guardian article as Warschauer’s recollections, Russell Stannard ‘says the trick is to put the pedagogy first, not the technology. “You’ve got to know why you’re using it. Teachers do need to learn to use new technology, but the driving force should always be the pedagogy behind it’. Nicky Hockly, Gavin Dudeney and Mark Pegrum (Hockly et al., 2013: 45) concur: ‘Content and pedagogy come before technology. We must decide on our content and pedagogical aims before determining whether our students should use pens or keyboards, write essays or blogs, or design posters or videos’. And Graham Stanley (2013: 1) in the introduction to his ‘Language Learning With Technology’ states that his ‘book makes a point of putting pedagogy at the forefront of the lesson, which is why content has been organised around specific learning content goals rather than specific technologies’.

But, Axel Krommer, of the Friedrich-Alexander University of Erlangen-Nürnberg, has argued that the principle of ‘pedagogy before technology’ is ‘trivial at best’. In a piece for the Goethe Institute he writes ‘a theory with which everyone agrees and whose opposite no-one believes true is meaningless’, although he adds that it may be useful as ‘an admonitory wake-up call when educational institutions risk being blinded by technological possibilities that cause them to neglect pedagogical principles that should really be taken for granted’. It was this piece that set me thinking more about ‘pedagogy before technology’.

Pedagogy before technology (on condition that there is technology)

Another person to lament the placing of technology before pedagogy is Nik Peachey. In an opinion piece for the Guardian, entitled ‘Technology can sometimes be wasted on English language teaching’, he complains about how teachers are left to sort out how to use technology ‘in a pedagogically effective way, often with very little training or support’. He appears to take it as given that technology is a positive force, and argues that it shouldn’t be wasted. The issue, he says, is that better teacher training is needed so that teachers’ ‘digital literacies’ are improved and to ensure that technological potential is fulfilled.

His position, therefore, cannot really be said to be one of ‘pedagogy before technology’. Like the other writers mentioned above, he comes to the pedagogy through and after an interest in the technology. The educational use of digital technology per se is never seriously questioned. The same holds true for almost the entirety of the world of CALL research.

confer

A Canadian conference ‘Pedagogy b4 Technology’ illustrates my point beautifully.

There are occasional exceptions. A recent example which I found interesting was an article by Herodotou et al (2019), in which the authors take as their starting point a set of OECD educational goals (quality of life, including health, civic engagement, social connections, education, security, life satisfaction and the environment), and then investigate the extent to which a variety of learning approaches (formative analytics, teachback, place-based learning, learning with robots, learning with drones, citizen inquiry) – not all of which involve technology – might contribute to the realisation of these goals.

Technology before pedagogy as policy

Some of the high school English teachers I work with have to use tablets in one lesson a week. Some welcome it, some accept it (they can catch up with other duties while the kids are busy with exercises on the tablet), others just roll their eyes at the mention of this policy. In the same school system, English language learning materials can only be bought if they come in digital versions (even if it is the paper versions that are actually used). The digital versions are mostly used for projecting pages onto the IWBs. Meanwhile, budgets and the time available for in-service training have been cut.

Elsewhere, a chain of universities decides that a certain proportion of all courses must be taught online. English language courses, being less prestigious than major subjects, are one of the first to be migrated to platforms. The staff, few of whom have tenure or time to spare, cope as best as they can, with some support from a department head. Training is provided in the mechanics of operating the platform, and, hopefully before too long, more training will become available to optimize the use of the platform for pedagogical purposes. An adequate budget has yet to be agreed.

The reasons why so many educational authorities introduce such policies are, at best, only superficially related to pedagogy. There is a belief, widely held, that technology cannot fail to make things better. In the words of Tony Blair: ‘Technology has revolutionised the way we work and is now set to transform education. Children cannot be effective in tomorrow’s world if they are trained in yesterday’s skills’. But there is also the potential of education technology to scale education up (i.e. increase student numbers), to reduce long-term costs, to facilitate accountability, to increase productivity, to restrict the power of teachers (and their unions), and so on.

In such circumstances, which are not uncommon, it seems to me that there are more pressing things to worry about than teachers who are not sufficiently thinking about the pedagogical uses to which they put the technology that they have to use. Working conditions, pay and hours, are all affected by the digitalisation of education. These things do get talked about (see, for example, Walsh, 2019), but only rarely.

Technology as pedagogy

Blended learning, described by Pete Sharma in 2010 as a ‘buzz word’ in ELT, remains a popular pedagogical approach. In a recent article (2019), he enthuses about the possibilities of blended learning, suggesting that teachers should use it all the time: ‘teaching in this new digital age should use the technologies which students meet in their everyday lives, such as the Internet, laptop, smartphone and tablet’. It’s also, he claims, time-efficient, but other pedagogical justifications are scant: ‘some language areas are really suited to be studied outside the classroom. Extensive reading and practising difficult phonemes, for instance’.

Blended learning and digital technology are inseparable. Hockley (2018) explains the spread of blended learning in ELT as being driven primarily by ‘the twin drivers of economics (i.e. lower costs) and increasingly accessible and affordable hardware and software’. It might be nice to believe that ‘it is pedagogy, rather than technology, that should underpin the design of blended learning programmes’ (McCarthy, 2016, back cover), but the technology is the pedagogy here. Precisely how it is used is almost inevitably an afterthought.

Which pedagogy, anyway?

We can talk about putting pedagogy before technology, but this raises the question of which particular pedagogy we want to put in the driving seat. Presumably not all pedagogies are of equal value.

One of the most common uses of digital technology that has been designed specifically for language learning is the IWB- or platform-delivered coursebook and its accompanying digital workbook. We know that a majority of teachers using online coursebook packages direct their students more readily to tasks with clear right / wrong answers (e.g. drag-and-drop or gap-fill grammar exercises) than they do to the forum facilities where communicative language use is possible. Here, technology is merely replicating and, perhaps (because of its ease of use), encouraging established pedagogical practices. The pedagogy precedes the technology, but it’s probably not the best pedagogy in the world. Nor does it make best use of the technology’s potential. Would the affordances of the technology make a better starting point for course design?

Graham Stanley’s book (2013) offers suggestions for using technology for a variety of purposes, ranging from deliberate practice of grammar and vocabulary to ways of facilitating opportunities for skills practice. It’s an eclectic mix, similar to the range of activities on offer in the average coursebook for adults or teenagers. It is pedagogy-neutral in the sense that it does not offer a set of principles of language learning or teaching, and from these derive a set of practices for using the technology. It is a recipe book for using technological tools and, like all recipe books, prioritises activities over principles. I like the book and I don’t intend these comments as criticism. My point is simply that it’s not easy to take pedagogical principles as a starting point. Does the world of ELT even have generally agreed pedagogical principles?

And what is it that we’re teaching?

One final thought … If we consider how learners are likely to be using the English they are learning in their real-world futures, technology will not be far away: reading online, listening to / watching online material, writing and speaking with messaging apps, writing with text, email or Google Docs … If, in designing pedagogical approaches, we wish to include features of authentic language use, it’s hard to see how we can avoid placing technology fairly near the centre of the stage. Technologically-mediated language use is inseparable from pedagogy: one does not precede the other.

Similarly, if we believe that it is part of the English teacher’s job to develop the digital literacy (e.g. Hockly et al., 2013), visual literacy (e.g. Donaghy, 2015) or multimodal literacy of their students – not, incidentally, a belief that I share – then, again, technology cannot be separated from pedagogy.

Pedagogy before technology, OK??

So, I ask myself what precisely it is that people mean when they say that pedagogy should come before technology. The locutionary force, or referential meaning, usually remains unclear: in the absence of a particular pedagogy and particular contexts, what exactly is being said? The illocutionary force, likewise, is difficult to understand in the absence of a particular addressee: is the message only intended for teachers suffering from Everest Syndrome? And the perlocutionary force is equally intriguing: how are people who make the statement positioning themselves, and in relation to which addressee? Along the lines of green-washing and woke-washing, are we sometimes seeing cases of pedagogy-washing?

REFERENCES

Carrier, M., Damerow, R. M. & Bailey, K. M. (2017) Digital Language Learning and Teaching: Research, theory, and practice. New York: Routledge

Donaghy, K. (2015) Film in Action. Peaslake, Surrey: DELTA Publishing

Herodotou, C., Sharples, M., Gaved, M., Kukulska-Hulme, A., Rienties, B., Scanlon, E. & Whitelock, D. (2019) Innovative Pedagogies of the Future: An Evidence-Based Selection. Frontiers in Education, 4 (113)

Hockly, N. (2018) Blended Learning. ELT Journal 72 (1): pp. 97 – 101

Hockly, N., Dudeney, G. & Pegrum, M. (2013) Digital Literacies. Harlow: Pearson

Li, L. (2017) New Technologies and Language Learning. London: Palgrave

McCarthy, M. (Ed.) (2016) The Cambridge Guide to Blended Learning for Language Teaching. Cambridge: Cambridge University Press

OECD (2015) Students, Computers and Learning: Making the Connection, PISA. Paris: OECD Publishing

Sharma, P. (2010) Blended Learning. ELT Journal, 64 (4): pp. 456 – 458

Sharma, P. (2019) The Complete Guide to Running a Blended Learning Course. Oxford University Press English Language Teaching Global Blog 17 October 2019. Available at: https://oupeltglobalblog.com/2019/10/17/complete-guidagogyde-blended-learning/

Stanley, G. (2013) Language Learning with Technology. Cambridge: Cambridge University Press

Surkamp, C. & Viebrock, B. (Eds.) (2018) Teaching English as a Foreign Language: An Introduction. Stuttgart: J. B. Metzler

Walsh, P. (2019) Precarity. ELT Journal, 73 (4): pp. 459–462

Watson, D. M. (2001) Pedagogy before Technology: Re-thinking the Relationship between ICT and Teaching. Education and Information Technologies 6:4: pp.251–26

Colloquium

At the beginning of March, I’ll be going to Cambridge to take part in a Digital Learning Colloquium (for more information about the event, see here ). One of the questions that will be explored is how research might contribute to the development of digital language learning. In this, the first of two posts on the subject, I’ll be taking a broad overview of the current state of play in edtech research.

I try my best to keep up to date with research. Of the main journals, there are Language Learning and Technology, which is open access; CALICO, which offers quite a lot of open access material; and reCALL, which is the most restricted in terms of access of the three. But there is something deeply frustrating about most of this research, and this is what I want to explore in these posts. More often than not, research articles end with a call for more research. And more often than not, I find myself saying ‘Please, no, not more research like this!’

First, though, I would like to turn to a more reader-friendly source of research findings. Systematic reviews are, basically literature reviews which can save people like me from having to plough through endless papers on similar subjects, all of which contain the same (or similar) literature review in the opening sections. If only there were more of them. Others agree with me: the conclusion of one systematic review of learning and teaching with technology in higher education (Lillejord et al., 2018) was that more systematic reviews were needed.

Last year saw the publication of a systematic review of research on artificial intelligence applications in higher education (Zawacki-Richter, et al., 2019) which caught my eye. The first thing that struck me about this review was that ‘out of 2656 initially identified publications for the period between 2007 and 2018, 146 articles were included for final synthesis’. In other words, only just over 5% of the research was considered worthy of inclusion.

The review did not paint a very pretty picture of the current state of AIEd research. As the second part of the title of this review (‘Where are the educators?’) makes clear, the research, taken as a whole, showed a ‘weak connection to theoretical pedagogical perspectives’. This is not entirely surprising. As Bates (2019) has noted: ‘since AI tends to be developed by computer scientists, they tend to use models of learning based on how computers or computer networks work (since of course it will be a computer that has to operate the AI). As a result, such AI applications tend to adopt a very behaviourist model of learning: present / test / feedback.’ More generally, it is clear that technology adoption (and research) is being driven by technology enthusiasts, with insufficient expertise in education. The danger is that edtech developers ‘will simply ‘discover’ new ways to teach poorly and perpetuate erroneous ideas about teaching and learning’ (Lynch, 2017).

This, then, is the first of my checklist of things that, collectively, researchers need to do to improve the value of their work. The rest of this list is drawn from observations mostly, but not exclusively, from the authors of systematic reviews, and mostly come from reviews of general edtech research. In the next blog post, I’ll look more closely at a recent collection of ELT edtech research (Mavridi & Saumell, 2020) to see how it measures up.

1 Make sure your research is adequately informed by educational research outside the field of edtech

Unproblematised behaviourist assumptions about the nature of learning are all too frequent. References to learning styles are still fairly common. The most frequently investigated skill that is considered in the context of edtech is critical thinking (Sosa Neira, et al., 2017), but this is rarely defined and almost never problematized, despite a broad literature that questions the construct.

2 Adopt a sceptical attitude from the outset

Know your history. Decades of technological innovation in education have shown precious little in the way of educational gains and, more than anything else, have taught us that we need to be sceptical from the outset. ‘Enthusiasm and praise that are directed towards ‘virtual education, ‘school 2.0’, ‘e-learning and the like’ (Selwyn, 2014: vii) are indications that the lessons of the past have not been sufficiently absorbed (Levy, 2016: 102). The phrase ‘exciting potential’, for example, should be banned from all edtech research. See, for example, a ‘state-of-the-art analysis of chatbots in education’ (Winkler & Söllner, 2018), which has nothing to conclude but ‘exciting potential’. Potential is fine (indeed, it is perhaps the only thing that research can unambiguously demonstrate – see section 3 below), but can we try to be a little more grown-up about things?

3 Know what you are measuring

Measuring learning outcomes is tricky, to say the least, but it’s understandable that researchers should try to focus on them. Unfortunately, ‘the vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them’ (Lai & Bower, 2019). Metrics such as student grades are hard to interpret, not least because of the large number of variables and the danger of many things being conflated in one score. Equally, or possibly even more, problematic, are self-reporting measures which are rarely robust. It seems that surveys are the most widely used instrument in qualitative research (Sosa Neira, et al., 2017), but these will tell us little or nothing when used for short-term interventions (see point 5 below).

4 Ensure that the sample size is big enough to mean something

In most of the research into digital technology in education that was analysed in a literature review carried out for the Scottish government (ICF Consulting Services Ltd, 2015), there were only ‘small numbers of learners or teachers or schools’.

5 Privilege longitudinal studies over short-term projects

The Scottish government literature review (ICF Consulting Services Ltd, 2015), also noted that ‘most studies that attempt to measure any outcomes focus on short and medium term outcomes’. The fact that the use of a particular technology has some sort of impact over the short or medium term tells us very little of value. Unless there is very good reason to suspect the contrary, we should assume that it is a novelty effect that has been captured (Levy, 2016: 102).

6 Don’t forget the content

The starting point of much edtech research is the technology, but most edtech, whether it’s a flashcard app or a full-blown Moodle course, has content. Research reports rarely give details of this content, assuming perhaps that it’s just fine, and all that’s needed is a little tech to ‘present learners with the ‘right’ content at the ‘right’ time’ (Lynch, 2017). It’s a foolish assumption. Take a random educational app from the Play Store, a random MOOC or whatever, and the chances are you’ll find it’s crap.

7 Avoid anecdotal accounts of technology use in quasi-experiments as the basis of a ‘research article’

Control (i.e technology-free) groups may not always be possible but without them, we’re unlikely to learn much from a single study. What would, however, be extremely useful would be a large, collated collection of such action-research projects, using the same or similar technology, in a variety of settings. There is a marked absence of this kind of work.

8 Enough already of higher education contexts

Researchers typically work in universities where they have captive students who they can carry out research on. But we have a problem here. The systematic review of Lundin et al (2018), for example, found that ‘studies on flipped classrooms are dominated by studies in the higher education sector’ (besides lacking anchors in learning theory or instructional design). With some urgency, primary and secondary contexts need to be investigated in more detail, not just regarding flipped learning.

9 Be critical

Very little edtech research considers the downsides of edtech adoption. Online safety, privacy and data security are hardly peripheral issues, especially with younger learners. Ignoring them won’t make them go away.

More research?

So do we need more research? For me, two things stand out. We might benefit more from, firstly, a different kind of research, and, secondly, more syntheses of the work that has already been done. Although I will probably continue to dip into the pot-pourri of articles published in the main CALL journals, I’m looking forward to a change at the CALICO journal. From September of this year, one issue a year will be thematic, with a lead article written by established researchers which will ‘first discuss in broad terms what has been accomplished in the relevant subfield of CALL. It should then outline which questions have been answered to our satisfaction and what evidence there is to support these conclusions. Finally, this article should pose a “soft” research agenda that can guide researchers interested in pursuing empirical work in this area’. This will be followed by two or three empirical pieces that ‘specifically reflect the research agenda, methodologies, and other suggestions laid out in the lead article’.

But I think I’ll still have a soft spot for some of the other journals that are coyer about their impact factor and that can be freely accessed. How else would I discover (it would be too mean to give the references here) that ‘the effective use of new technologies improves learners’ language learning skills’? Presumably, the ineffective use of new technologies has the opposite effect? Or that ‘the application of modern technology represents a significant advance in contemporary English language teaching methods’?

References

Bates, A. W. (2019). Teaching in a Digital Age Second Edition. Vancouver, B.C.: Tony Bates Associates Ltd. Retrieved from https://pressbooks.bccampus.ca/teachinginadigitalagev2/

ICF Consulting Services Ltd (2015). Literature Review on the Impact of Digital Technology on Learning and Teaching. Edinburgh: The Scottish Government. https://dera.ioe.ac.uk/24843/1/00489224.pdf

Lai, J.W.M. & Bower, M. (2019). How is the use of technology in education evaluated? A systematic review. Computers & Education, 133(1), 27-42. Elsevier Ltd. Retrieved January 14, 2020 from https://www.learntechlib.org/p/207137/

Levy, M. 2016. Researching in language learning and technology. In Farr, F. & Murray, L. (Eds.) The Routledge Handbook of Language Learning and Technology. Abingdon, Oxon.: Routledge. pp.101 – 114

Lillejord S., Børte K., Nesje K. & Ruud E. (2018). Learning and teaching with technology in higher education – a systematic review. Oslo: Knowledge Centre for Education https://www.forskningsradet.no/siteassets/publikasjoner/1254035532334.pdf

Lundin, M., Bergviken Rensfeldt, A., Hillman, T. et al. (2018). Higher education dominance and siloed knowledge: a systematic review of flipped classroom research. International Journal of Educational Technology in Higher Education 15, 20 (2018) doi:10.1186/s41239-018-0101-6

Lynch, J. (2017). How AI Will Destroy Education. Medium, November 13, 2017. https://buzzrobot.com/how-ai-will-destroy-education-20053b7b88a6

Mavridi, S. & Saumell, V. (Eds.) (2020). Digital Innovations and Research in Language Learning. Faversham, Kent: IATEFL

Selwyn, N. (2014). Distrusting Educational Technology. New York: Routledge

Sosa Neira, E. A., Salinas, J. and de Benito Crosetti, B. (2017). Emerging Technologies (ETs) in Education: A Systematic Review of the Literature Published between 2006 and 2016. International Journal of Emerging Technologies in Education, 12 (5). https://online-journals.org/index.php/i-jet/article/view/6939

Winkler, R. & Söllner, M. (2018): Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis. In: Academy of Management Annual Meeting (AOM). Chicago, USA. https://www.alexandria.unisg.ch/254848/1/JML_699.pdf

Zawacki-Richter, O., Bond, M., Marin, V. I. And Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education 2019