Chatbots

Posted: December 1, 2016 in ed tech
Tags: , , , , , , , ,

 

Here’s a conversation I had recently with Cleverbot.

cleverbot

User (me): Hi Cleverbot. What are your views on the future of chatbots in language learning?

Cleverbot: Are you talking to yourself?

User: No, I’m asking you a question.

Cleverbot: Then what’s the question again? I forgot.

User: What are your views on the future of chatbots in language learning?

Cleverbot: Don’t you want me to finish my story?

User: No, I want you to answer my question.

Cleverbot: What is your question?

User: Do you think there is a future for chatbots in language education?

Cleverbot: Sure, if you are talking about the moon.

aliceI also tried to have a conversation with Alice, the avatar from EFL Classroom Bot, listed by Larry Ferlazzo as one of ‘the best online chatbots for practicing English’. I didn’t get any more sense out of her than out of Cleverbot.

Chatbots, apparently, are the next big thing. Again. David Mattin, head of trends and insights at trendwatching.com, writes (in the September 2016 issue of ‘Business Life’) that ‘the chatbot revolution is coming’ and that chatbots are a step towards the dream of an interface between user and technology that is so intuitive that the interface ‘simply fades away’. Chatbots have been around for some time. Remember Clippy – the Microsoft Office bot in the late 1990s – which you had to disable in order to stop yourself punching your computer screen? Since then, bots have become ubiquitous. There have been problems, such as Microsoft’s Tay bot that had to be taken down after sixteen hours earlier this year, when, after interacting with other Twitter users, it developed into an abusive Nazi. But chatbots aren’t going away and you’ve probably interacted with one to book a taxi, order food or attempt to talk to your bank. In September this year, the Guardian described them as ‘the talk of the town’ and ‘hot property in Silicon Valley’.

The real interest in chatbots is not, however, in the ‘exciting interface’ possibilities (both user interface and user experience remain pretty crude), but in the way that they are leaner, sit comfortably with the things we actually do on a phone and the fact that they offer a way of cutting out the high fees that developers have to pay to app stores . After so many start-up failures, chatbots offer a glimmer of financial hope to developers.

It’s no surprise, of course, to find the world of English language teaching beginning to sit up and take notice of this technology. A 2012 article by Ben Lehtinen in PeerSpectives enthuses about the possibilities in English language learning and reports the positive feedback of the author’s own students. ELTJam, so often so quick off the mark, developed an ELT Bot over the course of a hackathon weekend in March this year. Disappointingly, it wasn’t really a bot – more a case of humans pretending to be a bot pretending to be humans – but it probably served its exploratory purpose. duolingoAnd a few months ago Duolingo began incorporating bots. These are currently only available for French, Spanish and German learners in the iPhone app, so I haven’t been able to try it out and evaluate it. According to an infomercial in TechCrunch, ‘to make talking to the bots a bit more compelling, the company tried to give its different bots a bit of personality. There’s Chef Robert, Renee the Driver and Officer Ada, for example. They will react differently to your answers (and correct you as necessary), but for the most part, the idea here is to mimic a real conversation. These bots also allow for a degree of flexibility in your answers that most language-learning software simply isn’t designed for. There are plenty of ways to greet somebody, for example, but most services will often only accept a single answer. When you’re totally stumped for words, though, Duolingo offers a ‘help my reply’ button with a few suggested answers.’ In the last twelve months or so, Duolingo has considerably improved its ability to recognize multiple correct ways of expressing a particular idea, and its ability to recognise alternative answers to its translation tasks. However, I’m highly sceptical about its ability to mimic a real conversation any better than Cleverbot or Alice the EFL Bot, or its ability to provide systematically useful corrections.

My reasons lie in the current limitations of AI and NLP (Natural Language Processing). In a nutshell, we simply don’t know how to build a machine that can truly understand human language. Limited exchanges in restricted domains can be done pretty well (such as the early chatbot that did a good job of simulating an encounter with an evasive therapist, or, more recently ordering a taco and having a meaningless, but flirty conversation with a bot), but despite recent advances in semantic computing, we’re a long way from anything that can mimic a real conversation. As Audrey Watters puts it, we’re not even close.

When it comes to identifying language errors made by language learners, we’re not really much better off. Apps like Grammarly are not bad at identifying grammatical errors (but not good enough to be reliable), but pretty hopeless at dealing with lexical appropriacy. Much more reliable feedback to learners can be offered when the software is trained on particular topics and text types. Write & Improve does this with a relatively small selection of Cambridge English examination tasks, but a free conversation ….? Forget it.

So, how might chatbots be incorporated into language teaching / learning? A blog post from December 2015 entitled AI-powered chatbots and the future of language learning suggests one plausible possibility. Using an existing messenger service, such as WhatsApp or Telegram, an adaptive chatbot would send tasks (such as participation in a conversation thread with a predetermined topic, register, etc., or pronunciation practice or translation exercises) to a learner, provide feedback and record the work for later recycling. At the same time, the bot could send out reminders of work that needs to be done or administrative tasks that must be completed.

Kat Robb has written a very practical article about using instant messaging in English language classrooms. Her ideas are interesting (although I find the idea of students in a F2F classroom messaging each other slightly bizarre) and it’s easy to imagine ways in which her activities might be augmented with chatbot interventions. The Write & Improve app, mentioned above, could deploy a chatbot interface to give feedback instead of the flat (and, in my opinion, perfectly adequate) pop-up boxes currently in use. Come to think of it, more or less any digital language learning tool could be pimped up with a bot. Countless revisions can be envisioned.

But the overwhelming question is: would it be worth it? Bots are not likely, any time soon, to revolutionise language learning. What they might just do, however, is help to further reduce language teaching to a series of ‘mechanical and scripted gestures’. More certain is that a lot of money will be thrown down the post-truth edtech drain. Then, in the not too distant future, this latest piece of edtech will fall into the trough of disillusionment, to be replaced by the latest latest thing.

 

 

Advertisements
Comments
  1. eflnotes says:

    thanks Philip

    Chomsky says that the best current approaches in AI & NLP can be said to offer is moderately useful devices such as Google translate:

    “It’s like having a better bulldozer, that’s a good thing. If you want to build a road, it’s nice to have a big bulldozer. But it’s of no scientific interest that I can see. Though I’m sure it’s going to continue to be very popular for the reasons I mentioned. It’s intellectually almost vacuous you don’t have to know anything. And it’s in principle vacuous because the claim is there’s nothing to know.”

    A Dialogue with Noam Chomsky (1) [https://youtu.be/ZQa2X88PtYY?t=24m59s]

    hang on…maybe a nice fit in the end with ELT : )

    ta
    mura

    • philipjkerr says:

      Thanks, Mura. Interesting and, at times, highly contentious stuff … a universal lexicon in addition to a universal grammar?

      • eflnotes says:

        hehe don’t think universal lexicon is Chomskyian : )

        there is a second recent talk he did in Paris which goes into how current AI approaches blindly follow a “representationalist doctrine” which “cannot in principle discover the meaning of words” [https://youtu.be/cVKPd7edpII?list=PL4lIUw9TMdkVUXsIfxx7f4KGhUI5CGZd3]

  2. Ben Knight says:

    interesting post, Philip. I think there’s more potential value in chatbots than simply jumping on a tech bandwagon. They can provide opportunities to practise interactive conversation with ‘someone’ who appears to speak English, but without judging orlaughing at them. They don’t have to understand what the learners say with great accuracy, just be close enough to motivate the learner to continue interacting/practising. There’s an opportunity for a developer with the ELT market in mind….

    • philipjkerr says:

      Your comment, Ben, made me think about the Cambridge English 360 project with Google Cardboard. One could imagine combining that technology with AI-driven bots. But the cost / efficacy / externalities equation raises the ‘would it be worth it?’ question. For anyone who doesn’t know about the 360 project, they can find out more here: http://www.cambridgeenglish.org/beta/360/

  3. Interesting article, Philip. And they were such fun to play with when they first came out. There are a John Lennon one that was a lot of fun 😉 !

    WhatsApp, now THAT’s a great tool. Had some fantastic feedback from learners and teachers using it in a group for their class, as well as having used it myself for that. But with real people you actually know, a LOT more interesting!

  4. philipjkerr says:

    In this post, I made passing reference to wanting to punch the computer screen in response to the appearance of Clippy, the Microsoft ‘social agent’ (that’s academese for ‘agent provocateur’). I was unaware at the time of a piece of research by Antonella De Angeli and Rollo Carpenter from the School of Informatics University of Manchester. It’s called ‘Stupid computer! Abuse and social identities’ and is available here: http://www.agentabuse.org/deangeli.pdf
    It concludes that verbal abuse is the social norm in human / chatbot interaction. In a study of a corpus made up of interactions with the Jabberwacky chatbot, the researchers looked at the lexical items of statistical significance (ranging from ‘you’ to ‘vagina’), and found that, at 9th position in the position was the lemma ‘fuck’, immediately preceded by ‘well’ and ‘love’, and followed by ‘computer’ and ‘good’. ‘Sex’ came in at #16.
    10% of the stems in the corpus were insulting and offensive language, including body parts, mental abnormalities or inadequacy, immoral behaviour, and sexuality.
    Unfortunately, the researchers don’t consider the consequences of their findings for language learning, but they do make one interesting suggestion, which I would hope Duolingo will take up on. That is, the programming of chatbots could be modified so both humans and chatbots can aggress, thus providing us with interesting material for future study.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s