Posts Tagged ‘Pearson’

Ok, let’s be honest here. This post is about teacher training, but ‘development’ sounds more respectful, more humane, more modern. Teacher development (self-initiated, self-evaluated, collaborative and holistic) could be adaptive, but it’s unlikely that anyone will want to spend the money on developing an adaptive teacher development platform any time soon. Teacher training (top-down, pre-determined syllabus and externally evaluated) is another matter. If you’re not too clear about this distinction, see Penny Ur’s article in The Language Teacher.

decoding_adaptive jpgThe main point of adaptive learning tools is to facilitate differentiated instruction. They are, as Pearson’s latest infomercial booklet describes them, ‘educational technologies that can respond to a student’s interactions in real-time by automatically providing the student with individual support’. Differentiation or personalization (or whatever you call it) is, as I’ve written before  , the declared goal of almost everyone in educational power these days. What exactly it is may be open to question (see Michael Feldstein’s excellent article), as may be the question of whether or not it is actually such a desideratum (see, for example, this article ). But, for the sake of argument, let’s agree that it’s mostly better than one-size-fits-all.

Teachers around the world are being encouraged to adopt a differentiated approach with their students, and they are being encouraged to use technology to do so. It is technology that can help create ‘robust personalized learning environments’ (says the White House)  . Differentiation for language learners could be facilitated by ‘social networking systems, podcasts, wikis, blogs, encyclopedias, online dictionaries, webinars, online English courses,’ etc. (see Alexandra Chistyakova’s post on eltdiary ).

But here’s the crux. If we want teachers to adopt a differentiated approach, they really need to have experienced it themselves in their training. An interesting post on edweek  sums this up: If professional development is supposed to lead to better pedagogy that will improve student learning AND we are all in agreement that modeling behaviors is the best way to show people how to do something, THEN why not ensure all professional learning opportunities exhibit the qualities we want classroom teachers to have?

Differentiated teacher development / training is rare. According to the Center for Public Education’s Teaching the Teachers report , almost all teachers participate in ‘professional development’ (PD) throughout the year. However, a majority of those teachers find the PD in which they participate ineffective. Typically, the development is characterised by ‘drive-by’ workshops, one-size-fits-all presentations, ‘been there, done that’ topics, little or no modelling of what is being taught, a focus on rotating fads and a lack of follow-up. This report is not specifically about English language teachers, but it will resonate with many who are working in English language teaching around the world.cindy strickland

The promotion of differentiated teacher development is gaining traction: see here or here , for example, or read Cindy A. Strickland’s ‘Professional Development for Differentiating Instruction’.

Remember, though, that it’s really training, rather than development, that we’re talking about. After all, if one of the objectives is to equip teachers with a skills set that will enable them to become more effective instructors of differentiated learning, this is most definitely ‘training’ (notice the transitivity of the verbs ‘enable’ and ‘equip’!). In this context, a necessary starting point will be some sort of ‘knowledge graph’ (which I’ve written about here ). For language teachers, these already exist, including the European Profiling Grid , the Eaquals Framework for Language Teacher Training and Development, the Cambridge English Teaching Framework and the British Council’s Continuing Professional Development Framework (CPD) for Teachers  . We can expect these to become more refined and more granularised, and a partial move in this direction is the Cambridge English Digital Framework for Teachers  . Once a knowledge graph is in place, the next step will be to tag particular pieces of teacher training content (e.g. webinars, tasks, readings, etc.) to locations in the framework that is being used. It would not be too complicated to engineer dynamic frameworks which could be adapted to individual or institutional needs.cambridge_english_teaching_framework jpg

This process will be facilitated by the fact that teacher training content is already being increasingly granularised. Whether it’s an MA in TESOL or a shorter, more practically oriented course, things are getting more and more bite-sized, with credits being awarded to these short bites, as course providers face stiffer competition and respond to market demands.

Visible classroom home_page_screenshotClassroom practice could also form part of such an adaptive system. One tool that could be deployed would be Visible Classroom , an automated system for providing real-time evaluative feedback for teachers. There is an ‘online dashboard providing teachers with visual information about their teaching for each lesson in real-time. This includes proportion of teacher talk to student talk, number and type of questions, and their talking speed.’ John Hattie, who is behind this project, says that teachers ‘account for about 30% of the variance in student achievement and [are] the largest influence outside of individual student effort.’ Teacher development with a tool like Visible Classroom is ultimately all about measuring teacher performance (against a set of best-practice benchmarks identified by Hattie’s research) in order to improve the learning outcomes of the students.Visible_classroom_panel_image jpg

You may have noticed the direction in which this part of this blog post is going. I began by talking about social networking systems, podcasts, wikis, blogs and so on, and just now I’ve mentioned the summative, credit-bearing possibilities of an adaptive teacher development training programme. It’s a tension that is difficult to resolve. There’s always a paradox in telling anyone that they are going to embark on a self-directed course of professional development. Whoever pays the piper calls the tune and, if an institution decides that it is worth investing significant amounts of money in teacher development, they will want a return for their money. The need for truly personalised teacher development is likely to be overridden by the more pressing need for accountability, which, in turn, typically presupposes pre-determined course outcomes, which can be measured in some way … so that quality (and cost-effectiveness and so on) can be evaluated.

Finally, it’s worth asking if language teaching (any more than language learning) can be broken down into small parts that can be synthesized later into a meaningful and valuable whole. Certainly, there are some aspects of language teaching (such as the ability to use a dashboard on an LMS) which lend themselves to granularisation. But there’s a real danger of losing sight of the forest of teaching if we focus on the individual trees that can be studied and measured.

Back in December 2013, in an interview with eltjam , David Liu, COO of the adaptive learning company, Knewton, described how his company’s data analysis could help ELT publishers ‘create more effective learning materials’. He focused on what he calls ‘content efficacy[i]’ (he uses the word ‘efficacy’ five times in the interview), a term which he explains below:

A good example is when we look at the knowledge graph of our partners, which is a map of how concepts relate to other concepts and prerequisites within their product. There may be two or three prerequisites identified in a knowledge graph that a student needs to learn in order to understand a next concept. And when we have hundreds of thousands of students progressing through a course, we begin to understand the efficacy of those said prerequisites, which quite frankly were made by an author or set of authors. In most cases they’re quite good because these authors are actually good in what they do. But in a lot of cases we may find that one of those prerequisites actually is not necessary, and not proven to be useful in achieving true learning or understanding of the current concept that you’re trying to learn. This is interesting information that can be brought back to the publisher as they do revisions, as they actually begin to look at the content as a whole.

One commenter on the post, Tom Ewens, found the idea interesting. It could, potentially, he wrote, give us new insights into how languages are learned much in the same way as how corpora have given us new insights into how language is used. Did Knewton have any plans to disseminate the information publicly, he asked. His question remains unanswered.

At the time, Knewton had just raised $51 million (bringing their total venture capital funding to over $105 million). Now, 16 months later, Knewton have launched their new product, which they are calling Knewton Content Insights. They describe it as the world’s first and only web-based engine to automatically extract statistics comparing the relative quality of content items — enabling us to infer more information about student proficiency and content performance than ever before possible.

The software analyses particular exercises within the learning content (and particular items within them). It measures the relative difficulty of individual items by, for example, analysing how often a question is answered incorrectly and how many tries it takes each student to answer correctly. It also looks at what they call ‘exhaustion’ – how much content students are using in a particular area – and whether they run out of content. The software can correlate difficulty with exhaustion. Lastly, it analyses what they call ‘assessment quality’ – how well  individual questions assess a student’s understanding of a topic.

Knewton’s approach is premised on the idea that learning (in this case language learning) can be broken down into knowledge graphs, in which the information that needs to be learned can be arranged and presented hierarchically. The ‘granular’ concepts are then ‘delivered’ to the learner, and Knewton’s software can optimise the delivery. The first problem, as I explored in a previous post, is that language is a messy, complex system: it doesn’t lend itself terribly well to granularisation. The second problem is that language learning does not proceed in a linear, hierarchical way: it is also messy and complex. The third is that ‘language learning content’ cannot simply be delivered: a process of mediation is unavoidable. Are the people at Knewton unaware of the extensive literature devoted to the differences between synthetic and analytic syllabuses, of the differences between product-oriented and process-oriented approaches? It would seem so.

Knewton’s ‘Content Insights’ can only, at best, provide some sort of insight into the ‘language knowledge’ part of any learning content. It can say nothing about the work that learners do to practise language skills, since these are not susceptible to granularisation: you simply can’t take a piece of material that focuses on reading or listening and analyse its ‘content efficacy at the concept level’. Because of this, I predicted (in the post about Knowledge Graphs) that the likely focus of Knewton’s analytics would be discrete item, sentence-level grammar (typically tenses). It turns out that I was right.

Knewton illustrate their new product with screen shots such as those below.

Content-Insight-Assessment-1

 

 

 

 

 

Content-Insight-Exhaustion-1

 

 

 

 

 

 

 

They give a specific example of the sort of questions their software can answer. It is: do students generally find the present simple tense easier to understand than the present perfect tense? Doh!

It may be the case that Knewton Content Insights might optimise the presentation of this kind of grammar, but optimisation of this presentation and practice is highly unlikely to have any impact on the rate of language acquisition. Students are typically required to study the present perfect at every level from ‘elementary’ upwards. They have to do this, not because the presentation in, say, Headway, is not optimised. What they need is to spend a significantly greater proportion of their time on ‘language use’ and less on ‘language knowledge’. This is not just my personal view: it has been extensively researched, and I am unaware of any dissenting voices.

The number-crunching in Knewton Content Insights is unlikely, therefore, to lead to any actionable insights. It is, however, very likely to lead (as writer colleagues at Pearson and other publishers are finding out) to an obsession with measuring the ‘efficacy’ of material which, quite simply, cannot meaningfully be measured in this way. It is likely to distract from much more pressing issues, notably the question of how we can move further and faster away from peddling sentence-level, discrete-item grammar.

In the long run, it is reasonable to predict that the attempt to optimise the delivery of language knowledge will come to be seen as an attempt to tackle the wrong question. It will make no significant difference to language learners and language learning. In the short term, how much time and money will be wasted?

[i] ‘Efficacy’ is the buzzword around which Pearson has built its materials creation strategy, a strategy which was launched around the same time as this interview. Pearson is a major investor in Knewton.

There are a number of reasons why we sometimes need to describe a person’s language competence using a single number. Most of these are connected to the need for a shorthand to differentiate people, in summative testing or in job selection, for example. Numerical (or grade) allocation of this kind is so common (and especially in times when accountability is greatly valued) that it is easy to believe that this number is an objective description of a concrete entity, rather than a shorthand description of an abstract concept. In the process, the abstract concept (language competence) becomes reified and there is a tendency to stop thinking about what it actually is.

Language is messy. It’s a complex, adaptive system of communication which has a fundamentally social function. As Diane Larsen-Freeman and others have argued patterns of use strongly affect how language is acquired, is used, and changes. These processes are not independent of one another but are facets of the same complex adaptive system. […] The system consists of multiple agents (the speakers in the speech community) interacting with one another [and] the structures of language emerge from interrelated patterns of experience, social interaction, and cognitive mechanisms.

As such, competence in language use is difficult to measure. There are ways of capturing some of it. Think of the pages and pages of competency statements in the Common European Framework, but there has always been something deeply unsatisfactory about documents of this kind. How, for example, are we supposed to differentiate, exactly and objectively, between, say, can participate fully in an interview (C1) and can carry out an effective, fluent interview (B2)? The short answer is that we can’t. There are too many of these descriptors anyway and, even if we did attempt to use such a detailed tool to describe language competence, we would still be left with a very incomplete picture. There is at least one whole book devoted to attempts to test the untestable in language education (edited by Amos Paran and Lies Sercu, Multilingual Matters, 2010).

So, here is another reason why we are tempted to use shorthand numerical descriptors (such as A1, A2, B1, etc.) to describe something which is very complex and abstract (‘overall language competence’) and to reify this abstraction in the process. From there, it is a very short step to making things even more numerical, more scientific-sounding. Number-creep in recent years has brought us the Pearson Global Scale of English which can place you at a precise point on a scale from 10 to 90. Not to be outdone, Cambridge English Language Assessment now has a scale that runs from 80 points to 230, although Cambridge does, at least, allocate individual scores for four language skills.

As the title of this post suggests (in its reference to Stephen Jay Gould’s The Mismeasure of Man), I am suggesting that there are parallels between attempts to measure language competence and the sad history of attempts to measure ‘general intelligence’. Both are guilty of the twin fallacies of reification and ranking – the ordering of complex information as a gradual ascending scale. These conceptual fallacies then lead us, through the way that they push us to think about language, into making further conceptual errors about language learning. We start to confuse language testing with the ways that language learning can be structured.

We begin to granularise language. We move inexorably away from difficult-to-measure hazy notions of language skills towards what, on the surface at least, seem more readily measurable entities: words and structures. We allocate to them numerical values on our testing scales, so that an individual word can be deemed to be higher or lower on the scale than another word. And then we have a syllabus, a synthetic syllabus, that lends itself to digital delivery and adaptive manipulation. We find ourselves in a situation where materials writers for Pearson, writing for a particular ‘level’, are only allowed to use vocabulary items and grammatical structures that correspond to that ‘level’. We find ourselves, in short, in a situation where the acquisition of a complex and messy system is described as a linear, additive process. Here’s an example from the Pearson website: If you score 29 on the scale, you should be able to identify and order common food and drink from a menu; at 62, you should be able to write a structured review of a film, book or play. And because the GSE is so granular in nature, you can conquer smaller steps more often; and you are more likely to stay motivated as you work towards your goal. It’s a nonsense, a nonsense that is dictated by the needs of testing and adaptive software, but the sciency-sounding numbers help to hide the conceptual fallacies that lie beneath.

Perhaps, though, this doesn’t matter too much for most language learners. In the early stages of language learning (where most language learners are to be found), there are countless millions of people who don’t seem to mind the granularised programmes of Duolingo or Rosetta Stone, or the Grammar McNuggets of coursebooks. In these early stages, anything seems to be better than nothing, and the testing is relatively low-stakes. But as a learner’s interlanguage becomes more complex, and as the language she needs to acquire becomes more complex, attempts to granularise it and to present it in a linearly additive way become more problematic. It is for this reason, I suspect, that the appeal of granularised syllabuses declines so rapidly the more progress a learner makes. It comes as no surprise that, the further up the scale you get, the more that both teachers and learners want to get away from pre-determined syllabuses in coursebooks and software.

Adaptive language learning software is continuing to gain traction in the early stages of learning, in the initial acquisition of basic vocabulary and structures and in coming to grips with a new phonological system. It will almost certainly gain even more. But the challenge for the developers and publishers will be to find ways of making adaptive learning work for more advanced learners. Can it be done? Or will the mismeasure of language make it impossible?

Pearson’s ‘Efficacy’ initiative is a series of ‘commitments designed to measure and increase the company’s impact on learning outcomes around the world’. The company’s dedicated website  offers two glossy brochures with a wide range of interesting articles, a good questionnaire tool that can be used by anyone to measure the efficacy of their own educational products or services, as well as an excellent selection of links to other articles, some of which are critical of the initiative. These include Michael Feldstein’s long blog post  ‘Can Pearson Solve the Rubric’s Cube?’ which should be a first port of call for anyone wanting to understand better what is going on.

What does it all boil down to? The preface to Pearson’s ‘Asking More: the Path to Efficacy’ by CEO John Fallon provides a succinct introduction. Efficacy in education, says Fallon, is ‘making a measurable impact on someone’s life through learning’. ‘Measurable’ is the key word, because, as Fallon continues, ‘it is increasingly possible to determine what works and what doesn’t in education, just as in healthcare.’ We need ‘a relentless focus’ on ‘the learning outcomes we deliver’ because it is these outcomes that can be measured in ‘a systematic, evidence-based fashion’. Measurement, of course, is all the easier when education is delivered online, ‘real-time learner data’ can be captured, and the power of analytics can be deployed.

Pearson are very clearly aligning themselves with recent moves towards a more evidence-based education. In the US, Obama’s Race to the Top is one manifestation of this shift. Britain (with, for example, the Education Endowment Foundation) and France (with its Fonds d’Expérimentation pour la Jeunesse ) are both going in the same direction. Efficacy is all about evidence-based practice.

Both the terms ‘efficacy’ and ‘evidence-based practice’ come originally from healthcare. Fallon references this connection in the quote two paragraphs above. In the UK last year, Ben Goldacre (medical doctor, author of ‘Bad Science’ and a relentless campaigner against pseudo-science) was commissioned by the UK government to write a paper entitled ‘Building Evidence into Education’ . In this, he argued for the need to introduce randomized controlled trials into education in a similar way to their use in medicine.

As Fallon observed in the preface to the Pearson ‘Efficacy’ brochure, this all sounds like ‘common sense’. But, as Ben Goldacre discovered, things are not so straightforward in education. An excellent article in The Guardian outlined some of the problems in Goldacre’s paper.

With regard to ELT, Pearson’s ‘Efficacy’ initiative will stand or fall with the validity of their Global Scale of English, discussed in my March post ‘Knowledge Graphs’ . However, there are a number of other considerations that make the whole evidence-based / efficacy business rather less common-sensical than might appear at first glance.

  • The purpose of English language teaching and learning (at least, in compulsory education) is rather more than simply the mastery of grammatical and lexical systems, or the development of particular language skills. Some of these other purposes (e.g. the development of intercultural competence or the acquisition of certain 21st century skills, such as creativity) continue to be debated. There is very little consensus about the details of what these purposes (or outcomes) might be, or how they can be defined. Without consensus about these purposes / outcomes, it is not possible to measure them.
  • Even if we were able to reach a clear consensus, many of these outcomes do not easily lend themselves to measurement, and even less to low-cost measurement.
  • Although we clearly need to know what ‘works’ and what ‘doesn’t work’ in language teaching, there is a problem in assigning numerical values. As the EduThink blog observes, ‘the assignation of numerical values is contestable, problematic and complex. As teachers and researchers we should be engaging with the complexity [of education] rather than the reductive simplicities of [assigning numerical values]’.
  • Evidence-based medicine has resulted in unquestionable progress, but it is not without its fierce critics. A short summary of the criticisms can be found here .  It would be extremely risky to assume that a contested research procedure from one discipline can be uncritically applied to another.
  • Kathleen Graves, in her plenary at IATEFL 2014, ‘The Efficiency of Inefficiency’, explicitly linked health care and language teaching. She described a hospital where patient care was as much about human relationships as it was about medical treatment, an aspect of the hospital that went unnoticed by efficiency experts, since this could not be measured. See this blog for a summary of her talk.

These issues need to be discussed much further before we get swept away by the evidence-based bandwagon. If they are not, the real danger is that, as John Fallon cautions, we end up counting things that don’t really count, and we don’t count the things that really do count. Somehow, I doubt that an instrument like the Global Scale of English will do the trick.

‘Adaptive’ is a buzzword in the marketing of educational products. Chris Dragon, President of Pearson Digital Learning, complained on the Pearson Research blog. that there are so many EdTech providers claiming to be ‘adaptive’ that you have to wonder if they are not using the term too loosely. He talks about semantic satiation, the process whereby ‘temporary loss of meaning [is] experienced when one is exposed to the uninterrupted repetition of a word or phrase’. He then goes on to claim that Pearson’s SuccessMaker (‘educational software that differentiates and personalizes K-8 reading and math instruction’) is the real adaptive McCoy.

‘Adaptive’ is also a buzzword in marketing itself. Google the phrase ‘adaptive marketing’ and you’ll quickly come up with things like Adaptive Marketing Set to Become the Next Big Thing or Adaptive marketing changes the name of the game. Adaptive marketing is what you might expect: the use of big data to track customers and enable ‘marketers to truly tailor their activities in rapid and unparalleled ways to meet their customers’ interests and needs’ (Advertising Age, February 2012). It strikes me that this sets up an extraordinary potential loop: students using adaptive learning software that generates a huge amount of data which could then be used by adaptive marketers to sell other products.

I decided it might be interesting to look at the way one adaptive software company markets itself. Knewton, for example, which claims its products are more adaptive than anybody else’s.

Knewton clearly spend a lot of time and money on their marketing efforts. There is their blog and a magazine called ‘The Knerd’. There are very regular interviews by senior executives with newspapers, magazines and other blogs. There are very frequent conference presentations. All of these are easily accessible, so it is quite easy to trace Knewton’s marketing message. And even easier when they are so open about it. David Liu, Chief Operating Officer has given an interview  in which he outlines his company’s marketing strategy. Knewton, he says, focuses on driving organic interests and traffic. To that end, we have a digital marketing group that’s highly skilled and focused on creating content marketing so users, influencers and partners alike can understand our product, the value we bring and how to work with us. We also use a lot of advanced digital and online lead generation type of techniques to target potential partners and users to be able to get the right people in those discussions.

The message consists of four main strands, which I will call EdTech, EduCation, EduBusiness and EdUtopia. Depending on the audience, the marketing message will be adapted, with one or other of these strands given more prominence.

1 EdTech

Hardly surprisingly, Knewton focuses on what they call their ‘heavy duty infrastructure for an adaptive world’. They are very proud of their adaptive credentials, their ‘rigorous data science’. The basic message is that ‘only Knewton provides true personalization for any student, anywhere’. They are not shy of using technical jargon and providing technical details to prove their point.

2 EduCation

The key message here is effectiveness (Knewton also uses the term ‘efficacy’). Statistics about growth in pass rates and reduction in withdrawal rates at institutions are cited. At the same time, teachers are directly appealed to with statements like ‘as a teacher, you get tools you never had before’ and ‘teachers will be able to add their own content, upload it, tag it and seamlessly use it’. Accompanying this fairly direct approach is a focus on buzz words and phrases which can be expected to resonate with teachers. Recent blog posts include in their headlines: ‘supporting creativity’, ‘student-centred learning’, ‘peer mentoring’, ‘formative evaluation’, ‘continuous assessment’, ‘learning styles’, ‘scaffolding instruction’, ‘real-world examples’, ‘enrichment’ or ‘lifelong learning’.

There is an apparent openness in Knewton’s readiness to communicate with the rest of the world. The blog invites readers to start discussions and post comments. Almost no one does. But one blog post by Jose Ferreira called ‘Rebooting Learning Styles’  provoked a flurry of highly critical and well-informed responses. These remain unanswered. A similar thing happened when David Liu did a guest post at eltjam. A flurry of criticism, but no response. My interpretation of this is that Knewton are a little scared of engaging in debate and of having their marketing message hijacked.

3 EduBusiness

Here’s a sample of ways that Knewton speak to potential customers and investors:

an enormous new market of online courses that bring high margin revenue and rapid growth for institutions that start offering them early and declining numbers for those who do not.

Because Knewton is trying to disrupt the traditional industry, we have nothing to lose—we’re not cannibalising ourselves—by partnering.

Unlike other groups dabbling in adaptive learning, Knewton doesn’t force you to buy pre-fabricated products using our own content. Our platform makes it possible for anyone — publishers, instructors, app developers, and others — to build her own adaptive applications using any content she likes.

The data platform industries tend to have a winner-take-all dynamic. You take that and multiply it by a very, very high-stakes product and you get an even more winner-take-all dynamic.

4 EdUtopia

I personally find this fourth strand the most interesting. Knewton are not unique in adopting this line, but it is a sign of their ambition that they choose to do so. All of the quotes that follow are from Jose Ferreira:

We can’t improve education by curing poverty. We have to cure poverty by improving education.

Edtech is our best hope to narrow — at scale — the Achievement Gap between rich and poor. Yet, for a time, it will increase that gap. Society must push past that unfortunate moment and use tech-assisted outcome improvements as the rationale to drive spending in poor schools.

I started Knewton to do my bit to fix the world’s education system. Education is among the most important problems we face, because it’s the ultimate “gateway” problem. That is, it drives virtually every global problem that we face as a species. But there’s a flip-side: if we can fix education, then we’ll dramatically improve the other problems, too. So in fact, I started Knewton not just to help fix education but to try to fix just about everything.

What if the girl who invents the cure for ovarian cancer is growing up in a Cambodian fishing village and otherwise wouldn’t have a chance? As distribution of technology continues to improve, adaptive learning will give her and similar students countless opportunities that they otherwise wouldn’t have.

But our ultimate vision – and what really motivated me to start the company – is to solve the access problem for the human race once and for all. Only 22% of the world finishes high school; only 55% finish sixth grade. This is a preventable tragedy. Adaptive learning can give students around the world access to high-quality education they wouldn’t otherwise have.

In Part 9 of the ‘guide’ on this blog (neo-liberalism and solutionism), I suggested that the major advocates of adaptive learning form a complex network of vested neo-liberal interests. Along with adaptive learning and the digital delivery of educational content, they promote a free-market, for-profit, ‘choice’-oriented (charter schools in the US and academies in the UK) ideology. The discourses of these advocates are explored in a fascinating article by Neil Selwyn, ‘Discourses of digital ‘disruption’ in education: a critical analysis’ which can be accessed here.

Stephen Ball includes a detailed chart of this kind of network in his ‘Global Education Inc.’ (Routledge 2012). I thought it would be interesting to attempt a similar, but less ambitious, chart of my own. Sugata Mitra’s plenary talk at the IATEFL conference yesterday has generated a lot of discussion, so I thought it would be interesting to focus on him. What such charts demonstrate very clearly is that there is a very close interlinking between EdTech advocacy and a wider raft of issues on the neo-liberal wish list. Adaptive learning developments (or, for example, schools in the cloud) need to be understood in a broader context … in the same way that Mitra, Tooley, Gates et al understand these technologies.

In order to understand the chart, you will need to look at the notes below. Many more nodes could be introduced, but I have tried my best to keep things simple. All of the information here is publicly available, but I found Stephen Ball’s work especially helpful.

mitra chart

People

Bill Gates is the former chief executive and chairman of Microsoft, co-chair of the Bill and Melinda Gates Foundation.

James Tooley is the Director of the E.G. West Centre. He is a founder of the Educare Trust, founder and chairman of Omega Schools, president of Orient Global, chairman of Rumi School of Excellence, and a former consultant to the International Finance Corporation. He is also a member of the advisory council of the Institute of Economic Affairs and was responsible for creating the Education and Training Unit at the Institute.

Michael Barber is Pearson’s Chief Education Advisor and Chairman of Pearson’s $15 million Affordable Learning Fund. He is also an advisor on ‘deliverology’ to the International Finance Corporation.

Sugata Mitra is Professor of Educational Technology at the E.G. West Centre and he is Chief Scientist, Emeritus, at NIIT. He is best known for his “Hole in the Wall” experiment. In 2013, he won the $1 million TED Prize to develop his idea of a ‘school-in-the-cloud’.

Institutions

Hiwel (Hole-in-the-Wall Education Limited) is the company behind Mitra’s “Hole in the Wall” experiment. It is a subsidiary of NIIT.

NIIT Limited is an Indian company based in Gurgaon, India that operates several for-profit higher education institutions.

Omega Schools is a privately held chain of affordable, for-profit schools based in Ghana.There are currently 38 schools educating over 20,000 students.

Orient Global is a Singapore-based investment group, which bought a $48 million stake in NIIT.

Pearson is … Pearson. Pearson’s Affordable Learning Fund was set up to invest in private companies committed to innovative approaches. Its first investment was a stake in Omega Schools.

Rumi Schools of Excellence is Orient Global’s chain of low-cost private schools in India, which aims to extend access and improve educational quality through affordable private schooling.

School-in-the-cloud is described by Mitra as’ a learning lab in India, where children can embark on intellectual adventures by engaging and connecting with information and mentoring online’. Microsoft are the key sponsors.

The E.G. West Centre of the University of Newcastle is dedicated to generating knowledge and understanding about how markets and self organising systems work in education.

The Educare Trustis a non-profit agency, formed in 2002 by Professor James Tooley of the University of Newcastle Upon Tyne, England, and other members associated with private unaided schools in India.It is advised by an international team from the University of Newcastle. It services include the running of a loan scheme for schools to improve their infrastructure and facilities.

The Institute of Economic Affairs is a right-wing free market think tank in London whose stated mission is to improve understanding of the fundamental institutions of a free society by analysing and expounding the role of markets in solving economic and social problems.

The International Finance Corporation is an international financial institution which offers investment, advisory, and asset management services to encourage private sector development in developing countries. The IFC is a member of the World Bank Group.

The Templeton Foundation is a philanthropic organization that funds inter-disciplinary research about human purpose and ultimate reality. Described by Barbara Ehrenreich as a ‘right wing venture’, it has a history of supporting the Cato Institute (publishers of Tooley’s most well-known book) , a libertarian think-tank, as well as projects at major research centers and universities that explore themes related to free market economics.

Additional connections

Barber is an old friend of Tooley’s from when both men were working in Zimbabwe in the 1990s.

Omega Schools are taking part in Sugata Mitra’s TED Prize Schools in the Cloud project.

Omega Schools use textbooks developed by Pearson.

Orient Global sponsored an Education Development fund at Newcastle University. The project leaders were Tooley and Mitra. They also sponsored the Hole-in-the-Wall experiment.

Pearson, the Pearson Foundation, Microsoft and the Gates Foundation work closely together on a wide variety of projects.

Some of Tooley’s work for the Educare Trust was funded by the Templeton Trust. Tooley was also winner of the 2006 Templeton Freedom Prize for Excellence.

The International Finance Corporation and the Gates Foundation are joint sponsors of a $60 million project to improve health in Nigeria.

The International Finance Corporation was another sponsor of the Hole-in-the-Wall experiment.

Let’s take a look at the business of adaptive learning from a publisher’s perspective. Not an ELT publisher, but someone a few rungs higher on the ladder with strategic responsibilities. You might not know a great deal about ELT. It is, after all, only one of a number of divisions you are responsible for, and not an especially profitable one at that. You will, however, know a lot about creative destruction, the process by which one industry is replaced by another. The decline and demise of printed magazines, newspapers and books, of book reviewers and traditional booksellers, and their replacement by digital products will be such a part of your professional knowledge that they hardly need to be mentioned. Graphs such as the one below from PricewaterhouseCoopers (PwC) will be terribly familiar. You will also be aware that the gales of creative destruction in publishing are blowing harder than ever before.

2014-03-31_1020

In fact, you probably owe your job to your ability to talk convincingly about creative destruction and how to profit from it. Whatever your particular strategy for the future might be, you will have noted the actions of others. You will have evaluated advice, such as the following, from Publishing Perspectives

  • Do not delay in taking action when there are clear signals of a decline in market value.
  • Trade your low-profit print assets (even though some may have limited digital value) for core product that has a higher likelihood of success and can be exploited digitally.
  • Look for an orderly transition from print to digital product which enables a company to reinvent itself.

You will be looking to invest in technology, and prioritizing the acquisition of technological expertise (through partnerships or the purchase of start-ups) over the development of traditional ELT products. Your company will be restructured, and possibly renamed, to facilitate the necessary changes.

You will also know that big data and analytics have already transformed other industries. And you will know that educational publishing is moving towards a winner-take-all business market, where ‘the best performers are able to capture a very large share of the rewards, and the remaining competitors are left with very little’ (Investopedia). Erik Brynjolfsson and Andrew McAfee’s new book, The Second Machine Age (New York: Norton, 2014), argues that ‘each time a market becomes more digital, winner-take-all economics become a little more compelling …Digitization creates winner-take-all markets because [there are] enormous economies of scale, giving the market leader a huge cost advantage and room to beat the price of any competitor while still making a good profit’ (pp.153-155).

the second machine age

It is in this light that we need to understand the way that companies like Pearson and Macmillan are banking everything on a digital future. Laurie Harrison’s excellent blog post at eltjam  summarises the Pearson position: ‘the world’s biggest education publisher is spending £150m on a total restructure which involves an immediate move to digital learning, a focus on emerging markets, and a transformation from publisher to education services provider. If the English language learning market is worth $4billion a year, then Pearson still only have a very small chunk of it. And if you’re a company as successful and ambitious as Pearson, that just isn’t good enough – so a change of direction is needed. In order to deliver this change, the company have recently announced their new senior management team.’

Adaptive learning fits the new business paradigm perfectly. If the hype is to be believed, adaptive learning will be a game-changer. ‘The shifting of education from analog to digital is a one-time event in the history of the human race. At scale, it will have as big an effect on the world as indoor plumbing or electricity,’ writes Jose Ferreira of Knewton. ‘True disruption,’ he says elsewhere, ‘happens when entrepreneurs aim big and go after a giant problem, a problem that, if solved, would usher in an era of large-scale transformation across industries and nations. … Education is the last of the information industries to move online,’ he goes on. ‘When it breaks, it breaks fast. And that’s going to happen in the next five years. All the education content will go online in the next 10 years. And textbooks will go away. … Ultimately, all learning materials will be digital and they will all be adaptive.’

Ferreira clearly knows all about creative disruption. He also knows about winner-take-all markets. ‘The question is who is going to power [the] platform,’ he writes. ‘It’s probably going to be one or two companies’. He states his ambition for Knewton very clearly: ‘Knewton’s goal is to be like Amazon Web Services for education’. ‘It’s pretty clear to us,’ he writes, ‘that there’s going to be one dominant data platform for education, the way there’s one dominant data platform for search, social media, etailing. But in education, it’s going to be even more winner-take-all; there will be a number of companies that make up the platform, like Wintel. People might make a perverse choice to use Bing for search because they don’t like Google. But no one’s going to make the choice to subject their kid to the second-best adaptive learning platform, if that means there’s a 23% structural disadvantage. The data platform industries tend to have a winner-take-all dynamic. You take that and multiply it by a very, very high-stakes product and you get an even more winner-take-all dynamic.’

What is at stake in this winner-take-all market? Over to Jose Ferreira one more time: ‘The industry is massive. It’s so massive that virtually nobody I’ve met truly grasps how big it is. It’s beyond their frame of reference. The total amount of money (both public and private) spent annually exceeds all spending, both online and offline, of every other information industry combined: that is, all media, entertainment, games, news, software, Internet and mobile media, e-tailing, etc.’

But, still, a few questions continue to nag away at me. If all of this is so certain, why does Jose Ferreira feel the need to talk about it so much? If all of this is so certain, why don’t all the ELT publishers jump on the bandwagon? What sort of track record does economic forecasting have, anyway?

In a recent interesting post on eltjam, Cleve Miller wrote the following

Knewton asks its publishing partners to organize their courses into a “knowledge graph” where content is mapped to an analyzable form that consists of the smallest meaningful chunks (called “concepts”), organized as prerequisites to specific learning goals. You can see here the influence of general learning theory and not SLA/ELT, but let’s not concern ourselves with nomenclature and just call their “knowledge graph” an “acquisition graph”, and call “concepts” anything else at all, say…“items”. Basically our acquisition graph could be something like the CEFR, and the items are the specifications in a completed English Profile project that detail the grammar, lexis, and functions necessary for each of the can-do’s in the CEFR. Now, even though this is a somewhat plausible scenario, it opens Knewton up to several objections, foremost the degree of granularity and linearity.

In this post, Cleve acknowledges that, for the time being, adaptive learning may be best suited to ‘certain self-study material, some online homework, and exam prep – anywhere the language is fairly defined and the content more amenable to algorithmic micro-adaptation.’ I would agree, but its value / usefulness will depend on getting the knowledge graph right.

Which knowledge graph, then? Cleve suggests that it could be something like the CEFR, but it couldn’t be the CEFR itself because it is, quite simply, too vague. This was recognized by Pearson when they developed their Global Scale of English (GSE), an instrument which, they claim, can provide ‘for more granular and detailed measurements of learners’ levels than is possible with the CEFR itself, with its limited number of wide levels’. This Global Scale of English will serve as ‘the metric underlying all Pearson English learning, teaching and assessment products’, including, therefore, the adaptive products under development.

gse2

‘As part of the GSE project, Pearson is creating an associated set of Pearson Syllabuses […]. These will help to link instructional content with assessments and to create a reference for authoring, instruction and testing.’ These syllabuses will contain grammar and vocabulary inventories which ‘will be expressed in the form of can-do statements with suggested sample exponents rather than as the prescriptive lists found in more traditional syllabuses.’ I haven’t been able to get my hands on one of these syllabuses yet: perhaps someone could help me out?

Informal feedback from writer colleagues working for Pearson suggests that, in practice, these inventories are much more prescriptive than Pearson claim, but this is hardly surprising, as the value of an inventory is precisely its more-or-less finite nature.

Until I see more, I will have to limit my observations to two documents in the public domain which are the closest we have to what might become knowledge graphs. The first of these is the British Council / EAQUALS Core Inventory for General EnglishScott Thornbury, back in 2011, very clearly set out the problems with this document and, to my knowledge, the reservations he expressed have not yet been adequately answered. To be fair, this inventory was never meant to be used as a knowledge graph: ‘It is a description, not a prescription’, wrote the author (North, 2010). But presumably a knowledge graph would look much like this, and it would have the same problems. The second place where we can find what a knowledge graph might look like is English Profile and this is mentioned by Cleve. Would English Profile work any better? Possibly not. Michael Swan’s critique of English Profile (ELTJ 68/1 January 2014 pp.89-96) asks some big questions that have yet, to my knowledge, to be answered.

Knewton’s Sally Searby has said that, for ELT, knowledge graphing needs to be ‘much more nuanced’. Her comment suggests a belief that knowledge graphing can be much more nuanced, but this is open to debate. Michael Swan quotes Prodeau, Lopez and Véronique (2012): ‘the sum of pragmatic and linguistic skills needed to achieve communicative success at each level makes it difficult, if not impossible, to find lexical and grammatical means that would characterize only one level’. He observes that ‘the problem may, in fact, simply not be soluble’.

So, what kind of knowledge graph are we likely to see? My best bet is that it would look a bit like a Headway syllabus.

Talk the big data talk

Posted: March 7, 2014 in big data
Tags: ,

Pearson’s Efficacy document has a chapter called ‘A New Era of Learning Efficacy on a Planet of Smarter Systems’ by Jon Iwata. It’s basically a paean of praise to the potential of big data.  I threw the text into a word cloud program to see what would come up. And what we get is a handy little guide for anyone who wants to bluff their way in a discussion about adaptive learning. Alternatively, you could use it for buzz word bingo.

Efficacy

Some of the words you won’t be needing at all are ‘teach’, ‘teachers’, ‘classrooms’ or ‘lessons’. Sorry about the blur in the image: it’s my laptop having an emotional response.

The drive towards adaptive learning is being fuelled less by individual learners or teachers than it is by commercial interests, large educational institutions and even larger agencies, including national governments. How one feels about adaptive learning is likely to be shaped by one’s beliefs about how education should be managed.

Huge amounts of money are at stake. Education is ‘a global marketplace that is estimated conservatively to be worth in excess of $5 trillion per annum’ (Selwyn, Distrusting Educational Technology 2013, p.2). With an eye on this pot, in one year, 2012, ‘venture capital funds, private equity investors and transnational corporations like Pearson poured over $1.1 billion into education technology companies’[1] Knewton, just one of a number of adaptive learning companies, managed to raise $54 million before it signed multi-million dollar contracts with ELT publishers like Macmillan and Cambridge University Press. In ELT, some publishing companies are preferring to sit back and wait to see what happens. Most, however, have their sights firmly set on the earnings potential and are fully aware that late-starters may never be able to catch up with the pace-setters.

The nexus of vested interests that is driving the move towards adaptive learning is both tight and complicated. Fuller accounts of this can be found in Stephen Ball’s ‘Education Inc.’ (2012) and Joel Spring’s ‘Education Networks’ (2012) but for this post I hope that a few examples will suffice.

Leading the way is the Bill and Melinda Gates Foundation, the world’s largest private foundation with endowments of almost $40 billion. One of its activities is the ‘Adaptive Learning Market Acceleration Program’ which seeks to promote adaptive learning and claims that the adaptive learning loop can defeat the iron triangle of costs, quality and access (referred to in The Selling Points of Adaptive Learning, above). It is worth noting that this foundation has also funded Teach Plus, an organisation that has been lobbying US ‘state legislatures to eliminate protection of senior teachers during layoffs’ (Spring, 2012, p.51). It also supports the Foundation for Excellence in Education, ‘a major advocacy group for expanding online instruction by changing state laws’ (ibid., p.51). The chairman of this foundation is Jeb Bush, brother of ex-president Bush, who took the message of his foundation’s ‘Digital Learning Now!’ program on the road in 2011. The message, reports Spring (ibid. p.63) was simple: ‘the economic crises provided an opportunity to reduce school budgets by replacing teachers with online courses.’ The Foundation for Excellence in Education is also supported by the Walton Foundation (the Walmart family) and iQity, a company whose website makes clear its reasons for supporting Jeb Bush’s lobbying. ‘The iQity e-Learning Platform is the most complete solution available for the electronic search and delivery of curriculum, courses, and other learning objects. Delivering over one million courses each year, the iQity Platform is a proven success for students, teachers, school administrators, and district offices; as well as state, regional, and national education officials across the country.[2]

Another supporter of the Foundation for Excellence in Education is the Pearson Foundation, the philanthropic arm of Pearson. The Pearson Foundation, in its turn, is supported by the Gates Foundation. In 2011, the Pearson Foundation received funding from the Gates Foundation to create 24 online courses, four of which would be distributed free and the others sold by Pearson the publishers (Spring, 2012, p.66).

The campaign to promote online adaptive learning is massively funded and extremely well-articulated. It receives support from transnational agencies such as the World Bank, WTO and OECD, and its arguments are firmly rooted in the discourse ‘of international management consultancies and education businesses’ (Ball, 2012, p.11-12). It is in this context that observers like Neil Selwyn connect the growing use of digital technologies in education to the corporatisation and globalisation of education and neo-liberal ideology.

Adaptive learning also holds rich promise for those who can profit from the huge amount of data it will generate. Jose Fereira, CEO of Knewton, acknowledges that adaptive learning has ‘the capacity to produce a tremendous amount of data, more than maybe any other industry’[3]. He continues ‘Big data is going to impact education in a big way. It is inevitable. It has already begun. If you’re part of an education organization, you need to have a vision for how you will take advantage of big data. Wait too long and you’ll wake up to find that your competitors (and the instructors that use them) have left you behind with new capabilities and insights that seem almost magical.’ Rather paradoxically, he then concludes that ‘we must all commit to the principle that the data ultimately belong to the students and the schools’. It is not easy to understand how such data can be both the property of individuals and, at the same time, be used by educational organizations to gain competitive advantage.

The existence and exploitation of this data may also raise concerns about privacy. In the same way that many people do not fully understand the extent or purpose of ‘dataveillance’ by cookies when they are browsing the internet, students cannot be expected to fully grasp the extent or potential commercial use of the data that they generate when engaged in adaptive learning programs.

Selwyn (Distrusting Educational Technology 2013, p.59-60) highlights a further problem connected with the arrival of big data. ‘Dataveillance’, he writes, also ‘functions to decrease the influence of ‘human’ experience and judgement, with it no longer seeming to matter what a teacher may personally know about a student in the face of his or her ‘dashboard’ profile and aggregated tally of positive and negative ‘events’. As such, there would seem to be little room for ‘professional’ expertise or interpersonal emotion when faced with such data. In these terms, institutional technologies could be said to be both dehumanizing and deprofessionalizing the relationships between people in an education context – be they students, teachers, administrators or managers.’

Adaptive learning in online and blended programs may well offer a number of advantages, but these will need to be weighed against the replacement or deskilling of teachers, and the growing control of big business over educational processes and content. Does adaptive learning increase the risk of transforming language teaching into a digital diploma mill (Noble, Digital Diploma Mills: The automation of higher education 2002)?

Solutionism

Evgeney Morozov’s 2013 best-seller, ‘To Save Everything, Click Here’, takes issue with our current preoccupation with finding technological solutions to complex and contentious problems. If adaptive learning is being presented as a solution, what is the problem that it is the solution of? In Morosov’s analysis, it is not an educational problem. ‘Digital technologies might be a perfect solution to some problems,’ he writes, ‘but those problems don’t include education – not if by education we mean the development of the skills to think critically about any given issue’ (Morosov, 2013, p.8). Only if we conceive of education as the transmission of bits of information (and in the case of language education as the transmission of bits of linguistic information), could adaptive learning be seen as some sort of solution to an educational problem. The push towards adaptive learning in ELT can be seen, in Morosov’s terms, as reaching ‘for the answer before the questions have been fully asked’ (ibid., p.6).

The world of education has been particularly susceptible to the dreams of a ‘technical fix’. Its history, writes Neil Selwyn, ‘has been characterised by attempts to use the ‘power’ of technology in order to solve problems that are non-technological in nature. […] This faith in the technical fix is pervasive and relentless – especially in the minds of the key interests and opinion formers of this digital age. As the co-founder of the influential Wired magazine reasoned more recently, ‘tools and technology drive us. Even if a problem has been caused by technology, the answer will always be more technology’ (Selwyn, Education in a Digital World 2013, p.36).

Morosov cautions against solutionism in all fields of human activity, pointing out that, by the time a problem is ‘solved’, it becomes something else entirely. Anyone involved in language teaching would be well-advised to identify and prioritise the problems that matter to them before jumping to the conclusion that adaptive learning is the ‘solution’. Like other technologies, it might, just possibly, ‘reproduce, perpetuate, strengthen and deepen existing patterns of social relations and structures – albeit in different forms and guises. In this respect, then, it is perhaps best to approach educational technology as a ‘problem changer’ rather than a ‘problem solver’ (Selwyn, Education in a Digital World 2013, p.21).


[1] Philip McRae Rebirth of the Teaching Machine through the Seduction of Data Analytics: This time it’s personal April 14, 2013 http://philmcrae.com/2/post/2013/04/rebirth-of-the-teaching-maching-through-the-seduction-of-data-analytics-this-time-its-personal1.html (last accessed 13 January 2014)

[2] http://www.iq-ity.com/ (last accessed 13 January, 2014)