Friday, July 03, 2015
Sure to make most educationalists and teachers splutter with indignation, robot teachers (or at least teaching assistants) may not be far off. I wrote a piece Robot teachers five years ago, and have been following to progress ever since. Teachers will always be with us but can robots extend and enhance learning in the home and classroom? The point, in the short term, is not to replace teachers but to replace the teaching of SOME TEACHING TASKS. Cheap, consumer tech, with brilliant AI and UX software point towards a future where robots will have the affordances necessary for this role.
You already use robots
Robots already play a major role in your world. At any given time you are likely to be wearing, watching, listening to, eating, driving, flying or using something that has been partly made by robots. They are already your manufacturing servants. Why? In some tasks, they outperform humans in precision, consistency, endurance, strength and speed. The lesson is clear. When robots can perform a task more precisely, consistently and with greater endurance and speed than humans, then, given reasonable manufacturing costs and social acceptability, they will be used. Robots have now started to emerge in education and learning.
In terms of learning theory that backs up the possibility of robot teaching, let’s start with some research. Bloom is famous for his taxonomy but, in my view, his more substantial contribution was his famous paper, The 2 Sigma Problem where he compared the effectiveness of the lecture (conventional), formative feedback lecture (mastery learning) and one-to-one tuition (tutorial). Taking the straight lecture as the mean, he found an 84% increase in mastery above the mean for a formative approach to teaching and an astonishing 98% increase in mastery for one-to-one tuition. Google’s Peter Norvig famously said that if you only have to read one paper to support online learning, this is it. In other words, the increase in efficacy for one-to-one, because of the increase in on-task learning is immense. This paper deserves to be read by anyone looking at improving the efficacy of
learning as it shows hugely significant improvements by simply altering the way teachers interact with learners. Online learning, in the widest sense of the word, promises what Bloom called ‘one-to-one learning’, whether it’s through self-paced structured learning, scenario-based learning, simulations or informal learning. This points towards the future of learning as being individual, personalized, one-to-one teaching, if possible.
So what about research that backs up the use of robots? Nass & Reeves published a remarkable book The Media Equation: How People Treat Computers, Television and New Media Like Real People and Places. They completed 35 experiments to show that people confuse media with real life. This conclusion shows that we have a disposition towards suspension of disbelief when interacting with most media, to see media as having human dimensions. It’s hard-wired. The conclusion we can draw from this, and I did when designing online learning programmes, is that learning experiences, if delivered in a suitably human manner (not necessarily by real humans) can be successful. The key is that learning experience must conform to certain social and physical rules. If we can deliver learning as if it were being delivered by a real person in a realistic way, it works. What Nass & Reeves researched was the role of simple physical and social rules. We respond to politeness, avoiding unnatural pauses, flattery, don’t like harsh judgments and so on. These are the things that we expect in dialogue. If this can be created through robots, AI and good UX, we will will have come a long way towards the one-to-one tuition that Bloom shows is the most effective way to teach.
There is one more dimension to robot learning that matters – design. Donald Norman laid the ground rules for successful technology, the touchstone being its ‘invisibility’. This is true ergonomically but also true of interfaces, which need to be cognitively ergonomic. Norman saw our emotional responses to design in terms of:
1 Visceral (appearance)
2 Behavioural (performance)
3 Reflective (memories and experience)
Interestingly he thinks Americans value 2 more than 1&3, whereas Europeans, at least the cultural classes, value 1&3. He claims that different people buy things with different fuel mixtures of the three emotions. Different companies design to different types of emotions. Great companies, like Apple, deliver all three. As he explains in Living with Complexity, it is not that technology delivers too much complexity. The fact is, we live in a world of complexity, with complex technologies that do complex things. Live with it – that’s reality. The enemy is not complexity, it is dreadful design. Complexity needs to be tamed, masked or made invisible with good design. This is precisely what robots are starting to deliver – great design that makes the mechanics of learning invisible.
Robots meet AI &UX
Most great technology is a combination of technology The iPhone is a cluster of existing technologies where the sum is greater than the parts. Brian Arthur, in The Nature of Technology, thinks that this is an essential feature of technology, the coalescence of existing technologies to create something exciting and new. This is what makes the confluence between robots and AI so exciting.
With adaptive learning (AI in learning), where what the robot delivers (using algorithms) is always in response to who you are, what you know and what you do, we get somewhere towards the necessary one-to-one, personalized and more human-like delivery necessary for robot teaching. Remember also, that machine learning, the ability of the robot to learn, even learn on the back of aggregated data from many learners from a network of robots, could see rapid progress in effectiveness. These adaptive systems are already here. I work with them in real educational institutions with tens of thousands of learners.
Search, speech recognition, gesture recognition, touch screens, text to speech, automatic translation – all of these are now available and only getting better. They are being be embedded into robots. Speech recognition is the big leap but some are also taking advantage of touchscreen interfaces on the chest of the robot. With sensors, movement, balance, vision and hearing, these robots are getting pretty serious.
We’ve had simple robots that allow teachers to teach from a distance in rural schools. The Nexus Academy of Columbia allows teachers to Skype in and control this robot from their computer, going up to student’s desks and checking work.
Robosem is a Korean Robot used to teach English. It takes a hybrid approach, either using a real teacher via teleconferencing or through autonomous, adaptive lessons that use speech recognition and motion tracking. There are many of these examples but we are now seeing some serious commercial activity.
Time to introduce NAO, now used in education in 70 countries. You can teach NAO and NAO can teach you. This is nowhere near the dream I’ve outlined above but it’s a start and it works. With speech recognition, NAO will call you by your first name, teach your child multiplication tables, wake you up, monitor your home if you’re out. NAO will be able to recognize members of the family, your friends, judge moods, know your preferences. This is just one of a slew of early robots that will eventually find a role in education.
Now let me introduce Pepper with it’s four microphones, two color video cameras, 3D sensor, touch sensors, bump sensors, lasers, sonar, and gyros positioned in the head, body, arms, and legs, voice recognition and a touchscreen on its chest, with wifi. This companion learns from its environment and draws from it’s cloud-based, updated, master algorithms. It has emotion recognition reading visual expressions and from speech tones. Price: $1900.
The ‘cute’ factor is the current focus, as these are aimed at young children but the long-term goal is much bolder. I see the first advances in early literacy, numeracy, simple programming, as well as second languages. I also see real applications for children with autism and other differences in learning, where the pressure on parents is immense and the needs often different from traditional classroom learning. Children with autism have been shown to respond positively to synchronised behaviour from a robot. This is used to move the child on to other types of social interaction. Robots have being used to with autistic children using mimicry to establish trust.
Looking to the future, check out this amazing example of robots teaching other robots. Berlin researcher Luc Steels, has developed the robots that learn, speak to each other and speak back.
Beyond this we have Google investing madly in AI and robotics, with Deepmind, created by a learning expert (here for some background to relevance to learning). Google was the obvious buyer for this type of company, as it thinks in terms of deep, basic problems, looking for generic solutions. Google has the eyeballs, brains and behavioural traits of billions of humans. They have the data that matters. This means they can apply Deep Learning solutions on scale. But they’re not the only game in town. IBM’s Watson is set to deliver AI on tap, Microsoft are using AI in their services and the Chinese search giant Baidu are all hiring, buying and experimenting.
As Stanford’s Nass & Reeves showed, we need teaching and learning services to at least appear to be friendly, patient, efficient, polite, relevant, relevant, personalised and…. well social and human. Parents spend hundreds of millions on one-to-one tuition for their kids. Much, much more is spent on teachers and teaching. Robots can give unlimited amounts of attention and help, in ways no teacher, or even parent, can. Robots don’t get hangovers, don’t take holidays, never discriminate on grounds of gender, race or accent. They’re patient, scalable and consistent. Wouldn’t it be wise to pay attention to something that takes at least some of the pain away?
Monday, June 15, 2015
All of the Above – 20 ways to cheat Multiple Choice questions
Forget objectives, start with questions
I never liked the 'start with the learning objectives' advice on designing learning content, and preferred to start with writing and polishing up these objectives as test items. This seemed that much more real, practical and relevant than forced 'Mager-like' objectives. Get the assessment right and the rest seemed to follow. This, I suspect, is why students much prefer to get past test papers to establish the real contents of a course.
Of course, writing good test items is far more difficult than many imagine, which is why many tests are not really tests of understanding, merely tests of recall. An interesting way of coming at this problem is to do some reverse engineering.
If you think this doesn’t work, think again. Poundstone number crunched 100 tests with a total of 2456 questions to get some of these statistical biases. I have 30 years experience in writing the damn things.
Second-guessing the test designer
Many multiple choice questions are poorly written. What better way to expose these errors than write a crib sheet for learners? So here goes with my 20 ways to cheat Multiple Choice tests:
1. Skip the hard questions, mark them with a cross, and go back to them. This means you’ll not lose marks for unanswered easy questions.
2. Cover the options and try to answer. Prevents being misled by clever wrong options.
3. If in doubt choose ‘B’, poor questions designers do not truly randomise the right options and have a bias towards ‘B’. Next best is ‘C’.
4. If in doubt choose the ‘longest option’. Question designers often cannot make a right option any shorter, but have complete freedom with wrong options.
5. If in doubt choose TRUE, in true/false questions, as they come easier to mind for designers.
6. Reverse answers. There is more T/F alteration in tests than in truly randomized sequences. So, if you’re sure you’ve got one right, reverse the next answer.
7. Eliminate the outlier. Look for similarities in options and eliminate outliers (in bold) e.g. 4p-q, 2p+q, 4p+q, 3p+q. Look for these internal patterns.
8. If two options are opposites, one is likely to be correct. Designers first made up option is likely to be the correct option’s opposite.
9. Favour options with careful qualifiers, such as ‘sometimes, occasionally etc.’ as tested knowledge usually has more finite than absolute qualities.
10. Be wary of options with absolute qualifiers, such as ‘always, never etc’. As these are often too definite to be reasonably correct.
11. Choose a middle order option i.e. out of 100, 150. 200, 250, choose 150 or 200. Designers tend to have a bias, where right answers tend to be lower than the highest and higher than the lowest option.
12. For questions that demand an ‘except’ or ‘not’, mark each option with a T for true and F for false against each option. And underline the word ‘not’ as it’s sometimes missed.
13. ‘All of the above’ and ‘None of the above’ are both significantly likely to be correct. is likely to be correct. For it to be correct the writer has to design options that were all correct, so, if you can’t spot any wrong answers, or see that two or more are correct, it increases the probability of ‘All of the above’ being correct. Similarly with ‘None of the above’.
14. Typo or punctuation error, the option is likely to be wrong. Writers tend to proofread correct answers only.
15. Look for grammatical agreement between the question and its options; ‘An.....’ and words starting with vowels or agreement between subject, object or verb.
16. If you’re stuck, go with the ‘Least bad rule’. Eliminate least likely answers first.
17. Look for clues about answers from other questions. Designers often, unintentionally, put clues, even answers, to questions in other questions.
18. Ignore never heard of answers. If you’ve never heard of the answer, it’s likely to be made up and incorrect.
19. Go with your first impression. The more you read, the more you tend to read into the wrong options.
20. Always guess, unless there is a penalty. It’s a 1 in 4 chance, so don’t give it up.
This crib sheet can be used by students or question designers to improve their tests. Good students put themselves in the shoes of the test designer to improve their chance, so the more you know about their techniques, the better designer you’ll be.
Next post - how to write a great MC question...
School of Athens: explains a lot about modern schooling?
If one artwork captures the roots of our Western intellectual tradition it is The School of Athens (Scuola di Atene) by Raphael. Note the title. The figures are set within a ‘school’ both the place, and metaphorically, the golden thread of a tradition that still has he influence on education today. The school is actually Roman architecture, not Greek, but is meant to echo the schools of the two central, principle figures; Plato (Academy) and Aristotle (Lyceum).
Plato and Aristotle
Plato steps forward and points to the sky (heavens), while Aristotle stands still with his hand level, palm down to the ground (real world). This represents two different philosophical traditions that were to shape, not only western philosophy but also religion and learning, both theory and practice. In their hands, Plato holds his Timaeus, Aristotle, his Ethics. This shows a divergence between the theoretical, cosmological and metaphysical concerns of Plato and the grounded, earthly and practical approach of Aristotle. They represent two schools of thought but also two approaches to schooling. This is a simplification but Plato, the rationalist is contrasted with Aristotle, the empiricist. This persists today in the arts/academic versus science/vocational debate around curricula and educational policy.
Another figure, stands off to the left, dressed simply in green, a secular colour in the Renaissance, in deep dialogue with a young man, with his back to Plato and Aristotle. Although the figure behind looks across to Plato, as it is through the Platonic dialogues that we know most about this man - Socrates. He had a profound influence on the western approach to learning that is still alive today. The sceptic, whose educational approach was to deconstruct through dialogue, strip away pre-conceptions and expose ignorance. He doesn’t conform to any of the traditions around him and survives today, in the Socratic method, as someone who believes in an approach that eschews lectures for dialogue, feedback and reflection. (see Socrates as learning theorist)
There is in this image, another theme, related to both Plato and Aristotle, but also other figures, such as Euclid and Pythagoras. Pythagoras is the figure writing in a book in the foreground on the left, surrounded by acolytes. He represents abstract mathematics and the idea that learning is about the master transmitting immutable knowledge to their students. His parallel figure in the foreground on the right is Euclid (some say Archimedes), leaning down to demonstrate his proofs, on what looks like a slate, with callipers, where the students are in discussion, working through the proofs in their heads. Again, this contrast exists between the didactic teaching of a canon and the more learner-centric view of the learner as someone who has to learn by doing and reflection.
Diogenes sits as a sceptic, alone, looking at no one, in front of Plato and Aristotle. He’s a check on these systematic thinkers, representing another learning thread that was by this time coming alive in the University system and certainly came from the Greeks – scepticism, and its close relative, cynicism. There’s a host of other characters, such a Zoroaster and Averroes, showing non Greek threads but the main pantheon of teachers are mostly Greek.
That an intellectual tradition is represented as a great work of art is one thing, but Raphael also injected another theme into the fresco. He represents some of the figures from known representations of busts, others, it is speculated, have the faces of famous artists, Plato (Leonardo da Vinci), Aristotle (Giuliano da Sangallo), Heraclitus (Michelangelo), Plotinus (Donatello). Raphael is thought to have included himself, as the figure at the elbow of Epicurus (on left lifting the bowl from the plinth). The sculptures behind the figures are Apollo (left), God of music and light, and Athena (right) Goddess of wisdom, again reflecting rhetorically the arts and knowledge as underlying themes in learning. Again, we have a lasting theme in education, the role of the arts.
From philosophy to theology
It may seem odd that this painting was commissioned by a Pope and is to be found in the Vatican. However, remember that this fresco is one of many frescos in this room, and adjoining rooms, that represent largely Christian and theological issues. Theology had, well before this point and for many centuries, held an iron grip on the educational process, that was to continue, and never really disappear, even in our supposedly secular age.
There’s no large-scale lecturing in this image, although nascent technology in the several books (3), scroll (1), pens and notebooks in which notes are being taken (3), compasses (1), globes (2) and what appear to be slates (2), are already being used to assist learning and teaching.
The main triumvirate of Greek philosophers define the strands for learning and educational theory that are alive today. The great schism between the academic and practical was set in motion and the Socratic tradition defined, but, so often ignored.
Eurozzzzone - a giant squeezyjet, tax-funded, city-break, gravy train
Have a look at this ELIG 2015 conference. Check out the speaker list. Notice how non-pan-European it is? Notice its almost complete focus on Germany and he fact that Microsoft & IBM are there but almost no one from the rest of Europe? Their slogan is 'We change the way Europe learns' - oh yeah!
A few years ago I was invited to attend a one week trip to Spain, paid for by some European grant. I was the UK rep and there were reps from each of the other 14 countries. It was a revelation, in that it was a complete waste of time. At the end of the fact-finding trip we got together to discuss our findings, which to paraphrase the discussion, showed that Spain was milking European grants like a starved piglet at the teat, but to no great effect in terms of entrepreneurial progress (the trip was about entrepreneurship). I was the only business person of the 14, and was politely told to forget the criticisms and write the report free from any negative conclusions. It turned out to be a massive fraud with ghost employees and huge sums extracted from the system. Many are now in jail.
It was a shocking introduction to the neverneverland that is the European Union. As the Euroland implodes in a sort of low-key rerun of the second world war, one wonders what all that money spent on ‘euro-learning’ actually brought us?
Answer - nothing. Academics have been sqeezyjetting around Europe for years to meetings that were little more than excuses for short-breaks or a nosh-up. The collaborative projects weren’t designed around competences or goals, merely a bunch of people who were good at form-filling. Then there’s that obnoxious group of middlemen, no better than street drug runners, who promise to get you a chunk of the motherload (for a fee of course). The whole sorry tale is largely one of useless research on useless projects set up by worse than useless hustlers. In practice, the real work was being done by hard-working people in real companies and organisations doing things with real people in the real world. European projects are like the Eurovision Song Contest, countries send their least talented people to a contest that is best known as a parody of the real world and the output is woeful. It used to be something to laugh at, now it’s a politicised, block vote idiotfest. I’m positive about Europe as a single market but sad that so much money has been spent with so little meaningful output. The real action is in hte commercial conferences, such as Online Educa, that genuinely try to attract a pan-European audience and do so with little or no government support.
One market myth
Europe is not a single market in education and training because people learn best in their first language, and in the UK we don’t have a second. Almost all education and training in Europe is delivered by local and national suppliers. There has huge effort to create pan-European companies by supporting pan-European research, but it hasn’t worked. Giunti Labs seem to have been on some sort of permanent financial drip from Brussels, along with several other companies that would never have survived in the commercial world.
Let’s take the ‘E’ out of e-learning
ELEARNINGEUROPA, EU4ALL, ERGO, ECLO, ELIG, EDEN, EFQUEL, EIFEL, EMDEL and on and on it goes…..Dozens of crap acronyms all staring with ‘E’ and hundreds of administrators, unread reports, AGMs, meetings and conferences. When I ran an e-learning company I had absolutely nothing to do with any of these or any other European quangos. Have they delivered? I think not. We're far more likely to look to the US and the E-learning Guild, that attempts to dig deep into real practice, than any European organisation, wieghed down by beurocrats. Would the world miss them, I think not. Let’s take the ‘E’ out of e-learning – Europe that is!