Friday, July 25, 2014

MOOC points from a real learner

For the last 9 weeks I have been enrolled in a Coursera MOOC ‘An Introduction to Marketing’, run by Wharton at the University of Pennsylvania. Here’s the question; was it worth it and have my marketing skills improved? YES & YES!
I have to admit, nine weeks ago, I was skeptical and slightly reluctant to set aside  time over nine weeks for this course, as I had looked at others that were not so good and weaker on content. Strangely, the thing that attracted me to this course was the nice certificate at the end that I could link to my LinkedIn profile. To do this I had to sign up for the ‘Signature track’. This cost £30, didn’t break the bank, but gave me a goal, made me care and kept me going.
Context matters
I had a quick look at some of the lectures and was hooked. They were not hour-long, boring videos of a dull Prof talking at me, they were 10 minute videos, well made in different consumer locations, with interesting people, explaining things in context. For example, I really appreciated the insights into the difference between product and customer-centric companies. It was this contextual approach that worked for me. I liked this real-world application side as that’s the world in which I have to apply my skills.
Quizzes kept me going
Every so often, the video would stop and I had to answer a question on what I had just seen, keeping me engaged and not allowing me to simply go with the flow, immersing me in theory until I drowned. The module quizzes also keep you on your marketing toes as it’s easy to just drift along without reflection.
Useful app
I quickly downloaded the app so I could dip in and out when I had a free 10 minutes. That is exactly what I did. Every night, before I went to bed, I watched one quick lecture, at my own pace, until I understood the concepts. This was the perfect amount of learning for me as I have a job, play the drums and all that stuff. Whenever I had a bit of free time, I’d do it – even at 3/4am - that suits me, as when I felt productive, I’d get a load done. I felt in control. 
The Prof got back to me!
I could interact with different students on the forums, reading what they had written from their experiences. I didn’t spend a huge amout of time on the forums but they were interesting. I did ask some questions, to which the professors promptly replied. It was cool to get a reply from the Prof on a question I asked about ‘Celebrity endorsements’. For once learning was a pleasant experience! 
Tests not tricks
Tests came frequently and I scored 100% in all of them (honestly!). These really made me focus on the content and commit to remembering what I had learnt. Now I look back on it, I realize that these first tests weren’t trying to trick me , judge me or confuse me (that’s what tests and exams have always felt like), but asking me if I really knew something. If I found it hard, I could flip to the lecture and seal up any gaps. Only the last and longest test was a struggle, asking me more in-depth questions and making me project what I had learnt onto different situations. This was challenging but that’s exactly what I needed, as I was doing this, not as a course, but as a way of improving my marketing skills for my job.
Overall, a good MOOC has a good set of people behind it, professors who are clearly excited by the fact they are teaching an unlimited amount of people and are passionate about that. I like the idea that I can shop around, find the MOOC that suits me, and the convenience of doing it when it suited me, usually late at night.
Final thoughts
Did I like the MOOC? Hell yeah. Did I learn much? Absolutely. Was it useful? My job is in social media marketing and this gave me some depth of understanding on mainstream marketing. Would I do another? Already started.
This is my LinkedIn profile with the certificate. I am a marketing person after all ;)

I missed something. My employer encouraged me to take this sort of course and the fact that I can take my certificate (with distinction!) back to him is a big plus.
    Carl Clark (20)

 Subscribe to RSS

Sunday, July 06, 2014

6 reasons why we don't need ‘mentors’

I’ve never had a mentor. I don’t want a mentor. I don’t like mentoring. I know this is swimming against the tide of liberal orthodoxy but I value liberal values more than I value fads, groupthink or orthodoxy. But there’s many reasons why I’m both suspicious of and reject mentoring.
1. Fictional constructs
Mentor was a character in Homer’s The Odyssey and it is often assumed that his role was one of a guiding, experienced hand for his son and family. This is false. Mentor was simply an old acquaintance, ill-qualified play a protective role to his family, and worse, turned out to be a patsy for a hidden force, the God Athena. A similar tale has unfolded in recent times, with mentoring being revived on the back of late 19th century psychoanalytic theory, where the original theory has been abandoned but the practice upon which it is based survives.
There is another later work of fiction that resurrected the classical model as a source for the word ‘mentor’ in education, Fenelon’s Les Adventures de Telemaque (1699). This is a tale about limiting the excesses of a king but it did reinforce the presence of the word ‘mentor’ in both French, then English. Yet Mentor in this ponderous novel is prone to didactic speeches about how a king should rule (aided by the aristocracy), hardly the egalitarian text one would expect to spark a revolution in education. Interestingly, it pops up again as one of two books given to Emile in the novel of the same name, by Rousseau.
2. Psychoanalytic veneer
Mentoring came out of the psychanalytic movement in education with Freud and Rogers. Nothing survives of Freud’s theories on the mind, education, dreams, humour or anything else for that matter. But Rogers is different. His legacy is more pernicious, like pollution seeping into the water table. His work has resulted in institutional practice that has hung around many decades after the core theories have been abandoned. We need to learn how to abandon practice when the theories are defunct.
3. Mentoring is a trap
As Homer actually showed, one person is not enough. To limit your path, in work or life, to one person is to be feeble when it comes to probability. Why choose one person (often that person is chosen for you) when there are lots of good people out there. It stands to reason that a range of advice on a range of diverse topics (surely work and life are diverse) needs a range of expertise. Spread your network, speak to a range and variety of people. Don’t get caught in one person’s spider’s web. Mentoring is a trap.
4.  People, social media, books etc. are better
You don’t need a single person, you need advice and expertise. That is to be found in a range of resources. Sure, a range of people can do the job and hey - the best write books. Books are cheap, so buy some of the best and get reading. You can do it where and when you want and they’re written by the world’s best, not just the person who has been chosen in your organisation or a local life coach. And if you yearn for that human face, try video – TED and YouTube – they’re free! I’d take a portion of the training budget and allow people to buy from a wide reading list, arther than institute expensive mentoring programmes. Then there's socil media a rich source of advice and guidance provided daily. This makes people more self-reliant, rather than being infantalised.
5. Absence of proof
Little (1990:297) warned us, on mentoring, that, “relative to the amount of pragmatic activity, the volume of empirical enquiry is small [and]... that rhetoric and action have outpaced both conceptual development and empirical warrant.”  This, I fear, is not unusual in the learning world.
Where such research is conducted the results are disappointing. Mentors are often seen as important learning resources in teacher education and in HE teaching development. Empirical research shows, however, that the potential is rarely realised (Edwards and Protheroe, 2003: 228; Boice, 1992: 107). The results often reveal low level "training" that simply instruct novices on the "correct" way to teach (Handal and Lauvas, 1988: 65; Hart-Landsberg et al., 1992: 31). Much mentoring has been found to be rather shallow and ineffective (Edwards, 1998: 55-56).
6. Fossilised practice
Practice gets amplified and proliferates through second-rate train the trainer and teacher training courses, pushing orthodoxies long after their sell-by, even retirement, date. Mentoring has become a lazy option and alternative for hard work, effort, real learning and reflection. By all means strive to acquire knowledge, skills and competences, but don’t imagine that any of this will come through mentoring.
Conclusion: get a life, not a coach
I know that many of you will feel uncomforted by these arguments but work and life are not playthings. It’s your life and career, so don’t for one minute imagine that the HR department has the solutions you need. Human resources is there to protect organisations from their employees, so is rarely either human or resourceful. Stay away from this stuff if you really want to remain independently human and resourceful.
English translation of Les Adventures de Telemaque
Little, J.W. (1990) ‘The Mentor Phenomenon and the Social Organisation of Teaching’, in: Review of Research in Education. Washington D.C: American Educational Research Association.
Warhurst R (2003)Learning to lecture Paper presented at the British Educational Research Association Annual Conference, Heriot-Watt University, Edinburgh.

 Subscribe to RSS

Thursday, June 12, 2014

What does the learning game have to learn from the World Cup?

Most professional sports employ data to improve performance. Yet football (soccer), in data terms, is not so much the beautiful game as a rather messy and random affair. Unlike many sports, such as basketball, American football and baseball, in soccer the ball changes sides so often it is difficult to identify patterns in the numbers. That’s not to say they don’t exist. As usual, the data, although messy, reveals some surprising facts:
1. Corners don’t matter that much. Mourino was amazed when English supporters cheered corners, as he knew they rarely led to goals. The stats support this. There is no correlation between corners and goals – the correlation is essentially zero.
2. Then there’s the old myth that teams are at their most vulnerable after scoring a goal. Teams are not more vulnerable immediately after scoring goal. In fact the numbers show that this is the least likely time that a goal will be conceded.
3. Coin toss is the most significant factor in penalty-shootout success. 60% of all penalty shootouts have been won by coin toss winners. Goalkeepers who mess about on the line and hold their hands high to look bigger also have an effect, making a miss more likely. Standing 10 cms to one side also has a significant, almost unconscious effect on the goalscorer, making one side look more tempting.
4. It’s a game of turnovers. The vast amount of moves never go beyond four passes. This has huge consequences – ‘pressing’ matters, especially in final third of field. Avoiding turnovers is perhaps the most important tactic in football.
These are just a few of the secrets revealed by Chris Anderson and David Sally, two academics, from Cornell and Dartmouth, in their book The Numbers Game – Why Everything You Know About Football is Wrong.
Seasoned managers, coaches, trainers, players often get it wrong because in football our cognitive biases exaggerate individual events. We exaggerate the positives and what is obvious and seen at the expense of the hidden, subtle and negative. A good example is defending. Mancini may have been the greatest defender ever because of what he never did – tackle. We prize tackling, yet it is often a weakness not a strength. We think that corners matter when they don’t. Similarly in education, we prize the opinions of seasoned practitioners over the data: exams, uniforms, one hour lectures, one hour lessons and all sorts of specious things just because they’re part of the traditional game.
Soccer and learning
If a sport like football, which is random and chaotic, can benefit from data and algorithms that guide action such as buying players, picking players, strategy, and tactics, then surely something far more predictable, such as learning will benefit from such an approach? What we can learn is that data about the ‘players’ is vital, what they do, when they do it and what leads to positive outcomes. It is this focus on the performance of the people who really count, learners, that is so often missing in learning.
Education gathers wrong data
Education has, perhaps, been gathering the wrong data – bums on seats, contact time, course completion, results of summative assessments, even happy sheets. What is missing is the more fine grained data about what works and doesn’t work. Data about the learner’s progress. Here. We can lever data, through algorithms to improve each student’s performance as they take a learning journey. We need the sort of data that a satnav uses to identify where they start, where they’re going and, when they go off-piste, how to get them back on track.

Just as the ‘nay-sayers’ in football claimed that the numbers would have no role to play in performance, as it was all down to good coaches, trainers and scouts, so education claims that it’s all down to good teachers. This is a stupid, silver-bullet response to a complex set of problems. It is partly down to good teachers but aided by good data, learners have the most to gain from other interventions. Education needs to take a far more critical look at pedagogic change and admit that critical analysis leads to better outcomes. This means using data, especially personal data, in real time to improve learner performance

 Subscribe to RSS

Saturday, June 07, 2014

Africa - the mobile continent

Africa is a mobile only continent. Phones were never meant to be tethered to the wall like a goat. Mobile is their natural state and everywhere you go in Africa, you see people with $10 mobiles. There’s kiosks everywhere, that offer phone charging, airtime, money transfer, recycling and repair. We have a lot to learn from Africa in this regard.
Mobile is lifeline
Why Africa? Mobile is far more important to the poor than the rich. It’s a lifeline to work, money transfer, running a small business, communications with family, medical advice, vetinary advice, market prices and increasingly knowledge and education.
Sustainable success
There is a strong relationship between internet access and economic growth. In a donor-dependent continent where agents of virtue, often compound, rather than solve problems, the ubiquitous use of mobile is one of Africa’s great sustainable successes. It’s cheap, compelling and continues to grow as it’s so damn useful. Small businesses can thrive, money managed and progress made in people’s lives.
Mobiles & literacy
Last year in Namibia I participated in a discussion about mobiles and literacy. Cornelia Koku Muganda, from Tanzania, explained why mobiles were pushing a ride in basic literacy. Every child in Africa WANTS to read and write, as they want to TXT and read TXTS. We now know that this constant writing leads to better literacy, a fuller phonetic understanding of the language and more social skills. There’s even phonics apps to txt in local languages. School is not cool but mobiles are as cool as it comes.
Mobiles & education
I’ve always been rather sceptical about m-learning in the developed world but in the developing world necessity is the mother of mobile learning, with Dr Maths through Mxit, Wikipedia Zero, even SMS requests for SMS delivery of Wikipedia in Kenya. There’s a vibrant, home grown m-learning industry emerging.
Political transparency
Africa has its share of problems, with cronyism and corruption but remember that eth Arab Spring happened largely in Africa and young people across the continent are finding their voices leading to gains in transparency and political action.
Leapfrogged landlines
They have leapfrogged the landline infrastructure and with 635m mobile subscriptions rising to 930 million by 2019. $10 simple phones, often with FM radio and torches are still dominant but $50 smartphones have hit the market. Samsung are the market leader but cheaper Chinese phones are gaining ground. (Note that Apple has only 3% share of the smartphone market.) This will make a huge difference as internet access in Africa is largely through mobiles. 70% of internet use is via mobile. According to a World Bank study, An astonishing 1 in 5 would forgo basic necessities, such as food, for extra airtime, it’s that valuable a commodity.

This is a good news story from Africa, something simple and sustainable that has emerged across the whole continent. This is something that Africans use with other Africans to improve their lives. For Africa, the future is mobile. It’s powerful, personal and portable; perfect in a continent of huge distances with huge problems and huge demand. Scalable internet access offers a cheap infrastructure with access to free content. (This is a sort of summary of a contribution made to BBC Radio Scotland this morning). 

 Subscribe to RSS

Tuesday, June 03, 2014

From Rift Valley to Oculus Rift (7 lessons learnt at eLearning Africa)

A mind blowing week in central Africa at the eLearning Africa Conference in Uganda, where I was flipped so many times, both mentally and physically. My mind was repeatedly flipped when I saw the wrong solutions being forced into the wrong contexts. Conversely, I saw unexpected solutions in the right context. Physically, I was flipped into the Nile on a White Water raft trip – more of that later.
Technology, learning and Africa
My opening gambit in a talk to Ministers from across Africa was to show that we were not far from the Rift Valley, where the first technology was invented by man – the stone axe (see full article on its importance as a learning technology). This handheld device was to last for the next 1.5 million years and is a window into the mind of early man. It showed, intent, planning, ability to find resources, hand-eye co-ordination and a culture of teaching and learning. We were close to the source of the Nile, and it was in Egypt that the first writing was invented (see full article), the big-bang in learning technology, far more important then the printing press. Papyrus technology was also invented in Africa (see full article). So what does Africa need now?
1. Innovation is not innovation unless it is sustainable
In asking what Africa needs now from technology and learning I believe we must stick to a simple mantra – that innovation is not innovation unless it is sustainable. Sugatra Mitra’s Holes in the walls are now just that – holes in walls, with no computers, no lasting impact, a waste oftime and money. Tablets may prove to be less than useful, especialy Negroponte's Ethiopian experiment. Before any initiative is funded or started, do a cost-effectiveness and sustainability analysis.
By sustainability I mean the big 6:
Sustainable stakeholders
Sustainable learners
Sustainable change management

Sustainable teacher skills
Sustainable design
Sustainable culture
Sustainable electricity/Sustainable wiring
Sustainable resources
Sustainable on cost
Sustainable technology
Sustainable maintenance
2. Infrastructure not devices
Flip the mindset away from devices to infrastructure, and focus spending on bandwidth so that accessibility and prices fall. $50 a month for an unreliable 126K connection is way too high. This means deregulation and getting networks built along with free tariffs, such as Wikipedia Zero, for educational content. Economic growth is closely correlated with internet penetration.
3. Projects not pilots
Africa is littered with short-term, funded pilots. A donor-led, pilot mentality means too many pilots are really ‘doomed to succeed’ and fall flat when finished. Pilots are thinly disguised research projects, often led by academics whose real goal is simply publication not pragmatism. Fund projects that have real feasibility objectives and sustainability as their goal.
4. Vocational not academic
Africa has schooling and Universities but a huge hole in the middle – vocational colleges. Yet what Africa desperately needs is not more Universities but more vocational learning. Economic growth will come from practical skills agenda not building expensive educational institutions. Why copy a University system that doesn’t work in the developed world. It’s expensive, elitist and graduate unemployment is rising. Only 6% of Africans will even have a chance of a  University education, what about the other 94%. Let’s focus on them, as they are Africa’s future.
5. Learning not schooling
The Millennium goals focus too much on simple schooling, yet all of us eventually leave school. What happens then is important. Employability and job creation is vital, not the Anglo-Saxon liberal-arts, colonial agenda. It’s not schools that matter but what is taught and learnt in schools. Improve the quality of teacher training (take it out of Universities) and focus on what is required locally.
6. Leapfrog don’t follow
Africa has the highest growth in mobile penetration in the world. Everywhere, people have cheap phones and use them to transfer money, communicate and get on with their lives. Mobile griwt has been the big success story and new, cheap smartphones will accelerate internat access via mobile. They’re cheap and compelling because they’re useful. Africa needs to do the same with learning, leapfrog with good infrastructure projects that use the BYOD devices. Fascinating things are happening on leapfrog infrastructure –  a geostationary satellite above the Congo with pan-African reach – one way internet access but a start. Then there’s Facebook’s solar powered drones using infra-red to provide internet access, easy to launch and maintain. Finally Google’s balloons.
7. Focus on the free
They say that information wants to be free, well education now wants to be free. We have Wikipedia, Khan Academy, OER, MOOCs (see articles on MOOCs) and so on. Africa would be mad not to take this stuff, as it’s free. MOOCs are now being produced by the likes of EPFL, Kepler and the African Virtual University in relevant languages on relevant topics – and they’re free. With MOOCs Africa has bandwidth problems, even on campuses, so well designed offline solutions are needed. We also need to integrate MOOCs into local curricula, blend involving local faculty, collaborate at the teacher level. Academic regulations need to be amended and MOOCs bundled.
Africa is rising and needs, not the failed models of the developed world but new models that are more suited to the massive demand that already exists for education and training. This is not more universities but more vocational learning. The great opportunity here, is to use the great gifts of the internet, that are already there, for free. 
This conference is a small miracle, but it's in Africa and well attended by Africans from across the continent. Once again, Rebecca Stromeyer and her fantastic team pull together a fantastic conference that focuses wholly on Africa and is not scared to ask hard questions and seek out new and radical answers.  
News from China may be the greatest boost Africa has seen in a long time. A major Chinese Solar Tech CEO says, "We are not far away from the cost of (solar energy) production for conventional energy. We are sure that by 2016 - or at the latest 2017 - the cost of solar PV will be the same as coal-fired generation in China". If true, at that moment each ail drive economic and educational growth in Africa.

 Subscribe to RSS

Thursday, May 22, 2014

Game of Phones – 7 DOs & DON’Ts in m-learning

Mobile learning is confusing. In theory it sounds great, in practice it’s often misattributed hype. Different devices have different patterns of use. The fact that you make ‘responsive’ e-learning simply means that it can be delivered on different devices NOT that it will be used on different devices. M-learning is, therefore, often more fiction than fact.
1. DON’T expect people to play tennis in a cupboard
People don’t do long e-learning courses on mobile phones. It’s a device for short, episodic activity, not long, deep, reflective learning experiences. Large e-learning courses on mobiles are a waste of time. It’s like playing tennis in a cupboard.
2. DO use for informal learning
Formal stuff is hard on mobiles, so focus on informal, such as fast facts, flashcards, quick quizzes, comms and support for students, social media and so on. People use mobiles informally, so deliver informal learning. 
3. DO use for projects
Long used for gathering material and evidence for assessment in vocational learning. Mobiles are great at gathering data, whether notes, images, audio or video. It gives impetus to learning by doing.
4. DO use for performance support
OK, you’re stuck and only have a mobile to hand. That’s when you need the right answer to a question or solution to a problem. This is short, sharp and useful. Learning at the point of need.
5. DO use for spaced practice
The most ignored piece of theory in the psychology of learning is the Ebbinghaus forgetting curve – we forget most of what people suppose we learn – fact. The solution is spaced practice and mobiles are powerful, portable and personal devices in our pockets, so we can deliver spaced practice by delivering cues from any learning experience across a defined time.
6. DON’T expect people to pay
Yes, non-wifi use costs money. Don’t expect learners to pay unless they have agreed to this approach. This sounds obvious but it’s a fact that is too often forgotten in designing m-learning.
7. DON’T cheat on mobile metrics
First, there’s confusion about what ‘mobile’ devices are, a confusion that is confusing the hell out of everyone. When I say “he’s using a mobile”, I don’t mean he’s using a ‘tablet’. Otherwise, I’d say “he’s using a tablet”.  Yet people are reporting mobile use as phones plus tablets. If the answer is, tablets are taken around by people and used as second screens, then those two criteria also apply to my laptop. This is sleight of hand. Don’t cheat on mobile metrics

In the same way that tablets have been hyped in schools, as they are limited in terms of complex learning tasks such as long-form writing, coding, tools etc., mobiles are hyped in formal learning. Don’t treat all devices as delivery devices. Different devices have different learning attributes.

 Subscribe to RSS

Friday, May 02, 2014

Educational research largely useless - no costs, no bite

A question that is often asked is why education seems to never change in relation to to technology, compared, for example, to health? One answer is that there's no real consequences of failure for deliverers in education, there is in health. But another problem is the lack of a culture of cost-effectiveness analysis (CEA).
A wonderful paper by Levin and Blefield ‘Guiding the development and use ofcost-effectiveness analysis in education’ (2013) shows how education, unlike other areas of massive expenditure fails to apply Cost-Effective Analysis (CEA) to research projects. In practice, this leaves a massive vacuum at the centre of arguments around educational interventions and explains why progress is glacial.
Given the massive costs to society for education this is a puzzle. Surely, say the authors,“alternatives that show greater productivity relative to their costs, i.e., that are more cost-effective and more efficient in the use of social resources, should be preferred for adoption and implemented more intensively.” But how often do you see costs mentioned and if they are, they are of such low quality as to be meaningless.
The problem, as the authors of the report claim, is endemic. There’s little understanding of some basic economic concepts among educational researchers. No real distinction is made between Cost Benefit Analysis (CBA) and Cost Effectiveness Analysis (CEA). There is rarely any conceptual understanding of Opportunity Costs or realistic Sensitivity Analysis. This leaves research incomplete and hanging. A good CEA can even shift marginal and even negative research results into positive territory, as one delivery method may result in marginal or even negative learning effectiveness, but at a much lower cost, allowing other interventions to take place, making the overall system better.
Examples - MOOCs, Blended, Tablets
Let me give you an example. In debates around MOOCs, many claim that MOOCs are no substitute for face-to-face campus courses. But that is not the point. If they result in the same measurable academic outcomes, or even lower educational outcomes, the real win is in the fact that MOOCs can deliver a scalable solution at a tiny fraction of the 'cost per learner' of traditional campus courses. We’re not talking about shaving percentages off the costs but coming down to a tiny fraction of the original costs. This, in turn frees up resources to do better teaching and even more research.
One more example is blended learning, where effective alternatives are rarely, truly calculated in terms of the costs components in relation to effective interventions in a blended solution. Blended learning often turns out to be just variations on blended ‘teaching’, with no real appreciation of the true costs of delivery.
Yet another example is the purchase of tablets in schools. I doubt of there is a single exhaustive cost-effective analysis on any of these large-scale purchases in the UK. In fact, disasters due to inadequate procurement have already been reported from the US.
Rarity of true costs
An additional problem is resource-based costing. In practice, estimates of costs based on resources are often hopelessly inadequate, as the institutions often doesn’t actually know the true costs of delivery. The actual costs of personnel and the all important issue use of buildings and accompanying resources is rarely calculable. Yet we know that many educational institutions, schools, colleges and especially Universities have woeful occupancy rates, making this a substantial figure. This is not easy as ‘rental’ rates are not applicable, as they are in many other areas of the economy but with a little effort one can calculate the costs of the building and its amortisation and maintenance costs. There may also be other hidden funding costs from Government or other sources. It is important to be exhaustive here. The true costs of any control group must be clear if cost-effectiveness is to be measured.
Once you have the true costs you really can determine a ‘cost per learner’, a metric I like. The next step is to also include the marginal cost per learner as further participants in online courses are normally at marginal costs. You can go further and look at the distribution of these costs but few get as far as this first metric.

The truth may be hard to bear here, but much educational research is meaningless in the sense of having no real chance of impact and change, as it does not carry through to Cost Effectiveness Analysis (CEA). The anti-corporate attitudes in our Universities is one problem, the lack of actual fiscal skills among researchers is another. But the main problem is the lack of commissioned research that demand rigour on costs. Without a truly rigorous Cost Effectiveness Analysis in education we will continue to spend huge amounts of money on fruitless research. The lesson is clear link effectiveness to costs. If you don’t the research will fall into the category of ‘inconclusive’. That means no evidence-based change will happen.

 Subscribe to RSS