I previously covered a study which found social scientists can’t predict better than laymen. As you may recall, ‘experts’ and laymen were asked to predict the size and direction of social change in the U.S. between April and October of 2020. Overall, the two groups were equally inaccurate – there was no overall advantage for the ‘experts’.
A new study finds that social scientists can’t predict better than simple models either. (The study is a pre-print, so hasn’t yet been peer reviewed.)
Dillon Bowen collated data from several previous studies in which ‘experts’ had been asked to predict the impact of various interventions on people’s behaviour. His analysis covered five previous studies, but I’ll focus on the first three since the other two are rather complicated.
In the first study, 90 ‘experts’ were asked to predict how effective 53 different nudges would be at encouraging gym users to exercise more often. These nudges included things like administering a free audiobook and a bonus for returning after missed workouts.
In the second study, 24 ‘experts’ were asked to predict how effective 22 text-message interventions would be at encouraging Walmart customers to get a flu vaccine. These interventions included things like a simple reminder and a joke about the flu.
In the third study, 237 ‘experts’ were asked to predict how effective interventions carried out by two government nudge units would be at changing citizens’ behaviour. These interventions encompassed domains like healthcare, state benefits and community engagement.
For all three studies, Bowen compared the ‘expert’ predictions to a null model that said none of the interventions would work (i.e., none of them would change subjects’ behaviour by a statistically significant amount).
Remarkably, he found that in all three cases the null model performed better than the ‘experts’.
In other words, the ‘experts’ overestimated the interventions’ effects by such an extent that they would have been more accurate by simply guessing ‘no effect’ for every single one. Or put in other way, the effects of the interventions were generally closer to zero than they were to the ‘experts’ predictions.
Behavioural interventions are widely used by governments and other interventions to tweak people’s behaviour without coercing them. Yet practitioners consistently overestimate how well these interventions work. As it turns out, ‘they don’t work at all’ is a safer assumption than whatever the ‘experts’ are saying.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.
Governments use of coercion has gone far beyond encouragement or tweaking. It is in the realm of propaganda and should be stopped.
As I recall it was the Liberal Democrat David Cameron who introduced the “nudge unit” into Downing Street.
Nudge: if persuasion doesn’t work, use force.
A huge problem here is that the ‘social scientists’ haven’t included an understanding of how over-use of their ‘nudges’ could result in widescale mistrust of any future ‘nudges’.
I suggest that the extreme propaganda and ‘psychological meddling’ that went on in 2020 and 2021 will in time result in extreme social unrest that could well have a huge impact.
Are you aware of this publication from Amnesty in 1973? The coercions utilised relentlessly are recognised as forms of torture. Slide 26 p49
https://www.amnesty.org/en/documents/ACT40/001/1973/en/
There’s only one reliable predictor of human behaviour at our disposal and that’s the stockmarket.
So powerful that the US military seriously considered using a market to bet on the likelihood of future terrorist attacks. Yes, seeing where the money would go would be used to try and get one step ahead of terrorists across the globe. Obviously it was quickly shelved as being unethical and never used.
One curious aspect of Covid was that on March 23rd the stockmarket, in the midst of a new apparently deadly virus outbreak, started one of the biggest bull runs in history. At its peak in November 2021 it had run up a 149%.
Put £1 per point (Margin required about £500) on March 23rd 2020 in a spread betting account and you’d have had £10,000 by November 2021 or a staggering annual return of 25,000%!
Why, when the world was facing devastation from a new deadly virus that could decimate the labour force, did the stockmarket skyrocket?
Part may have been the prospect of stimulus but that still wouldn’t have helped if the workforce was going to be dead or seriously ill.
The other explanation is that if you, as a massive institutional investor, are in charge of billions of pounds or dollars you do your own research and don’t rely on the ‘experts’ to tell you whats happening.
That’s what, I believe, happened, the ‘smart’ money did their own research on Covid and quickly concluded that it was a complete load of B******* and that, with the Governments (who WERE listening to the experts) about to pump in trillions that the party was just about to begin.
Well, they seem to have manipulated the behaviour of the Chancellor – according to the propaganda world service this morning, he has to come back from America a day early; money talks.
“the ‘smart’ money did their own research on Covid”
Or, more plausibly the ‘smart’ money was actually backing / running the scam.
One of the initial aims of the Scamdemic was to siphon money from ordinary people and in to the pockets of the “elites.” A plan which appears to have worked quite well so far.
Ah yes, the often overlooked Null Hypothesis. No money in advising politicians to do nothing.
That’s true because in our ‘Democratic’ system the profit from buying a slice of political power and influence is privatised, concentrated among a relative few, whereas the cost is socialised spread across the many.
The politicians are middle-men and take their commission
So governments take note, this implies the people, not reacting, recognise propaganda when they see it, and the net result will be an increase in cynicism in government. And isn’t that precisely what we now have?
It is perhaps, but it doesn’t seem to stop people from still obeying without question.
Society isn’t a science. It is the natural emergence of spontaneous order brought about by shared language, heritage, family, culture, customs, traditions, manners and economic interdependence.
It did this very well for tens of thousands of years prior to social scientists and politicians.
All social problems are the result of interventions to shape society according to whatever fashionable ideology, crazy, half-wit notion, and the political agenda du jour.
I’ve got a problem with the term “social scientist”. Are they into psychology (https://dictionary.cambridge.org/dictionary/english/psychology ), or is it just a made up term to justify their activities, or make them look more valuable (financially, perhaps)? As we are social animals, normal “psychology” would fit the bill, maybe with an element of speciality on group behaviour.
Behaviour manipulation goes on all the time; that’s politics, managing a business etc – a list as long as your arm.
“Behavioural interventions are widely used by governments and other interventions to tweak people’s behaviour without coercing them”
Thank god it doesn’t work.. always.. It could be argued that these sorts of behavioural interventions, or ‘nudge’ represent the most insidious, damaging and evil form of coercion.
Bob holds a gun to his neighbour Gary’s head and tells him to give up his garden, as he needs the space for his children to play. The dynamics of this interaction are clear to both parties – the incentive for Gary to comply is fear of being shot. Bob wants the garden and has the leverage to get it – his actions in using the threat of force make it harder to hide from his own motivations. If he shoots Gary, he’ll have to live with the consequences – in doing so being forced to examine whether the threat or his course of action was proportionate and fair.
If Bob instead uses tools of psychological manipulation to persuade Gary that it’s in his best interests to surrender his garden, convincing him he’ll be happier without the responsibility of caring for it; that he’s selfish and bad for having such a big garden when his neighbours are suffering; that saving water will prevent climate change – he absolves himself not only of any responsibility for his bullying, but also any need to examine his own motives or reasoning – in a sense giving Gary a gun to shoot himself.
If Gary then refuses to comply with this manipulation, it’s far easier for Bob to escalate to, and justify violence or intimidation to get what he wants. Because he’s already placed the burden of responsibility for compliance on Gary’s shoulders, he has no reason to question the morality or sanity of his own demands, and so finds it easy to convince himself they’re reasonable in the framework of his coercive arguments. In his mind, because Gary has refused to see sense, shooting him would be a justifiable action.
If Gary does comply of his own volition, there’s no reason for Bob to examine whether he acted morally in obtaining the garden, and hence no reason for him not to go on to demand Gary’s lawnmower, then his house too.
It’s bizarre that the use of this sort of psychological manipulation forms the backbone of the most murderous forms of totalitarianism, and has been well understood for centuries, yet is still accepted without question; dressed up in sparkly clothes and presented as something beneficial to society. Broadcasters in the UK (looking at you, Comcast-owned Sky and the BBC) have boasted of enthusiastically adopting it as official policy.
Orwell had a great handle on it – it’s why the Party requires Winston Smith not only to denounce himself as a traitor, but wholeheartedly believe his denunciation before execution. Orwell was only saying what he saw – it was also what the Russian Communist party demanded before dispatching anyone deemed ‘problematic’ – if only for being part of the required quota of traitors on that day.
C.S. Lewis also understood it well:
“Of all tyrannies, a tyranny sincerely exercises for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron’s cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience. They may be more likely to go to Heaven yet at the same time likelier to make a Hell of earth. This very kindness stings with intolerable insult. to be “cured” against one’s will and cured of states which we may not regard as disease is to be put on a level of those who have not yet reached the age of reason or those who never will; to be classed with infants, imbeciles and domestic animals.”
Where does the quote from Lewis come from? I’m curious as to who he was addressing his remarks to at the time.
It’s from God in the Dock: Essays on Theology (Making of Modern Theology).
Maybe they are no good at prediction because social science is not a science.
Excellent article, especially enjoyed ‘Remarkably, he found that in all three cases the null model performed better than the ‘experts’.’
The reality is that genuine science deals with the observation, understanding and manipulation of purely physical substances and processes (atoms, molecules, sound waves, gravity etc) whose behaviour is fixed and therefore predictable / repeatable.
Above and beyond our physical bodies (which certainly can be approached scientifically via eg medical pathology) human beings possess free will (via a non-material and eternal soul) therefore our thoughts and behaviour are both non pre-determined and non predictable.
Which is a very long winded way of saying that all ‘social science’ is pseudo science (as a For a fist full of roubles already pointed out);
And because of its fundamentally deceptive, statistical and dehumanising approach a very dangerous type at that (as a glance at its history in practice will immediately reveal).
“Social Scientist” – the ultimate oxymoron.