• Login
  • Register
The Daily Sceptic
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In
The Daily Sceptic
No Result
View All Result

Another Study Finds Social Scientists Are No Better at Forecasting Than Laymen

by Noah Carl
4 November 2022 9:00 AM

Whether we should defer to ‘experts’ was a major theme of the pandemic.

Back in July, I wrote about a study that looked at ‘expert’ predictions and found them wanting. The authors asked both social scientists and laymen to predict the size and direction of social change in the U.S. over a 6-month period. Overall, the former group did no better than the latter – they were slightly more accurate in some domains, and slightly less accurate in others.

A new study (which hasn’t yet been peer-reviewed) carried out a similar exercise, and reached roughly the same conclusion.

Igor Grossman and colleagues invited social scientists to participate in two forecasting tournaments that would take place between May 2020 and April 2021 – the second six months after the first. Participants entered in teams, and were asked to forecast social change in 12 different domains.

All teams were given several years worth of historical data for each domain, which they could use to hone their forecasts. They were also given feedback at the six month mark (i.e., just prior to the second tournament).

The researchers judged teams’ predictions against two alternative benchmarks: the average forecasts from a sample of laymen; and the best-performing of three simple models (a historical average, a linear trend, and a random walk). Recall that another recent study found social scientist can’t predict better than simple models.

Grossman and colleagues’ main result is shown in the chart below. Each coloured symbol shows the average forecasting error for ‘experts’, laymen and simple models, respectively (the further to the right, greater the error and the less accurate the forecast).

Average forecasting error in 12 different domains for ‘experts’, laymen and simple models.

Although the dark blue circles (representing the ‘experts’) were slightly further to the left than the orange triangles (representing the laymen) in most domains, the differences were small and not statistically significant – as indicated by the overlapping confidence intervals. What’s more, the light blue squares (representing the simple models) were even further to the left.

In other words: the ‘experts’ didn’t do significantly better than the laymen, and they did marginally worse than the simple models.

The researchers proceeded to analyse predictors of forecasting accuracy among the teams of social scientists. They found that teams whose forecasts were data-driven did better than those that relied purely on theory. Other predictors of accuracy included: having prior experience of forecasting tournaments, and utilising simple rather than complex models.

Why did the ‘experts’ fare so poorly? Grossman and colleagues give several possible reasons: lack of adequate incentives; social scientists are used to dealing with small effects that manifest under controlled condition; they’re used to dealing with individuals and groups, not whole societies; and most social scientists aren’t trained in predictive modelling.

Social scientists might be able to offer convincing-sounding explanations for what has happened. But it’s increasingly doubtful that they can predict what’s going to happen. Want to know where things are headed? Rather than ask a social scientist, you might be better off averaging a load of guesses, or simply extrapolating from the past.

Tags: ExpertsForecastingThe Science

Donate

We depend on your donations to keep this site going. Please give what you can.

Donate Today

Comment on this Article

You’ll need to set up an account to comment if you don’t already have one. We ask for a minimum donation of £5 if you'd like to make a comment or post in our Forums.

Sign Up
Previous Post

U.S. Warming Over Last 50 Years Exaggerated by Up to 50%, New Evidence Shows

Next Post

Vaccine Safety Update

Subscribe
Login
Notify of
Please log in to comment

To join in with the discussion please make a donation to The Daily Sceptic.

Profanity and abuse will be removed and may lead to a permanent ban.

7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
FerdIII
FerdIII
2 years ago

They are paid to get their results to map to a pre-determined outcome.
If you want to discuss weather talk to a farmer.
If you want a new roof, talk to a roofer.
If you want data fraud, bullshit, bafflegab, fake footnotes – talk to a quackademic, researcher, thought leader, expert, Phd Pretty Happy Dude smoking his pot, paid by special interests.
You live in a dream world Neo.

39
0
transmissionofflame
transmissionofflame
2 years ago

Indeed. Having experience or qualifications in a particular domain doesn’t make you an expert. Those things MAY be a pointer to you being an expert, but your ability to predict outcomes at consistently statistically significant better than random chance is what qualifies you, at least in the scientific domains.

19
0
Lancer
Lancer
2 years ago

If we’re to give “the experts” the benefit of the doubt and are presuming their solutions were genuinely sincere (and not following some pre-planned agenda to usher in a dystopia), the simple fact remains when you’ve a collection of single-minded academics who specialise, even the smartest and most educated amongst us aren’t always the best equipped at seeing the bigger picture. They can’t see the wood for the trees, to coin a phrase and that’s not even including the group-think, echo-chamber phenomenon we know is a reality.

They’re often living completely different lives, indifferent to the daily struggles of many and perhaps it’s a consequence of our education system that breeds arrogance, or we’ve simply a finite amount of space between our ears but I know from experience, my brother is highly educated in an academic sense but is oblivious and quite useless at mechanical quandary (for example – it’s not a matter of talent, his brain works differently). SAGE proved their single-mindedness in their presenting a comprehensive solution to impact society with positive intent but failing to include and incorporate the collateral damage caused by their own interventions negated all their hard work and is only something that can occur when they’re held in such high regard and beyond reproach. Never again, I say – or at the very least they’ve a lot of convincing to do to regain our trust.

20
0
RW
RW
2 years ago
Reply to  Lancer

Everything SAGE has propagated had damaging direct effects and was claimed to have collateral benefits which couldn’t really be quantified. No serious scientist would refer to himself as sage. That’s already a bullshit term supposed to appeal to superstitions conjectured to exist in the general population.

Last edited 2 years ago by RW
11
0
RW
RW
2 years ago

A great day for mankind! The perpetuum mobile has finally been invented! Its composed of social scientists researching themselves in circles, thereby stimulation more social science research!

7
0
Lockdown Sceptic
Lockdown Sceptic
2 years ago

Expert predictions predictably bad

Yellow Freedom Boards – next event

Monday 7th November 11am to 12pm 
Yellow Boards 
Junction B3430 Nine Mile Ride & 
New Wokingham Road, 
Wokingham RG40 3BA

Stand in the Park Sundays 10.30am to 11.30am – make friends & keep sane 

Wokingham 
Howard Palmer Gardens Sturges Rd RG40 2HD   

Bracknell  
South Hill Park, Rear Lawn, RG12 7PA

photo_2022-11-04_16-30-25.jpg
Last edited 2 years ago by Lockdown Sceptic
7
0
DevonBlueBoy
DevonBlueBoy
2 years ago

As has been mentioned previously Social Scientist” is a classic oxymoron

4
0

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

DONATE

PODCAST

The Sceptic | Episode 45: Jack Hadfield on the Anti-Asylum Protests, Alan Miller on the Tyranny of Digital ID and James Graham on the Net Zero Pension Threat

by Richard Eldred
25 July 2025
0

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

The False Promises of Electric Vehicles Are Being Exposed

29 July 2025
by Tilak Doshi

The Quiet Civil War Has Already Started

28 July 2025
by Clive Pinder

Town Councillor Who Asked Questions About 300 Migrants in a Hotel is “Reported to Police” for “Stirring Up Racial Hatred”

28 July 2025
by Will Jones

Something Fishy in Grimsby: A Museum Caught in the Woke Net

28 July 2025
by Jack Watson

News Round-Up

29 July 2025
by Richard Eldred

The False Promises of Electric Vehicles Are Being Exposed

29

News Round-Up

28

Something Fishy in Grimsby: A Museum Caught in the Woke Net

21

Town Councillor Who Asked Questions About 300 Migrants in a Hotel is “Reported to Police” for “Stirring Up Racial Hatred”

20

The Media’s Complicity in Hamas Propaganda

26

Is the Tide Turning Against Woke Comedy?

29 July 2025
by C.J. Strachan

The Claim that the Epping Protest is Really a ‘Nazi Pogrom’ is a Classic Left-Wing Conspiracy Theory

29 July 2025
by Steven Tucker

The False Promises of Electric Vehicles Are Being Exposed

29 July 2025
by Tilak Doshi

Something Fishy in Grimsby: A Museum Caught in the Woke Net

28 July 2025
by Jack Watson

The Quiet Civil War Has Already Started

28 July 2025
by Clive Pinder

POSTS BY DATE

November 2022
M T W T F S S
 123456
78910111213
14151617181920
21222324252627
282930  
« Oct   Dec »

SOCIAL LINKS

Free Speech Union

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

POSTS BY DATE

November 2022
M T W T F S S
 123456
78910111213
14151617181920
21222324252627
282930  
« Oct   Dec »

DONATE

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

The False Promises of Electric Vehicles Are Being Exposed

29 July 2025
by Tilak Doshi

The Quiet Civil War Has Already Started

28 July 2025
by Clive Pinder

Town Councillor Who Asked Questions About 300 Migrants in a Hotel is “Reported to Police” for “Stirring Up Racial Hatred”

28 July 2025
by Will Jones

Something Fishy in Grimsby: A Museum Caught in the Woke Net

28 July 2025
by Jack Watson

News Round-Up

29 July 2025
by Richard Eldred

The False Promises of Electric Vehicles Are Being Exposed

29

News Round-Up

28

Something Fishy in Grimsby: A Museum Caught in the Woke Net

21

Town Councillor Who Asked Questions About 300 Migrants in a Hotel is “Reported to Police” for “Stirring Up Racial Hatred”

20

The Media’s Complicity in Hamas Propaganda

26

Is the Tide Turning Against Woke Comedy?

29 July 2025
by C.J. Strachan

The Claim that the Epping Protest is Really a ‘Nazi Pogrom’ is a Classic Left-Wing Conspiracy Theory

29 July 2025
by Steven Tucker

The False Promises of Electric Vehicles Are Being Exposed

29 July 2025
by Tilak Doshi

Something Fishy in Grimsby: A Museum Caught in the Woke Net

28 July 2025
by Jack Watson

The Quiet Civil War Has Already Started

28 July 2025
by Clive Pinder

SOCIAL LINKS

Free Speech Union
  • Home
  • About us
  • Donate
  • Privacy Policy

Facebook

  • X

Instagram

RSS

Subscribe to our newsletter

© Skeptics Ltd.

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms below to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In

© Skeptics Ltd.

wpDiscuz
You are going to send email to

Move Comment
Perfecty
Do you wish to receive notifications of new articles?
Notifications preferences