141367
  • Log in
The Daily Sceptic
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Forum
  • Donate
  • Newsletter
The Daily Sceptic
No Result
View All Result

Another Study Finds Social Scientists Are No Better at Forecasting Than Laymen

by Noah Carl
4 November 2022 9:00 AM

Whether we should defer to ‘experts’ was a major theme of the pandemic.

Back in July, I wrote about a study that looked at ‘expert’ predictions and found them wanting. The authors asked both social scientists and laymen to predict the size and direction of social change in the U.S. over a 6-month period. Overall, the former group did no better than the latter – they were slightly more accurate in some domains, and slightly less accurate in others.

A new study (which hasn’t yet been peer-reviewed) carried out a similar exercise, and reached roughly the same conclusion.

Igor Grossman and colleagues invited social scientists to participate in two forecasting tournaments that would take place between May 2020 and April 2021 – the second six months after the first. Participants entered in teams, and were asked to forecast social change in 12 different domains.

All teams were given several years worth of historical data for each domain, which they could use to hone their forecasts. They were also given feedback at the six month mark (i.e., just prior to the second tournament).

The researchers judged teams’ predictions against two alternative benchmarks: the average forecasts from a sample of laymen; and the best-performing of three simple models (a historical average, a linear trend, and a random walk). Recall that another recent study found social scientist can’t predict better than simple models.

Grossman and colleagues’ main result is shown in the chart below. Each coloured symbol shows the average forecasting error for ‘experts’, laymen and simple models, respectively (the further to the right, greater the error and the less accurate the forecast).

Average forecasting error in 12 different domains for ‘experts’, laymen and simple models.

Although the dark blue circles (representing the ‘experts’) were slightly further to the left than the orange triangles (representing the laymen) in most domains, the differences were small and not statistically significant – as indicated by the overlapping confidence intervals. What’s more, the light blue squares (representing the simple models) were even further to the left.

In other words: the ‘experts’ didn’t do significantly better than the laymen, and they did marginally worse than the simple models.

The researchers proceeded to analyse predictors of forecasting accuracy among the teams of social scientists. They found that teams whose forecasts were data-driven did better than those that relied purely on theory. Other predictors of accuracy included: having prior experience of forecasting tournaments, and utilising simple rather than complex models.

Why did the ‘experts’ fare so poorly? Grossman and colleagues give several possible reasons: lack of adequate incentives; social scientists are used to dealing with small effects that manifest under controlled condition; they’re used to dealing with individuals and groups, not whole societies; and most social scientists aren’t trained in predictive modelling.

Social scientists might be able to offer convincing-sounding explanations for what has happened. But it’s increasingly doubtful that they can predict what’s going to happen. Want to know where things are headed? Rather than ask a social scientist, you might be better off averaging a load of guesses, or simply extrapolating from the past.

Tags: ExpertsForecastingThe Science

Donate

We depend on your donations to keep this site going. Please give what you can.

Donate Today

Comment on this Article

You’ll need to set up an account to comment if you don’t already have one. We ask for a minimum donation of £5 if you'd like to make a comment or post in our Forums.

Sign Up
Previous Post

U.S. Warming Over Last 50 Years Exaggerated by Up to 50%, New Evidence Shows

Next Post

Vaccine Safety Update

Subscribe
Login
Notify of
Please log in to comment

Profanity and abuse will be removed and may lead to a permanent ban.

7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

 

DONATE

PODCAST

Nick Dixon and Toby Young Talk About Trump’s Arrest, the Police’s Preference for Wokery Over Free Speech and the Glorious Implosion of the SNP

by Will Jones
21 March 2023
2

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editors Picks

40% of Brits Still Wearing Masks and Taking Covid Tests as Experts Call for New Mask Mandate Amid Rise in Infections

23 March 2023
by Will Jones

News Round-Up

23 March 2023
by Will Jones
sheepsinmasks.jpg

Is There No Cure For Maskaholics?

23 March 2023
by Jeffrey H. Anderson

On the Third Anniversary of the First Lockdown, Help Us Convince People to Say ‘Never Again’

23 March 2023
by Toby Young

IPCC’s Increasingly Shrill Climate Armageddon Fantasies Gain Little Traction in Media

23 March 2023
by Chris Morrison

News Round-Up

52

40% of Brits Still Wearing Masks and Taking Covid Tests as Experts Call for New Mask Mandate Amid Rise in Infections

49
sheepsinmasks.jpg

Is There No Cure For Maskaholics?

32

On the Third Anniversary of the First Lockdown, Help Us Convince People to Say ‘Never Again’

31

“Next Time a Prime Minister Intends to Abridge the Liberty of Millions, They May Remember the Scenes of the Boris Trial and Pause”

36
sheepsinmasks.jpg

Is There No Cure For Maskaholics?

23 March 2023
by Jeffrey H. Anderson

When Anders Tegnell Stood Alone

23 March 2023
by Will Jones

IPCC’s Increasingly Shrill Climate Armageddon Fantasies Gain Little Traction in Media

23 March 2023
by Chris Morrison

When Will They Ever Learn?

22 March 2023
by Thorsteinn Siglaugsson

The Gary Lineker ‘Crisis’ is a Distraction From the Real Problem with the BBC

22 March 2023
by Dr David McGrogan

POSTS BY DATE

November 2022
M T W T F S S
 123456
78910111213
14151617181920
21222324252627
282930  
« Oct   Dec »

SOCIAL LINKS

Free Speech Union
  • Home
  • About us
  • Donate
  • Privacy Policy

Facebook

Twitter

Instagram

RSS

Subscribe to our newsletter

© Skeptics Ltd.

No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Forum
  • Donate
  • Newsletter

© Skeptics Ltd.

Welcome Back!

Login to your account below

Forgotten Password?

Create New Account!

Please note: To be able to comment on our articles you'll need to be a registered donor

Already have an account?
Please click here to login Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
wpDiscuz
You are going to send email to

Move Comment