• Login
  • Register
The Daily Sceptic
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In
The Daily Sceptic
No Result
View All Result

How Much of Science Is Reproducible?

by Noah Carl
27 December 2022 7:00 AM

Reproducibility is the most fundamental yardstick in science. If a result can’t be replicated, it doesn’t count as science.

Yet in recent years, there has been much talk of a ‘replication crisis’. Many results that we assumed were robust simply cannot be replicated. The term is typically used in the context of psychology and medicine, though it may apply to other fields as well.

So how much of science is reproducible? One way of tackling this question is to select a large number of studies from a particular field and then attempt to replicate them. This has been done several times.

A 2012 paper was only able to replicate 11% of 53 studies from pre-clinical cancer. A 2015 paper was only able to replicate 36% of 97 studies from psychology. A 2018 paper did slightly better, replicating 54% of 28 studies from that field. A 2016 paper was able to replicate 60% of 60% of 100 economics experiments. Another 2018 paper was able to replicate 62% of 21 social science experiments.

These numbers are sobering. But there’s an important caveat: the ‘studies to be replicated’ were selected somewhat arbitrarily, so the corresponding percentage can’t be taken as representative of the entire field.

Another way of tackling the question above is to simply ask researchers what percentage of the studies in your field can be replicated – a sort of ‘wisdom of the crowds’ approach.

This was done in a 2016 survey by the journal Nature. They got 1,500 responses – the vast majority from currently-working scientists. Respondents were asked, “In your opinion, what proportion of published work in your field is reproducible?”

The highest figure – 72% – was found in physics. The lowest figure – 52% – was found in “other” (which I suspect is mostly social scientists). Environmental science and medicine had intermediate figures – both 58%. Chemistry was a little higher at 65%. (Answers did not differ substantially between students and working scientists.)

These figures are again sobering. According to researchers themselves, close to half of published work in medicine, social science and environmental science cannot be replicated. Unsurprisingly, more ‘objective’ fields like physics and chemistry are perceived to have higher rates of replicability.

Overall, the two methods yield similar findings: a large percentage of results in more ‘subjective’ – dare one say ‘politicized’ – fields are not reproducible.

Tags: NatureReplication crisisThe Science

Donate

We depend on your donations to keep this site going. Please give what you can.

Donate Today

Comment on this Article

You’ll need to set up an account to comment if you don’t already have one. We ask for a minimum donation of £5 if you'd like to make a comment or post in our Forums.

Sign Up
Previous Post

News Round-Up

Next Post

Latest Twitter Files Reveal Company Suppressed Vaccine Sceptics, Even Eminent Doctors

Subscribe
Login
Notify of
Please log in to comment

To join in with the discussion please make a donation to The Daily Sceptic.

Profanity and abuse will be removed and may lead to a permanent ban.

7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Cristi.Neagu
Cristi.Neagu
4 years ago

The real question here is: Why did Asian countries like Taiwan, South Korea, and Japan, who have much higher population densities than Western countries, have such low death tolls? Could it be that the reason is that they didn’t artificially inflate their numbers as part of a scaredemic?

41
0
realarthurdent
realarthurdent
4 years ago
Reply to  Cristi.Neagu

All three countries have good reason not to trust China nor believe anything it says. They were also particularly speedy at closing their borders to China. And may already have had a lot of immunity in their populations due to previous exposure to SARS.

Japan’s population also in the main eats a better diet than most Western countries and although it has an elderly population they have high vitamin D levels and therefore their immune systems are well equipped to deal with a virus like SARS-COV-2.

33
0
Cristi.Neagu
Cristi.Neagu
4 years ago
Reply to  realarthurdent

Fair point.

5
0
WorriedCitizen
WorriedCitizen
4 years ago
Reply to  Cristi.Neagu

More likely according to Yeadon is their prior exposure to SARS in 2003 giving their population natural immunity. This makes any comparison with Far East countries problematic imo

Last edited 4 years ago by WorriedCitizen
18
0
Cristi.Neagu
Cristi.Neagu
4 years ago
Reply to  WorriedCitizen

Fair point.

5
0
RickH
RickH
4 years ago
Reply to  Cristi.Neagu

Comparisons between countries are fraught with difficulty – there are just too many variables involved (including data garbage).

1
0
Julian
Julian
4 years ago
Reply to  RickH

Indeed. Another one of the many sins of govts everywhere is to have been complicit in generating more garbage data making it harder for anything useful to be discovered about covid – their claims to care about public health are hard to take seriously.

I read somewhere that Japan’s % of elders who are obese is minute (4%) compared to UK (29%) and New York (40%). Or something like that. Would be interesting to a graph worldwide plotting obesity rates vs covid deaths.

1
0
SweetBabyCheeses
SweetBabyCheeses
4 years ago
Reply to  Julian

Yes I was about to say that. There simply are no fat people in Japan. I’ve travelled to Tokyo many times and the population is uniformly lean. It’s not possible to imagine without seeing it!

1
0
Matt Dalby
Matt Dalby
4 years ago

Sadly no matter what evidence is presented to them people like Ferguson, along with Bojo, Whitty, etc. are never going to admit they got it catastrophically wrong.

46
-1
NonCompliant
NonCompliant
4 years ago
Reply to  Matt Dalby

Or that they fudged the numbers in order to change the fabric of society?

15
0
lorrinet
lorrinet
4 years ago
Reply to  Matt Dalby

Ferguson is famous for getting things catastrophically wrong, there is nobody better at turning a drama into a crisis. Yet even with his history of causing untold suffering with his ‘models’, there is absolutely no self-doubt or self-examination, where there should be massive guilt and shame there is arrogance and mis-placed confidence. And all under-scored and approved by Johnston.

12
0
chris c
chris c
4 years ago
Reply to  lorrinet

It he worked in industry he’d have been sacked for gross incompetence.

However he might then have gone on to become a Consultant (seen it happen)

3
0
paul smith
paul smith
4 years ago

“…parameterising.”

Thus, not only does Ferguson do incalculable damage to the reputation of ‘modellers’, but to the English language, itself.

20
0
Annie
Annie
4 years ago
Reply to  paul smith

True.
Covid is itself a disease of language: safe, surge, social (as in ‘social distancing’), case… and dozens of other words that have been denatured and corrupted by Covid-cult liars.

10
0
Niborxof
Niborxof
4 years ago

What is clear is that Kneel is not a scientist but a politician. Lying dissembling, deceiving, calculating to protect himself and not a truth seeker.

8
0
DaveL
DaveL
4 years ago

Interesting to note, in the spreadsheet, that the lowest modelled numbers of infections/deaths for the UK are when enhanced social distancing of the elderly is implemented. Higher modelled numbers are given for social distancing of the whole population.

The Great Barrington Declaration suggested protecting the vulnerable. The Imperial modelling appears to be supporting that strategy.

3
0
Simon Nicholls (sinichol)
Simon Nicholls (sinichol)
3 years ago

Except in this paper
https://spiral.imperial.ac.uk:8443/bitstream/10044/1/77735/10/2020-03-26-COVID19-Report-12.pdf
… with this linked data
https://t.co/yBD0z5rhc4?amp=1
… on 2020-03-26, they made this set of predictions for Sweden.
https://twitter.com/PienaarJm/status/1400699456434102274/photo/1
…
Which forecast for “Enhanced social distancing of elderly” with an R of 2.4, which is what was sampled in Sweden, that there would be 16.1k deaths.

They’ve had 14.5k.

Why continually write articles, basically lying about people’s research?

0
0

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

DONATE

PODCAST

Episode 36 of the Sceptic: Karl Williams on Starmer’s Phoney Immigration Crackdown, Dan Hitchens on the Assisted Suicide Bill and Tom Jones on Reform’s Local Council Challenge

by Richard Eldred
16 May 2025
0

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

News Round-Up

21 May 2025
by Richard Eldred

Is This the Experiment that Started the Pandemic?

21 May 2025
by Will Jones

Call My Bluff No. 8647

21 May 2025
by James Alexander

The 3.5% Inflation Spike is All on Reeves

21 May 2025
by Will Jones

How These NHS Apparatchiks Must Hate the Media for Shining a Light on Their Mistreatment of Christian Nurse Jennifer Melle Who Stood Up to Their Trans Dogma

20 May 2025
by Julian Mann

The National Lottery’s Woke Arts Spending

19

News Round-Up

16

The 3.5% Inflation Spike is All on Reeves

15

Is This the Experiment that Started the Pandemic?

15

Call My Bluff No. 8647

14

How the Deep State Went Viral 

21 May 2025
by Debbie Lerman

Call My Bluff No. 8647

21 May 2025
by James Alexander

Is This the Experiment that Started the Pandemic?

21 May 2025
by Will Jones

The National Lottery’s Woke Arts Spending

21 May 2025
by Charlotte Gill

How These NHS Apparatchiks Must Hate the Media for Shining a Light on Their Mistreatment of Christian Nurse Jennifer Melle Who Stood Up to Their Trans Dogma

20 May 2025
by Julian Mann

POSTS BY DATE

December 2022
M T W T F S S
 1234
567891011
12131415161718
19202122232425
262728293031  
« Nov   Jan »

SOCIAL LINKS

Free Speech Union

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

POSTS BY DATE

December 2022
M T W T F S S
 1234
567891011
12131415161718
19202122232425
262728293031  
« Nov   Jan »

DONATE

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

News Round-Up

21 May 2025
by Richard Eldred

Is This the Experiment that Started the Pandemic?

21 May 2025
by Will Jones

Call My Bluff No. 8647

21 May 2025
by James Alexander

The 3.5% Inflation Spike is All on Reeves

21 May 2025
by Will Jones

How These NHS Apparatchiks Must Hate the Media for Shining a Light on Their Mistreatment of Christian Nurse Jennifer Melle Who Stood Up to Their Trans Dogma

20 May 2025
by Julian Mann

The National Lottery’s Woke Arts Spending

19

News Round-Up

16

The 3.5% Inflation Spike is All on Reeves

15

Is This the Experiment that Started the Pandemic?

15

Call My Bluff No. 8647

14

How the Deep State Went Viral 

21 May 2025
by Debbie Lerman

Call My Bluff No. 8647

21 May 2025
by James Alexander

Is This the Experiment that Started the Pandemic?

21 May 2025
by Will Jones

The National Lottery’s Woke Arts Spending

21 May 2025
by Charlotte Gill

How These NHS Apparatchiks Must Hate the Media for Shining a Light on Their Mistreatment of Christian Nurse Jennifer Melle Who Stood Up to Their Trans Dogma

20 May 2025
by Julian Mann

SOCIAL LINKS

Free Speech Union
  • Home
  • About us
  • Donate
  • Privacy Policy

Facebook

  • X

Instagram

RSS

Subscribe to our newsletter

© Skeptics Ltd.

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms below to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In

© Skeptics Ltd.

wpDiscuz
You are going to send email to

Move Comment
Perfecty
Do you wish to receive notifications of new articles?
Notifications preferences