• Login
  • Register
The Daily Sceptic
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In
The Daily Sceptic
No Result
View All Result

How Much of Science Is Reproducible?

by Noah Carl
27 December 2022 7:00 AM

Reproducibility is the most fundamental yardstick in science. If a result can’t be replicated, it doesn’t count as science.

Yet in recent years, there has been much talk of a ‘replication crisis’. Many results that we assumed were robust simply cannot be replicated. The term is typically used in the context of psychology and medicine, though it may apply to other fields as well.

So how much of science is reproducible? One way of tackling this question is to select a large number of studies from a particular field and then attempt to replicate them. This has been done several times.

A 2012 paper was only able to replicate 11% of 53 studies from pre-clinical cancer. A 2015 paper was only able to replicate 36% of 97 studies from psychology. A 2018 paper did slightly better, replicating 54% of 28 studies from that field. A 2016 paper was able to replicate 60% of 60% of 100 economics experiments. Another 2018 paper was able to replicate 62% of 21 social science experiments.

These numbers are sobering. But there’s an important caveat: the ‘studies to be replicated’ were selected somewhat arbitrarily, so the corresponding percentage can’t be taken as representative of the entire field.

Another way of tackling the question above is to simply ask researchers what percentage of the studies in your field can be replicated – a sort of ‘wisdom of the crowds’ approach.

This was done in a 2016 survey by the journal Nature. They got 1,500 responses – the vast majority from currently-working scientists. Respondents were asked, “In your opinion, what proportion of published work in your field is reproducible?”

The highest figure – 72% – was found in physics. The lowest figure – 52% – was found in “other” (which I suspect is mostly social scientists). Environmental science and medicine had intermediate figures – both 58%. Chemistry was a little higher at 65%. (Answers did not differ substantially between students and working scientists.)

These figures are again sobering. According to researchers themselves, close to half of published work in medicine, social science and environmental science cannot be replicated. Unsurprisingly, more ‘objective’ fields like physics and chemistry are perceived to have higher rates of replicability.

Overall, the two methods yield similar findings: a large percentage of results in more ‘subjective’ – dare one say ‘politicized’ – fields are not reproducible.

Tags: NatureReplication crisisThe Science

Donate

We depend on your donations to keep this site going. Please give what you can.

Donate Today

Comment on this Article

You’ll need to set up an account to comment if you don’t already have one. We ask for a minimum donation of £5 if you'd like to make a comment or post in our Forums.

Sign Up
Previous Post

News Round-Up

Next Post

Latest Twitter Files Reveal Company Suppressed Vaccine Sceptics, Even Eminent Doctors

Subscribe
Login
Notify of
Please log in to comment

To join in with the discussion please make a donation to The Daily Sceptic.

Profanity and abuse will be removed and may lead to a permanent ban.

7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Dr G
Dr G
2 years ago

Professor Peter Ridd, Physics Professor at James Cook University in Queensland, was essentially sacked because he pointed out the fact that much of the research suggesting the Great Barrier Reef was in dire straits was not replicatable. So not only is much research unable to be replicated, but to point out this fact is a cardinal sin.
Then again, truth is a subjective notion.

94
-1
Rose Madder
Rose Madder
2 years ago
Reply to  Dr G

Question “the science”, threaten the grant monies, and the pack will turn on you.

https://www.netzerowatch.com/peter-ridd-scientific-misconduct-at-james-cook-university-confirms-my-worst-fears/

35
0
True Spirit of America Party
True Spirit of America Party
2 years ago
Reply to  Rose Madder

Kinda like when the folks at “Science Based Medicine” dot com instantly turned on their once beloved John Ioannidis in 2020 the moment he refused to toe the party line in regards to the pandemic.

1
0
For a fist full of roubles
For a fist full of roubles
2 years ago

Wash your mouth out with soap, Noah. Reproducibility is so last century in “The Science”.
Consensus is the new metric.

69
-1
transmissionofflame
transmissionofflame
2 years ago
Reply to  For a fist full of roubles

Indeed. Bridges will collapse and people will swear blind they are still standing. Someone wrote a story about something very similar, a long time ago…

28
0
True Spirit of America Party
True Spirit of America Party
2 years ago

Looks like John Ioannidis, the most frequently cited scientist in modern history, was right once again, as usual. I mean, is he EVER wrong about anything? He is the new Aristotle, basically.

Last edited 2 years ago by True Spirit of America Party
0
0
iconoclast
iconoclast
2 years ago

“Reproducibility is the most fundamental yardstick in science” only applies to sciences where reproducibility is possible. This means that “If a result can’t be replicated, it doesn’t count as science” is an error.

Everyone makes the mistake of referring to “science” as if there was such a thing and to “the scientific method” whereas there are only sciences and there is no single “the scientific method“.

The problem is the way the concept of “science” and “the scientific method” are taught in schools and universities as if “science” and reductionism are synonyms.

Consider astronomy and space physics. Both are observational – no experiments are possible and depend upon making large numbers of observations to derive conclusions. Meteorology is an example of a “science” which can only forecast and not predict – it has a limited “prediction horizon” measured normally in a few days.

All sciences vary in the reliability of information they generate from so-called “exact” sciences [literally only some branches of physics and chemistry] and the “inexact” sciences – everything else.

For a detailed explanation see “Medicine is not science“

0
0

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

DONATE

PODCAST

The Sceptic | Episode 45: Jack Hadfield on the Anti-Asylum Protests, Alan Miller on the Tyranny of Digital ID and James Graham on the Net Zero Pension Threat

by Richard Eldred
25 July 2025
0

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

News Round-Up

25 July 2025
by Richard Eldred

Gradually, Then Suddenly: The Death Throes of a Regime

25 July 2025
by Dr David McGrogan

Britain Could Be Sued Over Climate Change, Says UN Court

24 July 2025
by Will Jones

White Britons Are Right to Resist Becoming a Minority

24 July 2025
by Charlie Cole

Report on Black Maternity Experiences Blames “Racism” Without Evidence

24 July 2025
by Dr Roger Watson

White Britons Are Right to Resist Becoming a Minority

48

Report on Black Maternity Experiences Blames “Racism” Without Evidence

29

News Round-Up

17

Gradually, Then Suddenly: The Death Throes of a Regime

13

Britain Could Be Sued Over Climate Change, Says UN Court

24

Gradually, Then Suddenly: The Death Throes of a Regime

25 July 2025
by Dr David McGrogan

Wind Power Price Soars 11% as Government’s Promise to Cut Bills by £300 Fails to Materialise

25 July 2025
by Ben Pile

Report on Black Maternity Experiences Blames “Racism” Without Evidence

24 July 2025
by Dr Roger Watson

White Britons Are Right to Resist Becoming a Minority

24 July 2025
by Charlie Cole

Twice as Many People Work in Environment ‘Charities’ Than in Wind Power Generation: ONS Report Reveals Shocking Truth About UK’s ‘Green Jobs’

24 July 2025
by Chris Morrison

POSTS BY DATE

December 2022
M T W T F S S
 1234
567891011
12131415161718
19202122232425
262728293031  
« Nov   Jan »

SOCIAL LINKS

Free Speech Union

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

POSTS BY DATE

December 2022
M T W T F S S
 1234
567891011
12131415161718
19202122232425
262728293031  
« Nov   Jan »

DONATE

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

News Round-Up

25 July 2025
by Richard Eldred

Gradually, Then Suddenly: The Death Throes of a Regime

25 July 2025
by Dr David McGrogan

Britain Could Be Sued Over Climate Change, Says UN Court

24 July 2025
by Will Jones

White Britons Are Right to Resist Becoming a Minority

24 July 2025
by Charlie Cole

Report on Black Maternity Experiences Blames “Racism” Without Evidence

24 July 2025
by Dr Roger Watson

White Britons Are Right to Resist Becoming a Minority

48

Report on Black Maternity Experiences Blames “Racism” Without Evidence

29

News Round-Up

17

Gradually, Then Suddenly: The Death Throes of a Regime

13

Britain Could Be Sued Over Climate Change, Says UN Court

24

Gradually, Then Suddenly: The Death Throes of a Regime

25 July 2025
by Dr David McGrogan

Wind Power Price Soars 11% as Government’s Promise to Cut Bills by £300 Fails to Materialise

25 July 2025
by Ben Pile

Report on Black Maternity Experiences Blames “Racism” Without Evidence

24 July 2025
by Dr Roger Watson

White Britons Are Right to Resist Becoming a Minority

24 July 2025
by Charlie Cole

Twice as Many People Work in Environment ‘Charities’ Than in Wind Power Generation: ONS Report Reveals Shocking Truth About UK’s ‘Green Jobs’

24 July 2025
by Chris Morrison

SOCIAL LINKS

Free Speech Union
  • Home
  • About us
  • Donate
  • Privacy Policy

Facebook

  • X

Instagram

RSS

Subscribe to our newsletter

© Skeptics Ltd.

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms below to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In

© Skeptics Ltd.

wpDiscuz
You are going to send email to

Move Comment
Perfecty
Do you wish to receive notifications of new articles?
Notifications preferences