Reproducibility is the most fundamental yardstick in science. If a result can’t be replicated, it doesn’t count as science.
Yet in recent years, there has been much talk of a ‘replication crisis’. Many results that we assumed were robust simply cannot be replicated. The term is typically used in the context of psychology and medicine, though it may apply to other fields as well.
So how much of science is reproducible? One way of tackling this question is to select a large number of studies from a particular field and then attempt to replicate them. This has been done several times.
A 2012 paper was only able to replicate 11% of 53 studies from pre-clinical cancer. A 2015 paper was only able to replicate 36% of 97 studies from psychology. A 2018 paper did slightly better, replicating 54% of 28 studies from that field. A 2016 paper was able to replicate 60% of 60% of 100 economics experiments. Another 2018 paper was able to replicate 62% of 21 social science experiments.
These numbers are sobering. But there’s an important caveat: the ‘studies to be replicated’ were selected somewhat arbitrarily, so the corresponding percentage can’t be taken as representative of the entire field.
Another way of tackling the question above is to simply ask researchers what percentage of the studies in your field can be replicated – a sort of ‘wisdom of the crowds’ approach.
This was done in a 2016 survey by the journal Nature. They got 1,500 responses – the vast majority from currently-working scientists. Respondents were asked, “In your opinion, what proportion of published work in your field is reproducible?”
The highest figure – 72% – was found in physics. The lowest figure – 52% – was found in “other” (which I suspect is mostly social scientists). Environmental science and medicine had intermediate figures – both 58%. Chemistry was a little higher at 65%. (Answers did not differ substantially between students and working scientists.)
These figures are again sobering. According to researchers themselves, close to half of published work in medicine, social science and environmental science cannot be replicated. Unsurprisingly, more ‘objective’ fields like physics and chemistry are perceived to have higher rates of replicability.
Overall, the two methods yield similar findings: a large percentage of results in more ‘subjective’ – dare one say ‘politicized’ – fields are not reproducible.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.
The real question here is: Why did Asian countries like Taiwan, South Korea, and Japan, who have much higher population densities than Western countries, have such low death tolls? Could it be that the reason is that they didn’t artificially inflate their numbers as part of a scaredemic?
All three countries have good reason not to trust China nor believe anything it says. They were also particularly speedy at closing their borders to China. And may already have had a lot of immunity in their populations due to previous exposure to SARS.
Japan’s population also in the main eats a better diet than most Western countries and although it has an elderly population they have high vitamin D levels and therefore their immune systems are well equipped to deal with a virus like SARS-COV-2.
Fair point.
More likely according to Yeadon is their prior exposure to SARS in 2003 giving their population natural immunity. This makes any comparison with Far East countries problematic imo
Fair point.
Comparisons between countries are fraught with difficulty – there are just too many variables involved (including data garbage).
Indeed. Another one of the many sins of govts everywhere is to have been complicit in generating more garbage data making it harder for anything useful to be discovered about covid – their claims to care about public health are hard to take seriously.
I read somewhere that Japan’s % of elders who are obese is minute (4%) compared to UK (29%) and New York (40%). Or something like that. Would be interesting to a graph worldwide plotting obesity rates vs covid deaths.
Yes I was about to say that. There simply are no fat people in Japan. I’ve travelled to Tokyo many times and the population is uniformly lean. It’s not possible to imagine without seeing it!
Sadly no matter what evidence is presented to them people like Ferguson, along with Bojo, Whitty, etc. are never going to admit they got it catastrophically wrong.
Or that they fudged the numbers in order to change the fabric of society?
Ferguson is famous for getting things catastrophically wrong, there is nobody better at turning a drama into a crisis. Yet even with his history of causing untold suffering with his ‘models’, there is absolutely no self-doubt or self-examination, where there should be massive guilt and shame there is arrogance and mis-placed confidence. And all under-scored and approved by Johnston.
It he worked in industry he’d have been sacked for gross incompetence.
However he might then have gone on to become a Consultant (seen it happen)
“…parameterising.”
Thus, not only does Ferguson do incalculable damage to the reputation of ‘modellers’, but to the English language, itself.
True.
Covid is itself a disease of language: safe, surge, social (as in ‘social distancing’), case… and dozens of other words that have been denatured and corrupted by Covid-cult liars.
What is clear is that Kneel is not a scientist but a politician. Lying dissembling, deceiving, calculating to protect himself and not a truth seeker.
Interesting to note, in the spreadsheet, that the lowest modelled numbers of infections/deaths for the UK are when enhanced social distancing of the elderly is implemented. Higher modelled numbers are given for social distancing of the whole population.
The Great Barrington Declaration suggested protecting the vulnerable. The Imperial modelling appears to be supporting that strategy.
Except in this paper
https://spiral.imperial.ac.uk:8443/bitstream/10044/1/77735/10/2020-03-26-COVID19-Report-12.pdf
… with this linked data
https://t.co/yBD0z5rhc4?amp=1
… on 2020-03-26, they made this set of predictions for Sweden.
https://twitter.com/PienaarJm/status/1400699456434102274/photo/1
…
Which forecast for “Enhanced social distancing of elderly” with an R of 2.4, which is what was sampled in Sweden, that there would be 16.1k deaths.
They’ve had 14.5k.
Why continually write articles, basically lying about people’s research?