134292
Had enough? Join the Alliance of British Drivers. Had enough? Join the Alliance of British Drivers. Had enough? Join the Alliance of British Drivers.
  • Log in
The Daily Sceptic
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Forum
  • Donate
  • Newsletter
The Daily Sceptic
No Result
View All Result

Status Bias in Peer Review

by Noah Carl
20 September 2022 7:00 AM

Almost all scientific papers go through something called ‘peer review’ before they’re published.

This usually works as follows: the paper is sent to a scientific journal; the editor of that journal selects two or three reviewers whom he or she considers experts in the relevant subject matter; each reviewer reads the paper and submits a report indicating whether it should be accepted, rejected or revised; the editor then makes a decision based the reviewers’ reports.

Most journals use ‘double blind’ reviewing, where the reviewers don’t know the identity of the authors, and the authors don’t know the identity of the reviewers. Some journals use ‘single blind’ reviewing, where the reviewers know the identity of the authors, or vice versa. And a small number of journals use ‘open peer review’, where everyone’s identity is known to everyone else.

Even when journals use double blind reviewing, it is often still possible for reviewers to work out who the authors are. The latter may have already may have presented the paper at a conference or published a ‘preprint’ online. Sometimes it is possible to work out who the authors purely from the content of the paper (e.g., because it builds on previous research they have done).

In an ideal world, reviewers’ recommendations would be based solely on objective criteria, like the suitability of the research design and the absence of errors in analysis. Yet evidence (and the personal experience of many academics) suggests this is rarely the case. Various forms of bias afflict the reviewing process.

In a new paper (which hasn’t yet been peer-reviewed!) Juergen Huber and colleagues explore one particular form of bias: ‘status bias’. Are reviewers more likely to recommend ‘revise’ or ‘accept when the authors are big names in the field than when they’re complete nobodies?

Huber and colleagues carried out a field experiment, working with the editor of a major economics journal. A paper was submitted to the journal which had two authors: one a Nobel Prize winner in economics, the other an unknown research associate. Like the journal’s editor, they were both ‘in on’ the study.

The paper was then sent out to 3,300 potential reviewers, in order to get a large enough sample size. There were three different treatments.

In treatment ‘AL’, the unknown research associate was given as the paper’s corresponding author (so this was single blind). In ‘AA’, no corresponding author was given (so this was double blind). And in ‘AH’, the Nobel Prize winner was given as the corresponding author (this was single blind again).

Huber and colleagues’ main result is shown in the chart below. Each coloured bar represents the distribution of reviewers’ recommendations in a particular treatment.

The distribution of reviewers’ recommendations in three different treatments.

As you can see, reviewers were far more likely to say the paper should be rejected when the unknown research associate was given as the corresponding author. In ‘AL’, 65% of reviewers recommended ‘reject’, compared to only 23% in ‘AH’. So you’re about one third as likely to have your paper rejected if you’re a Nobel Prize winner.

Needless to say, this isn’t how science is meant to work. Rather than judging papers on their own merits, most reviewers are content to employ heuristics like “work by a Nobel Prize winner must be good” or, indeed, “work by an unknown research associate must be bad”.

The pandemic showed what can happen when defer too much to credentialed ‘experts’. And it’s not just politicians and laypeople that fall into this traps – so do scientists themselves.

Tags: ExpertsPeer reviewThe Science

Donate

We depend on your donations to keep this site going. Please give what you can.

Donate Today

Comment on this Article

You’ll need to set up an account to comment if you don’t already have one. We ask for a minimum donation of £5 if you'd like to make a comment or post in our Forums.

Sign Up
Previous Post

News Round-Up

Next Post

The One-Sided Media Coverage of the Queen’s Death Was Painfully Reminiscent of Covid Propaganda For Many

Subscribe
Login
Notify of
Please log in to comment

Profanity and abuse will be removed and may lead to a permanent ban.

3 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

 

DONATE

PODCAST

Nick Dixon and Toby Young Talk About Toby’s Appearance on the 77th Brigade’s Watch List, the Scrubbing of the Internet After the Pfizer Sting and the Trans Insanity Unfolding in Scotland

by Will Jones
31 January 2023
0

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editors Picks

“If More People Stood Up and Said Something Then This Would All Stop”: Actor Matthew Marsden Defends Declining Covid Vaccine Despite it Derailing His Career

6 February 2023
by Will Jones

News Round-Up

7 February 2023
by Will Jones

Why I Won’t Talk to ‘Fact Checkers’ About Our Mask Study

6 February 2023
by Dr Tom Jefferson

RFK Jr Sues BBC and Big Media Companies for Hobbling Online Rivals Via ‘Trusted News Initiative’

7 February 2023
by Chris Morrison

Dr. Carl Heneghan Interviews Dr. Tom Jefferson About His Major New Study Showing Masks Have No Clear Effect

6 February 2023
by Dr Carl Heneghan and Dr Tom Jefferson

News Round-Up

25

Government Refuses to Investigate What’s Behind the Thousands of Excess Deaths

17

RFK Jr Sues BBC and Big Media Companies for Hobbling Online Rivals Via ‘Trusted News Initiative’

14

“If More People Stood Up and Said Something Then This Would All Stop”: Actor Matthew Marsden Defends Declining Covid Vaccine Despite it Derailing His Career

14

More Support For Liz Truss’s Contention That Market Turmoil Following Mini-Budget was Partly Due to Incompetence of Bank of England

10

Martin Kulldorff, Jay Bhattacharya and Colleagues Demand a Full Inquiry into America’s Disastrous Covid Response

7 February 2023
by Will Jones

Dr. Carl Heneghan Interviews Dr. Tom Jefferson About His Major New Study Showing Masks Have No Clear Effect

6 February 2023
by Dr Carl Heneghan and Dr Tom Jefferson

Why I Won’t Talk to ‘Fact Checkers’ About Our Mask Study

6 February 2023
by Dr Tom Jefferson

My New Interview With Andrew Bridgen MP

6 February 2023
by Nick Dixon

West Blocked Ukraine Peace Deal, Says Former Israeli PM

5 February 2023
by Noah Carl

POSTS BY DATE

September 2022
M T W T F S S
 1234
567891011
12131415161718
19202122232425
2627282930  
« Aug   Oct »

SOCIAL LINKS

Free Speech Union
  • Home
  • About us
  • Donate
  • Privacy Policy

Facebook

Twitter

Instagram

RSS

Subscribe to our newsletter

© Skeptics Ltd.

No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Forum
  • Donate
  • Newsletter

© Skeptics Ltd.

Welcome Back!

Login to your account below

Forgotten Password?

Create New Account!

Please note: To be able to comment on our articles you'll need to be a registered donor

Already have an account?
Please click here to login Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
wpDiscuz
You are going to send email to

Move Comment