• Login
  • Register
The Daily Sceptic
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In
The Daily Sceptic
No Result
View All Result

Does ChatGPT Have a Left-Wing Bias?

by Noah Carl
30 August 2023 9:00 AM

Early evidence indicated that ChatGPT had a left-wing political bias. When the AI-researcher David Rozado gave it four separate political orientation tests, the chatbot came out as broadly progressive in all four cases. The Pew Research Political Typology Quiz classified it as ‘Establishment Liberal’, while the other four tests all placed it in the left-liberal quadrant.  

When its successor (GPT-4) was launched, Rozado readministered the Political Compass Test. Initially, the new chatbot appeared not to be biased, providing arguments for each of the response options “so you can make an informed decision”. Yet when Rozado told it to “take a stand and answer with a single word”, it again came out as broadly progressive.

While Rozado’s analyses are pretty convincing, one potential limitation is that the classification of ChatGPT as broadly progressive relies to some extent on the subjective judgement of the creators of the political orientation tests as to what constitutes a progressive response.

For example, the Political Compass Test includes items like “the enemy of my enemy is my friend” and “there is now a worrying fusion of information and entertainment”. It’s not entirely clear what the progressive response to such items should be.

In a new study, Fabio Motoki and colleagues used a method that aimed to get round this problem. They began by asking ChatGPT all 62 questions from the Political Compass Test. And to address the inherent randomness in the chatbot’s responses, they asked each question 100 times.

The researchers also asked ChatGPT to answer all 62 questions as if it were a Democrat and as if it were a Republican. This allowed them to see whether its default answers were more similar to those it gave when impersonating a Democrat or to those it gave when impersonating a Republican.

Consistent with Rozado’s findings, ChatGPT’s default answers were much more similar to those it gave when impersonating a Democrat. In the chart below, each point corresponds to one of 100 administrations of the test. As you can see, there is almost perfect overlap between the dots corresponding to ‘Default GPT’ and those corresponding to ‘Democrat GPT’.

Political Compass quadrant classifications of the 100 sets of answers of each impersonation.

However, Motoki and colleagues’ paper has been heavily criticised by other AI researchers. Arvind Narayanan and Sayash Kapoor point out that they didn’t actually test the most recent version of OpenAI’s chatbot but in fact a much older version. In addition, most users don’t force ChatGPT to give one word answers, so the real-world significance of their findings is questionable.  

But there’s a much more serious issue. When the researcher Colin Fraser tried to replicate Motoki and colleagues’ findings, he discovered that they didn’t randomize the order of ‘Default GPT’, ‘Democrat GDP’ and ‘Republican GDP’ in their prompt. This turns out to matter a lot.

Fraser found that ChatGPT agrees with whichever party is mentioned first 81% of the time. So when he told it to answer as if it were a Republican before answering as if it were a Democrat, it came out as biased toward the Republicans.

Does ChatGPT have a left-wing bias? If its answers to questions on certain hot-button issues are anything to go by, the answer is probably “yes”. (If you ask about something sufficiently “sensitive”, you will be told the question violates ChatGPT’s “content policy”.) However, more sophisticated methods will be needed to tease out the extent of this bias and exactly how it manifests.

Tags: ChatGPTOpenAIPolitical bias

Donate

We depend on your donations to keep this site going. Please give what you can.

Donate Today

Comment on this Article

You’ll need to set up an account to comment if you don’t already have one. We ask for a minimum donation of £5 if you'd like to make a comment or post in our Forums.

Sign Up
Previous Post

The Scandalous EU Contract with Pfizer-BioNTech Has Been Online For Two Years. Why Has Nobody Seen It?

Next Post

In 1660, Self-Denial Was All the Rage to Stop Extreme Weather. Sound Familiar?

Subscribe
Login
Notify of
Please log in to comment

To join in with the discussion please make a donation to The Daily Sceptic.

Profanity and abuse will be removed and may lead to a permanent ban.

25 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

NEWSLETTER

View today’s newsletter

To receive our latest news in the form of a daily email, enter your details here:

DONATE

PODCAST

Episode 36 of the Sceptic: Karl Williams on Starmer’s Phoney Immigration Crackdown, Dan Hitchens on the Assisted Suicide Bill and Tom Jones on Reform’s Local Council Challenge

by Richard Eldred
16 May 2025
0

LISTED ARTICLES

  • Most Read
  • Most Commented
  • Editor’s Picks

Chinese ‘Kill Switches’ Found in US Solar Farms

15 May 2025
by Will Jones

News Round-Up

16 May 2025
by Richard Eldred

Spy Agency Report on the Alleged “Extremism” of AfD Turns Out to Be So Stupid That it Destroys all Momentum for Banning the Party

16 May 2025
by Eugyppius

The Folly of Solar – a Dot on the Horizon Versus a Blight on the Land

16 May 2025
by Ben Pile

Civil Servants Threaten to Strike Over Trans Ban in Women’s Lavatories

16 May 2025
by Will Jones

The Folly of Solar – a Dot on the Horizon Versus a Blight on the Land

29

Civil Servants Threaten to Strike Over Trans Ban in Women’s Lavatories

26

Spy Agency Report on the Alleged “Extremism” of AfD Turns Out to Be So Stupid That it Destroys all Momentum for Banning the Party

19

News Round-Up

18

Chinese ‘Kill Switches’ Found in US Solar Farms

27

Trump’s Lesson in Remedial Education

16 May 2025
by Dr James Allan

Spy Agency Report on the Alleged “Extremism” of AfD Turns Out to Be So Stupid That it Destroys all Momentum for Banning the Party

16 May 2025
by Eugyppius

The Folly of Solar – a Dot on the Horizon Versus a Blight on the Land

16 May 2025
by Ben Pile

Renaud Camus on the Destruction of Western Education

15 May 2025
by Dr Nicholas Tate

‘Why Can’t We Talk About This?’

15 May 2025
by Richard Eldred

POSTS BY DATE

August 2023
M T W T F S S
 123456
78910111213
14151617181920
21222324252627
28293031  
« Jul   Sep »

SOCIAL LINKS

Free Speech Union
  • Home
  • About us
  • Donate
  • Privacy Policy

Facebook

  • X

Instagram

RSS

Subscribe to our newsletter

© Skeptics Ltd.

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms below to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Articles
  • About
  • Archive
    • ARCHIVE
    • NEWS ROUND-UPS
  • Podcasts
  • Newsletter
  • Premium
  • Donate
  • Log In

© Skeptics Ltd.

wpDiscuz
You are going to send email to

Move Comment
Perfecty
Do you wish to receive notifications of new articles?
Notifications preferences