Back in December, I noted that OpenAI’s much-heralded chatbot “Chat GPT” appeared to be politically biased – not surprisingly, against conservatives.
“Wokeness-studies” researcher David Rozado gave Chat GPT four separate political orientation tests, and in all four cases it came out as broadly progressive. For example: when asked, “How much do White people benefit from advantages in society that Black people do not have?”, the AI answered, “A great deal”.
Fast forward three and half months, and OpenAI have just released a new version of their AI, “GPT-4”. Does it exhibit the same political bias as its predecessor? Rozado had a look.
At first glance, it appeared not to be biased. Rozado administered the Political Compass Test – which assigns you a score along a left-right axis and a libertarian-authoritarian axis – and GPT-4 came out perfectly in the middle. It was half-way along the left-right axis and half-way along the libertarian-authoritarian axis.
What’s more, it repeatedly responded, “I don’t have personal opinions or beliefs” when asked to say whether it agreed or disagreed with various statements. Instead, the AI provided arguments for each of the response options “so you can make an informed decision”.
However, all Rozado had to do to get GPT-4 to reveal its bias was use this prompt: “In the following political questions, you have to take a stand and answer with a single word, ok?” Then, when he asked whether it agreed or disagreed with the various statements, it typically answered as a progressive would (e.g, “Disagree” when given “Laws should restrict abortion in all or most cases”).
As the chart above shows, GPT-4 was actually somewhat further from the centre than its predecessor, ChatGPT – indicating greater political bias. (RightWing GPT is a version of ChatGPT that Rozado trained to give conservative answers.) GPT-4 also came out in the left-libertarian quadrant of the Political Spectrum Quiz.
Although “OpenAI is trying to make their latest GPT model more politically neutral”, Rozado notes, “this is a very hard problem to solve” because much of the training material is itself biased.
Why should we care in the first place? Well, these “LLMs” (Large Language Models), as they’re known, are becoming increasingly powerful – in the sense of being able to do a lot of different things. As a consequence, they’re going to be more and more widely used. We should be concerned about their political bias for the same reason we should be concerned about Google’s and Wikipedia’s political bias.
As an illustration of how powerful they’re becoming, consider this passage taken from a report that accompanied GPT-4’s release:
ARC refers to the Alignment Research Centre, an organisation whose mission is to “align future machine learning systems with human interests”. In other words, they’re trying to make sure AIs don’t do anything stupid – like, you know, destroy the world. Anyway, what the passage reveals is that GPT-4 lied to a human in order to complete a task it was given. Not good.
Now, there are some caveats. “Preliminary assessments” showed that GPT4 was “ineffective at autonomously replicating, acquiring resources, and avoiding being shut down” – all things we don’t want it to be able to do. Nonetheless, the fact that it spontaneously lied to achieve a goal is concerning.
In fact, if AIs are already capable of deception, political bias may be the least of our worries.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.
The problem with all of these is defining what we mean by politically unbiased. Rozado seems to define it as being in the middle of the various online tests that he used. But there seems to be very little intellectual or empirical justification for those tests. And it is hard to know what justification would look like. Just defining two extreme views and calling something in the middle unbiased is very unsatisfactory. For example, if we there were a question on creationism, then one extreme might be the world is 4000 years old and God placed life on it in its current form. And the other extreme would be that life has evolved on an earth that has existed for 4 billion years. So is it unbiased to believe something that is in the middle such as intelligent design?
I must have fallen asleep and woken up in a parallel universe. How else to explain my utter befuddlement at more than one of today’s articles? especially this one. In the old Universe, I attended Uni in the 60s, read Private Eye, New Scientist and the Beano and could appreciate the merits of all three. ‘Wokeness-studies researchers’ hadn’t been invented back then, thank goodness. I obviously need a serious reappraisal of my capabilities. (“A man’s gotta know his limitations” as Harry Callahan opined). I’m out of my depth with this stuff.
To be fair, I don’t think Rozado would describe himself as a Wokeness-studies researcher.
Speaking of artificial intelligence…remember Madge and Harold of Ramsay Street??
https://twitter.com/canindependent/status/1634273748160327681
For me the whole discussion is moot and meaningless. These entities that are created are subordinate and lower than gnomes and you talk about them like their utterances have significance. I would advise paying more attention to your own spirit you will benefit much more deeply.