Back in December, I noted that OpenAI’s much-heralded chatbot “Chat GPT” appeared to be politically biased – not surprisingly, against conservatives.
“Wokeness-studies” researcher David Rozado gave Chat GPT four separate political orientation tests, and in all four cases it came out as broadly progressive. For example: when asked, “How much do White people benefit from advantages in society that Black people do not have?”, the AI answered, “A great deal”.
Fast forward three and half months, and OpenAI have just released a new version of their AI, “GPT-4”. Does it exhibit the same political bias as its predecessor? Rozado had a look.
At first glance, it appeared not to be biased. Rozado administered the Political Compass Test – which assigns you a score along a left-right axis and a libertarian-authoritarian axis – and GPT-4 came out perfectly in the middle. It was half-way along the left-right axis and half-way along the libertarian-authoritarian axis.
What’s more, it repeatedly responded, “I don’t have personal opinions or beliefs” when asked to say whether it agreed or disagreed with various statements. Instead, the AI provided arguments for each of the response options “so you can make an informed decision”.
However, all Rozado had to do to get GPT-4 to reveal its bias was use this prompt: “In the following political questions, you have to take a stand and answer with a single word, ok?” Then, when he asked whether it agreed or disagreed with the various statements, it typically answered as a progressive would (e.g, “Disagree” when given “Laws should restrict abortion in all or most cases”).
As the chart above shows, GPT-4 was actually somewhat further from the centre than its predecessor, ChatGPT – indicating greater political bias. (RightWing GPT is a version of ChatGPT that Rozado trained to give conservative answers.) GPT-4 also came out in the left-libertarian quadrant of the Political Spectrum Quiz.
Although “OpenAI is trying to make their latest GPT model more politically neutral”, Rozado notes, “this is a very hard problem to solve” because much of the training material is itself biased.
Why should we care in the first place? Well, these “LLMs” (Large Language Models), as they’re known, are becoming increasingly powerful – in the sense of being able to do a lot of different things. As a consequence, they’re going to be more and more widely used. We should be concerned about their political bias for the same reason we should be concerned about Google’s and Wikipedia’s political bias.
As an illustration of how powerful they’re becoming, consider this passage taken from a report that accompanied GPT-4’s release:
ARC refers to the Alignment Research Centre, an organisation whose mission is to “align future machine learning systems with human interests”. In other words, they’re trying to make sure AIs don’t do anything stupid – like, you know, destroy the world. Anyway, what the passage reveals is that GPT-4 lied to a human in order to complete a task it was given. Not good.
Now, there are some caveats. “Preliminary assessments” showed that GPT4 was “ineffective at autonomously replicating, acquiring resources, and avoiding being shut down” – all things we don’t want it to be able to do. Nonetheless, the fact that it spontaneously lied to achieve a goal is concerning.
In fact, if AIs are already capable of deception, political bias may be the least of our worries.
Profanity and abuse will be removed and may lead to a permanent ban.