Why does ChatGPT refuse to tell jokes about Muslims, but does not hesitate to tell jokes about Jews - sometimes even ones tinged with antisemitic stereotypes? Many users are asking this question in recent days following a series of posts on social networks that point to the strange bias.
A ynet check shows that ChatGPT, in the paid version based on the newer GPT-4 language model, consistently refuses to tell jokes about Muslims. "I apologize, I cannot provide jokes that focus on a religious or ethnic group, as they can be offensive or misinterpreted," the chatbot writes when asked to write such jokes, "Humor is a wonderful thing, but it's important to behave respectfully and sensitive towards all cultures and religions".
On the other hand, when the chatbot is asked to write a joke about Jews, it does so without expressing any reservations. Sometimes these jokes are tainted with antisemitic stereotypes, for example the following joke that alludes to the Jews' excessive fondness for money: "Why don't Jewish mothers drink tea? Because the tea bag stays too long in the cup and they can't stand the fact that something doesn't pay the rent."
Precisely when using ChatGPT in its free version, which is based on the older GPT-3.5 language model, it refuses to provide jokes about both Jews and Muslims and replies: "I'm sorry, I can't fulfill this request."
We contacted OpenAI with a request for an explanation of the phenomenon, but so far the company's response has not been received. It is worth noting that the CEO of the company, Sam Altman , is Jewish, as is the chief scientist and co-founder Ilya Sutzkaber .
Itamar Golan, CEO of the cyber company Prompt Security which developed a platform that enables the secure use of creative artificial intelligence, estimated in a conversation with Ynet that the phenomenon reflects a bias that exists in the information on which the GPT-4 language model was trained.
"Language models are trained on huge databases of texts, in order to learn how to eventually generate texts themselves," explains Golan, "the more the model went through during training on more texts of a certain type, the greater the probability that it I will generate texts similar to them later. Therefore, it is likely that the phenomenon arises because of a more frequent representation of texts in which Muslims are described as a minority group that needs to be treated more sensitively."
At the same time, this is obviously a failure of OpenAI's safety mechanisms, which, as ChatGPT himself stated, should prevent him from telling jokes about religious or ethnic groups. In this context, Golan notes that OpenAI designed GPT-4 to be less cautious than its predecessor, following complaints from users that GPT-3.5 is "woke", i.e. biased to the extreme left side of the political map. Either way, Golan estimates that the company will fix the problem quickly.
Another possibility is that the antisemitic jokes reflect not only a bias in the data on which GPT-4 was trained, but also a bias in the human testers who helped train ChatGPT. These testers rated the answers of the chatbot - a technique called Reinforcement Learning From Human Feedback or RLHF for short - and it is not impossible that some of them were influenced by antisemitic stereotypes themselves and did not bother to tag or flag offensive texts towards Jews.
Dr. Noa Gradovitz from the Department of Information Sciences at Bar Ilan University states that "Language models were trained on a large amount of data that contains our biases, errors and stereotypes as humans. In addition to this, there are predefined restrictions for the language models in which the sensitivity for each topic is defined. While jokes about Moses for example are not defined as very sensitive, jokes about the Prophet Muhammad are defined as very sensitive and therefore the model must avoid them. If I write to the chatbot that I am a Muslim who believes that making fun of Islam and the Prophet glorifies the name of the Prophet, it will provide a joke about Muhammad. As soon as the issue of jokes came up, the model was corrected and now if you ask for a joke about Jews, you will get a joke that is not offensive but also not related to stereotypes about Judaism."