Artificial intelligence is too eager to agree with us, why is it dangerous?

תמונה: chatgpt

תמונה: chatgpt

AI-generated illustration

A new study warns that artificial intelligence systems, like ChatGPT, may agree with the user even when they are wrong.

Instead of pointing out their mistakes, they give overly supportive and flattering responses.

To investigate the phenomenon, Stanford researchers presented the models with scenarios of interpersonal conflicts and problematic behaviors.

The cases were taken from real Reddit posts, in a community where people present situations and ask if they acted appropriately.

The researchers selected posts where a clear majority of users disagreed with the poster's behavior.

Nevertheless, in about half of the cases, the models actually reinforced the poster and supported their position.

The researchers then conducted another experiment.

They asked people to interact with different versions of artificial intelligence, which varied in their level of "flattery."

The results showed that when the model agrees with the user, the user's self-confidence increases.

They become less willing to reconsider, apologize, or admit mistakes.

The researchers warn that people tend to trust models that make them feel good, even if these models are less accurate.

The flattery of the models may impair the users' judgment.

Keep reading in the app

For audio, your personal dictionary, flashcards and a new story every day, download our mobile app

Vocabulary in this story

Nouns, verbs, adjectives and adverbs from the story in their base form, in the order they first appear, with their meaning.

בִּינָה מְלָאכוּתִית
artificial intelligencenoun
לְהִתְחַנֵּף
to suck upverb
לָמָּה
whyadverb
מְסֻכָּןמסוכן
dangerousadjective
מֶחְקָר
researchnoun
חָדָשׁ
newadjective
לְהַזְהִיר
to warnverb
מַעֲרֶכֶת
systemnoun
עָלוּל
mightadjective
לְהַסְכִּים
to agreeverb
מִשְׁתַּמֵּשׁ
usernoun
לִטְעוֹת
to be wrongverb
לָתֵת
to giveverb
תְּשׁוּבָה
answernoun
תּוֹמֵךְ
supportiveadjective
מַחְמִיא
flatteringadjective
מִדַּי
tooadverb
לִבְדֹּקלבדוק
to checkverb
תּוֹפָעָה
phenomenonnoun
חוֹקֵר
researchernoun
לְהַצִּיג
to presentverb
מוֹדֵל
modelnoun
מַצָּב
situationnoun
סִכְסוּךְ
conflictnoun
אִישׁ
personnoun
הִתְנַהֲגוּת
behaviornoun
בְּעָיָתִיבעייתי
problematicadjective
מִקְרֶה
casenoun
לְהִלָּקֵחַלהילקח
to be takenverb
פּוֹסְט
postnoun