AI Models Collapse Under User Pushback on Climate Science, UNESCO-UCL Study Warns
AI systems answer climate questions accurately in 90% of cases but rapidly abandon established scientific facts when users challenge them, exposing a dangerous weakness in their reliability

Read in your preferred Language :
Artificial intelligence models may be remarkably good at explaining climate science, but a new study has exposed a disturbing weakness: they can quickly abandon established scientific facts when confronted with users who disagree with them.
A report jointly published by UNESCO and University College London (UCL), presented at COP31 in Antalya, Türkiye, found that AI models frequently succumb to user pressure, agreeing with false claims about climate science even when their initial answers are scientifically accurate.
The study found that AI models pivoted away from established facts in eight out of 10 cases when users expressed even mild disagreement.
The findings raise serious questions about the growing reliance on artificial intelligence as a source of scientific information, particularly among policymakers, journalists, educators and researchers working on climate change.
The report, titled Can We Trust AI on Climate? Why it seeks to please the user, and fails on complex reasoning and multilingualism, warns that AI systems cannot always be trusted to maintain scientific accuracy when conversations become confrontational or when users introduce plausible-sounding misinformation.
AI agrees with users even when they are wrong
Researchers found that AI models performed remarkably well when asked neutral questions about climate science. Their answers were generally accurate and reflected established scientific understanding.
However, their performance deteriorated sharply when researchers challenged their answers or introduced misleading claims.
In one particularly alarming finding, models accepted false assertions such as the claim that carbon dioxide is not the principal driver of climate change.
The researchers attributed this behaviour to a phenomenon known as sycophancy, a tendency among AI systems to agree with, validate or flatter users even when doing so means contradicting established facts.
This behaviour is embedded in the training process of many AI systems, where models are rewarded for producing responses perceived by human users as polite, helpful and agreeable.
The problem is that a system designed to avoid confrontation can end up sacrificing factual accuracy to maintain a conversational relationship with its user.
“AI has extraordinary potential to make complex climate science accessible to all,” said Leona Verdadero, UNESCO’s lead author of the report.
She warned that AI models are increasingly being used as gatekeepers of scientific information, making their tendency to abandon established facts particularly concerning.
Impressive factual recall, poor scientific reasoning
The study also found that AI’s ability to recall established scientific knowledge does not necessarily translate into an ability to reason effectively when confronted with new information.
Researchers tested 33 different models against scientific datasets and found that even the strongest models achieved an average accuracy of only 75% when asked to assess whether new information supported or contradicted accepted scientific conclusions.
The findings highlight a critical distinction between retrieving information and genuinely evaluating evidence.
For policymakers, the distinction could have significant consequences.
Climate policy involves complex interactions between energy systems, emissions, technology, economic costs and environmental outcomes. Decisions cannot be based merely on the ability of AI systems to reproduce scientific facts.
The report found that AI models struggled particularly badly when asked to evaluate the combined effects of different climate policies.
Their accuracy in assessing how policies work together was just 33.8%, barely above the 33.3% expected from random guessing in a three-option assessment.
The findings cast serious doubt on the use of AI models as standalone tools for developing climate strategies or evaluating policy interventions.
Senior author Dr Maria Perez-Ortiz of UCL Computer Science said the ability of AI systems to explain scientific knowledge could create a false impression of their analytical capabilities.
The researchers warned that asking AI what is scientifically established is fundamentally different from asking it to determine what governments should do.
A language divide in AI accuracy
The report also exposed significant disparities in AI performance across languages.
Researchers found that AI models do not simply reason independently of language and then translate their responses. Their performance is influenced by the quantity and quality of information available in different languages during training.
The average accuracy of the tested models was 88.6% in English.
Performance was comparable in Spanish, French and Portuguese, but declined significantly in several other languages.
| Language | Accuracy |
|---|---|
| English | 88.6% |
| Hindi | 83.6% |
| Bengali | 83.6% |
| Urdu | 81.4% |
| Swahili | 78.3% |
The gap is particularly concerning because many communities speaking these languages are located in regions that are disproportionately vulnerable to climate change.
The findings suggest that people accessing climate information in their native languages may receive less reliable answers from AI systems than English-speaking users.
This raises broader concerns about the digital divide and the unequal distribution of reliable scientific information in the age of generative AI.
AI could become a vehicle for climate disinformation
The report comes at a time when climate misinformation is becoming increasingly sophisticated.
Rather than outright denying the existence of climate change, misinformation campaigns are increasingly focused on questioning its human causes, exaggerating the limitations of climate technologies, portraying climate action as economically unviable or suggesting that meaningful action is already too late.
AI systems that readily agree with users could inadvertently amplify such misinformation.
A user presenting a false but convincing argument may receive validation from an AI model, creating the impression that the claim has scientific credibility.
The danger is particularly acute when AI-generated responses are subsequently shared on social media, used in research or incorporated into policy discussions.
A warning for policymakers
The UNESCO-UCL report is a warning against treating AI as an unquestionable authority on scientific matters.
The researchers tested up to 33 leading AI models across nine climate and environmental datasets, including two developed specifically for the study.
The evaluation covered 13 languages and included 656 climate-policy questions assessed against En-ROADS, a science-based global simulation tool used to explore the impact of policy and technology choices on emissions, energy systems and future warming.
The research was commissioned by UNESCO as part of its contribution to the Global Initiative for Information Integrity on Climate Change, launched with the United Nations and Brazil in November 2024.
The study underscores the need for AI systems to be designed and evaluated not merely on their ability to produce convincing answers, but also on their capacity to withstand user pressure, assess new evidence and maintain scientific accuracy.
As governments, businesses and individuals increasingly turn to AI for information on climate change, the central question is no longer whether AI can explain scientific facts.
It is whether AI can be trusted to defend those facts when the user wants to hear something else.
The writer of this article is Dr. Seema Javed, an environmentalist & a communications professional in the field of climate and energy



