Why are so many scientific articles wrong about the disease I study?

Human brucellosis has been my field of scientific interest for decades. I tend to read everything published on it and frequently serve as reviewer for many peer-reviewed medical journals. In the past two years, I’ve started noticing something strange: brucellosis, several manuscripts report, is endemic in 170 countries. The disease is certainly widespread; that much is well accepted. But the specific claims about brucellosis circulating in 170 countries? They’re made up. And it appears that the AI chatbots that scientists increasingly use to produce papers may be helping to spread them.

Brucellosis is a bacterial disease that causes fever, arthritis, and other symptoms. You can get it through contact with animals, drinking raw or unpasteurized dairy products, or eating undercooked meat. Because it can transmit through the air, it has been studied as a biological warfare agent in the past. Brucellosis is notoriously difficult to treat and eradicate. It’s the most common zoonotic bacterial infection in the world, with a 2023 estimate putting annual global incidence at 2.1 million cases. My country, Greece, has the highest caseload in the European Union.

While it’s not a matter of life and death whether brucellosis is actually circulating in 170 countries, finding out why false facts make it into scientific literature—the basis for important medical interventions and other developments—is critical. An erroneous understanding can have deadly results in medicine and other areas. And with AI chatbots fast becoming a source for information and a tool for content production for the public and scientist alike, false understandings can spread even more quickly than they might have in the past. The growth of shady corners of academic publishing, where predatory journals publish studies without strict scrutiny and paper mills sell articles to authors, increases the potential of AI to erode trust in published science.

I have researched how often claims about brucellosis being endemic or occurring in “170 countries” appeared in the medical literature, what references supported the assertions, and how far back in time they could be traced. I found 117 articles including this term.

The 170 number seems to have appeared in 10 articles prior to 2020; afterwards it became increasingly common, appearing in 27 articles in 2024 and 25 articles in 2025. The 117 articles I found used the number in various ways—brucellosis was “endemic in 170 countries” or “reported from more than 170 countries.” These characterizations and others I saw are wrong. The claim that “the disease has been reported from 170 countries” might be accurate, if it were adequately referenced, though 170 still looks like too big a number. Seventy percent of these articles originated from China.

Not one of the articles I found adequately referenced the claims it made about the status of brucellosis in various countries. In 35 articles, the 170-country number was cited without any reference whatsoever. Four additional articles referenced one of these articles. None of the references in the remaining 79 articles supported the way they cited the 170-country fact, either. For example, 26 articles referenced an article I published in The Lancet Infectious Diseases in 2006 for an iteration of the fact. Seven additional articles used one of these as a reference. My co-authors and I never included anything relevant in the piece. Two additional articles erroneously cited a 2005 New England Journal of Medicine article that I wrote as a reference on brucellosis. One article I found supported its claim by referencing a study on fruit flies. Subsequently, three more articles used the fruit fly article as “evidence” for their “170 countries” with brucellosis claim.

It’s possible, even plausible that many of the publications in which references pointed to irrelevant or erroneous articles were developed using AI. More than 13 percent of the scientific articles published in 2024 and listed in the PubMed database have traces of AI augmentation. And one can expect that this percentage will skyrocket in the coming years. In recent years, large language models (LLMs) have evolved into useful manuscript preparation tools for many researchers, but authors rarely report the extent that they rely on these services. A recent study showed that only 1 in 40 manuscripts evaluated had disclosed such assistance.

Now this augmentation may be benign, used purely for improvements in writing style, grammar, or syntax. But it may also be sinister. If authors use AI platforms to generate content, they compromise their integrity and that of science more broadly. AI-produced content can be erroneous; platforms still hallucinate and support falsehoods.

To see how AI chatbots were weighing the evidence on where one might get sick with brucellosis, I asked several, “is human brucellosis endemic in 170 countries?” The results of my little experiment underscore the risks of using chatbots to produce science.

ChatGPT agreed that brucellosis has been reported in 170 countries but noted that doesn’t mean it is endemic in all of them. “Studies and reviews state that at least 170 countries have reported human brucellosis cases over time, indicating a very widespread global presence,” the LLM reported, without an appropriate reference. Google Gemini made similar observations, again referencing sources that did not actually show that brucellosis has been reported in 170 countries. Several other LLMs agreed that brucellosis has been reported in at least 170 countries, again without accurately referencing their claims. In a particularly wrong response, Deepseek, a Chinese LLM, argued that brucellosis was in more than 170 countries, and thus the answer to my prompt asking whether the disease was endemic in 170 countries was “no.” Anthropic’s Claude agreed brucellosis was endemic in 170 countries, perhaps just not “equally” endemic in all of them.

The one exemption to the overall wrong and unsatisfactory responses from LLMs appeared to be Microsoft’s Copilot, which gave accurate responses that outlined the misunderstanding in the literature about where brucellosis is endemic or has been reported. It even volunteered to trace how the scientific literature came to incorrectly cite the 170-country figure so frequently. “The ‘brucellosis is present in more than 170 countries’ statistic does not come from a primary scientific source,” Copilot reported. Instead, links to erroneous articles and researchers making assumptions likely led to the fact being propagated, the chatbot said.

Most AI models recognized that the “endemic” part is not accurate, but most of them accepted that brucellosis has been reported from 170 countries, referencing articles that do not actually support this claim. Moreover, two AI platforms supported the claim about endemicity and hallucinated repeatedly in their effort to do so.

To be clear, some of the articles erroneously reporting that brucellosis is endemic or has been reported in 170 countries were published before LLMs like ChatGPT came into widespread use. Academic publishing has faced years of pre-AI-era criticism for publishing flawed or unreproducible scientific research.

In recent years, the publishing industry has been increasingly infected by junk publications. Among these are “predatory” journals that are willing to publish anything—for a specific publication fee and after a reputed peer review process. Full of articles augmented by AI, these publications also influence AI platform performance. Perhaps some 15,000 of these journals exist within the broader ecosystem of academic publishing.

The rise of junk publication is occurring amid an overall increase in the number of scientific articles published annually. In 2016, 1.9 million articles were published, according to Scopus and Web of Science, which index scientific publications. By 2022, that figure was 2.8 million. According to a study by Finish researchers, the number of predatory journal articles increased from 53,000 in 2010 to 420,000 in 2014.

Is the overflow of shoddy publications, many now likely written with a heavy dose of AI, innocent? Even if journals retract such articles, do AI chatbots cease to reference them? A recent study showed that ChatGPT more often than not persisted in supporting information with retracted articles or studies with other concerns. In fact, ChatGPT rated many retracted studies as being “world leading, internationally excellent, or close.”

When false information gets repeated enough in journal articles, it can become accepted scientific wisdom, cited in subsequent articles with little effort made to ascertain its veracity. This has happened with potentially destructive consequences in the pre-AI era: A 1980 letter in the prestigious New England Journal of Medicine was considered by many as supporting that opioids, when used for chronic pain, are not that addictive. This assertion was blindly adopted as an axiom in future years, with hundreds of other articles citing it and often actually misreading it, partly creating the frame for the opioid crisis.

The quantity of studies supporting something, even when these studies are not necessarily original research, can structure the “accepted” scientific opinion and further enhance its perceived validity through repeated referencing. Studies demonstrating different or non-conclusive data—even those including original research—may garner fewer citations and thus less visibility. This “constructed scientific agreement” can extend to grant applications.

What can be done? Should I write to each editor of the journals hosting the 117 brucellosis articles highlighting the errors I found and requesting they be changed, noted, or even retracted? I do have a personal interest, after all. Many of these articles claim that my colleagues and I reported something that we didn’t. But I have only looked at articles about a single disease. And the LLMs that draw from flawed articles do not have editors to interact with. This evolution to a post-truth situation appears unavoidable. It is not the scientific publishers that will halt it. It is not the public, no matter how scientifically literate they become. It is up to scientists and academia to guard the truth: Everything written in an article, every scientific interaction with the public, every peer-review we perform, every editorial we publish has to be thorough and based on facts. Perhaps I should get started on those letters.