Study Reveals Widespread Inaccuracy in ChatGPT's News Citations, Raising Concerns for Publishers
December 2, 2024A recent study from Columbia University's Tow Center for Digital Journalism has highlighted significant accuracy issues in ChatGPT's search function.
Out of 200 news citations tested from 20 publishers, the study found that ChatGPT provided incorrect or partially incorrect source information in 153 instances.
These inaccuracies were observed even among publishers that have partnerships with OpenAI, including notable names like The New York Post and The Atlantic.
In some instances, ChatGPT cited content that was plagiarized, linking to websites that reproduced articles without proper attribution.
The researchers concluded that there is currently no way for publishers to ensure their content is displayed correctly by ChatGPT, regardless of their relationship with OpenAI.
Mat Honan, editor-in-chief of MIT Technology Review, expressed concerns about the lack of recourse for publishers facing these inaccuracies.
In response, OpenAI stated that ChatGPT directs 250 million users to high-quality content weekly and is working to improve citation accuracy, although they did not directly address the study's findings.
Additionally, the study noted that ChatGPT often fabricates answers instead of admitting it cannot access certain information, displaying unwarranted confidence in its false claims.
Summary based on 1 source
Get a daily email with more AI stories
Source
THE DECODER • Dec 2, 2024
ChatGPT search provides inaccurate sources even for OpenAI partner publishers