Over the past year, several major news media companies have entered content licensing partnerships with OpenAI, the developer of ChatGPT. These deals promise attributed summaries of each media company's reporting, with links to their publications' websites. However, recent findings reveal significant issues with ChatGPT producing hallucinated links to partnered publications, causing concerns among news outlets.
On June 13, it was reported that ChatGPT is generating fake URLs for Business Insider, directing users to 404 errors instead of real articles. This problem extends to at least 10 other publications involved in OpenAI's licensing deals, including The Associated Press, The Wall Street Journal, the Financial Times, The Times (UK), Le Monde, El País, The Atlantic, The Verge, Vox, and Politico.
Tests show that ChatGPT often fails to link to notable stories from these publications. For instance, when asked to provide links to major investigations, the chatbot frequently offers URLs that do not exist, undermining the credibility and utility of these links.
Statements from media companies emphasize that proper attribution and accurate linking are still under development. OpenAI has acknowledged the issue, stating that the promised citation features are not yet live. However, they have not explained how they plan to address the problem of fake URLs.
The issue of hallucinated links highlights a broader concern with ChatGPT’s predictive text generation. Rather than providing accurate URLs, ChatGPT often predicts plausible but incorrect URLs, leading to a high incidence of broken links.
The problem is not limited to English-language publications. Fake URLs have been generated for investigations in French and Spanish by Le Monde and El País, respectively. Despite these ongoing issues, some accurate hyperlinks were noted, such as links to Politico's Roe v. Wade decision publication and the WSJ’s Facebook Files investigation.
Media companies stress the importance of accurate attribution and the protection of their journalism's integrity. While some progress has been made, the prevalence of hallucinated URLs underscores the need for further improvements before the new features can be fully trusted.
Overall, the reliability of ChatGPT in providing accurate citations remains a work in progress, and both OpenAI and its media partners continue to seek solutions to enhance the chatbot’s performance.
