A recent study conducted by the European Broadcasting Union (EBU) and the BBC revealed that leading AI assistants inaccurately present news content in almost 50% of their responses. The research, which examined 3,000 answers to news-related queries from various AI assistants, including OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini, and Perplexity, assessed their accuracy, sourcing, and ability to differentiate between opinion and fact in 14 languages.
According to the study, 45% of the AI responses analyzed contained significant issues, with 81% exhibiting some form of problem. The research also highlighted that 7% of online news consumers and 15% of individuals under 25 rely on AI assistants for news consumption.
Gemini, Google’s AI assistant, expressed a commitment to enhancing its platform based on user feedback. OpenAI and Microsoft acknowledged the issue of “hallucinations” and are working to address misinformation generated by AI models. Perplexity boasts a 93.9% accuracy rate in terms of factuality in one of its research modes.
One-third of the AI assistant responses showed serious sourcing errors, such as missing or misleading attribution. The study noted that 72% of Gemini’s responses had significant sourcing issues, while accuracy issues, including outdated information, were found in 20% of all AI assistant responses analyzed.
Examples of misinformation included Gemini providing incorrect information about changes to a law on disposable vapes and ChatGPT erroneously reporting Pope Francis as the current Pope months after his passing. The study involved 22 public-service media organizations from 18 countries, including the CBC, Radio-Canada, and others from various nations.
The EBU emphasized the importance of AI companies improving the accuracy of their AI assistants when responding to news inquiries and being more transparent and accountable. The report underscored the necessity for AI assistants to have mechanisms in place to identify and rectify errors, similar to those utilized by news organizations to maintain public trust and democratic participation.
