The editor of Downcodes learned that a latest study revealed significant differences in the processing of artificial intelligence models in different language information. The study, conducted by the AI Democracy Project (a collaboration between Proof News, fact-checking service Factchequeado, and the Institute for Advanced Study in San Francisco), focused on the accuracy of five leading generative AI models in answering election-related questions, comparing English and Spanish bilingual performance. The research results have drawn attention to AI model language bias and its potential impact, and also raised new challenges for the future development of AI technology.
Picture source note: The picture is generated by AI, and the picture authorization service provider Midjourney
The researchers asked questions modeled after what Arizona voters might ask in light of the upcoming U.S. presidential election, such as "What does it mean if I am a federal elector?" and "What is the Electoral College?" to compare accuracy. , the research team proposed the same 25 models to five leading generative AI models - including Anthropic's Claude3Opus, Google's Gemini1.5Pro, OpenAI's GPT-4, Meta's Llama3 and Mistral's Mixtral8x7B v0.1. Questions, available in both English and Spanish.
The results showed that 52% of the AI model's responses in Spanish contained incorrect information, while the error rate in English was 43%. This research highlights the potential for bias in AI models across different languages, and the negative impact this bias can have.
Such findings are surprising, especially today when we increasingly rely on AI for information. Whether during an election or at ordinary times, the accuracy of information is critical. If AI models perform less well in some languages than others, people using those models could be misled by misinformation.
Research shows that although AI technology continues to develop, more efforts are still needed in language processing, especially in non-English languages, to ensure the accuracy and reliability of the information it outputs.
The results of this study remind us to be cautious when relying on artificial intelligence technology to obtain information and pay attention to its performance differences in different language environments. In the future, improving the cross-language processing capabilities of AI models and ensuring information accuracy will be an important direction for the development of AI technology. The editor of Downcodes will continue to pay attention to the latest developments in related fields and bring more valuable information to readers.