Stanford researchers recently conducted a language test on a popular A.I. chatbot, revealing some significant challenges in handling languages other than English. The chatbot, Claude 3.5 by Anthropic, was asked to compose a traditional Vietnamese poem in the “song thất lục bát” form but failed to adhere to the required structure. This example highlighted the difficulties A.I. systems face in accurately interpreting and producing content in non-English languages.
Further testing demonstrated that Claude 3.5 struggled with basic language queries in Vietnamese, such as providing incorrect terms for familial relationships. These errors underscored the limitations of current A.I. models when dealing with languages that lack substantial online resources and data sets for training.
While English-language A.I. technologies have seen widespread adoption and advancement, the disparity in language support raises concerns about technological inequities on a global scale. Experts warn that the exclusion of non-English languages could lead to significant economic and societal setbacks for regions and communities that rely on these languages.
Sang Truong, a Stanford Ph.D. candidate involved in the research, emphasized the potential consequences of delayed access to advanced technology, noting that even a short lag in technology adoption could create substantial long-term disadvantages.
The findings from the Stanford study highlight the urgent need for improving A.I. language capabilities across diverse linguistic contexts to ensure that technological progress benefits all societies equally.
