"Teams from EPFL’s Natural Language Processing Lab, Cohere Labs and collaborators across the globe have developed INCLUDE. This tool represents a significant step toward an AI more attuned to local contexts. The benchmark enables one to assess whether a LLM is not only accurate in a given language but also capable of integrating the culture and sociocultural realities associated with it. This approach aligns with the goals of the Swiss AI Initiative to create models that reflect Swiss languages and values." https://ai.epfl.ch/beyond-translation-making-ai-multicultural/

INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree
Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare,
Mohamed A. Haggag, Imanol Schlag
Marzieh Fadaee, Sara Hooker, Antoine Bosselut
https://doi.org/10.48550/arXiv.2411.19799