Language identification is a natural language processing task that determines the language associated with a given input by analyzing linguistic features extracted from textual or speech data. The task is formulated as a classification problem in which computational models assign an input sample to one or more predefined language categories based on patterns in lexical, character-level, phonological, or acoustic representations [1]. Language identification systems may operate on different input modalities, including written text and spoken language, and rely on statistical, machine learning, or other computational approaches to distinguish among languages [2]. The concept differs from broader language classification tasks because it specifically concerns recognizing the linguistic identity of an input rather than categorizing content, topics, or other non-linguistic properties [1][3].
- natural language processing
- language detection
- computational linguistics
- text classification
References
- Tommi Jauhiainen; Marco Lui; Marcos Zampieri; Timothy Baldwin; Krister Lindén; Automatic Language Identification in Texts: A Survey. J. Artif. Intell. Res. 2019, 65, 675–782, 10.1613/jair.1.11675.
- Zissman, Marc A. 1996. Comparison of Four Approaches to Automatic Language Identification of Telephone Speech. IEEE Transactions on Speech and Audio Processing. https://www.ll.mit.edu/sites/default/files/publication/doc/automatic-language-identification-telephone-speech-zissman-ja-7289.pdf
- Timothy Baldwin and Marco Lui. 2010. Language Identification: The Long and the Short of the Matter. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 229–237, Los Angeles, California. Association for Computational Linguistics. https://aclanthology.org/N10-1027/