Restoring Arabic vowels through omission-tolerant dictionary lookup: تشْكيل الكَلِمات عَبْرَ مَوارد حاسوبيّة
Vowels in Arabic are optional orthographic symbols written as diacritics above or below letters. In Arabic texts, typically more than 97 percent of written words do not explicitly show any of the vowels they contain; that is to say, depending on the author, genre and field, less than 3 percent of wo...
| Publicado en: | Language Resources & Evaluation Vol. 54; no. 2; pp. 487 - 552 |
|---|---|
| Autores principales: | , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Jun2020
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=143152356&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 143152356 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Jun2020 vid: 54 iid: 2 pid: 237 pub: Springer Nature artinfo: ui: 143152356 10.1007/s10579-019-09464-6 ppf: 487 ppct: 65 formats: fmt: – @attributes: type: T – @attributes: type: P size: 7.7MB tig: atl: Restoring Arabic vowels through omission-tolerant dictionary lookup: تشْكيل الكَلِمات عَبْرَ مَوارد حاسوبيّة aug: au: Neme, Alexis Amid Paumier, Sébastien affil: LIGM, UPEM, CNRS, ENPC, ESIEE, Université Paris-Est, 77454, Marne-la-Vallée, France su: Vowels Arabic language Prepositions Encyclopedias & dictionaries Nouns Electronic newspapers sug: subj: Vowels Arabic language Prepositions Encyclopedias & dictionaries Nouns Electronic newspapers keyword: Arabic language resources Arabic NLP Compression algorithm Finite state transducers Local grammars Pattern-and-root model Root-and-pattern model Semtic morphology Vocalisation Vowelization ab: Vowels in Arabic are optional orthographic symbols written as diacritics above or below letters. In Arabic texts, typically more than 97 percent of written words do not explicitly show any of the vowels they contain; that is to say, depending on the author, genre and field, less than 3 percent of words include any explicit vowel. Although numerous studies have been published on the issue of restoring the omitted vowels in speech technologies, little attention has been given to this problem in papers dedicated to written Arabic technologies. In this research, we present Arabic-Unitex, an Arabic Language Resource, with emphasis on vowel representation and encoding. Specifically, we present two dozens of rules formalizing a detailed description of vowel omission in written text. They are typographical rules integrated into large-coverage resources for morphological annotation. For restoring vowels, our resources are capable of identifying words in which the vowels are not shown, as well as words in which the vowels are partially or fully included. By taking into account these rules, our resources are able to compute and restore for each word form a list of compatible fully vowelized candidates through omission-tolerant dictionary lookup. In our previous studies, we have proposed a straightforward encoding of taxonomy for verbs (Neme in Proceedings of the international workshop on lexical resources (WoLeR) at ESSLLI, 2011) and broken plurals (Neme and Laporte in Lang Sci, 2013, http://dx.doi.org/10.1016/j.langsci.2013.06.002). While traditional morphology is based on derivational rules, our description is based on inflectional ones. The breakthrough lies in the reversal of the traditional root-and-pattern Semitic model into pattern-and-root, giving precedence to patterns over roots. The lexicon is built and updated manually and contains 76,000 fully vowelized lemmas. It is then inflected by means of finite-state transducers (FSTs), generating 6 million forms. The coverage of these inflected forms is extended by formalized grammars, which accurately describe agglutinations around a core verb, noun, adjective or preposition. A laptop needs one minute to generate the 6 million inflected forms in a 340-MB flat file, which is compressed in 2 min into 11 MB for fast retrieval. Our program performs the analysis of 5000 words/second for running text (20 pages/second). Based on these comprehensive linguistic resources, we created a spell checker that detects any invalid/misplaced vowel in a fully or partially vowelized form. Finally, our resources provide a lexical coverage of more than 99 percent of the words used in popular newspapers, and restore vowels in words (out of context) simply and efficiently. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2020. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2020 holdings: @attributes: islocal: N |
|---|