Mismatching-aware unsupervised translation quality estimation for low-resource languages.
Translation Quality Estimation (QE) is the task of predicting the quality of machine translation (MT) output without any reference. This task has gained increasing attention as an important component in the practical applications of MT. In this paper, we first propose XLMRScore, which is a cross-lin...
| Publicado en: | Language Resources & Evaluation Vol. 58; no. 4; pp. 1207 - 1232 |
|---|---|
| Autores principales: | , , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Dec2024
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=180627308&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 180627308 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Dec2024 vid: 58 iid: 4 pid: 237 pub: Springer Nature artinfo: ui: 180627308 10.1007/s10579-024-09727-x ppf: 1207 ppct: 25 formats: fmt: – @attributes: type: T – @attributes: type: P size: 955KB tig: atl: Mismatching-aware unsupervised translation quality estimation for low-resource languages. aug: au: Azadi, Fatemeh Faili, Heshaam Dousti, Mohammad Javad affil: https://ror.org/05vf56z40 School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran https://ror.org/04xreqs31 School of Computer Science, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran su: Machine translating Programming languages Language models Pearson correlation (Statistics) English language sug: subj: Machine translating Programming languages Language models Pearson correlation (Statistics) English language keyword: Cross-lingual word embeddings Low-resource languages Machine translation Pre-trained language models Quality estimation ab: Translation Quality Estimation (QE) is the task of predicting the quality of machine translation (MT) output without any reference. This task has gained increasing attention as an important component in the practical applications of MT. In this paper, we first propose XLMRScore, which is a cross-lingual counterpart of BERTScore computed via the XLM-RoBERTa (XLMR) model. This metric can be used as a simple unsupervised QE method, nevertheless facing two issues: firstly, the untranslated tokens leading to unexpectedly high translation scores, and secondly, the issue of mismatching errors between source and hypothesis tokens when applying the greedy matching in XLMRScore. To mitigate these issues, we suggest replacing untranslated words with the unknown token and the cross-lingual alignment of the pre-trained model to represent aligned words closer to each other, respectively. We evaluate the proposed method on four low-resource language pairs of the WMT21 QE shared task, as well as a new English → Persian (En-Fa) test dataset introduced in this paper. Experiments show that our method could get comparable results with the supervised baseline for two zero-shot scenarios, i.e., with less than 0.01 difference in Pearson correlation, while outperforming unsupervised rivals in all the low-resource language pairs for above 8%, on average. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2024. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2024 holdings: @attributes: islocal: N |
|---|