Sense representations for Portuguese: experiments with sense embeddings and deep neural language models.
Sense representations have gone beyond word representations like Word2Vec, GloVe and FastText and achieved innovative performance on a wide range of natural language processing tasks. Although very useful in many applications, the traditional approaches for generating word embeddings have a strict d...
| Publicado en: | Language Resources & Evaluation Vol. 55; no. 4; pp. 901 - 925 |
|---|---|
| Autores principales: | , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Dec2021
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=152947514&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 152947514 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Dec2021 vid: 55 iid: 4 pid: 237 pub: Springer Nature artinfo: ui: 152947514 10.1007/s10579-020-09525-1 ppf: 901 ppct: 24 formats: fmt: – @attributes: type: T – @attributes: type: P size: 427KB tig: atl: Sense representations for Portuguese: experiments with sense embeddings and deep neural language models. aug: au: Rodrigues da Silva, Jéssica Caseli, Helena de M. affil: Federal University of São Carlos (UFSCar), São Carlos, Brazil su: Natural language processing Ambiguity Portuguese language Language transfer (Language learning) sug: subj: Natural language processing Ambiguity Portuguese language Language transfer (Language learning) keyword: Deep neural language models Portuguese Sense embeddings Word embeddings Word sense disambiguation ab: Sense representations have gone beyond word representations like Word2Vec, GloVe and FastText and achieved innovative performance on a wide range of natural language processing tasks. Although very useful in many applications, the traditional approaches for generating word embeddings have a strict drawback: they produce a single vector representation for a given word ignoring the fact that ambiguous words can assume different meanings. In this paper, we explore unsupervised sense representations which, different from traditional word embeddings, are able to induce different senses of a word by analyzing its contextual semantics in a text. The unsupervised sense representations investigated in this paper are: sense embeddings and deep neural language models. We present the first experiments carried out for generating sense embeddings for Portuguese. Our experiments show that the sense embedding model (Sense2vec) outperformed traditional word embeddings in syntactic and semantic analogies task, proving that the language resource generated here can improve the performance of NLP tasks in Portuguese. We also evaluated the performance of pre-trained deep neural language models (ELMo and BERT) in two transfer learning approaches: feature based and fine-tuning, in the semantic textual similarity task. Our experiments indicate that the fine tuned Multilingual and Portuguese BERT language models were able to achieve better accuracy than the ELMo model and baselines. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2021. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2021 holdings: @attributes: islocal: N |
|---|