A task-performance evaluation of referring expressions in situated collaborative task dialogues.
Appropriate evaluation of referring expressions is critical for the design of systems that can effectively collaborate with humans. A widely used method is to simply evaluate the degree to which an algorithm can reproduce the same expressions as those in previously collected corpora. Several researc...
| Publicado en: | Language Resources & Evaluation Vol. 47; no. 4; pp. 1285 - 1305 |
|---|---|
| Autores principales: | , , , , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Dec2013
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=92719974&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 92719974 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Dec2013 vid: 47 iid: 4 pid: 237 pub: Springer Nature artinfo: ui: 92719974 10.1007/s10579-013-9240-5 ppf: 1285 ppct: 20 formats: fmt: @attributes: type: P size: 383KB tig: atl: A task-performance evaluation of referring expressions in situated collaborative task dialogues. aug: au: Spanger, Philipp Iida, Ryu Tokunaga, Takenobu Terai, Asuka Kuriyama, Naoko affil: Department of Computer Science, Tokyo Institute of Technology, Tokyo, Japan Department of Human System Science, Tokyo Institute of Technology, Tokyo, Japan su: Task performance Performance evaluation Referral centers (Information services) Collaborative learning Dialogue Japanese language sug: subj: Task performance Performance evaluation Referral centers (Information services) Collaborative learning Dialogue Japanese language keyword: Demonstrative pronouns Japanese Referring expressions Situated dialogue Task-performance evaluation ab: Appropriate evaluation of referring expressions is critical for the design of systems that can effectively collaborate with humans. A widely used method is to simply evaluate the degree to which an algorithm can reproduce the same expressions as those in previously collected corpora. Several researchers, however, have noted the need of a task-performance evaluation measuring the effectiveness of a referring expression in the achievement of a given task goal. This is particularly important in collaborative situated dialogues. Using referring expressions used by six pairs of Japanese speakers collaboratively solving Tangram puzzles, we conducted a task-performance evaluation of referring expressions with 36 human evaluators. Particularly we focused on the evaluation of demonstrative pronouns generated by a machine learning-based algorithm. Comparing the results of this task-performance evaluation with the results of a previously conducted corpus-matching evaluation (Spanger et al. in Lang Resour Eval, 2010b ), we confirmed the limitation of a corpus-matching evaluation and discuss the need for a task-performance evaluation. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2013. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2013 holdings: @attributes: islocal: N |
|---|