Evaluation of a rule-based approach to automatic factual question generation using syntactic and semantic analysis.
We present a rule-based approach to automatic factual question generation implemented in the Adaptive Courseware and Natural Language Tutor, a natural language-based intelligent tutoring system. Since machine-generated questions are intended for adaptive teaching, learning and assessment, their accu...
| Publicado en: | Language Resources & Evaluation Vol. 57; no. 4; pp. 1431 - 1462 |
|---|---|
| Autores principales: | , , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Dec2023
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=173723399&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 173723399 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Dec2023 vid: 57 iid: 4 pid: 237 pub: Springer Nature artinfo: ui: 173723399 10.1007/s10579-023-09672-1 ppf: 1431 ppct: 31 formats: fmt: – @attributes: type: T – @attributes: type: P size: 1.6MB tig: atl: Evaluation of a rule-based approach to automatic factual question generation using syntactic and semantic analysis. aug: au: Gašpar, Angelina Grubišić, Ani Šarić-Grgić, Ines affil: https://ror.org/00m31ft63 Catholic Faculty of Theology, University of Split, Zrinsko Frankopanska 19, 21000, Split, Croatia https://ror.org/00m31ft63 Faculty of Science, University of Split, Ruđera Boškovića 33, 21000, Split, Croatia su: National Institute of Standards & Technology (U.S.) Intelligent tutoring systems Language teachers English as a foreign language Feature extraction English language sug: subj: National Institute of Standards & Technology (U.S.) Intelligent tutoring systems Language teachers English as a foreign language Feature extraction English language keyword: Automatic evaluation Automatic factual question generation Domain-specific texts Human evaluation Intelligent tutoring ab: We present a rule-based approach to automatic factual question generation implemented in the Adaptive Courseware and Natural Language Tutor, a natural language-based intelligent tutoring system. Since machine-generated questions are intended for adaptive teaching, learning and assessment, their accuracy is of the utmost importance. However, the generation of high-quality questions is still challenging. The proposed approach relies on pre-processing techniques and syntactic and semantic feature extraction to transform declarative sentences and their segments into questions. The quality of questions, generated from domain specific texts, was evaluated by using mixed evaluation strategies: (1) human evaluation, (2) qualitative error analysis, (3) automatic evaluation, (4) human and automatic evaluation of machine-generated questions from paraphrases compared to a set of human-authored questions, (5) preliminary comparison to other approaches. The human evaluation involved two teachers of English as a foreign language who set up evaluation criteria (grammaticality, semantic accuracy, and answerability) and a group of 30 English language graduates. Student-generated questions were validated and used as reference questions for automatic evaluation based on similarity metrics (BLEU-4, METEOR, CHRF, NIST and ROUGE-L). Human and automatic evaluation results were satisfactory but improved significantly with the paraphrasing strategy. The preliminary comparison to other approaches showed that the proposed rule-based approach performed equally well despite its limitations. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2023. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2023 holdings: @attributes: islocal: N |
|---|