Automated identification of discourse markers using the NLP approach: The case of okay. - SANOSI, ABDULAZIZ; ABDALLA, MOHAMED (Australian Journal of Applied Linguistics, 2021)
This study aimed to examine the potentials of the NLP approach in detecting discourse markers (DMs), namely okay, in transcribed spoken data. One hundred thirty-eight concordance lines were presented to human referees to judge the functions of okay in them as a DM or Non-DM. After that, the researchers used a Python script written according to the POS tagging scheme of the NLTK library to set rules for identifying cases where okay is used as non-DM. The output of the script was compared to the reference human-annotated data. The results showed that the script could accurately identify the function of okay as DM or non-DM in 92% of the cases. The inaccuracy of detecting the rest was found to be caused by a lack of proper and detailed punctuations. The main implications of the results are that new NLP approaches can detect DMS; however, proper punctuation is required to enable the proper identification of DMs. In accordance with the findings, the researcher recommended adopting the approach after conducting further comprehensive studies.