Current limitations in cyberbullying detection: On evaluation criteria, reproducibility, and data scarcity.
The detection of online cyberbullying has seen an increase in societal importance, popularity in research, and available open data. Nevertheless, while computational power and affordability of resources continue to increase, the access restrictions on high-quality data limit the applicability of sta...
| Published in: | Language Resources & Evaluation Vol. 55; no. 3; pp. 597 - 634 |
|---|---|
| Main Authors: | , , , , , , , , |
| Format: | Article |
| Published: |
Springer Nature
Sep2021
|
| Subjects: | |
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=151686294&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 151686294 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Sep2021 vid: 55 iid: 3 pid: 237 pub: Springer Nature artinfo: ui: 151686294 10.1007/s10579-020-09509-1 ppf: 597 ppct: 37 formats: fmt: – @attributes: type: T – @attributes: type: P size: 890KB tig: atl: Current limitations in cyberbullying detection: On evaluation criteria, reproducibility, and data scarcity. aug: au: Emmery, Chris Verhoeven, Ben De Pauw, Guy Jacobs, Gilles Van Hee, Cynthia Lefever, Els Desmet, Bart Hoste, Véronique Daelemans, Walter affil: CSAI, Tilburg University, Tilburg, The Netherlands CLiPS, University of Antwerp, Antwerp, Belgium LT3, Ghent University, Ghent, Belgium su: Cyberbullying Scarcity Power resources Social dynamics Crowdsourcing Acquisition of data sug: subj: Cyberbullying Scarcity Power resources Social dynamics Crowdsourcing Acquisition of data keyword: Cross-domain evaluation Cyberbullying detection Data enrichment Reproducibility ab: The detection of online cyberbullying has seen an increase in societal importance, popularity in research, and available open data. Nevertheless, while computational power and affordability of resources continue to increase, the access restrictions on high-quality data limit the applicability of state-of-the-art techniques. Consequently, much of the recent research uses small, heterogeneous datasets, without a thorough evaluation of applicability. In this paper, we further illustrate these issues, as we (i) evaluate many publicly available resources for this task and demonstrate difficulties with data collection. These predominantly yield small datasets that fail to capture the required complex social dynamics and impede direct comparison of progress. We (ii) conduct an extensive set of experiments that indicate a general lack of cross-domain generalization of classifiers trained on these sources, and openly provide this framework to replicate and extend our evaluation criteria. Finally, we (iii) present an effective crowdsourcing method: simulating real-life bullying scenarios in a lab setting generates plausible data that can be effectively used to enrich real data. This largely circumvents the restrictions on data that can be collected, and increases classifier performance. We believe these contributions can aid in improving the empirical practices of future research in the field. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2021. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2021 holdings: @attributes: islocal: N |
|---|