Aratox: a multi-dialect, multi-label arabic dataset and model benchmark for toxicity detection.
Toxic language detection in Arabic remains an underexplored yet critical task, complicated by the language’s dialectal diversity, morphological richness, and the prevalence of overlapping toxicity types. In this paper, AraTox is presented as a large-scale, hand-annotated Arabic dataset designed for...
| Publicado en: | Language Resources & Evaluation Vol. 60; no. 2; pp. 1 - 39 |
|---|---|
| Autores principales: | , , , , , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Jun2026
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=193074424&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 193074424 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Jun2026 vid: 60 iid: 2 pid: 237 pub: Springer Nature artinfo: ui: 193074424 10.1007/s10579-026-09917-9 ppf: 1 ppct: 38 formats: tig: atl: Aratox: a multi-dialect, multi-label arabic dataset and model benchmark for toxicity detection. aug: au: Alshargi, Faisal Abulohoom, Ahmed Yagi, Sane Jabr, Furat Lulu, Leena Elnagar, Ashraf affil: Amazon Robotics, New York, NY, USA https://ror.org/00engpz63 Department of Computer Science, University of Sharjah, 27272, University City, Sharjah, UAE https://ror.org/00engpz63 Department of Foreign Languages, University of Sharjah, 27272, University City, Sharjah, UAE https://ror.org/05k89ew48 Department of English, University of Jordan, Queen Rania St., Amman, Jordan https://ror.org/00b98jc42 Department of Computer Science, University of AL Dhaid, AlDhaid, Sharjah, UAE sug: keyword: Annotated dataset Arabic NLP Dialectal arabic Meta-learning Multi-label classification Toxicity detection ab: Toxic language detection in Arabic remains an underexplored yet critical task, complicated by the language’s dialectal diversity, morphological richness, and the prevalence of overlapping toxicity types. In this paper, AraTox is presented as a large-scale, hand-annotated Arabic dataset designed for multi-label and multi-dialect toxicity classification. The dataset comprises around 36,700 social media comments spanning Gulf, Levantine, Nile Basin, North African, Yemeni, and Modern Standard Arabic (MSA) varieties, each manually labeled by expert annotators across seven categories, including hatred, cussing, racial, appearance-based, and sexual insults. The categories were derived from an iterative refinement of overlapping harassment types. AraTox is benchmarked using a range of deep learning models and classical classifiers, highlighting the superior performance of a stacked meta-learning ensemble over standalone models. The meta-classifier achieves a macro-averaged F1-score of 96% and a subset accuracy of 89.67%, significantly outperforming the Multilingual Arabic BERT, MARBERTv2, and other Bidirectional Encoder Representations from Transformers (BERT)-based baselines; i.e., the meta-classifier achieved +5.31% increase in subset accuracy and +5.23% improvement in precision over the strongest deep learning baseline (MARBERTv2). State-of-the-art large language models (LLMs), including GPT-4 and DeepSeek, are further evaluated in zero- and few-shot settings, demonstrating how they struggle with exact-match accuracy (subset accuracy) and exhibit label-set fragmentation. In addition to quantitative evaluation, a qualitative analysis of representative samples is conducted to identify key challenges in annotating and detecting nuanced forms of toxicity. AraTox sets a new benchmark for Arabic toxicity detection and provides a valuable resource for advancing safer and more inclusive Arabic-language moderation systems. Rather than proposing a novel architecture, this work provides a high-utility benchmark and a multi-label dataset. This includes classical learners, fine-tuned Arabic transformers, a stacked ensemble optimized for multi-label classification, and zero-/few-shot evaluation using LLMs. AraTox is released as an open-source resource to foster collaboration and innovation in the Arabic NLP community. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y dt: @attributes: year: 2026 holdings: @attributes: islocal: N |
|---|