POS tagging of low-resource Pashto language: annotated corpus and BERT-based model. - Haq, Ijazul; Zhang, Yingjie; Qadri, Intakhab Alam (Language Resources & Evaluation, 2025)
This paper presents the development of a comprehensive part-of-speech (POS) annotated corpus for the low-resource Pashto language, along with a deep learning model for automatic POS tagging. The corpus comprises approximately 700K words (30K sentences), labeled for word boundaries, considering Pashto lacks explicit delimiters for word segmentation. The corpus was then annotated for POS information using a concise and pragmatic tagset of 36 grammatical categories. We utilized this corpus to train a supervised POS tagging model. For the model development, we employed a multilingual BERT (Bidirectional Encoder Representations from Transformers), fine-tuning it for this specific task. The BERT-based model performance was evaluated against an RNN-based model that employs the BiLSTM-CRF network and word embeddings (Word2Vec, fastText, and GloVe). Experimental results demonstrated that the BERT-based model achieved the best performance, attaining an accuracy of 96.24% and an F1-score (weighted average) of 96.22%. The model's performance is highly satisfactory, making it useful in practical applications. Furthermore, the applications of the annotated corpus are not limited to this study only; it can be employed in various NLP applications, including named entity recognition (NER), text proofing, and constituency and dependency parsing.