A web-based Bengali news corpus for named entity recognition.

The rapid development of language resources and tools using machine learning techniques for less computerized languages requires appropriately tagged corpus. A tagged Bengali news corpus has been developed from the web archive of a widely read Bengali newspaper. A web crawler retrieves the web pages...

Full description

Bibliographic Details
Published in:Language Resources & Evaluation Vol. 42; no. 2; pp. 173 - 183
Main Authors: Ekbal, Asif, Bandyopadhyay, Sivaji
Format: Article
Published: Springer Nature May2008
Subjects:
Online Access:View this record in EBSCOhost
Description
Summary:The rapid development of language resources and tools using machine learning techniques for less computerized languages requires appropriately tagged corpus. A tagged Bengali news corpus has been developed from the web archive of a widely read Bengali newspaper. A web crawler retrieves the web pages in Hyper Text Markup Language (HTML) format from the news archive. At present, the corpus contains approximately 34 million wordforms. Named Entity Recognition (NER) systems based on pattern based shallow parsing with or without using linguistic knowledge have been developed using a part of this corpus. The NER system that uses linguistic knowledge has performed better yielding highest F-Score values of 75.40%, 72.30%, 71.37%, and 70.13% for person, location, organization, and miscellaneous names, respectively.