Ngram and Bayesian Classification of Documents for Topic and Authorship.

Large, real world, data sets have been investigated in the context of Authorship Attribution of real world documents. Ngram measures can be used to accurately assign authorship for long documents such as novels. A number of 5 (authors &times 5 (movies) arrays of movie reviews were acquired from the...

Full description

Bibliographic Details
Published in:Literary & Linguistic Computing Vol. 18; no. 4; pp. 423 - 448
Main Authors: Clement, Ross, Sharp, David
Format: Article
Published: Oxford University Press / USA Nov2003
Subjects:
Online Access:View this record in EBSCOhost
Description
Summary:Large, real world, data sets have been investigated in the context of Authorship Attribution of real world documents. Ngram measures can be used to accurately assign authorship for long documents such as novels. A number of 5 (authors &times 5 (movies) arrays of movie reviews were acquired from the Internet Movie Database. Both ngram and naive Bayes classifiers were used to classify along both the authorship and topic (movie) axes. Both approaches yielded similar results, and authorship was as accurately detected, or more accurately detected, than topic. Part of speech tagging and function-word lists were used to investigate the influence of structure on classification tasks on documents with meaning removed but grammatical structure intact.