N-gram Feature Selection for Authorship Identification

Abstract

Automatic authorship identification offers a valuable tool for supporting crime investigation and security. It can be seen as a multi-class, single-label text categorization task. Automatic authorship identification depends on selecting stylisticfeatures that would capture an authors writing style independent of the content or genre of text. Character n-grams have been used successfully to represent text for stylistic purposes in literature.They seem to be able to capture nuances in lexical, syntactical, and structural level. To date character n-grams of fixed length have been used for authorship identification. In this thesis: we introduce a new approach for selecting variable length n-grams inspired by previous work for selecting variable-length word sequences. We propose the use of variable-length n-grams to represent the stylistic information of the documents to be classified. We explore the significance of digits as stylistic features for distinguishing between authors and show that an increase in performance can be achieved using simple text pre-processing. Using a subset of the new Reuters corpus, consisting of texts on thesame topic by 50 different authors, we show that the proposed feature selection method is at least as effective as information gain for selecting the most significant n-grams, although the feature sets produced by the two methods have few common members.