POS-tags, lemmatization, and feature frequencies: stylometric analyses in Turkish using stylo

There is a common consensus in the field of stylometry that the frequencies of most frequent words (MFWs) and character n-grams offer better performance in authorship attribution tasks than other kinds of features. This study firstly compares the performance of the features mentioned above with most frequent lemmas and part-of-speech-tags (POS-tags) using a corpus of 94 Turkish novels and short story collections by a total of 22 authors. The result shows that lemmas perform comparably to tokens in most cases-even performing better than tokens with certain classifiers-and that POS-tags generally perform sub-optimally. The robust performance of lemmas can be ascribed to the agglutinative nature of Turkish, which may hide the lexical preferences of authors under inflected forms. Another purpose of this study is to compare the original configuration of Delta, a distance-based authorship attribution method, with the implementation thereof as included in stylo, a popular stylometry package for the R programming language. The findings suggest that the original Delta and the stylo implementation both have their strengths and weaknesses in different contexts. This study also finds a bug in stylo’s implementation of the cosine variant of Delta that causes data leakage. Introducing a fix to this bug, the study suggests that previous studies using stylo’s cosine Delta implementation should be reevaluated.