DOI: 10.11118/ecos2026_027

AUTHORSHIP VERIFICATION OF ENGLISH TEXT: TRADITIONAL STYLOMETRY, TRANSFORMERS, AND LARGE LANGUAGE MODELS COMPARED AND DEPLOYED

Jiří Pokorný¹, Michaela Veselá¹, Oldřich Trenz¹

¹ Department of Informatics, Faculty of Business and Economics, Mendel University in Brno, Zemědělská 1, 613 00 Brno, Czech Republic

This article compares three families of methods for authorship verification, the binary decision whether two texts share one author. We frame the task as pairwise classification and test traditional stylometric pipelines (bag-of-words and TF-IDF with scikit-learn classifiers), fine-tuned transformers (Longformer, BigBird-RoBERTa, ModernBERT), and large language models used through prompting. Training used the PAN 2020 corpus, testing the disjoint PAN 2021 corpus, together 295,800 English text pairs. A fine-tuned Longformer classification head reached the highest F1 of 92.38%, but a much cheaper TF-IDF pipeline with a linear classifier trailed it by less than one percentage point at 91.50%. The way two text vectors are combined into one feature vector mattered more than the classifier itself: absolute difference beat element-wise multiplication everywhere. The three open large language models we tried produced unusable output. Because hosting was constrained, the deployed web application runs TF-IDF with logistic regression (F1 91.28%), which returns the probability the interface needs.

Keywords: authorship verification, stylometry, TF-IDF, transformers, Longformer, large language models

pages: 257-263, online: 2026