Explainable Authorship Identification in Cultural Heritage Applications
Authors: 
Mattia Setzu, Silvia Corbara, Anna Monreale, 
Alejandro Moreo, 
Fabrizio SebastianiAuthors Info & Claims
Mattia Setzu, Silvia Corbara, Anna Monreale, Alejandro Moreo, Fabrizio SebastianiAuthors Info & ClaimsArticle No.: 44, Pages 1 - 23
Abstract
While a substantial amount of work has recently been devoted to improving the accuracy of computational Authorship Identification (AId) systems for textual data, little to no attention has been paid to endowing AId systems with the ability to explain the reasons behind their predictions. This substantially hinders the practical application of AId methods, since the predictions returned by such systems are hardly useful unless they are supported by suitable explanations. In this article, we explore the applicability of existing general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId, with a focus on explanations addressed to scholars working in cultural heritage. In particular, we assess the relative merits of three different types of XAI techniques (feature ranking, probing, factual and counterfactual selection) on three different AId tasks (authorship attribution, authorship verification and same-authorship verification) by running experiments on real AId textual data. Our analysis shows that, while these techniques make important first steps towards XAI, more work remains to be done to provide tools that can be profitably integrated into the workflows of scholars.
References
[1]
Charu C. Aggarwal and ChengXiang Zhai. 2012. A survey of text classification algorithms. In Mining Text Data. Charu C. Aggarwal and ChengXiang Zhai (Eds.), Springer, Heidelberg, 163–222.
[2]
Albert R. Ascoli. 1997. Access to authority: Dante in the epistle to cangrande. In Seminario Dantesco Internazionale/International Dante Seminar 1. Zygmunt G. Baranski (Ed.), Le Lettere, Firenze, 309–52.
[3]
Luca Azzetta. 2016. Nuova edizione commentata delle opere di Dante, Vol. 5. Salerno Editrice, Roma, IT, Chapter “Epistola XIII”, 271–487.
[4]
Shane Barratt. 2017. Interpnet: Neural introspection for interpretable deep learning. Retrieved from
[5]
Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics 48, 1 (2022), 207–219. DOI:
[6]
Dario Benedetto and Mirko Degli Esposti. 2016. Dynamics of style and the case of the “Diario Postumo” by Eugenio Montale: A quantitative approach. In Creativity and Universality in Language. Mirko Degli Esposti, Eduardo G. Altmann and Francois Pachet (Eds.), Springer Nature, Cham, CH, 157–176.
[7]
Barbara Berti, Andrea Esuli, and Fabrizio Sebastiani. 2023. Unravelling interlanguage facts via explainable machine learning. Digital Scholarship in the Humanities 38, 3 (2023), 953–977. DOI:
[8]
José Nilo G. Binongo. 2003. Who wrote the 15th book of Oz? An application of multivariate analysis to authorship attribution. Chance 16, 2 (2003), 9–17.
[9]
Sebastian Bischoff, Niklas Deckers, Marcel Schliebs, Ben Thies, Matthias Hagen, Efstathios Stamatatos, Benno Stein, and Martin Potthast. 2020. The importance of suppressing domain style in authorship analysis. Retrieved from
[10]
Benedikt T. Boenninghoff, Steffen Hessler, Dorothea Kolossa, and Robert M. Nickel. 2019. Explainable authorship verification in social media via attention-based similarity learning. In Proceedings of the IEEE International Conference on Big Data (BigData ’19). IEEE, 36–45. DOI:
[11]
Michael C. Bromby. 2011. Juries and their understanding of forensic science: Are jurors equipped? International Journal of Science in Society 2, 2 (2011), 247–256.
[12]
Diogo V. Carvalho, Eduardo M. Pereira, and Jaime S. Cardoso. 2019. Machine learning interpretability: A survey on methods and metrics. Electronics 8 (2019), 832. DOI:
[13]
Alberto Casadei. 2016. Sempre contro l’autenticita dell’Epistola a Cangrande. Studi danteschi LXXXI (2016), 215–46.
[14]
Carole E. Chaski. 2005. Who’s at the keyboard? Authorship attribution in digital evidence investigations. International Journal of Digital Evidence 4, 1 (2005), 1–13.
[15]
Silvia Corbara, Alejandro Moreo, and Fabrizio Sebastiani. 2023. Same or different? Diff-vectors for authorship analysis. ACM Transactions on Knowledge Discovery from Data 18, 1 (2023), Article 12. DOI:
[16]
Silvia Corbara, Alejandro Moreo, and Fabrizio Sebastiani. 2023. Syllabic quantity patterns as rhythmic features for Latin authorship attribution. Journal of the Association for Information Science and Technology 74, 1 (2023), 128–141. DOI:
[17]
Silvia Corbara, Alejandro Moreo, Fabrizio Sebastiani, and Mirko Tavoni. 2020. L’epistola a Cangrande al vaglio della computational authorship verification: Risultati preliminari (con una postilla sulla cosiddetta “XIV Epistola di Dante Alighieri”). In Atti del Seminario “Nuove Inchieste sull’Epistola a Cangrande”. Alberto Casadei (Ed.), Pisa University Press, Pisa, IT, 153–192.
[18]
Silvia Corbara, Alejandro Moreo, Fabrizio Sebastiani, and Mirko Tavoni. 2022. MedLatinEpi and MedLatinLit: Two datasets for the computational authorship analysis of medieval Latin texts. ACM Journal of Computing and Cultural Heritage 15, 3 (2022), 571–57:15. DOI:
[19]
Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable AI for natural language processing. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL/IJCNLP ’20). 447–459.
[20]
Christopher W. Forstall, Sarah L. Jacobson, and Walter J. Scheirer. 2011. Evidence of intertextuality: Investigating Paul the Deacon’s Angustae Vitae. Literary and Linguistic Computing 26, 3 (2011), 285–296.
[21]
Jack Grieve. 2007. Quantitative authorship attribution: An evaluation of techniques. Literary and Linguistic Computing 22, 3 (2007), 251–270.
[22]
Yuling Gu, Bhavana D. Mishra, and Peter Clark. 2021. DREAM: Uncovering mental models behind language models. Retrieved from
[23]
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM Computing Surveys (CSUR) 51, 5 (2018), 1–42.
[24]
Ralph G. Hall and Madison U. Sowell. 1989. ‘Cursus’ in the can grande epistle: A forger shows his hand? Lectura Dantis 5 (1989), 89–104.
[25]
Oren Halvani. 2021. Practice-Oriented Authorship Verification. Ph. D. Dissertation. Technische Universität Darmstadt, Darmstadt, DE.
[26]
Oren Halvani, Christian Winter, and Lukas Graner. 2019. Assessing the applicability of authorship verification methods. In Proceedings of the 14th International Conference on Availability, Reliability and Security (ARES ’19). 1–10. DOI:
[27]
Ronan Hamon, Henrik Junklewitz, and Ignacio Sanchez. 2020. Robustness and Explainability of Artificial Intelligence. Technical Report EUR 30040. Publications Office of the European Union, Luxembourg, LU. DOI:
[28]
David I. Holmes. 1998. The evolution of stylometry in humanities scholarship. Literary and Linguistic Computing 13, 3 (1998), 111–117.
[29]
Giles Hooker and Lucas Mentch. 2019. Please stop permuting features: An explanation and alternatives. Retrieved from
[30]
John Hwitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP ’19). 2733–2743.
[31]
Fereshteh Jafariakinabad, Sansiri Tarnpradab, and Kien A. Hua. 2020. Syntactic neural model for authorship attribution. In Proceedings of the 33rd International Conference of the Florida Artificial Intelligence Research Society (FLAIRS ’20). Virtual Event, 234–239.
[32]
Mael Jullien, Marco Valentino, and Andre Freitas. 2022. Do transformers encode a foundational ontology? Probing abstract classes in natural language. Retrieved from
[33]
Patrick Juola. 2006. Authorship attribution. Foundations and Trends in Information Retrieval 1, 3 (2006), 233–334. DOI:
[34]
Jakub Kabala. 2020. Computational authorship attribution in medieval Latin corpora: The case of the Monk of Lido (ca. 1101–08) and Gallus Anonymous (ca. 1113–17). Language Resources and Evaluation 54, 1 (2020), 25–56. DOI:
[35]
Mike Kestemont. 2014. Function words in authorship attribution. From black magic to theory?. In Proceedings of the 3rd EACL Workshop on Computational Linguistics for Literature (CLFL ’14). 59–66.
[36]
Mike Kestemont, Enrique Manjavacas, Ilia Markov, Janek Bevendorff, Matti Wiegmann, Efstathios Stamatatos, Benno Stein, and Martin Potthast. 2021. Overview of the cross-domain authorship verification task at PAN 2021. In Proceedings of the Working Notes of the 2021 Conference and Labs of the Evaluation Forum (CLEF ’21). 1743–1759.
[37]
Mike Kestemont, Sara Moens, and Jeroen Deploige. 2015. Collaborative authorship in the twelfth century: A stylometric study of Hildegard of Bingen and Guibert of Gembloux. Digital Scholarship in the Humanities 30, 2 (2015), 199–224. DOI:
[38]
Mike Kestemont, Efstathios Stamatatos, Enrique Manjavacas, Walter Daelemans, Martin Potthast, and Benno Stein. 2019. Overview of the cross-domain authorship attribution task at PAN 2019. In Proceedings of the Working Notes of the 2019 Conference and Labs of the Evaluation Forum (CLEF ’19). 1–15.
[39]
Mike Kestemont, Justin A. Stover, Moshe Koppel, Folgert Karsdorp, and Walter Daelemans. 2016. Authenticating the writings of Julius Caesar. Expert Systems with Applications 63 (2016), 86–96. DOI:
[40]
Moshe Koppel, Jonathan Schler, and Shlomo Argamon. 2009. Computational methods in authorship attribution. Journal of the American Society for Information Science and Technology 60, 1 (2009), 9–26. DOI:
[41]
Orestis Lampridis, Laura State, Riccardo Guidotti, and Salvatore Ruggieri. 2023. Explaining short text classification with diverse synthetic exemplars and counter-exemplars. Machine Learning 112, 11 (2023), 4289–4322.
[42]
Samuel Larner. 2014. Forensic Authorship Analysis and the World Wide Web. Springer, Heidelberg, DE.
[43]
Thai Le, Suhang Wang, and Dongwon Lee. 2020. GRACE: Generating concise and informative contrastive sample to explain neural network model’s prediction. In Proceedings of the 26th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD ’20). Virtual Event, 238–248. DOI:
[44]
Piyawat Lertvittayakumjorn and Francesca Toni. 2019. Human-grounded evaluations of explanation methods for text classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP ’19). 5195–5205.
[45]
Bill Y. Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. 2020. Birds have four legs?! NumerSense: Probing numerical commonsense knowledge of pre-trained language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP ’20). Online Event, 6862–6868. DOI:
[46]
Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2020. Explainable AI: A review of machine learning interpretability methods. Entropy 23, 1 (2020), 18.
[47]
Hui Liu, Qingyu Yin, and William Yang Wang. 2019. Towards explainable NLP: A generative explanation framework for text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL ’19). 5570–5581.
[48]
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a. Linguistic knowledge and transferability of contextual representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL ’19). 1073–1094. DOI:
[49]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. RoBERTa: A robustly optimized BERT pretraining approach. Retrieved from
[50]
Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In Proceedings of the 6th International Conference on Learning Representations (ICLR ’19). DOI: https://openreview.net/forum?id=Bkg6RiCqY7
[51]
Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence 2, 1 (2020), 56–67.
[52]
Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS ’17). 4765–4774.
[53]
Thomas C. Mendenhall. 1887. The characteristic curves of composition. Science 9, 214 (1887), 237–249.
[54]
Frederick Mosteller and David L. Wallace. 1963. Inference in an authorship problem: A comparative study of discrimination methods applied to the authorship of the disputed Federalist Articles. Journal of the American Statistical Association 58, 302 (1963), 275–309.
[55]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.
[56]
Ria Perkins. 2015. Native language identification (NLID) for forensic authorship analysis of weblogs. In New Threats and Countermeasures in Digital Crime and Cyber Terrorism. Maurice Dawson and Marwan Omar (Eds.), IGI Global, Hershey, US, 213–234. DOI:
[57]
Dheeraj Rajagopal, Vidhisha Balachandran, Eduard H. Hovy, and Yulia Tsvetkov. 2021. SELFEXPLAIN: A self-explaining architecture for neural text classifiers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP ’21). 836–850. DOI:
[58]
Marco T. Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16). 1135–1144.
[59]
Laura Rieger and Lars K. Hansen. 2020. IROF: A low-resource evaluation metric for explanation methods. In Proceedings of the ICLR 2020 Workshop on AI for Affordable Healthcare. DOI: https://orbit.dtu.dk/en/publications/irof-a-low-resource-evaluation-metric-for-explanation-methods
[60]
Anderson Rocha, Walter J. Scheirer, Christopher W. Forstall, Thiago Cavalcante, Antonio Theophilo, Bingyu Shen, Ariadne Carvalho, and Efstathios Stamatatos. 2017. Authorship attribution for social media forensics. IEEE Transactions on Information Forensics and Security 12, 1 (2017), 5–33. DOI:
[61]
Upendra Sapkota, Steven Bethard, Manuel Montes, and Thamar Solorio. 2015. Not all character n-grams are created equal: A study in authorship attribution. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL ’15). 93–102.
[62]
Yunita Sari, Mark Stevenson, and Andreas Vlachos. 2018. Topic or style? Exploring the most useful features for authorship attribution. In Proceedings of the 27th International Conference on Computational Linguistics (COLING ’18). 343–353.
[63]
Gennaro Sasso. 2013. Sull’Epistola a Cangrande. La Cultura 3 (2013), 359–446.
[64]
Jacques Savoy. 2019. Authorship of Pauline epistles revisited. Journal of the Association for Information Science and Technology 70, 10 (2019), 1089–1097. DOI:
[65]
Michael R. Schmid, Farkhund Iqbal, and Benjamin C. Fung. 2015. E-mail authorship attribution using customized associative classification. Digital Investigation 14, 1 (2015), S116–S126. DOI:
[66]
Prasha Shrestha, Sebastian Sierra, Fabio A. González, Manuel Montes, Paolo Rosso, and Thamar Solorio. 2017. Convolutional neural networks for authorship attribution of short texts. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL ’17). 669–674.
[67]
Efstathios Stamatatos. 2009. A survey of modern authorship attribution methods. Journal of the American Society for Information Science and Technology 60, 3 (2009), 538–556. DOI:
[68]
Efstathios Stamatatos. 2016. Authorship verification: A review of recent advances. Research in Computing Science 123 (2016), 9–25.
[69]
Efstathios Stamatatos, Mike Kestemont, Krzysztof Kredens, Piotr Pezik, Annina Heini, Janek Bevendorff, Benno Stein, and Martin Potthast. 2022. Overview of the authorship verification task at PAN 2022. In Proceedings of the Working Notes of the Conference and Labs of the Evaluation Forum (CLEF ’22). 2301–2313.
[70]
Justin A. Stover, Yaron Winter, Moshe Koppel, and Mike Kestemont. 2016. Computational authorship verification method attributes a new work to a major 2nd century African author. Journal of the American Society for Information Science and Technology 67, 1 (2016), 239–242. DOI:
[71]
Antonio Theophilo, Rafael Padilha, Fernanda A. Andaló, and Anderson Rocha. 2022. Explainable artificial intelligence for authorship attribution on social media. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP ’22). 2909–2913.
[72]
Enrico Tuccinardi. 2017. An application of a profile-based method for authorship verification: Investigating the authenticity of Pliny the Younger’s letter to Trajan concerning the Christians. Digital Scholarship in the Humanities 32, 2 (2017), 435–447. DOI:
[73]
Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP ’19). 11–20. DOI:
[74]
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP ’20). Online Event, 38–45.
[75]
Haiyan Wu, Zhiqiang Zhang, and Qingfeng Wu. 2021. Exploring syntactic and semantic features for authorship attribution. Applied Soft Computing 111 (2021), 107815. DOI:
[76]
Rong Zheng, Jiexun Li, Hsinchun Chen, and Zan Huang. 2006. A framework for authorship identification of online messages: Writing-style features and classification techniques. Journal of the American Society for Information Science and Technologies 57, 3 (2006), 378–393.
Index Terms
- Explainable Authorship Identification in Cultural Heritage Applications
Recommendations
Accessible Cultural Heritage through Explainable Artificial Intelligence
UMAP '20 Adjunct: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and PersonalizationEthics Guidelines for Trustworthy AI advocate for AI technology that is, among other things, more inclusive. Explainable AI (XAI) aims at making state of the art opaque models more transparent, and defends AI-based outcomes endorsed with a rationale ...
Constructing a semantic wiki for living cultural heritage: case study of Korean tea culture
IMCOM '15: Proceedings of the 9th International Conference on Ubiquitous Information Management and CommunicationIn recent years, the amount of digital content related to cultural heritage resources has been growing to such a degree that it can now be easily published and distributed. However, such digital content is limited to material manifestations, such as ...
Predictive crowd simulations for Cultural Heritage applications
GRAPHITE '05: Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East AsiaComputer technologies and digital recreations have been widely used in the field of Cultural Heritage in the past decade. However, most of the effort has concentrated in accurate data gathering and geometrical representation of buildings and sites. Only ...
Comments
Information & Contributors
Information
Published In
September 2024
382 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 June 2024
Online AM: 20 April 2024
Accepted: 15 March 2024
Revised: 10 March 2024
Received: 03 August 2023
Published in JOCCH Volume 17, Issue 3
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- SoBigData++
- AI4Media
- European Commission
- ERC-2018-ADG
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 220Total Downloads
- Downloads (Last 12 months)220
- Downloads (Last 6 weeks)21
Reflects downloads up to 01 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in