Perfect Density Models Cannot Guarantee Anomaly Detection
Abstract
Thanks to the tractability of their likelihood, several deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities through the lens of reparametrization and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for anomaly detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.
- Publication:
-
Entropy
- Pub Date:
- December 2021
- DOI:
- arXiv:
- arXiv:2012.03808
- Bibcode:
- 2021Entrp..23.1690L
- Keywords:
-
- deep generative modeling;
- probabilistic modeling;
- anomaly detection;
- Computer Science - Machine Learning;
- Statistics - Machine Learning
- E-Print:
- Accepted to the Special Issue "Probabilistic Methods for Deep Learning" of the Journal Entropy. 14 pages and 10 figures in main content, 4 pages of bibliography, and 2 pages in Appendix