AbstractsMathematics

Dependency-based anomaly detection:

by C.F. Brenkman




Institution: Delft University of Technology
Department:
Year: 2015
Keywords: anomaly; dependency; log
Record ID: 1260437
Full text PDF: http://resolver.tudelft.nl/uuid:441c5611-5072-454a-909d-fc20c94c5ef5


Abstract

Anomaly (or outlier) detection techniques can be used to find occurrences in data that are surprising or unusual, arousing the suspicion of being generated by an aberrant mechanism. A collective anomaly is a collection of data instances of which the individual data instances may not be anomalies by themselves, but their occurrence together is anomalous with respect to the entire data set. An essential part of a collective anomaly detection technique is the manner in which instances are grouped. A dependency is a statistical relation between two events. When two events are dependent, the occurrence of one event affects the probability of occurrence of the other. For multi-dimensional data, dependencies can be used as a method to group instances for collective anomaly detection. When used in this manner, a dependency becomes an abstract object describing a collection of instances that are related in a statistically significant way. This exploratory study demonstrates the potential of using dependencies to find collective anomalies. During the research, anomalies have been found that no other method currently available would have been able to detect.