Detect Outlier (Distances) (RapidMiner Studio Core)

Description

This operator performs outlier search according to the outlier detection approach recommended by Ramaswamy, Rastogi and Shim in "Efficient Algorithms for Mining Outliers from Large Data Sets". In their paper, a formulation for distance-based outliers is proposed that is based on the distance of a point from its k-th nearest neighbor. Each point is ranked on the basis of its distance to its k-th nearest neighbor and the top n points in this ranking are declared to be outliers. The values of k and n can be specified by the number of neighbors and number of outliers parameters respectively. This search is based on simple and intuitive distance-based definitions for outliers by Knorr and Ng which in simple words is: 'A point p in a data set is an outlier with respect two parameters k and d if no more than k points in the data set are at a distance of d or less from p'.

This operator adds a new boolean attribute named 'outlier' to the given ExampleSet. If the value of this attribute is true that example is an outlier and vice versa. n examples will have the value true in the 'outlier' attribute (where n is the value specified in the number of outliers parameter). Different distance functions are supported by this operator. The desired distance function can be selected by the distance function parameter.

An outlier is an example that is numerically distant from the rest of the examples of the ExampleSet. An outlying example is one that appears to deviate markedly from other examples of the ExampleSet. Outliers are often (not always) indicative of measurement error. In this case such examples should be discarded.