Search

Unsupervised k -nearest-neighbor-based outlier detectors play a vital role in data science research. However, the detectors' performance relies on the choice of the parameter k. However, autonomous selection of the optimal k is poorly documented in literature as it is very challenging. Conventional methods prove ineffective and lack universality as they fail to account for both application and detector factors simultaneously. This article proposes neighborhood consistency , a new concept which tackles the existing issues of selecting the optimal k by considering both application and detector factors. This concept was used to develop a method termed k finder based on neighborhood c onsistency (KFC). KFC does not rely on any extra parameter and has linear time complexity. Simulations show that KFC outperformed baselines and had a good generality to different datasets and detectors. The implementation of the proposed methods can be found on www.OutlierNet.com for reproducibility. • Neighborhood consistency (NC) that an object and its k-NN should have consistent outlier scores is introduced. • Based on NC, we proposed a framework called KFC for automatically selecting k for k-NN-based outlier detectors. • We reviewed the optimum-k-searching methods published between 2000 and 2022. [ABSTRACT FROM AUTHOR]

Trajectory outlier detection is one of the fundamental data mining techniques used to analyze the trajectory data of the Global Positioning System. A comprehensive literature review of trajectory outlier detectors published between 2000 and 2022 led to a conclusion that conventional trajectory outlier detectors suffered from drawbacks, either due to the detectors themselves or the pre-processing methods for the variable-length trajectory inputs utilized by detectors. To address these issues, we proposed a feature extraction method called middle polar coordinates (MiPo). MiPo extracted tabular features from trajectory data prior to the application of conventional outlier detectors to detect trajectory outliers. By representing variable-length trajectory data as fixed-length tabular data, MiPo granted tabular outlier detectors the ability to detect trajectory outliers, which was previously impossible. Experiments with real-world datasets showed that MiPo outperformed all baseline methods with 0.99 AUC on average; however, it only required approximately 10% of the computing time of the existing industrial best. MiPo exhibited linear time and space complexity. The features extracted by MiPo may aid other trajectory data mining tasks. We believe that MiPo has the potential to revolutionize the field of trajectory outlier detection. [ABSTRACT FROM AUTHOR]

• We analyzed the isolation-forest-based methods' problem of lacking efficacy in selecting suitable hyperplanes to split data. • We proposed to use random projection techniques to improve isolation forest's efficacy in selecting suitable hyperplanes. • Experimental results on 30 real-world datasets show that the proposed methods outperform 12 SOTA outlier detectors. [Display omitted] Isolation Forest has a low computational complexity, hence has been widely applied to detect outliers in large-scale data. However, it suffers from the artifacts caused by the hyperplanes chosen, thereby failing to detect outliers in some specific regions. To tackle this problem, we propose the random-projection-based Isolation Forest, which works in two steps. First, we transform the data using the random projection technique. Then, we employ the Isolation Forest to identify outliers using the transformed data. Experimental results show that the proposed methods outperform 12 state-of-the-art outlier detectors. [ABSTRACT FROM AUTHOR]

Traditional outlier detection methods create a model for data and then label as outliers for objects that deviate significantly from this model. However, when dat has many outliers, outliers also pollute the model. The model then becomes unreliable, thus rendering most outlier detectors to become ineffective. To solve this problem, we propose a mean-shift outlier detector. This detector employs a mean-shift technique to modify data and cancel the bias caused by the outliers. The mean-shift technique replaces every object by the mean of its k -nearest neighbors which essentially removes the effect of outliers before clustering without the need to know the outliers. In addition, it also detects outliers based on the distance shifted. Our experiments show that the proposed method works well regardless of the number of outliers in the data. This method outperforms all state-of-the-art methods tested, with both real-world numeric datasets as well as generated numeric and string datasets. [ABSTRACT FROM AUTHOR]