1. Outlier detection: How to Select [formula omitted] for [formula omitted]-nearest-neighbors-based outlier detectors.
- Author
-
Yang, Jiawei, Tan, Xu, and Rahardja, Sylwan
- Subjects
- *
OUTLIER detection , *DETECTORS , *TIME complexity , *DATA science , *K-nearest neighbor classification - Abstract
Unsupervised k -nearest-neighbor-based outlier detectors play a vital role in data science research. However, the detectors' performance relies on the choice of the parameter k. However, autonomous selection of the optimal k is poorly documented in literature as it is very challenging. Conventional methods prove ineffective and lack universality as they fail to account for both application and detector factors simultaneously. This article proposes neighborhood consistency , a new concept which tackles the existing issues of selecting the optimal k by considering both application and detector factors. This concept was used to develop a method termed k finder based on neighborhood c onsistency (KFC). KFC does not rely on any extra parameter and has linear time complexity. Simulations show that KFC outperformed baselines and had a good generality to different datasets and detectors. The implementation of the proposed methods can be found on www.OutlierNet.com for reproducibility. • Neighborhood consistency (NC) that an object and its k-NN should have consistent outlier scores is introduced. • Based on NC, we proposed a framework called KFC for automatically selecting k for k-NN-based outlier detectors. • We reviewed the optimum-k-searching methods published between 2000 and 2022. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF