Search

Internet of Things (IoT) realizes the interconnection of heterogeneous devices by the technology of wireless and mobile communication. The data of target regions are collected by widely distributed sensing devices and transmitted to the processing center for aggregation and analysis as the basis of IoT. The quality of IoT services usually depends on the accuracy and integrity of data. However, due to the adverse environment or device defects, the collected data will be anomalous. Therefore, the effective method of anomaly detection is the crucial issue for guaranteeing service quality. Deep learning is one of the most concerned technology in recent years which realizes automatic feature extraction from raw data. In this article, the integrated model of the convolutional neural network (CNN) and recurrent autoencoder is proposed for anomaly detection. Simple combination of CNN and autoencoder cannot improve classification performance, especially, for time series. Therefore, we utilize the two-stage sliding window in data preprocessing to learn better representations. Based on the characteristics of the Yahoo Webscope S5 dataset, raw time series with anomalous points are extended to fixed-length sequences with normal or anomaly label via the first-stage sliding window. Then, each sequence is transformed into continuous time-dependent subsequences by another smaller sliding window. The preprocessing of the two-stage sliding window can be considered as low-level temporal feature extraction, and we empirically prove that the preprocessing of the two-stage sliding window will be useful for high-level feature extraction in the integrated model. After data preprocessing, spatial and temporal features are extracted in CNN and recurrent autoencoder for the classification in fully connected networks. Empiric results show that the proposed model has better performances on multiple classification metrics and achieves preferable effect on anomaly detection. [ABSTRACT FROM AUTHOR]

Anomaly detection, which is a method of intrusion detection, detects anomaly behaviors and protects network security. Data mining technology has been integrated to improve the performance of anomaly detection and some algorithms have been improved for anomaly detection field. We think that most data mining algorithms are analyzed on static data sets and ignore the influence of dynamic data streams. Data stream is the potentially unbounded, ordered sequence of data objects which arrive over time. The entire data objects cannot be stored and they need to be handled in one-time scanning. The data distribution of data stream may change over time and this phenomenon is called concept drift. The properties of data stream make analysis method different from the method based on data set and the analysis model is required to be updated immediately when concept drift occurs. In this paper, we summarize the characteristics of data stream, compare the difference between data stream and data set, discuss the problems of data stream mining and propose some corresponding strategies. [ABSTRACT FROM AUTHOR]

Intrusion detection provides important protection for network security and anomaly detection as a type of intrusion detection, which can recognize the pattern of normal behaviors and label the behaviors which departure from normal pattern as anomaly behaviors. The updating of network equipment and broadband speed makes the data mining object change from static data sets to dynamic data streams. We think that the traditional methods based on data set do not satisfy the needs of dynamic network environment. The network data stream is temporal and cannot be treated as static data set. The concept and distribution of data objects is variety in different time stamps and the changing is unpredictable. Therefore, we propose an improved data stream clustering algorithm and design the anomaly detection model according to the improved algorithm. The established model can be modified with the changing of data stream and detect anomaly behaviors in time. [ABSTRACT FROM AUTHOR]

Anomaly detection based on data mining is one of the key technologies to be applied to intelligent detection. K-means is a classic clustering algorithm which is efficient for anomaly detection. Traditional K-means is sensitive to the selection of initial clustering centers. Different initial value can cause different clustering results. We combine improved DD algorithm with information entropy to improve the performance of K-means. Improved K-means can optimize the selection of initial clustering centers; automatically decide the number of clusters and output stable clustering results. After the pretreatment of PCA, the adaptability of improved K-means has a distinct progress. To solve the problem of massive data processing time, we adopt the technology of cloud computing and modify the algorithm for parallel processing. We analyze the performance of improved K-means by using different data sets, KDD Cup99 and public mobile malware data set (i.e. MalGenome). The experimental results illustrate that improved K-means has accurate results and can be applied to anomaly detection in mobile networks. This improved K-means also can be applied for image retrieval by calculating the similarity between each image. [ABSTRACT FROM AUTHOR]