Search

This paper concerns a new incremental approach to mining data streams. It is known that patterns in a data stream may evolve over time. In many cases, we need to track and analyze the nature of these changes. In the paper, the probabilistic neural networks are considered as basic models for tracking changes in data streams. We present globally convergent stream data mining algorithms applied to problems of regression, classification, and density estimation in a time-varying (drifting) environment. The algorithms are derived from the Parzen kernel-based probabilistic neural networks working in the online mode. For each problem, a theorem is presented ensuring the L 2 convergence of the algorithm designed for tracking drifting regression, density, or discriminant functions. Illustrative examples explain in detail how to choose the bandwidth of the Parzen kernel and the learning rate of the online algorithm. The performance of all algorithms is shown in exemplary simulations. It should be noted that this paper is one of very few, in the existing literature, presenting mathematically justified stream data mining algorithms. • The incremental version of the Generalized Regression Neural Network (IGRNN) able to track drifting regression functions. • The incremental version of the Probabilistic Neural Network (IPNN) working in non-stationary environments. • Application of IPNN for tracking drifting discriminant functions. • Mathematical proofs of the L 2 convergence of all the proposed estimators. [ABSTRACT FROM AUTHOR]

Frequent pattern mining (FM) has a wide range of applications in the real world. But FM sometimes discovers many uninteresting patterns at the same time. Constraint-based FM, especially periodic constraint FM, becomes of interest, and it reduces redundant patterns. Most of the state-of-the-art algorithms for mining periodic frequent patterns (PFPs) are designed to find PFPs only in binary temporal databases. However, vast information in real situations is more suitable to be modelized as quantitative databases, in which most existing methods are inapplicable. Patterns obtained by those methods may change due to a tiny perturbation in the database, which is not expected in many situations. In this paper, we propose an algorithm (FP2M) to mine fuzzy-driven periodic frequent patterns and an algorithm (SFP2M) to find stable fuzzy-driven periodic frequent patterns in quantitative temporal databases. Fuzzy sets are used in our algorithms to deal with quantitative data since they provide a relaxed interval segmentation and are easy to implement. Novel pruning strategies and a new structure called Estimated Period Co-occurrence Structure (EPCS) are designed to speed up the mining process and improve efficiency. An improved method (SFP2M) is also presented by using the lability measurement to find stable patterns. Experimental evaluations on both real and synthetic datasets show good performance of the designed algorithms. [ABSTRACT FROM AUTHOR]

Blockchain distributed ledger technology (DLT) has widespread applications in society 5.0 because it improves service efficiency and significantly reduces labor costs. However, employing blockchain DLT entails considerable energy consumption in the mining process. This paper proposes a blockchain accelerator (BCA) with ultralow power consumption and a high processing rate to address the problem. The BCA focuses on accelerating the double secure hash algorithm (SHA) 256 function required in the mining process at a system-on-chip (SoC) level. We propose three ideas, namely, multiple local memories (multimem), double-cell processing element (D-PE), and nonce autoupdate (NAU), to reduce the external data transfer time and improve the BCA hardware efficiency. We propose a cascaded multiple BCA chip model to enhance the system throughput by several-fold. Our experiments on an ASIC and FPGA prove that the proposed BCA successfully performs the mining process for multiple blockchain networks with much lower power consumption than that of the state-of-the-art CPUs and GPUs. The BCA is laid out with Renesas 65 nm technology with a chip area of $25~mm^{2}$ and consumes $530~mW$ at 100 MHz. The power efficiency of the layout chip is improved by 2428 and 143 times compared with that of the fastest CPU Intel i9-10940X and GPU RTX 3090, respectively. [ABSTRACT FROM AUTHOR]

• The existing methods may lead to a contradictory in the transitivity. • A novel neighborhood geometric similarity was firstly defined. • Four new kinds of neighborhood similarities were proposed. • A new VPNRS model is constructed, and a novel decision tree algorithm is proposed. The decision tree algorithm has been widely used in data mining and machine learning due to its high accuracy, low computational cost and high interpretability. However, when dealing with the continuous data, the classical decision tree algorithm needs to replace continuous attributes with discretized attributes by the strategy of discretization. Discretization may cause a loss of information structure, which will affect the performance of classification. To tackle this problem, many researchers have proposed different decision tree methods based on variable precision neighborhood rough sets. However, these methods do not consider the geometric structure of neighborhood systems, which may lead to a contradiction in the transitivity of the equivalence relation. In this paper, we first define a novel neighborhood geometric similarity in a neighborhood system from the perspective of geometry. Second, by combining the neighborhood geometric similarity and the neighborhood algebraic similarity, we propose four new kinds of neighborhood similarities, which can solve the contradictory transitivity of the equivalence relation. Third, a variable precision neighborhood rough set model is constructed using the new similarities, and a novel decision tree algorithm is proposed based on this model, where the degree of attribute dependence is used as the partition measure. Experimental results on 14 selected datasets from the UCI Machine Learning Repository show that our algorithm is effective. The average accuracy of our algorithm is over 90%, which is 10% higher than the classical decision tree algorithms, and the number of leaf nodes increases slightly. [ABSTRACT FROM AUTHOR]

• We propose to use fractional order correlation to measure the relationship between two sequences and apply the normalized form of the results to create the fractional order shape-based distance between two sequences. • We use the average sequence calculation method DBA to determine the cluster center and then combine it with our distance to cluster the time series. • We also combine our distance with the center determination strategy of other clustering algorithms to execute comparative experiments. • Experiments show that our method has improved the clustering accuracy. Our proposed distance can achieve better results when combined with a variety of strategies. At the same time, in the shape-based clustering algorithm, compared with the best KShape algorithm, we can also achieve better results. Time series clustering is a research hotspot in data mining. Most of the existing clustering algorithms combine with the classical distance measure which ignore the offset of sequence shape. As a result, shape-based clustering algorithms are becoming increasingly popular. On the majority of data sets, the most representative shape-based clustering algorithm, KShape, which defines a shape-based distance with shift invariance, has been shown to outperform other algorithms. In this paper, we propose a new shape-based clustering algorithm named Fractional Order Shape-based k-cluster(FrOKShape), which defines a multi-variable shape-based distance by normalized fractional order cross-correlation and uses the DTW Barycenter Averaging (DBA) as a center computation strategy. Our distance exhibits excellent shape shift deviating properties and good compatibility integrated with a variety of existing clustering center strategies so that it can provide more potential good results. Experiments show that combining our distance with a traditional clustering algorithm produces excellent clustering indicators. In a series of comparative experiments, FrOKShape also exhibits a comparable result to the existing better shape-based clustering algorithm KShape. [ABSTRACT FROM AUTHOR]

Clustering, as an essential technique in unsupervised learning, plays a pivotal role in the fields of data mining and machine learning. However, the classic K -means clustering algorithm has intrinsic drawbacks such as sensitivity to initial cluster centers, susceptibility to a local optimal solution, and challenges in handling data uncertainty. To address these problems, this paper proposes an artificial hummingbird algorithm (AHA)-based three-way K -means clustering algorithm, called AHA-3WKM. First, AHA is introduced to address the problems of sensitivity to initial cluster centers and local optima. Second, a fitness function of AHA is specifically constructed to find the best initial clustering centers so that the hummingbirds can search for high-quality food sources, i.e., the global optimum cluster centers. Third, a three-way clustering approach is utilized to capture information about data uncertainty. In this way, the results of clustering are divided into three distinct regions based on the relationship between objects and clusters. The experimental results demonstrate that AHA-3WKM has good performance, and enhances the stability and the accuracy of clustering results. • AHA is introduced to address the problems of the sensitivity to initial cluster centers and the proneness to local optima. Hummingbirds are treated as data points, which dynamically update their strategies and effectively find the optimal cluster centers during multiple iterations. • A fitness function is designed based on the clustering principle of "birds of a feather flock together", with the aim of simplifying calculations, which enhances the specificity and practicality of K-means algorithm. • An AHA-based three-way K-means clustering algorithm (i.e., AHA-3WKM) is proposed. The clustering process is initialized with cluster centers optimized by AHA, and the results are represented in three regions, which can capture the uncertainty within the datasets. [ABSTRACT FROM AUTHOR]

PageRank is a classic measure that effectively evaluates the importance of nodes in large graphs. It has been applied in numerous applications spanning data mining, Web algorithms, recommendation systems, load balancing, search and connectivity structures identification. Computing PageRank for large graphs is challenging and this has motivated the studies of distributed algorithms to compute PageRank. Previously, little works have been spent on the distributed PageRank algorithms with strong guarantees on both complexity and accuracy. In this paper, we focus on the theoretical aspect and study the complexity of distributed PageRank computation based on the well-known congested-clique model with a bandwidth generalization. An existing algorithm proposed by Sarma et al. (2015) can be applied in this model, which estimates PageRanks in an n -node graph using, with high probability, O (log n) communication rounds and a bandwidth of O ((log n) 3) bits. We present Radar-Push (RP), which is a distributed PageRank algorithm that is strictly better in round complexities. Specifically, Radar-Push uses O (log log n) communication rounds and an edge bandwidth of O ((log n) 3) bits. We further show that Radar-Push can be adapted to efficiently compute an important variant of PageRank, namely, the batch one-hop personalized PageRank, in O (log log n) communication rounds. [ABSTRACT FROM AUTHOR]

More training instances could lead to better classification accuracy. However, accuracy could also degrade if more training instances mean further noises and outliers. Additional training instances arguably need additional computational resources in future data mining operations. Instance selection algorithms identify subsets of training instances that could desirably increase accuracy or at least do not decrease accuracy significantly. There exist many instance selection algorithms, but no single algorithm, in general, dominates the others. Moreover, existing instance selection algorithms do not allow direct controlling of the instance selection rate. In this paper, we present a simple and generic cluster-oriented instance selection algorithm for classification problems. Our proposed algorithm runs an unsupervised K Means Clustering algorithm on the training instances and with a given selection rate, selects instances from the centers and the borders of the clusters. On 24 benchmark classification problems, when very similar percentages of instances are selected by various instance selection algorithms, K Nearest Neighbours classifiers achieve more than 2%–3% better accuracy when using instances selected by our proposed method than when using those selected by other state-of-the-art generic instance selection algorithms. [ABSTRACT FROM AUTHOR]

Pattern mining is one of the most studied fields in data mining. Being mostly motivated by practitioners, pattern mining algorithms are often based on heuristics and are lacking suitable formalization. In this paper, we are revisiting pattern mining, and especially itemset mining, which allows one to analyze binary datasets in searching for interesting and meaningful itemsets and respective association rules. We introduce a concise representation –the closure structure– based on closed itemsets and their minimum generators (called "passkeys") for capturing the intrinsic content of a dataset. The closure structure allows one to understand the content of the dataset in terms of closed sets and equivalence classes of itemsets. We discuss theoretical properties of passkeys which are concise representatives of closed itemsets. We propose a formalization of the closure structure and passkeys in terms of Formal Concept Analysis, which is well adapted to studying such elements. Besides theoretical results, we present the GDPM algorithm for enumerating passkeys and discovering the closure structure. GDPM is rather unique as it returns a characterization of a dataset content in terms of complexity levels, highlighting the diversity and the distribution of the itemsets. Finally, some experiments show how the GDPM algorithm and the closure structure can be practically used. [ABSTRACT FROM AUTHOR]

K-nearest neighbors search (K-NNS) is a fundamental problem in many areas of machine learning and data mining. In an attempt to solve NNS problems by locality-sensitive hashing (LSH)-based algorithms and avoid the F1-trap, in this paper, we theoretically prove the necessity of maintaining close edges of each dimension, the independence of the partition location and the precision of retrieval. Based on the theories, a new method is proposed for constructing the hash function family. Then, a volume-balancing locality-sensitive hashing (VBLSH) algorithm is implemented. In this algorithm, the problem of constructing the hash function family is transformed into an optimization problem under a fixed number of hypercubes. By solving the optimization problem, different hyperplane intervals are set for each dimension to ensure that the hash functions can evenly segment the data space, where each segmented hypercube has a close volume and close side lengths. Finally, the analysis of the algorithm shows that the VBLSH algorithm has a high index coding efficiency and hash code calculation efficiency. The experimental results show that the VBLSH algorithm can simultaneously take into account the efficiency, precision and recall rate, and avoid the F1-trap; it shows excellent effectiveness on 4 datasets. [ABSTRACT FROM AUTHOR]

In this study, we focus on semi-supervised data stream classification tasks. With the advent of applications that generate vast streams of data, data stream mining algorithms are required to handle the intrinsic issues of learning patterns from this type of non-stationary data. The main challenges are fast arriving, infinite length, limited labeled data, concept drift, and concept evolution. Although many algorithms have been used to overcome these issues, there are still many challenges that need to be addressed and discussed in practical domains. This paper mainly focuses on the limited labeled data and concept drift detection issues. Therefore, a semi-supervised framework called CPSSDS is proposed, which uses an incremental classifier as a base learner and a self-training framework to handle the scarcity of labeled samples. The proposed approach utilizes a form of Conformal Predictors to discover a set of informative unlabeled data instances to add to the original training set in each training procedure which is the main challenge in the standard self-training approach. In terms of this framework, the Kolmogorov-Smirnov test is adopted to detect concept drift by comparing the outputs of conformal prediction for two sequences of data chunks. Finally, experimental results on several synthetic and real datasets demonstrate that the proposed learning framework is comparable to the state-of-the-art semi-supervised algorithms for data streams. The implementation source code of the CPSSDS is available at https://github.com/neginsmd/CPSSDS.git. [ABSTRACT FROM AUTHOR]

Often, machine learning applications have to cope with dynamic environments where data are collected in the form of continuous data streams with potentially infinite length and transient behavior. Compared to traditional (batch) data mining, stream processing algorithms have additional requirements regarding computational resources and adaptability to data evolution. They must process instances incrementally because the data's continuous flow prohibits storing data for multiple passes. Ensemble learning achieved remarkable predictive performance in this scenario. Implemented as a set of (several) individual classifiers, ensembles are naturally amendable for task parallelism. However, the incremental learning and dynamic data structures used to capture the concept drift increase the cache misses and hinder the benefit of parallelism. This paper proposes a mini-batching strategy that can improve memory access locality and performance of several ensemble algorithms for stream mining in multi-core environments. With the aid of a formal framework, we demonstrate that mini-batching can significantly decrease the reuse distance (and the number of cache misses). Experiments on six different state-of-the-art ensemble algorithms applying four benchmark datasets with varied characteristics show speedups of up to 5X on 8-core processors. These benefits come at the expense of a small reduction in predictive performance. [ABSTRACT FROM AUTHOR]

• A search space partitioning approach for frequent high utility sequence mining with constraints. • The C-FHUSM algorithm to mine these patterns from generators and closed sequences. • The FGenHUSM algorithm to mine all frequent generator high utility sequences. An important data mining task is to discover all high utility sequences in a quantitative sequence database. Although useful, the number of discovered sequences is often very large. To find patterns that are more tailored to a user's needs, this paper studies the problem of mining frequent high utility sequences satisfying item constraints. This article proposes a novel algorithm named C-FHUSM to quickly obtain these sequences from two concise representations discovered from a quantitative sequence database, namely frequent generator high utility sequences and frequent closed high utility sequences. The first set is extracted using a novel algorithm named FGenHUSM, while an existing algorithm is applied to extract the second set. C-FHUSM integrates novel pruning techniques to ignore sequences that do not satisfy item constraints early by checking only a small number of representative sequences at the beginning of the mining process. Experimental results show that C-FHUSM can be more than ten times faster and has better scalability than a modified version of the state-of-the-art EHUSM algorithm for mining sequences with item constraints. Moreover, it is found that using C-FHUSM is beneficial when a user frequently changes constraints as results can be updated without rescanning the database. [ABSTRACT FROM AUTHOR]