Search

Knowledge graph (KG) modeling constructs a connected network of hazard/fault events by drawing on historical reports, helping extract risk factors and propagation features in railway signaling safety. For the huge graph structure of the knowledge system, efficiently and accurately transforming the event information with text for risk level prediction is the crucial topic of this paper. To establish a quantitative risk assessment method based on KG modeling, we innovatively introduce text clustering technology to intelligently divide the entity short text data set in the KG, assign standardized entity names to the cluster partitions, and then complete the calculation and analysis according to the characteristic parameter formula, greatly reducing the labor and time consumption of data annotation and approximate text repetitive processing. Significantly, a two-stage clustering ensemble algorithm (TSCEA) integrating base clusterer optimization and hierarchical clustering prediction is proposed, whose innovative approach lies in describing sample similarity by the co-association matrix, while considering the contribution discrepancy of samples for decision-making. Specifically, Stage One selects and optimizes various base clusterers with different method kernels, while Stage Two emphasizes the contribution discrepancy of the cluster core and the cluster halo to make the final clustering decision, with the ensemble strategy serving as the bridge between these two. The KG established on field investigation is used to complete the experiment and analysis of the novel algorithm. Another clustering ensemble algorithm (based on a direct weighting strategy) and each base clusterer are applied to verify clustering performance, while the evaluation depends on external indicators. The results indicate that the proposed clustering ensemble algorithm outperforms others in various indicators, significantly improving clustering accuracy and robustness. [ABSTRACT FROM AUTHOR]

Multi-label learning has garnered much research interest due to its wide range of real-world applications. Many multi-label learning methods have been proposed; however, few have addressed the class imbalance problem existing in multi-label data. Even though some studies have taken this issue into account, most of them have ignored the label correlations or only considered random correlations between them. In this study, we propose a novel partition-based imbalanced multi-label learning algorithm, named Multi-label Learning based on Hierarchical Clustering (MLHC), to tackle this problem. MLHC first carries out hierarchical clustering on the original label space to divide it into several disconnected subspaces, each of which contains several labels that are strongly correlated with each other. Then, for each label subspace, we use the problem transformation strategy to convert it into a multi-class problem by binary coding. Any multi-class imbalance learning algorithm can be applied to the transformed multi-class data. Finally, the classification results will be decoded to retrieve the corresponding label subspace, and all label subspace results are combined to show the predicted label vector in the original label space. We conducted experiments not only on thirteen benchmark multi-label datasets but also carried out them on XJTU-SY which is a multi-label engineering application dataset, and the results indicated that our proposed MLHC learning algorithm outperforms several state-of-the-art class imbalance multi-label learning algorithms, demonstrating the effectiveness and necessity of discovering label correlations and transforming the original imbalanced multi-label learning problem into multiple strongly correlated multi-class imbalanced learning problems. [ABSTRACT FROM AUTHOR]

Technologies such as Big Data and IoT have shown the need for intelligent unsupervised processing of Multivariate Time Series (MTS), MTS clustering among them. The challenges in MTS clustering includes not only the selection of the algorithm but also the MTS representation and the similarity measurement among the instances. This study proposes an ensemble of MTS clustering methods that merges different MTS representations and distance functions, aggregating them to obtain a similarity measurement. Furthermore, a proposal for prior knowledge representation is propose to balance the aggregation of the distances. The final clustering is performed either using k-means or hierarchical clustering. The experimentation set up includes the implementation of the ensemble with either 4 or 5 different methods, including an MTS extension of k-Shape. The results show that the ensemble is biased towards the best methods, which helps the clustering practitioner in the selection of the most suitable prototypes. Moreover, the evaluation of the ensemble with the number of clusters set to the number of labels shows that metrics, such as the sensitivity and specificity, must drive the rule of the elbow; alternatively, this value represents the most interesting prior knowledge bit in MTS clustering. Further work includes the study of digital markers to compare MTS representations and distance functions and the use of external metrics to balance the aggregation of the methods. [ABSTRACT FROM AUTHOR]

Density Peak clustering (DPC) as a novel algorithm can fast identify density peaks. But it comes along with two drawbacks: its allocation strategy may produce some non-adjacent associations that may lead to poor clustering results and even cause the malfunction of its cluster center selection method to mistakenly identify cluster centers; it may perform poorly with its high complex O (n 2) when comes to large-scale data. Herein, a fast hierarchical clustering of local density peaks via an association degree transfer method (FHC-LDP) is proposed. To avoid DPC's drawbacks caused by non-adjacent associations, FHC-LDP only considers the association between neighbors and design an association degree transfer method to evaluate the association between points that are not neighbors. FHC-LDP can fast identify local density peaks as sub-cluster centers to generate sub-clusters automatically and evaluate the similarity between sub-clusters. Then, by analyzing the similarity of sub-cluster centers, a hierarchical structure of sub-clusters is built. FHC-LDP replaces DPC's cluster center selection method with a bottom-up hierarchical approach to ensure sub-clusters in each cluster are most similar. In FHC-LDP, only neighbor information of data is required, so by using a fast KNN algorithm, FHC-LDP can run about O (nlog (n)). Experimental results demonstrate FHC-LDP is remarkably superior to traditional clustering algorithms and other variants of DPC in recognizing cluster structure and running speed. [ABSTRACT FROM AUTHOR]

The aim of this study is to design a novel machine learning model named Agglomerative Hierarchical Clustering algorithm based on Overlapped Interval Divergence distance measure (AHC-OLID), for modelling and assessing landslide susceptibility. The AHC-OLID algorithm is proposed to combat the limitations of many clustering algorithms in modelling and assessing landslide susceptibility, including: pre-defining the number of clusters; sensitivity to the clusters properties and noisy data; as well as difficulty in processing rainfall data. The proposed algorithm addresses these issues by integrating the traditional Agglomerative Hierarchical Clustering (AHC) and Overlapped Interval Divergence distance measure (OLID) methods into landslide susceptibility assessment to enhance its performance. It considers the clusters sizes as well the distances between them and tends to avoid taking into consideration small clusters which are very far from other clusters in the dataset. It is also insensitive to the variation in sizes, variances and shapes of the clusters, which makes the algorithm more advantageous. Besides, the AHC-OLID algorithm makes use of the OLID distance function to process the rainfall data, which takes two factors into consideration: distance between their centers as well as relative size of their overlapped area. Applying this new approach in Baota District, China, produced significant improvement in assessment of landslide susceptibility than previous models. Moreover, the Landslide susceptibility map constructed based on AHC-OLID algorithm can be a useful tool for landslide controlling strategies for proper land use and planning. [ABSTRACT FROM AUTHOR]

Recent developments in sensors and data processing made the structural health monitoring (SHM) sector expanding to big-data field, particularly when continuous long-term strategies are implemented. Nevertheless, main shortcomings are due to the identification and extraction of modal features. In fact, although machine learning methods have been implemented to automate modal identification processes, intense user interaction and time-consuming procedures are still required, limiting the extensive use of these techniques. In order to provide a fully automated procedure capable of identifying and extracting modal properties from covariance driven SSI analyses, an innovative and flexible algorithm for Iterative Hierarchical Clustering Analysis (IHCA) is proposed. To evaluate the stability and robustness of the IHCA method, a Variance-Based Global sensitivity Analysis (VBGA) was performed considering a numerical and experimental case study. The outcomes demonstrated that the IHCA is stable in clustering the physical structural modes and selecting the most representative modal features. • An innovative iterative hierarchical clustering method is proposed to support automated modal identification of structures • The robustness of the algorithm is validated through variance-based sensitivity analyses performed on numerical and experimental data • The proposed algorithm is able to detect outliers in the identified modes despite the noise in the recorded accelerations • The proposed algorithm is reliable for automated continuous structural health monitoring and for supporting decision making [ABSTRACT FROM AUTHOR]

We consider a novel clustering task in which clusters can have compositional relationships, e.g., one cluster contains images of rectangles, one contains images of circles, and a third (compositional) cluster contains images with both objects. In contrast to hierarchical clustering in which a parent cluster represents the intersection of properties of the child clusters, our problem is about finding compositional clusters that represent the union of the properties of the constituent clusters. This task is motivated by recently developed few-shot learning and embedding models (Alfassy et al., 2019; Li et al., 2021) that can distinguish the label sets , not just the individual labels, assigned to the examples. We propose three new algorithms – Compositional Affinity Propagation (CAP), Compositional k -means (CKM), and Greedy Compositional Reassignment (GCR) – that can partition examples into coherent groups and infer the compositional structure among them. We show promising results, compared to popular algorithms such as Gaussian mixtures, Fuzzy c -means, and Agglomerative Clustering, on the OmniGlot and LibriSpeech datasets. Our work has applications to open-world multi-label object recognition and speaker identification & diarization with simultaneous speech from multiple speakers. • We explore a novel clustering task in which clusters have compositional relationships. • We propose 3 new clustering algorithms (CKM, CAP, GCR) to tackle this new problem. • The proposed algorithms outperform existing methods on LibriSpeech and OmniGlot data. [ABSTRACT FROM AUTHOR]

• This paper proposes a hierarchical clustering algorithm robust to noise. • It includes a single linkage improvement that involves local density. • It forbids the merging of representative clusters. • Performance is assessed using data with known ground truth. Hierarchical clustering is widely used in data mining. The single linkage criterion is powerful, as it allows for handling various shapes and densities, but it is sensitive to noise 1 1 A sample code is available at: http://frederic.rosresearch.free.fr/mydata/homepage/.. Two improvements are proposed in this work to deal with noise. First, the single linkage criterion takes into account the local density to make sure the distance involves core points of each group. Second, the hierarchical algorithm forbids the merging of representative clusters, higher than a minimum size, once identified. The experiments include a sensitivity analysis to the parameters and a comparison of the available criteria using datasets known in the literature. The latter proved that local criteria yield better results than global ones. Then, the three single linkage criteria were compared in more challenging situations that highlighted the complementariness between the two levels of improvement: the criterion and the clustering algorithm. [ABSTRACT FROM AUTHOR]

Abstract Although many divisive hierarchical clustering methods for processing categorical data have been presented in the literature, none have been systematically or comprehensively investigated. This paper presents a systematic analysis of existing methods, with respective advantages and disadvantages summarized to develop a unified divisive hierarchical clustering framework that follows three general steps: (1) select attributes for splitting a selected cluster; (2) based on these attributes, generate bipartitions of the cluster; and (3) determine which of the resulting clusters should be further split. Using the proposed framework, representative existing algorithms are compared, and better-performing algorithms are produced through improvements relevant to each step of the unified framework. Experimental results on fifteen UCI benchmark datasets reveal that application of the proposed framework significantly improves the clustering performance of a number of algorithms relative to baseline. [ABSTRACT FROM AUTHOR]

Many real-world applications propose the request for sharing knowledge among different tasks or datasets. Transfer learning has been proposed to solve this kind of problems and it has been successfully applied in supervised learning and semi-supervised learning settings. However, its adoption in clustering, one of the most classical research problems in machine learning and data mining, is still scarce. Spectral clustering, as a major clustering algorithm with wide applications and better performance than k -means typically, has not been well incorporated with knowledge transfer. In this paper, we first consider the problem of learning from only one auxiliary unlabeled dataset for spectral clustering and propose a novel algorithm called transfer spectral clustering (TSC). Then, it is extended to the settings with multiple auxiliary tasks. TSC assumes the feature embeddings being shared with the auxiliary tasks and utilizes co-clustering to extract useful information from the auxiliary datasets to improve the clustering performance. TSC involves not only the data manifold information of individual task but also the feature manifold information shared between related tasks. An in-depth explanation of our algorithm together with a convergence analysis are provided. As demonstrated by the extensive experiments, TSC can effectively improve the clustering performance by using auxiliary unlabeled data when compared with other state-of-the-art clustering algorithms. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we propose an unsupervised face clustering algorithm called “Proximity-Aware Hierarchical Clustering” (PAHC) that exploits the local structure of deep representations. In the proposed method, a similarity measure between deep features is computed by evaluating linear SVM margins, which are learned using nearest neighbors of sample data. Clusters are then formed by applying agglomerative hierarchical clustering (AHC). We evaluate the clustering performance using four unconstrained face datasets, including Celebrity in Frontal-Profile (CFP), Labeled Faces in the Wild (LFW), IARPA JANUS Benchmark A (IJB-A), and IARPA JANUS Benchmark B (IJB-B) datasets. Experimental results demonstrate that the proposed approach can achieve improved performance over state-of-the-art methods. Moreover, we show the proposed clustering algorithm has the potential to be applied to actively learn robust deep face representations by first harvesting sufficient number of unseen face images through curation of large-scale dataset, e.g. the MS-Celeb-1 M dataset. By training DCNNs on the curated MS-Celeb-1 M dataset which contains over three million face images, improved representation for face images are learned. Highlights • Deep representations and clustering can be combined to group faces with distinct identities. • Local structure of deep features should be exploited for improved clustering performance. • Proposed clustering algorithm can be applied to curate training data for deep networks without human annotation. [ABSTRACT FROM AUTHOR]

• Providing the condition under which an AHC does not effectively extract clusters. • Introducing the entanglement to measure the dendrogram quality. • Identifying the root cause of a density bias of traditional AHC algorithms. • Improving AHC performance with a data-dependent kernel. Agglomerative hierarchical clustering (AHC) is one of the popular clustering approaches. AHC generates a dendrogram that provides richer information and insights from a dataset than partitioning clustering. However, a major problem with existing distance-based AHC methods is: it fails to effectively identify adjacent clusters with varied densities, regardless of the cluster extraction methods applied to the resultant dendrogram. This paper aims to reveal the root cause of this issue and provides a solution by using a data-dependent kernel. We analyse the condition under which existing AHC methods fail to effectively extract clusters, and give the reason why the data-dependent kernel is an effective remedy. This leads to a new approach to kernerlise existing hierarchical clustering algorithms including the traditional AHC algorithms, HDBSCAN, GDL, PHA and HC-OT. Our extensive empirical evaluation shows that the recently introduced Isolation Kernel produces a higher quality or purer dendrogram than distance, Gaussian Kernel and adaptive Gaussian Kernel in all the above mentioned AHC algorithms. [ABSTRACT FROM AUTHOR]

Due to the continuous growth of social networks the textual information available has increased exponentially. Data warehouses (DW) and online analytical processing (OLAP) are some of the established technologies to process and analyze structured data. However, one of their main limitations is the lack of automatic processing and analysis of unstructured data (specifically, textual data), and its integration with structured data. This paper proposes the creation, integration and implementation of a new dimension called Contextual Dimension from texts obtained from social networks into a multidimensional model. Such a dimension is automatically created after applying hierarchical clustering algorithms and is fully independent from the language of the texts. This dimension allows the inclusion of multidimensional analysis of texts using contexts and topics integrated with conventional dimensions into business decisions. The experiments were carried out by means of a freeware OLAP system (Wonder 3.0) using real data from social networks. [ABSTRACT FROM AUTHOR]

• Designing an ensemble hierarchical clustering framework based on cluster consensus selection approach. • Aggregation of primary clusters using the clusters clustering technique as a consensus function. • Development of NMI robustness measure for calculating merit at cluster and partition levels. • Defining an innovative criterion for calculating similarity based on merit scores and clusters size. • Perform extensive experiments to demonstrate the efficacy of the proposed clustering algorithm and give credence to our idea. Ensemble clustering has emerged as a combination of several basic clustering algorithms to achieve high quality final clustering. However, this technique is challenging due to the complexities in primary clusters such as overlapping, vagueness, instability and uncertainty. Typically, ensemble clustering uses all the primary clusters into partitions for consensus, where the merits of a cluster or a partition can be considered to improve the quality of the consensus. In general, the robustness of a partition may be poorly measured, while having some high-quality clusters. Inspired by the evaluation of cluster and partition, this paper proposes an ensemble hierarchical clustering algorithm based on the cluster consensus selection approach. Here, the selection of a subset of primary clusters from partitions based on their merit level is emphasized. Merit level is defined using the development of Normalized Mutual Information measure. Clusters of basic clustering algorithms that satisfy the predefined threshold of this measure are selected to participate in the final consensus. In addition, the consensus of the selected primary clusters to create the final clusters is performed based on the clusters clustering technique. In this technique, the selected primary clusters are re-clustered to create hyper-clusters. Finally, the final clusters are formed by assigning instances to hyper-clusters with the highest similarity. Here, an innovative criterion based on merit and cluster size for defining similarity is presented. The performance of the proposed algorithm has been proven by extensive experiments on real-world datasets from the UCI repository compared to state-of-the-art algorithms such as CPDM, ENMI, IDEA, CFTLC and SSCEN. [ABSTRACT FROM AUTHOR]

This paper considers the combined parameter and state estimation problem of a bilinear state space system with moving average noise. There are product terms of state variables and control variables in bilinear systems, which brings difficulties to parameter and state estimation. By designing a bilinear state estimator based on Kalman filter and using input–output data to estimate the state, a hierarchical multi-innovation stochastic gradient (i.e., H-MISG) algorithm based on the state estimator is proposed to jointly estimate unknown states and parameters. In addition, compared with the hierarchical stochastic gradient algorithm, H-MISG algorithm introduces the innovation length parameter, makes full use of the system input and output data information, and improves the accuracy of parameter estimation. Numerical simulation examples verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Cluster analysis is a crucial data mining technology widely used in image segmentation, language processing, and pattern recognition. Most existing clustering algorithms cannot identify complex shapes in manifold data sets and data sets with varying-density distribution, especially when clusters with significant differences in density are close to each other. Hierarchical clustering algorithms can identify data sets of arbitrary shapes. However, hierarchical clustering algorithms not only cannot cluster datasets with significant density variations but also have a high time cost. So in this paper, we propose a novel hierarchical clustering algorithm based on density-distance cores, called HCDC. It first selects the density-distance representative points for each point from the set of candidate representative points. Then it selects density-distance cores from all density-distance representatives. And it replaces the whole data set with density-distance cores and uses a new distance between them to apply hierarchical clustering. To avoid the influence of noise points in the dataset when finding density-distance cores, we also propose the noise point detection method and verify the feasibility of this method. In this paper, we compare our proposed algorithm with existing classical and novel algorithms on synthetic and real datasets. Experiments show that our algorithm clusters better than existing algorithms on complex-shaped datasets and datasets with different densities. On datasets with sparse and dense clusters close to each other, the ARI score of HCDC is more than 0.1 higher than that of LDP-MST. In particular, on the grid dataset, HCDC's ARI score is 0.997 higher than LDP-MST. On DS3 and DS8, HCDC's ARI score is more than 0.14 higher than the second-best algorithm, RNN-DBSCAN. Moreover, on the zoo dataset, HCDC's ARI score is 0.15 and 0.6 higher than RNN-DBSCAN and LDP-MST, respectively. On the olivetti face dataset, HCDC is the only algorithm with an NMI score above 0.9 on photo1 and photo2 datasets. [ABSTRACT FROM AUTHOR]

Density based clustering methods are proposed for clustering spatial databases with noise. Density Based Spatial Clustering of Applications with Noise (DBSCAN) can discover clusters of arbitrary shape and also handles outliers effectively. DBSCAN obtains clusters by finding the number of points within the specified distance from a given point. It involves computing distances from given point to all other points in the dataset. The conventional index based methods construct a hierarchical structure over the dataset to speed-up the neighbor search operations. The hierarchical index-structures fail to scale for datasets of dimensionality above 20. In this paper, we propose a novel graph-based index structure method Groups that accelerates the neighbor search operations and also scalable for high dimensional datasets. Experimental results show that the proposed method improves the speed of DBSCAN by a factor of about 1.5–2.2 on benchmark datasets. The performance of DBSCAN degrades considerably with noise due to unnecessary distance computations introduced by noise points while the proposed method is robust to noise by pruning out noise points early and eliminating the unnecessary distance computations. The cluster results produced by our method are exactly similar to that of DBSCAN but executed at a much faster pace. [ABSTRACT FROM AUTHOR]

Despite the increased availability of high resolution satellite image data, their operational use for mapping urban land cover in Sub-Saharan Africa continues to be limited by lack of computational resources and technical expertise. As such, there is need for simple and efficient image classification techniques. Using Bamenda in North West Cameroon as a test case, we investigated two completely unsupervised pixel based approaches to extract tree/shrub (TS) and ground vegetation (GV) cover from an IKONOS derived soil adjusted vegetation index. These included: (1) a simple Jenks Natural Breaks classification and (2) a two-step technique that combined the Jenks algorithm with agglomerative hierarchical clustering. Both techniques were compared with each other and with a non-linear support vector machine (SVM) for classification performance. While overall classification accuracy was generally high for all techniques (>90%), One-Way Analysis of Variance tests revealed the two step technique to outperform the simple Jenks classification in terms of predicting the GV class. It also outperformed the SVM in predicting the TS class. We conclude that the unsupervised methods are technically as good and practically superior for efficient urban vegetation mapping in budget and technically constrained regions such as Sub-Saharan Africa. [ABSTRACT FROM AUTHOR]

We investigate the feasibility of using the Dynamic Time Warping (DTW) technique to cluster continuous GNSS displacements in Taiwan. Using the DTW distance as the measure for waveform similarity, we combine the DTW method with the Hierarchical Agglomerative Clustering (HAC) algorithm. This is in contrast to the conventional clustering approach that uses the Euclidean distance, considering the average long-term crustal motion, but inherently neglects full-waveform temporal variations. Here we apply the DTW-based HAC algorithm adopting DTW distance as the waveform similarity measure on 11 years worth of 3-D displacement data from 115 continuous GNSS network stations in Taiwan. We demonstrate the efficacy of the DTW-based HAC method in distinguishing the GNSS spatiotemporal variabilities that are consistent with the known, complex tectonic behavior of the region. An open-source Python package has been developed and made available to perform the HAC analysis. • Dynamic Time Warping (DTW) based Hierarchical Agglomerative Clustering (HAC) technique is highly effective in clustering the geodetic time-series data. • We apply the DTW-based HAC algorithm adopting DTW distance as the waveform similarity metric on 11 years' worth of 3-D data from 115 continuous GNSS network stations in Taiwan. • DTW effectively captures the patterns of distinct tectonic motions and potential incoherency in GNSS displacement time series. • An open-source Python package, dtwhaclustering to perform the HAC analysis has been developed and made available.. [ABSTRACT FROM AUTHOR]

• This paper develops a data aggregation technique using optimization model for WSN-based IoT network. The overall procedure involves simulation of WSN-based IoT, multipath routing and finally data aggregation. • At first, the WSN-based IoT nodes are simulated, which are subjected to multipath routing process. The multipath routing is done using DEMFO, which is devised by combining Dolphin Echolation (DE) and Moth fly optimization technique (MFO). From the obtained paths, the data routing is performed wherein data are subjected to base station. • In base station, the data aggregation is done using proposed Hierarchical autoregressive Bidirectional Least-Mean-Square algorithm (HABLMS). The proposed HABLMS is devised by combining Hierarchical autoregressive (HA) using Caviar model and Bidirectional Least-Mean-Square algorithm (BLMS). Thus, the weights are updated in an optimal manner. • Thus, the proposed HABLMS algorithm helps to adapt optimal aggregation of data in WSN-based IoT network. The Internet of Things (IoT) is emerging that acquired the interest of several researchers, as it inspires each object to offer services and data to users by collaborating with each other. The Wireless sensor network (WSN) is a major constituent of IoT in transmitting data. Various wireless sensor-based protocols are developed for path selection. This paper develops a data aggregation technique using an optimization model for WSN-based IoT network. The overall procedure involves simulation of WSN-based IoT, multipath routing and finally, data aggregation. At first, the WSN-based IoT nodes are simulated, which are subjected to a multipath routing process. The multipath routing is done using DEMFO, which is devised by combining Dolphin Echolation (DE) and Moth fly optimization technique (MFO). From the obtained paths, the data routing is performed wherein data are subjected to the base station. In the base station, the data aggregation is done using the proposed Hierarchical autoregressive Bidirectional Least-Mean-Square algorithm (HABLMS). The proposed HABLMS is devised by combining Hierarchical autoregressive (HA) using the Caviar model and Bidirectional Least-Mean-Square algorithm (BLMS). Thus, the weights are updated in an optimal manner. Thus, the proposed HABLMS algorithm helps to adapt optimal aggregation of data in a WSN-based IoT network. The developed DECSA+HABLMS offered superior performance with minimal prediction error of 0.0147. [ABSTRACT FROM AUTHOR]

Community detection from social network data gains much attention from academia and industry since it has many real-world applications. The Girvan–Newman (GN) algorithm is a divisive hierarchical clustering algorithm for community detection, which is regarded as one of the most popular algorithms. It exploits the concept of edge betweenness to divide a network into multiple communities. Though it is being widely used, it has limitations in supporting large-scale networks since it needs to calculate the shortest path between every pair of vertices in a network. In this paper, we develop two parallel versions of the GN algorithm to support large-scale networks. First, we propose a new algorithm, which we call S hortest P ath B etweenness M ap R educe A lgorithm (SPB-MRA), that utilizes the MapReduce model. Second, we propose another new algorithm, which we call S hortest P ath B etweenness V ertex- C entric A lgorithm (SPB-VCA), that utilizes the vertex-centric model. An approximation technique is also developed to further speed up community detection processes. We implemented SPB-MRA using Hadoop and SPB-VCA using GraphChi, and then evaluated the performance of SPB-MRA on Amazon EC2 instances and that of SPB-VCA on a single commodity PC. The evaluation results showed that the elapsed time of SPB-MRA decreased almost linearly as the number of reducers increased, SPB-VCA outperformed SPB-MRA just on a single PC by 4–6 times, and the approximation technique introduced negligible errors. [ABSTRACT FROM AUTHOR]

The Automatic Quasi-Clique Merger algorithm is a new algorithm adapted from early work published under the name QCM (introduced by Ou and Zhang (2007)). The AQCM algorithm performs hierarchical clustering in any data set for which there is an associated similarity measure quantifying the similarity of any data i and data j. Importantly, the method exhibits two valuable performance properties: (1) the ability to automatically return either a larger or smaller number of clusters depending on the inherent properties of the data rather than on a parameter. (2) the ability to return a very large number of relatively small clusters automatically when such clusters are reasonably well defined in a data set. In this work we present the general idea of a quasi-clique agglomerative approach, provide the full details of the mathematical steps of the AQCM algorithm, and explain some of the motivation behind the new methodology. The main achievement of the new methodology is that the agglomerative process now unfolds adaptively according to the inherent structure unique to a given data set, and this happens without the time-costly parameter adjustment that drove the previous QCM algorithm. For this reason we call the new algorithm automatic. We provide a demonstration of the algorithm's performance at the task of community detection in a social media network of 22,900 nodes. [ABSTRACT FROM AUTHOR]

This paper focuses on density-based clustering, particularly the Density Peak (DP) algorithm and the one based on density-connectivity DBSCAN; and proposes a new method which takes advantage of the individual strengths of these two methods to yield a density-based hierarchical clustering algorithm. We first formally define the types of clusters DP and DBSCAN are designed to detect; and then identify the kinds of distributions that DP and DBSCAN individually fail to detect all clusters in a dataset. These identified weaknesses inspire us to formally define a new kind of clusters and propose a new method called DC-HDP to overcome these weaknesses to identify clusters with arbitrary shapes and varied densities. In addition, the new method produces a richer clustering result in terms of hierarchy or dendrogram for a better understanding of cluster structures. Our empirical evaluation results show that DC-HDP produces the best clustering results on 28 datasets in comparison with 8 state-of-the-art clustering algorithms. • Formalising a new type of clusters that DP clustering algorithms is designed to detect. • Introducing a different view of DP clustering as a hierarchical procedure. • Proposing a density-connected hierarchical DP to overcome the identified weaknesses. • DC-HDP produces the best clustering results on 28 synthetic and real datasets. [ABSTRACT FROM AUTHOR]

News recommendations distinguishes from general content recommendations as it takes in consideration news freshness, sparsity, monotony and time. Recent works approach these features using hybrid Collaborative-Content-based Filtering methods, adapting clustering techniques to handle sparsity and monotony without considering the effects that different clustering methods may have over recommendation results. Such studies often evaluate the results of varying different parameters individually, ignoring possible interaction effects between them. They also base their results on metrics such as accuracy and recall that are sensitive to bias. To investigate the importance of clustering method selection to News Recommender System results we evaluated the effects of different traditional techniques in recommending news articles. We implemented an algorithm that used a hybrid Collaborative-Content-based Filtering method to incorporate user behavior, user interest, article popularity and time effect. The system uses an article selection method that built the recommendation set based on content features. With this algorithm, we examined the existence of interaction effects between the input parameters. We used a Gaussian regression process to explore the response surface while sequentially optimizing parameters. To avoid being misled by underlying biases we used Informedness, an accuracy metric that captures both positive and negative information from prediction results. Our results demonstrated that different clustering methods had a significant influence on the recommendation results. It was also found that a traditional hierarchical method outperformed optimization methods with important performance improvement. In addition, we demonstrated that parameters may interact with each other and that analyzing them separately may mislead interpretation. • The clustering method choice considerably affects News Recommender Systems results. • Interaction effect may mislead results when single parameter optimization is used. • Traditional hierarchical clustering methods may outperform optimization methods. [ABSTRACT FROM AUTHOR]

Clustering non-Euclidean data is difficult, and one of the most used algorithms besides hierarchical clustering is the popular algorithm Partitioning Around Medoids (PAM), also simply referred to as k -medoids clustering. In Euclidean geometry the mean – as used in k -means – is a good estimator for the cluster center, but this does not exist for arbitrary dissimilarities. PAM uses the medoid instead, the object with the smallest dissimilarity to all others in the cluster. This notion of centrality can be used with any (dis-)similarity, and thus is of high relevance to many domains and applications. A key issue with PAM is its high run time cost. We propose modifications to the PAM algorithm that achieve an O (k) -fold speedup in the second ("SWAP") phase of the algorithm, but will still find the same results as the original PAM algorithm. If we relax the choice of swaps performed (while retaining comparable quality), we can further accelerate the algorithm by eagerly performing additional swaps in each iteration. With the substantially faster SWAP, we can now explore faster initialization strategies, because (i) the classic ("BUILD") initialization now becomes the bottleneck, and (ii) our swap is fast enough to compensate for worse starting conditions. We also show how the CLARA and CLARANS algorithms benefit from the proposed modifications. While we do not study the parallelization of our approach in this work, it can easily be combined with earlier approaches to use PAM and CLARA on big data (some of which use PAM as a subroutine, hence can immediately benefit from these improvements), where the performance with high k becomes increasingly important. In experiments on real data with k = 100 , 200 , we observed a 458 × respectively 1191 × speedup compared to the original PAM SWAP algorithm, making PAM applicable to larger data sets, and in particular to higher k. • Faster k-Medoids (PAM) clustering algorithm. • Scalable for large number of clusters (large k). • Sampling-based approximations for large data sets (large n). • Same quality as previous state-of-the-art techniques (PAM). • Included in popular clustering tools such as ELKI and R. [ABSTRACT FROM AUTHOR]

The use of the hesitant fuzzy linguistic term sets (HFLTSs) has recently become an important trend in fuzzy decision making, and aggregating HFLTSs and their extensions has now become crucial for making decisions. Previous approaches to aggregating possibility distributions for HFLTSs were based on the paradigm of computing with words, whereas few proposals have been made to aggregate HFLTS possibility distributions under the framework of statistical data analysis so as to reduce information loss and distortion. An initial attempt was the similarity-measure-based agglomerative hierarchical clustering (SM-AggHC) two-stage aggregation paradigm for HFLTS possibility distributions, which, however, presents some important performance limitations from time complexity and memory requirement perspectives. Thereby, this paper introduces a new approach, so called, " N -two-stage algorithmic aggregation paradigm driven by the K -means clustering" (N2S-KMC) to overcome these limitations by cardinality reduction in the first stage of the aggregation process. The subsequent stage uses the similarity-measure-based K -means clustering algorithm to outperform the SM-AggHC algorithm. Such an outperformance, from run time and memory usage, is demonstrated by experimental results. [ABSTRACT FROM AUTHOR]

Clustering is the process of partitioning objects of a dataset into some groups according to similarities and dissimilarities between its objects. DBSCAN is one of the most important clustering algorithms in the density based approach of clustering. In spite of the numerous advantages of the DBSCAN algorithm, it has two important input parameters, MinPts and Eps, which determining their values is still a great challenge. This problem arises because values of these parameters are heavily dependent on data distribution. To overcome this challenge, firstly features of these parameters are investigated and the data distribution are analyzed. Then a DBSCAN-based hierarchical clustering (DBHC) method is proposed in this paper in order to fix this challenge. For this purpose, DBHC first determines values of these parameters using the notion of k nearest neighbor and k-dist plot. Because most of the real world data is not distributed uniformly, it is needed to be produced several values for the Eps parameter. Then, DBHC executes the DBSCAN algorithm several times based on the number of Eps produced earlier. Finally, DBHC method merges obtained clusters if the number of produced clusters is larger than the number which has estimated by the user. To evaluate the performance of the DBHC method, several experiments were performed on some of benchmark datasets of UCI database. Obtained results were compared with other previous works. The obtained results consistently showed that the DBHC method led to better results in comparison to the other works. [ABSTRACT FROM AUTHOR]

Data collection through the mobile sink in wireless sensor networks (WSNs) is an effective solution to the hot-spot or sink-hole problem caused by multi-hop routings using the static sink. However, most of the existing research focuses on energy balance of the global network, but ignores the impact of local energy imbalance on the network lifetime. Therefore, this paper proposes an energy-efficient data collection algorithm to extend the network lifetime by balancing inter-cluster and inner-cluster energy (BIIE). In the proposed BIIE, we design an improved hierarchical clustering algorithm to reduce communication costs. To balance the energy between clusters, we design an efficient mechanism to select the optimal rendezvous node (RN) for each cluster and construct the traveling path of the mobile sink to access all RNs by particle swarm optimization (PSO). To balance the energy of the inner cluster, a local re-clustering mechanism is designed according to the residual energy level of sensor nodes. We also conducted simulation tests which confirm that the proposed BIIE can increase the network lifetime by approximately 46% and shorten the path length of the mobile sink by approximately 7% in comparison with other commonly-used algorithms (i.e., WRP and EAPC). • Proposal of a new algorithm for constructing energy-balanced path of mobile sink in wireless sensor networks. • Presentation of a new particle encoding method. • Emphasis on the local network energy. • Acquisition of better performance than commonly-used algorithms through simulations. [ABSTRACT FROM AUTHOR]

• A novel MS-RFKM clustering method based on shape features is proposed to depict the energy consumption pattern. • A new distance measure approach is proposed to measure the similarity of time series sequences. • The optimal numbers of clusters can be adaptively determined by integrating hierarchical clustering into the proposed MS-RFKM framwork. • The proposed MS-RFKM improves the clustering accuracy with strong robustness and stability. Clustering algorithms have been proven to be an effective method to identify representative energy consumption patterns, as well as being a pre-processing step for other applications (such as demand response, load prediction). This paper proposes a novel moving shape-based robust fuzzy K-modes (MS-RFKM) clustering method, aiming to accurately identify shape patterns in time-series sequences. Specifically, a novel distance measurement-shape feature matrix (SFM) is proposed, which is directly derived from the original load profiles and can accurately depict the shape features of load profiles. Besides, SFM helps to reduce the computation complexity and decrease the adverse impact of noise/ amplitude distortion. Meanwhile, the number of clusters is optimally determined by integrating moving procedure of hierarchical algorithm into the proposed shape-based robust fuzzy K-modes (S-RFKM) method. And the optimal centroids of clusters can be optimally fixed by dynamic time warping (DTW) based fuzzy K-modes (D-FKM). The presented algorithm is validated using users' metering data from China. The simulation results demonstrate that the proposed method can better capture the energy usage patterns and improve the clustering stability and robustness, compared with conventional clustering methods. [ABSTRACT FROM AUTHOR]

We present an efficient computational approach composed of forward and inverse analyses that can neatly capture multiple flaw clusters with high accuracy yet low computational effort. The inverse analysis consists of three continuous steps with distinct targets. Firstly, a combination of improved discrete artificial bee colony (IDABC) algorithm and hierarchical clustering analysis (HCA) is applied to seize sub-domains that contain flaw clusters with a conservative number of sensors. Secondly, additional sensors are introduced into the captured sub-domains so that flaw numbers and their rough geometries can be achieved by the IDABC-HCA optimizer. Eventually, the initial guesses of approximate geometries are turned towards the true geometries by using Broyden-Fletcher-Goldfarb-Shanno approach. In addition, a "Queue and Kill" method is used to determine and eliminate incorrect candidate flaws and/or clusters of flaws in the first two steps. The forward analysis is solved through a combination of dynamic extended finite element method (XFEM) enhanced by local mesh refinement, which highly matches the three-step strategy. The local mesh refinement not only improves the computational accuracy of concerning areas, i.e., sub-domains, but also controls the computation amount. Numerical examples demonstrate the effectiveness and robustness of the developed method in detecting multiple flaw clusters. • An effective and accurate computational method is developed for detection of multiple complicated flaw clusters. • The approach is composed of forward and inverse analyses. • The inverse analysis is divided into three continuous steps with distinct targets. • The forward analysis is solved through a dynamic XFEM with local mesh refinement, which matches the three-step strategy. • The local mesh refinement realizes high accuracy with low computational efforts. [ABSTRACT FROM AUTHOR]

• Timeseries representational method that presents the collective trend of the data. • A user defined segmentation method of time-series data. • Probabilistic approach of forming representational building blocks for timeseries. • Unsupervised trend based hierarchical clustering of timeseries data. Time series data is a sequence of values recorded systematically over a period which are mostly used for prediction, clustering, and analysis. The two essential features of a time series data are trend and seasonality. Preprocessing of the time series data is necessary for performing prediction tasks. In most of the cases, the trend and the seasonality are removed before applying the regression algorithms. The accuracy of such algorithms depends upon the functions used for the removal of trend and seasonality. Clustering of an unlabeled time series data with the presence of trend and seasonality is challenging. In this paper, we propose a probabilistic representational learning method for grouping the time series data. We introduce five terminologies in our method of clustering namely the trendlets, uplets, downlets, equalets and trendlet string. These elements are the representational building blocks of our proposed method. Experiments on the proposed algorithm are performed with the renewable energy data on the electricity supply system of continental Europe which includes the demand and inflow of renewable energy for the term 2012 to 2014 and UCR-2018 time series archive containing 128 datasets. We compared our proposed representational method with various clustering algorithms using the silhouette score. Mini-batch k-means and agglomerative hierarchical clustering algorithms show better performance in terms of quality, logical accordance with data and time taken for clustering. [ABSTRACT FROM AUTHOR]