Search

The entity reconciliation (ER) problem aroused much interest as a research topic in today's Big Data era, full of big and open heterogeneous data sources. This problem poses when relevant information on a topic needs to be obtained using methods based on: (i) identifying records that represent the same real world entity, and (ii) identifying those records that are similar but do not correspond to the same real-world entity. ER is an operational intelligence process, whereby organizations can unify different and heterogeneous data sources in order to relate possible matches of non-obvious entities. Besides, the complexity that the heterogeneity of data sources involves, the large number of records and differences among languages, for instance, must be added. This paper describes a Systematic Mapping Study (SMS) of journal articles, conferences and workshops published from 2010 to 2017 to solve the problem described before, first trying to understand the state-of-the-art, and then identifying any gaps in current research. Eleven digital libraries were analyzed following a systematic, semiautomatic and rigorous process that has resulted in 61 primary studies. They represent a great variety of intelligent proposals that aim to solve ER. The conclusion obtained is that most of the research is based on the operational phase as opposed to the design phase, and most studies have been tested on real-world data sources, where a lot of them are heterogeneous, but just a few apply to industry. There is a clear trend in research techniques based on clustering/blocking and graphs, although the level of automation of the proposals is hardly ever mentioned in the research work. [ABSTRACT FROM AUTHOR]

The arrival of new technologies related to smart grids and the resulting ecosystem of applications and management systems pose many new problems. The databases of the traditional grid and the various initiatives related to new technologies have given rise to many different management systems with several formats and different architectures. A heterogeneous data source integration system is necessary to update these systems for the new smart grid reality. Additionally, it is necessary to take advantage of the information smart grids provide. In this paper, the authors propose a heterogeneous data source integration based on IEC standards and metadata mining. Additionally, an automatic data mining framework is applied to model the integrated information. [ABSTRACT FROM AUTHOR]

Recommender systems try to predict the preferences of users for specific items, based on an analysis of previous consumer preferences. In this paper, we propose SCoR, a Synthetic Coordinate based Recommendation system which is shown to outperform the most popular algorithmic techniques in the field, approaches like matrix factorization and collaborative filtering. SCoR assigns synthetic coordinates to nodes (users and items), so that the distance between a user and an item provides an accurate prediction of the user’s preference for that item. The proposed framework has several benefits. It is parameter free, thus requiring no fine tuning to achieve high performance, and is more resistance to the cold-start problem compared to other algorithms. Furthermore, it provides important annotations of the dataset, such as the physical detection of users and items with common and unique characteristics as well as the identification of outliers. SCoR is compared against nine other state-of-the-art recommender systems, sever of them based on the well known matrix factorization and two on collaborative filtering. The comparison is performed against four real datasets, including a brief version of the dataset used in the well known Netflix challenge. The extensive experiments prove that SCoR outperforms previous techniques while demonstrating its improved stability and high performance. [ABSTRACT FROM AUTHOR]

As a new data representation in the big data era, a multi-scale decision information system (MSDIS) realizes "multi-level, multi-angle and multi-view" evaluations in various problems. However, most of existing multi-scale data analysis models are built for complete information systems (CISs), whereas the research on incomplete multi-scale decision information systems (I-MSDISs) is not perfect. Additionally, the irrational behavior of one decision-maker (DM) often has an impact on decision outcomes. On this basis, this paper develops a prospect-regret theory-based three-way decision (3WD) model with intuitionistic fuzzy numbers (IFNs) under an I-MSDIS, which is abbreviated as PR-3WD-I-MSDIS. Specifically, we first select the optimal scale combination of I-MSDISs by the matrix theory to complete the information extraction of incomplete optimal sub-systems. Then, we further propose an attribute weight evaluation strategy under the incomplete optimal subsystem, which fully considers the neighborhood information of each object. Subsequently, the concept and calculation method of aggregated weighted fuzzy conditional probabilities are constructed from the perspective of intuitionistic fuzzy sets. Furthermore, a prospect-regret theory-based relative profit function is proposed, which sufficiently takes into account the loss and utility in decision-making processes. Finally, a novel three-way classification and ranking method is developed for solving incomplete multi-scale problems. The experimental results on real-world datasets demonstrate that the PR-3WD-I-MSDIS model achieves excellent performance. • A prospect-regret theory-based 3WD in an I-MSDIS is designed. • An aggregated weighted fuzzy conditional probability is proposed. • Relative profit function of each object is defined by comprehensively two perspectives. • The selection of the optimal scale combination by matrix theory is explored. [ABSTRACT FROM AUTHOR]

• A multi-output soft sensor modeling method is established. • A self-coding NN for mining the relationship between multivariable time series. • The MOSVR achieve the data prediction of multiple product quality indicators. • A black hole algorithm with Levy flight is used to optimize the parameters. • The simulation dataset of the penicillin program is used for model verification. The product quality indicators of the penicillin fermentation process have multiple semantics and are interrelated. There is a complex nonlinear mapping relationship between input characteristics and multiple-output objectives, and the time dependence is strong. As a result, the prediction accuracy of existing soft sensor models is poor, and it is difficult to meet the needs of industrial sites. To solve the above problems, this paper proposes a multi-output soft sensor modeling method for the penicillin fermentation process based on big data feature analysis. In this method, the process data is divided into several batches in order, and then the data features of multivariable and time-dependent datasets are extracted according to the deep sparse self-coding neural network method to realize the effective mining of the relationship between multivariable time series factors, based on the multi-output support vector regression method, several soft sensor models for different prediction targets are established. Meanwhile, to improve the Predictive performance of the soft sensor model, the improved black hole algorithm is used to optimize the model parameters. Finally, a simulation experiment is carried out based on the simulation dataset of the penicillin fermentation process to verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Customer support constitutes the face of a brand and plays a critical role in shaping customers retention and loyalty. To facilitate customer contacts and to reduce the corresponding operating costs, organizations resort to self-service interaction channels. However, dealing with self-service customer support platforms has been identified to be one of the most frustrating aspects of a poor customer contact experience. This paper builds on real data from contact centers of a major U.S. insurance company, covering about 10 million calls, to study transfer rate at the interactive voice-response platform. We develop an empirical model to analyze features that exert significant impact on the likelihood that a call arrived to the platform is eventually transferred to a live agent. We find evidence that caller types, specific caller intents, trying distinct channels, and payment attempts can help predicting the transfer outcome. Our study confirms the importance of a caller's location attribute and evinces that the size and composition of the set of effective features and their directions vary substantially across different states. Our results provide managers with several actionable insights to enhance the customer contact experience. For example, our feature selection analysis enables designing personalized interactive voice-response systems based on customer-contact features. Identifying those calls expected to be transferred at any time can guide recruiting and scheduling a workforce with appropriate skill sets. In addition, the transfer rate is a key input variable for various decision making problems arising in the management of call center operations. • Analyzing feature selection on individual-level call transfer rate using big data. • Developing a machine learning model to predict a customer's call transfer outcome. • Formulating actionable insights to introduce personalized customer support systems. [ABSTRACT FROM AUTHOR]

• Propose probability based knowledge reasoning (case retrieval) method. • Propose a new parallel data processing method for big data processing. • Propose a new probabilistic learning algorithm for adapting to big data. To develop an advanced CBR system to well adapt to the intelligence implementation of new engineering process in the big data environment, Bayesian network (BN) model is introduced to CBR system for knowledge reasoning. However, as engineering application is becoming more and more complicated, the number of parameters used to define engineering application grows larger and larger, leading to the seriously reduced efficiency as well as the accuracy of the integrated model. For the problem of reduced efficiency, this paper proposes In-External (IE) algorithm to perform the assignment of big data distribution for parallel data processing, which can fully utilize the capacity of Hadoop system and attain the best efficiency of knowledge reasoning. For the problem of reduced accuracy, in view of the fact that traditional probability learning methods are unfit for the proposed CBR system, this paper proposes Discount Exponential Coefficients of Multivariate Beta Distribution (DECMBD) algorithm to conduct the probability learning of proposed system. In DECMBD algorithm, a discount ratio is given to each exponential coefficient of multivariate Beta distribution to improve the occurrence times counting of all problem features and then gain better effect of probability learning. Finally, lots of experiments are performed to validate the effectiveness of the proposed advanced CBR system. [ABSTRACT FROM AUTHOR]

The rapid growth of social network services has produced a considerable amount of data, called big social data. Big social data are helpful for improving personalized recommender systems because these enormous data have various characteristics. Therefore, many personalized recommender systems based on big social data have been proposed, in particular models that use people relationship information. However, most existing studies have provided recommendations on special purpose and single-domain SNS that have a set of users with similar tastes, such as MovieLens and Last.fm; nonetheless, they have considered closeness relation. In this paper, we introduce an appropriate measure to calculate the closeness between users in a social circle, namely, the friendship strength. Further, we propose a friendship strength-based personalized recommender system that recommends topics or interests users might have in order to analyze big social data, using Twitter in particular. The proposed measure provides precise recommendations in multi-domain environments that have various topics. We evaluated the proposed system using one month's Twitter data based on various evaluation metrics. Our experimental results show that our personalized recommender system outperforms the baseline systems, and friendship strength is of great importance in personalized recommendation. [ABSTRACT FROM AUTHOR]

In financial markets, it is both important and challenging to forecast the daily direction of the stock market return. Among the few studies that focus on predicting daily stock market returns, the data mining procedures utilized are either incomplete or inefficient, especially when a large amount of features are involved. This paper presents a complete and efficient data mining process to forecast the daily direction of the S&P 500 Index ETF (SPY) return based on 60 financial and economic features. Three mature dimensionality reduction techniques, including principal component analysis ( PCA ), fuzzy robust principal component analysis ( FRPCA ), and kernel-based principal component analysis ( KPCA ) are applied to the whole data set to simplify and rearrange the original data structure. Corresponding to different levels of the dimensionality reduction, twelve new data sets are generated from the entire cleaned data using each of the three different dimensionality reduction methods. Artificial neural networks ( ANN s) are then used with the thirty-six transformed data sets for classification to forecast the daily direction of future market returns. Moreover, the three different dimensionality reduction methods are compared with respect to the natural data set. A group of hypothesis tests are then performed over the classification and simulation results to show that combining the ANN s with the PCA gives slightly higher classification accuracy than the other two combinations, and that the trading strategies guided by the comprehensive classification mining procedures based on PCA and ANN s gain significantly higher risk-adjusted profits than the comparison benchmarks, while also being slightly higher than those strategies guided by the forecasts based on the FRPCA and KPCA models. [ABSTRACT FROM AUTHOR]

• Developing a framework for the wellness tourism supply chain. • Designing a mathematical model for the wellness tourism supply chain. • Observing the concept of sustainability in wellness tourism. • Using the 3 V big data concept in data collection. • Considering a real case study from Sarein spa in Iran. Nowadays, the most successful and prominent wellness tourism centers are those that, consider the aspects of sustainable development, while providing memorable services with the best quality and reasonable prices. A wellness tourism center is seeking to meet the purposeful journey of wellness tourists by improving their mental, physical, and spiritual happiness. This paper presents a three-objective mathematical model to formulate the supply chain of services provided by a wellness tourism center. The purpose of this model is to establish a balance between sustainability aspects in this supply chain. The first objective seeks to maximize profits from the supply chain and the second and third objectives are to maximize customer satisfaction concerning environmental and social impacts. The proposed model includes quantitative parameters of operating costs, capacity increase cost, service capacity, and service demand. Due to the multi-objective nature of the model, the improved multi-choice goal programming method is used. Real data are collected from the Ardabil-Sarein wellness tourism center in Iran and used as a case study to validate and analyze the model. In collecting the dataset for the parameters, the three aspects of big data (3 V's) are considered. Findings show that the model offers different types of discounts for various periods following the tourist demand. Additionally, the wellness tourism center can use the original cost to expand its range of services by offering different tours. Finally, the limitations and managerial suggestions are discussed. [ABSTRACT FROM AUTHOR]

• Two algorithms that determine different initial cluster centers are proposed. • It is aimed to perform clustering by considering existing fixed cluster centers. • The proposed algorithms are applied to real-life spatial data having fixed centers. • FC-Kmeans converged to K-means performance at fixed center ratio of 43% and below. Clustering is one of the data mining methods that partition large-sized data into subgroups according to their similarities. K-means clustering algorithm works well in spherical or convex data distribution of large-sized data sets. Most of the algorithms based on K-means have generally been interested in an initial cluster centers selection or cluster distribution. However, these algorithms may not meet satisfy some requirements in practice. This paper presents the FC-Kmeans algorithm, which enables clustering by fixing some cluster centers considering real conditions. Thus, while some of the cluster centers are fixed, it is tried to obtain the most appropriate cluster centers for the others and the best distribution of the data to the clusters. The K-means clustering algorithm is compared with two different fixed-centered clustering algorithms which are FC-Kmeans and FC-Kmeans 2. The experimental results show that although the FC-Kmeans algorithm has more limitations than K-means, it converges the performance of K-means algorithm according to some performance indicators such as SSE, DB Index and Silhouette Index. [ABSTRACT FROM AUTHOR]

Building accurate classifiers for predicting group membership is made difficult when using data that is skewed or imbalanced which is typical of real world data sets. The classifier has a tendency to be biased towards the over represented or majority group as a result. Re-sampling techniques offer simple approaches that can be used to minimize the effect. Over-sampling methods aim to combat class imbalance by increasing the number of minority group samples also refereed to as members of the minority group. Over the last decade SMOTE based methods have been used and extended to overcome this problem. There has been little emphasis on improvements to this approach with consideration to data intrinsic properties beyond that of class imbalance alone. In this paper we introduce modifications to a priori based methods Safe Level OUPS and OUPS that result in improvement for sensitivity measures over competing approaches using the SMOTE based method such as the Local neighborhood extension to SMOTE (LN-SMOTE), Borderline-SMOTE and Safe-Level-SMOTE. [ABSTRACT FROM AUTHOR]

Cluster analysis, or clustering, refers to the analysis of the structural organization of a data set. This analysis is performed by grouping together objects of the data that are more similar among themselves than to objects of different groups. The sampled data may be described by numerical features or by a symbolic representation, known as categorical features. These features often require a transformation into numerical data in order to be properly handled by clustering algorithms. The transformation usually assigns a weight for each feature calculated by a measure of importance (i.e., frequency, mutual information). A problem with the weight assignment is that the values are calculated with respect to the whole set of objects and features. This may pose as a problem when a subset of the features have a higher degree of importance to a subset of objects but a lower degree with another subset. One way to deal with such problem is to measure the importance of each subset of features only with respect to a subset of objects. This is known as co-clustering that, similarly to clustering, is the task of finding a subset of objects and features that presents a higher similarity among themselves than to other subsets of objects and features. As one might notice, this task has a higher complexity than the traditional clustering and, if not properly dealt with, may present an scalability issue. In this paper we propose a novel co-clustering technique, called HBLCoClust, with the objective of extracting a set of co-clusters from a categorical data set, without the guarantees of an enumerative algorithm, but with the compromise of scalability. This is done by using a probabilistic clustering algorithm, named Locality Sensitive Hashing, together with the enumerative algorithm named InClose. The experimental results are competitive when applied to labeled categorical data sets and text corpora. Additionally, it is shown that the extracted co-clusters can be of practical use to expert systems such as Recommender Systems and Topic Extraction. [ABSTRACT FROM AUTHOR]

Text summarization and classification are core techniques to analyze a huge amount of text data in the big data environment. Moreover, as the need to read texts on smart phones, tablets and television as well as personal computers continues to grow, text summarization and classification techniques become more important and both of them do essential processes for text analysis in many applications. Traditional text summarization and classification techniques have individually been considered as different research fields in this literature. However, we find out that they can help each other as text summarization makes use of category information from text classification and text classification does summary information from text summarization. Therefore, we propose an effective integrated learning framework using both of summary and category information in this paper. In this framework, the feature-weighting method for text summarization utilizes a language model to combine feature distributions in each category and text, and one for text classification does the sentence importance scores estimated from the text summarization. In the experiments, the performances of the integrated framework are better than ones of individual text summarization and classification. In addition, the framework has some advantages of easy implementation and language independence because it is based on only simple statistical approaches and POS tagger. [ABSTRACT FROM AUTHOR]

Probabilistic topic models could be used to extract low-dimension aspects from document collections, and capture how the aspects change over time. However, such models without any human knowledge often produce aspects that are not interpretable. In recent years, a number of knowledge-based topic models and dynamic topic models have been proposed, but they could not process concept knowledge and temporal information in Wikipedia. In this paper, we fill this gap by proposing a new probabilistic modeling framework which combines both data-driven topic model and Wikipedia knowledge. With the supervision of Wikipedia knowledge, we could grasp more coherent aspects, namely, concepts, and detect the trends of concepts more accurately, the detected concept trends can reflect bursty content in text and people’s concern. Our method could detect events and discover events specific entities in text. Experiments on New York Times and TechCrunch datasets show that our framework outperforms two baselines. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

As user-designer interactive products, smart, connected products (SCPs) have been highly focused. User satisfaction, being an important indicator to judge the performance of products, might be discussed through online customer reviews in the era of Big Data. Therefore, this paper tries to assess user satisfaction of SCPs with the aid of online reviews and then proposes a novel textual data-driven evaluation method. We first mine core attributes and evaluations by topics extraction and sentiment analysis instead of market survey approaches. Combining frequency weights and position weights, we then introduce an integrated approach to obtain the final importance of attributes. Specifically, a new frequency-oriented formula and a new position score-based formula are defined to calculate frequency and position weights, separately. Entire and partial user satisfaction degrees are subsequently analyzed by probabilistic linguistic-regret theory-multiplicative multi-objective optimization by ratio analysis (PL-RT-MULTIMOORA) and a formula with weighted rejoicing–regret values, respectively. Finally, the designed decision-making method is successfully applied to evaluate user satisfaction of smart speakers. Some useful suggestions are given to customers and manufacturers in the light of entire and partial user satisfaction degrees. • Design a textual data-driven evaluation method assessing user satisfaction of SCPs. • Mine core attributes and evaluations by data analysis techniques. • Determine final importance of attributes by frequency weights and position weights. • Entire and partial user satisfaction degrees are both discussed. • Provide some advice to customers and manufacturers. [ABSTRACT FROM AUTHOR]

Social networks have taken an irreplaceable role in our lives. They are used daily by millions of people to communicate and inform themselves. This success has also led to a lot of irrelevant content and even misinformation on social media. In this paper, we propose a user-centred framework to reduce the amount of irrelevant content in social networks to support further stages of data mining processes. The system also helps in the reduction of misinformation in social networks, since it selects credible and reputable users. The system is based on the belief that if a user is credible then their content will be credible. Our proposal uses word embeddings in a first stage, to create a set of interesting users according to their expertise. After that, in a later stage, it employs social network metrics to further narrow down the relevant users according to their credibility in the network. To validate the framework, it has been tested with two real Big Data problems on Twitter. One related to COVID-19 tweets and the other to last United States elections on 3rd November. Both are problems in which finding relevant content may be difficult due to the large amount of data published during the last years. The proposed framework, called NOFACE, reduces the number of irrelevant users posting about the topic, taking only those that have a higher credibility, and thus giving interesting information about the selected topic. This entails a reduction of irrelevant information, mitigating therefore the presence of misinformation on a posterior data mining method application, improving the obtained results, as it is illustrated in the mentioned two topics using clustering, association rules and LDA techniques. • A new framework for filtering irrelevant content in Twitter has been proposed. • First time that word embeddings and bios are used to obtain user expertise. • The framework filters content of social networks keeping only those relevant to the topic intend to study. • The proper performance has been tested on two real problems. [ABSTRACT FROM AUTHOR]

Big Data Cyber Security Analytics (BDCA) systems leverage big data technologies to collect, store, and analyze a large volume of security event data for detecting cyber-attacks. Big data analytical frameworks (e.g, Apache Hadoop and Apache Spark) are used for building BDCA systems. These frameworks come with dozens of configurable parameters that need to be tuned for each security dataset to use the BDCA system to the maximum of its ability. Manually tuning these many parameters for each dataset is a time-consuming and resource-intensive task. Therefore, tuning techniques are proposed to automatically tune framework parameters. However, previous tuning techniques not only require users to manually initiate and execute the tuning process but are also computationally expensive. In this paper, we present ADAPTER , a parameter-driven adaptation system that automatically triggers the tuning process, tunes the framework's configuration parameters for different security datasets, and finally executes the BDCA system with the adapted configuration. ADAPTER uses empirically designed fuzzy rules that tune each configuration parameter based on the parameter's impact on resource utilization such as CPU and disk utilization. We have evaluated ADAPTER for fully distributed Spark-based and Hadoop-based BDCA systems using five security datasets and several adaptation scenarios. Our evaluation shows that with respect to default settings, ADAPTER (i) reduces the training time by 35.9% and 31.1% for Spark and Hadoop, respectively (ii) reduces the testing time by 15.3% and 10.1% for Spark and Hadoop, respectively (iii) improves resource utilization by 36.6% for Spark and 55% for Hadoop (iii) reduces tuning time by 499 times and 135 times for Spark and Hadoop and (iv) improves data locality by 4%. [ABSTRACT FROM AUTHOR]

• The paper proposes a purity-based weighted consensus clustering. • The proposed approach demonstrates the efficient integration of clustering methods. • The best result shows the proposed approach with the squared Euclidean metric. • The proposed approach compensates for the shortcomings of each considered method. • Weighted consensus will allow experts to use it when making group decisions. The aim of this study is the development of a weighted consensus clustering that assigns weights to single clustering methods using the purity utility function. In the case of Big data that does not contain labels, the utility function based on the Davies-Bouldin index is proposed in this paper. The Banknote authentication, Phishing, Diabetic, Magic04, Credit card clients, Covertype, Phone accelerometer, and NSL-KDD datasets are used to assess the efficiency of the proposed consensus approach. The proposed approach is evaluated using the Euclidean, Minkowski, squared Euclidean, cosine, and Chebychev distance metrics. It is compared with single clustering algorithms (DBSCAN, OPTICS, CLARANS, k-means, and shared nearby neighbor clustering). The experimental results show the effectiveness of the proposed approach to the Big data clustering in comparison to single clustering methods. The proposed weighted consensus clustering using the squared Euclidean distance metric achieves the highest accuracy, which is a very promising result for Big data clustering. It can be applied to expert systems to help experts make group decisions based on several alternatives. The paper also provides directions for future research on consensus clustering in this area. [ABSTRACT FROM AUTHOR]

• This paper studies the knapsack problem with setups. • This paper presents an algorithm based on a Lagrangean relaxation of the problem. • We used very large data sets to test the algorithm. • The algorithm can solve near optimally the problem in reasonable amount of time. We consider the knapsack problem with setups which is a generalization of the classical knapsack problem where the items belong to families and an item can be placed in the knapsack only if its family is selected. The problem has received increasing attention by researchers because of its theoretical significance and practical applications related to resource allocation. This paper presents an algorithm based on a Lagrangean relaxation of the problem that produces solutions whose quality can be assessed automatically with the algorithm itself without ever knowing the optimal solutions. We report results of an extensive computational study which show that the method can solve near optimally very large instances of the problem with up to 500 families and 2,000,000 items in reasonable amount of time. This study shows the merit of using the Lagrangean relaxation method to solve the current problem when the constraints to relax are chosen properly. [ABSTRACT FROM AUTHOR]

• We review systematically the hybridization of decision-making techniques for supplier selection. • We summarize minutely the state-of-the-art developments of supplier selection. • We analyze the penetration of big data into supplier selection. • We emphasize and elaborate six aspects of transitions for future researches. Supplier selection (SS) is considered a sophisticated, application-oriented, decision-making (DM) problem and has received considerable attention. In the past two decades, DM theories and techniques continue to be incorporated into and contribute to the development of SS applications. Maintaining the pace of the rapid transitions in this field, this paper systematically reviews the relevant articles published between 2013 and 2018. Articles that orient various DM techniques are selected and analyzed under a well-established framework. State-of-the-art developments in the adoption of DM techniques are summarized in a SS process. We pay particular attention to promising directions that can dominate future research in this field. This paper further extends the history of several interacting fields, including big data and economic theories, toward methodological rather than application dimensions. The potential of such fields for SS is discussed from an interdisciplinary perspective. [ABSTRACT FROM AUTHOR]

• Proposing a fast and accurate version of DBSCAN algorithm for big data. • Calculating density based on a small subset of data called operational set. • Creating and updating the operational set at a very low computational cost. • Comprehensive evaluation of the proposed method in compared to other recent works. Today, data is being generated with a high speed. Managing large volume of data has become a challenge in the current age. Clustering is a method to analyze data that is generated in the Internet. Various approaches have been presented for data clustering until now. Among them, DBSCAN is a most well-known density-based clustering algorithm. This algorithm can detect clusters of different shapes and does not require prior knowledge about the number of clusters. A major part of the DBSCAN run-time is spent to calculate the distance of data from each other to find the neighbors of each sample in the dataset. The time complexity of this algorithm is O(n2); Therefore, it is not suitable for processing big datasets. In this paper, DBSCAN is improved so that it can be applied to big datasets. The proposed method calculates accurately each sample density based on a reduced set of data. This reduced set is called the operational set. This collection is updated periodically. The use of local samples to calculate the density has greatly reduced the computational cost of clustering. The empirical results on various datasets of different sizes and dimensions show that the proposed algorithm increases the clustering speed compared to recent related works while having similar accuracy as the original DBSCAN algorithm. [ABSTRACT FROM AUTHOR]

A number of marketing phenomena are too complex for conventional analytical or empirical approaches. This makes marketing a costly process of trial and error: proposing, imagining, trying in the real world, and seeing results. Alternatively, Agent-based Social Simulation (ABSS) is becoming the most popular approach to model and study these phenomena. This research paradigm allows modeling a virtual market to: design, understand, and evaluate marketing hypotheses before taking them to the real world. However, there are shortcomings in the specialized literature such as the lack of methods, data, and implemented tools to deploy a realistic virtual market with ABSS. To advance the state of the art in this complex and interesting problem, this paper is a seven-fold contribution based on a (1) method to design and validate viral marketing strategies in Twitter by ABSS. The method is illustrated with the widely studied problem of rumor diffusion in social networks. After (2) an extensive review of the related works for this problem, (3) an innovative spread model is proposed which rests on the exploratory data analysis of two different rumor datasets in Twitter. Besides, (4) new strategies are proposed to control malicious gossips. (5) The experimental results validate the realism of this new propagation model with the datasets and (6) the strategies performance is evaluated over this model. (7) Finally, the article is complemented by a free and open-source simulator. [ABSTRACT FROM AUTHOR]

Clustering aims to classify different patterns into groups called clusters . Many algorithms for both hard and fuzzy clustering have been developed to deal with exploratory data analysis in many contexts such as image processing, pattern recognition, etc. However, we are witnessing the era of big data computing where computing resources are becoming the main bottleneck to deal with those large datasets. In this context, sequential algorithms need to be redesigned and even rethought to fully leverage the emergent massively parallel architectures. In this paper, we propose a parallel implementation of the fuzzy minimals clustering algorithm called Parallel Fuzzy Minimal (PFM). Our experimental results reveal linear speed-up of PFM when compared to the sequential counterpart version, keeping very good classification quality. [ABSTRACT FROM AUTHOR]

Classifiers and imputation methods have played crucial parts in the field of big data analytics. Especially, when using data sets characterized by horizontal scattering, vertical scattering, level of spread, compound metric, imbalance ratio and missing ratio, how to combine those classifiers and imputation methods will lead to significantly different performance. Therefore, it is essential that the characteristics of data sets must be identified in advance to facilitate selection of the optimal combination of imputation methods and classifiers. However, this is a very costly process. The purpose of this paper is to propose a novel method of automatic, adaptive selection of the optimal combination of classifier and imputation method on the basis of features of a given data set. The proposed method turned out to successfully demonstrate the superiority in performance evaluations with multiple data sets. The decision makers in big data analytics could greatly benefit from the proposed method when it comes to dealing with data set in which the distribution of missing data varies in real time. [ABSTRACT FROM AUTHOR]

The availability of encyclopedic Linked Open Data (LOD) paves the way to a new generation of knowledge-intensive applications able to exploit the information encoded in the semantically-enriched datasets freely available on the Web. In such applications, the notion of relatedness between entities plays an important role whenever, given a query, we are looking not only for exact answers but we are also interested in a ranked list of related ones. In this paper we present an approach to build a relatedness graph among resources in the DBpedia dataset that refer to the IT domain. Our final aim is to create a useful data structure at the basis of an expert system that, looking for an IT resource, returns a ranked list of related technologies, languages, tools the user might be interested in. The graph we created is a basic building block to allow an expert system to support the user in entity search tasks in the IT domain (e.g. software component search or expert finding) that goes beyond string matching typical of pure keyword-based approaches and is able to exploit the explicit and implicit semantics encoded within LOD datasets. The graph creation relies on different relatedness measures that are combined with each other to compute a ranked list of candidate resources associated to a given query. We validated our tool through experimental evaluation on real data to verify the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper describes the design and implementation of the Data Quality Query System (DQ 2 S), a query processing framework and tool incorporating data quality profiling functionality in the processing of queries involving quality-aware query language extensions. DQ 2 S supports the combination of performance and quality-oriented query optimizations, and a query processing platform that enables advanced data profiling queries to be formulated based on well established query language constructs, often used to interact with relational database management systems. DQ 2 S encompasses a declarative query language and a data model that provides users with the capability to express constraints on the quality of query results as well as query quality-related information; a set of algebraic operators for manipulating data quality-related information, and optimization heuristics. The proposed query language and algebra represent seamless extensions to SQL and relational database engines, respectively. The constructs of the proposed data model are implemented at the user’s view level and are internally mapped into relational model constructs. The quality-aware extensions and features are extremely useful when users need to assess the quality of relational data sets and define quality constraints for acceptable data prior to using candidate data sources in decision support systems and conducting big data analytical tasks. [ABSTRACT FROM AUTHOR]

The recent developments in cellular networks, along with the increase in services, users and the demand of high quality have raised the Operational Expenditure (OPEX). Self-Organizing Networks (SON) are the solution to reduce these costs. Within SON, self-healing is the functionality that aims to automatically solve problems in the radio access network, at the same time reducing the downtime and the impact on the user experience. Self-healing comprises four main functions: fault detection, root cause analysis, fault compensation and recovery. To perform the root cause analysis (also known as diagnosis), Knowledge-Based Systems (KBS) are commonly used, such as fuzzy logic. In this paper, a novel method for extracting the Knowledge Base for a KBS from solved troubleshooting cases is proposed. This method is based on data mining techniques as opposed to the manual techniques currently used. The data mining problem of extracting knowledge out of LTE troubleshooting information can be considered a Big Data problem. Therefore, the proposed method has been designed so it can be easily scaled up to process a large volume of data with relatively low resources, as opposed to other existing algorithms. Tests show the feasibility and good results obtained by the diagnosis system created by the proposed methodology in LTE networks. [ABSTRACT FROM AUTHOR]

The worldwide growth of e-commerce has created new challenges for logistics companies, such as delivering products quickly and cheaply. This paper presents a heuristic to solve the last-mile route creation problem dynamically. The heuristic is based on a multi-agent system integrated with trajectory data mining techniques to extract territorial patterns and use them to solve the Dynamic Capacitated Vehicle Routing Problem with Stochastic Customers. Our solution approach is focused on a linear-time heuristic that depends only on the Warehouse system configurations and not on the total number of packages processed, which is suitable for express delivery logistics companies that must process a large number of packages per day. We compare our proposal with benchmark algorithms from the literature; additionally, we evaluate its performance and robustness under different scenarios. Results show that our solution approach is effective for scenarios in which routes must be set dynamically from a continuous stream of packages. • Trajectory data mining can improve last-mile routes. • Stochastic and dynamic VRP solutions make cross-docking operations more efficient. • Multi-agent systems allow efficiently implementing the dynamic VRP in warehouse systems. [ABSTRACT FROM AUTHOR]

Labeling samples on large data sets is a demanding task prone to different sources of errors. Those errors, denoted as noise, can significantly impact the performance of a classification algorithm due to overfitting of wrongly labeled data. So far, this problem has been treated by avoiding the overfitting and correcting mislabeled data through similarity analysis. The former approach can be affected by the curse of dimensionality and some mislabeled data will not be corrected. In this paper, we investigate the use of a biclustering approach to capture local models of coherence across subsets of instances and attributes. Those models are used to replace and augment the attributes of the original dataset. Through a systematic series of experiments, we have assessed the performance of the proposed approach, referred to as BicNoise , by considering different rates and types of label noise, and also different types of classifiers, binary datasets, and evaluation metrics. The good results achieved suggest that the transformed data can alleviate the dimensionality problem, reduce the redundancy of correlated features and improve the separability of the data, thus improving the classifier performance (most noticeably, in the highest noise settings). [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of mining structured data to find potentially useful patterns by association rule mining. Different than the traditional find-all-then-prune approach, a heuristic method is proposed to extract mostly associated patterns (MASPs). This approach utilizes a maximally-association constraint to generate patterns without searching the entire lattice of item combinations. This approach does not require a pruning process. The proposed approach requires less computational resources in terms of time and memory requirements while generating a long sequence of patterns that have the highest co-occurrence. Furthermore, k -item patterns can be obtained thanks to the sub-lattice property of the MASPs. In addition, the algorithm produces a tree of the detected patterns; this tree can assist decision makers for visual analysis of data. The outcome of the algorithm implemented is illustrated using traffic accident data. The proposed approach has a potential to be utilized in big data analytics. [ABSTRACT FROM AUTHOR]

The relevant documents from large data sets are retrieved with the help of ranking function in Information Retrieval system. In this paper, a new fuzzy logic based ranking function is proposed and implemented to enhance the performance of Information Retrieval system. The proposed ranking function is based on the computation of different terms of term-weighting schema such as term frequency, inverse document frequency and normalization. Fuzzy logic is used at two levels to compute relevance score of a document with respect to the query in present work. All the experiments are performed on CACM and CISI benchmark data sets. The experimental results reveal that the performance of our proposed ranking function is much better than the fuzzy based ranking function developed by Rubens along with other widely used ranking function Okapi-BM25 in terms of precision, recall and F-measure. [ABSTRACT FROM AUTHOR]

Naive Bayes (NB) is a popular machine learning tool for classification, due to its simplicity, high computational efficiency, and good classification accuracy, especially for high dimensional data such as texts. In reality, the pronounced advantage of NB is often challenged by the strong conditional independence assumption between attributes, which may deteriorate the classification performance. Accordingly, numerous efforts have been made to improve NB, by using approaches such as structure extension, attribute selection, attribute weighting, instance weighting, local learning and so on. In this paper, we propose a new Artificial Immune System (AIS) based self-adaptive attribute weighting method for Naive Bayes classification. The proposed method, namely AISWNB, uses immunity theory in Artificial Immune Systems to search optimal attribute weight values, where self-adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. One noticeable advantage of AISWNB is that the unique immune system based evolutionary computation process, including initialization, clone, section, and mutation, ensures that AISWNB can adjust itself to the data without explicit specification of functional or distributional forms of the underlying model. As a result, AISWNB can obtain good attribute weight values during the learning process. Experiments and comparisons on 36 machine learning benchmark data sets and six image classification data sets demonstrate that AISWNB significantly outperforms its peers in classification accuracy, class probability estimation, and class ranking performance. [ABSTRACT FROM AUTHOR]

The conventional Linear Discriminant Analysis (LDA) model has some challenges, such as sensitivity to the outlier, the singularity problem of the within-class scatter matrix, and Gaussian assumption of data within the same class. This paper proposes a robust LDA method that tries to solve the sensitivity to outliers and singularity problems. Specifically, we first use Bayesian risk to design the proposed method optimization problem. Then, the proposed Density-oriented LDA (DLDA) method used the data density as prior knowledge for robustness against outliers. The proposed method can classify non-linear and multi-mode distribution data sets. Furthermore, the proposed method can be employed for big data classification using the AdaBoost approach. Experimental results on synthetic and real data sets demonstrate the proposed DLDA method's superiority over other competing methods. • LDA model has some challenges, e.g., sensitivity to outlier and singularity problem. • Using the data density as prior knowledge for robustness against outliers. • Bayesian risk is used to design the proposed method optimization problem. • The proposed method can be employed for big data using the AdaBoost approach. [ABSTRACT FROM AUTHOR]

• A cellular-based evolutionary fuzzy system for the extraction of emerging patterns in high-speed, massive data streams is proposed. • Smart triggering of the learning method which updates the model only when required. • A reinitialisation and filtering strategy for reducing the extraction of redundant patterns is also defined. • The quality of knowledge extracted outperforms state-of-the-art methods. • The proposed method is able to process batches of data up to 750,000 instances. Today, the number of existing devices generates immense amounts of data on a continuous basis that must be processed by new distributed data stream mining approaches. In this paper we present a new approach for extracting descriptive emerging patterns in massive data streams from different sources through Apache Kafka and Apache Spark Streaming whose objective is to monitor the state of the system with respect to a variable of interest. For this purpose, the proposed algorithm is a cellular-based multi-objective evolutionary fuzzy system that uses an informed strategy for efficient data processing and a re-initialisation and filtering mechanism to eliminate redundant and low-reliable patterns. The experimental study carried out demonstrates an interpretability improvement of 25% in the extraction of high-interest knowledge by the proposed algorithm, which would make it easier for experts to analyse the problem. Finally, the proposed algorithm is up to five times faster than another proposal on the processing of the same amount of data. In this experimental study, up to 750,000 instances have been processed in approximately four seconds. [ABSTRACT FROM AUTHOR]

• Distributed Heterogeneous Ensemble is designed for big data classification. • Classifiers are pruned from the ensemble to increase the diversity. • A Spark version of DHBoost is presented based on MapReduce programming paradigm. • DHBoost outperforms the state-of-the-art ensemble classifiers in the Spark library. In this era of big data, processing large scale data efficiently and accurately has become a challenging problem. Ensemble classification is a type of supervised learning that uses multiple experts to generate the final output. It provides a way to classify data more accurately. As a result of using multiple classifiers, they are often more complicated than single classifiers, especially for big data problems. Apache Spark is a unified analytics engine for big data processing which provides a scalable framework to analyze the data. In this paper, we first extend our previous work and design a distributed heterogeneous ensemble classifier inspired by the boosting approach, which is capable of dealing with big datasets. Using heterogeneous classifiers makes it possible to have more diverse classifiers, and consequently, a more accurate classifier is obtained. Then, we present the Spark version of the proposed approach to speed up our heterogeneous ensemble classifier using the MapReduce paradigm. In order to evaluate our approach, we have applied it to seven big datasets. Extensive experimental results indicate the superiority of the proposed method over the existing ensemble algorithms implemented by Spark MLlib in terms of the classification accuracy, performance, and scalability. [ABSTRACT FROM AUTHOR]

• Some ELMs are used in parallel as experts to map data into discriminative space. • The training data are easily grouped into some clusters in the new feature space. • An index is calculated per group to help finding corresponding groups of a sample. • A tree is used to find corresponding groups of a sample and applying kNN on it. • The method is evaluated on 4 big datasets in absence and presence of noise. kNN algorithm, as an effective data mining technique, is always attended for supervised classification. On the other hand, the previously proposed kNN finding methods cannot be considered as efficient methods for dealing with big data. As there is daily generated and expanded big datasets on different online and offline servers, the efficient methods for such data must be introduced to find kNN. Moreover, massive amounts of data contain more noise and imperfection data samples that significantly increase the need for a robust kNN finding method. In this paper, a new fast and robust kNN finding framework is introduced to deal with the big datasets. In this method, a group of most relevant data samples to an input data sample are detected and the original kNN method is applied on them for finding the final nearest neighbors. The main goal of this method is dealing with the big datasets in an accurate, fast, and robust manner. Here, the training data samples of each label are grouped into some partitions based on the output of some mini-classifiers (i.e. ELM classifier). In fact, the behavior of the mini-classifiers is the basis of partitioning the training data samples. These mini-classifiers are trained using non-overlapping subsets of the training set in the form of each mini-classifier a subset. Here, an index is calculated for each partition to make the corresponding partition finding faster using a tree structure in which each partition index is fallen into a leaf. Then, outputs of the mini-classifiers for an input test sample are used to find the corresponding group of most relevant data samples to the input data sample on the tree. Experimental results indicate that the proposed method has better performance in most cases and comparable performance on other cases of original and noisy big data problems. [ABSTRACT FROM AUTHOR]

• The challenge of algorithm runtime has been addressed. • The graph modification step of the algorithm is focused. • The proposed algorithm has two steps of adding and deleting edges. • It uses an efficient algorithm to select the most appropriate edges. • It improves the algorithm runtime and the utility of the graph. Privacy on social networks is one of the most important and well-known issues. Various algorithms have been proposed to preserve the privacy of social network, all of which try to change the graph structure such that the utility of the graph is maintained. Although these algorithms have been successful in protecting the privacy and the utility of social networks, they are not suitable for anonymizing big data, because of the high cost of processing. Some of these algorithms have a high runtime. In addition, they should be improved from the aspect of preserving graph utility. In this paper, an effective algorithm has been introduced to increase the anonymization speed, as well as improving the graph utility. This algorithm uses number factorization to remove the best edges from the graph in the graph modification step of the algorithm. Since the appropriate edges are selected just through one scan of the edges, the runtime is reduced. In order to add edges to the graph, all the appropriate edges are selected simultaneously and added to the graph using NaFa algorithm. Some limitations are applied to the selection of edges to choose the correct ones. Taking into account the appropriate criteria to select edges in both the removal and addition steps improves the graph utility. The evaluation results of the proposed algorithm on real data sets show the efficiency of the algorithm to sharply reduce the runtime and increase the graph utility, simultaneously. [ABSTRACT FROM AUTHOR]

• Empirical study aimed at "scaling up" data quality (DQ) management applications. • Execution of data quality-aware queries over sensor-collected traffic data sets. • Exploration of Apache Spark and Pandas library to speed-up query execution. • Insights on choice of computational infrastructure to deploy DQ management tools. Combining query processing techniques with data quality management approaches enables enforcement of quality constraints, such as timeliness, accuracy and completeness, as part of ad-hoc query specification and execution, improving the quality of query results. Despite the emergence of novel data quality processing tools, there is a dearth of studies assessing performance and scalability in the execution of data quality assessment tasks during query processing. This paper reports on an empirical study aiming to investigate the extent to which a big data computing framework (Spark) can offer significant gains in performance and scalability when executing data quality querying tasks over a range of computational platforms including a single commodity multi-core machine and a cluster-based platform for a wide range of workloads. Our results show that substantial performance and scalability gains can be obtained by using optimized data science libraries combined with the parallel and distributed capabilities of big data computing. We also provide guidelines on choosing the appropriate computational infrastructure for executing DQ-aware queries. [ABSTRACT FROM AUTHOR]

• Develop ARIMA based time-series anomaly detection approach to handle vehicle operating status identification. • Use immersive Virtual Reality to interpret obtained results. • Use large scale vehicle operating time series dataset to validate proposed approach. Nowadays, with the increasing complexity of vehicle operating environment, vehicle operating reliability becomes growing essential and has attracted significant attention in recent decades. The majority of vehicle operating activities are organized in the format of time series data. Many conventional approaches are proposed to deal with time-series data. Autoregressive Integrated Moving Average (ARIMA) approach is one of the most widely used methods to analyze time-series data. In this paper, an ARIMA-based anomaly detection framework is developed to identify abnormal states of the vehicles based on the multiple-channel operating time series data. The state anomaly is captured by the deviation of real-time values at different channels from the predictions. After abnormal operating states are identified, a novel immersive Virtual Reality (VR) tool is applied to visualize the difference between normal and abnormal status. The combination of ARIMA anomaly detection and VR visualization for vehicle multiple-channel operating time-series data would provide a robust way to present the results of vehicle operating status. A large scale data set of 14 days operational performance channels of a specific vehicle is utilized to validate the performance of the proposed approach. [ABSTRACT FROM AUTHOR]

• Proposed computation framework for Big hyperspectral data discriminate analytics. • Identified Big data challenges in remote sensing applications. • Employed divide-conquer scalability for implementation on parallel architectures. • Discussed transforming the divide-conquer mechanism to be meta -scalable. • Showed discriminate analytics with proposed mechanism can give optimal solution. Recent technology developments in hyperspectral sensing has made it possible to acquire several hundred spectral bands that cover the electromagnetic spectrum of an observational scene in a single acquisition. The resulting hyperspectral data cube contains a large volume of spatial-spectral information. It has several concrete and special characteristics such as being multi-source, multi-scale, high dimensional and nonlinear. The hyperspectral video with temporal information further increases the data generation velocity and volume which lead to the Big data challenges especially in remote sensing applications. We term this type of Big data as Big hyperspectral data to differentiate it from the Big data generated from internet and multimedia-based sources. This paper presents a novel data computation framework for Big hyperspectral data discriminate analytics. This framework consists of some essential modules like tree-based divide-conquer (Tree-DC) mechanism, hierarchical spatial-spectral domain (HSSD) decomposition, global scalable and locally fast discriminative analytics (GSLF-DA), tree-based divide-conquer-merge (DCM), and temporal hyperspectral data decomposition. The challenge of the framework is to sustain the divide-conquer scalability for implementation on rapidly evolving parallel computing architectures i.e., transforming the divide-conquer mechanism to be meta -scalable. Moreover, the discriminate analytics in conjunction with the proposed mechanism can give the optimal solution in the final merging stage. Experiments are performed to validate the performance of the mechanisms in the framework. [ABSTRACT FROM AUTHOR]

• Statistical and semantic features based text clustering technique is proposed. • Combining the statistical and semantic features improves text clustering. • The formation of lexical chains from WordNet captures important terms. • Semantic relations such as synonym and hypernym etc. help to reduce dimensionality. Document clustering in text mining is a problem that is heavily researched upon. It is observed that individual approaches based on statistical features and semantic features have been extensively used to solve this problem. However, techniques combining the advantages of both types of features have not been frequently researched upon. Specifically, when the growth in the size of textual data is immense, there is a need for such an approach that combines the advantages of both types of features to give more accurate results within an acceptable range of time. In this paper, a document clustering technique is proposed that combines the effectiveness of the statistical features (using TF-IDF) and semantic features (using lexical chains). It is designed to use a fewer number of features while maintaining a comparable and even better accuracy for the task of document clustering. [ABSTRACT FROM AUTHOR]

The amount of available multimedia data in different formats and from different sources increases everyday. From an information retrieval point of view, this high volume and heterogeneity of data involves several issues to be addressed related to information overload and lacks of well structured information. Even if modern information retrieval systems offer to the user manifold search options, it is still hard to find systems with optimal performances in the document seeking process starting from a given topic. In recent years, several frameworks have been proposed and developed to support this task based on different models and techniques. In this paper we propose a semantic approach to document classification using both textual and visual topic detection techniques based on deep neural networks and multimedia knowledge graph. A semantic multimedia knowledge base has been exploited and several experimental results show the effectiveness of our proposed approach. • A combination of textual and visual information is suitable for topic detection. • Semantic analysis and deep neural networks are used in our approach. • Knowledge is represented by a Multimedia Knowledge Graph. • A web document collection is a real scenario to test our strategy. [ABSTRACT FROM AUTHOR]