Search

With the rapid growth of massive data in all walks of life, massive data faces enormous challenges such as storage capacity and computing power. In Chinese universities, traditional data analysis of student course cannot meet the growing demand for increasing data size and real-time computation of big data. In this paper, a parallel FP-Growth algorithm based on split is proposed. The established FP-Tree is split into blocks, and the split FP-Trees are equally divided into different nodes. The monitoring point is set up to monitor the operation of other nodes, dynamically migrate tasks and maintain load balancing. The experiment proves that each node has good load balancing with the given support degree, and the improved algorithm has better running performance than the classic FP-Growth algorithm in parallel processing. Finally, the parallel FP-Growth algorithm based on split is implemented on Hadoop to mine association rules between course grades. The mining process includes data preprocessing, mining results and analysis. The association rules between course grades provide suggestions for the way students learn and the way teachers teach. [ABSTRACT FROM AUTHOR]

Hot topic identification from micro-blog is very important for detection and control of the public opinion. When using Single-pass algorithm to cluster hot topics for Chinese micro-blog, Chinese word segmentation technology is a necessary preprocessing, but it will introduce inevitable segment errors. This kind of errors will make topic identification has low clustering precision. To solve this problem, this paper proposed an improved algorithm based on Single-pass which combines CS (Cosine Similarity) and LCS (Longest Common Subsequences) to calculate the similarity between Chinese words. Experiments on three different micro-blog data sets for hot topic identification are made, and the results show that the improved algorithm has both higher recall rate and precision rate than the original ones. The proposed algorithm is feasible and effective. [ABSTRACT FROM AUTHOR]

Attribute reduction is one of the key issues in rough set theory, and many algorithms have been proposed for static data set. Very little work has been done for incremental attribute reduction algorithm in era of big data. In this paper, the weakness of the existing incremental attribute reduction are analyzed. Then, two strategies of incremental attribute reduction algorithm for big data are designed, and an incremental attribute reduction algorithm using MapReduce is proposed. In order to reduce the computational complexity, our algorithm reuse the former Map results to speedup the computations of the equivalence classes. A new reduct can be updated by the old reduct effectively. This study gives some insights into how to conduct incremental attribute reduction for big data. [ABSTRACT FROM AUTHOR]

With technological advancements, massive amount of data is being collected in various domains. For instance, since the advent of digital image technology and remote sensing imagery (RSI), NASA and U.S. Geological Survey through the Landsat Data Continuity Mission, has been capturing images of Earth down to 15 meters resolution. Likewise, consider the Internet, where, growth of social media, blog Web sites , etc. generates exponential amount of textual data on a daily basis. Since clustering of data is time-consuming, much of these data is archived even before proper analysis. In this paper, we propose two novel and extremely fast algorithms called imgFAUST or Fast Attribute-based Unsupervised and Supervised Table Clustering for images and a variation called docFAUST for textual data. Both these algorithms are based on Predicate-Trees which are compressed, lossless and data-mining-ready data structures. Without compromising much on the accuracy, our algorithms are fast and can be effectively used in high-speed image data and document analysis. [ABSTRACT FROM AUTHOR]