Search

With the development of network and big data technology, how to dig out the effective information from massive data is the problem that we are facing. Traditional data mining algorithms in the face of big data, it is powerless. With the ability of parallel computing, data is processed on several machines, so the efficiency of data processing will be greatly improved. In this paper, the author analyzes the parallel data mining algorithm application in the evaluation of college counselors' psychological guidance ability, and puts forward the strategies of improving the ability of college counselors' psychological guidance. Based on the statistical analysis, the result shows that there were no significant differences in gender, but in the context of communication, female were better than male. The male instructors are superior to female instructors in the understanding ability of the observation and diagnosis and the plan of action. Obviously, training is a very effective way to improve the ability of College Counselors' psychological counseling, and it should be promoted and applied.

With the rapid development of information technology, the data is becoming more and more large, the data mining technology has been applied to many fields in the life. In view of the existing data mining algorithms complexity is too high, the execution time too long, the article applies association rules in data mining technology, analysis summarized the existing advantages and disadvantages of the classic data mining association rules algorithm, is proposed for mining maximum frequent itemsets MFP - Miner algorithm. At last, by experimental simulation result shows that the proposed algorithm performance superiority, its complexity and execution time significantly shortened, the practical guiding significance.

Frequent Itemset Mining (FIM) is one of most fundamental techniques in data mining with extensive applications to a variety of data mining problems such as association rule mining, correlations, clustering and classification. Since the first proposal of frequent itemset mining, numerous serial algorithms have been proposed in order to improve mining performance, yet most of them cannot scale to massive datasets which are very common nowadays. In this paper, we propose a new parallel FIM algorithm named PFIN based on Nodeset which is a more efficient data structure for mining frequent itemsets. PFIN can intelligently decompose a large-scale FIM problem into a set of tasks, where each task can be executed in parallel without unnecessary communication overheads. Moreover, a hash-based load balancing strategy has been adopted to optimize resource use and maximize throughput. For evaluating the performance of PFIN, we have conduct extensive experiments on Spark which is an emerging distributed in-memory processing framework to compare it against PFP which is one of state-of-the-art parallel FIM algorithms on a range of real datasets. The experimental results demonstrate that our proposed PFIN are highly competitive with PFP in scalability performance, outperforming PFP in speed performance.