Search

Clustering algorithms are part of algorithms of unsupervised-learning that are commonly used in different fields. Two major impacts on the data clustering algorithm have been the rapid developments and creation of electronic data. This includes the view of how the big data is stored as well as how it is processed. A cluster has a high level of resemblance to the same cluster and a low level of resemblance to other clusters. Clustering algorithms are commonly used in all fields, such as retail, banking, development, etc.. Even though different methodologies are proposed with different scenarios, but none of the existing methodologies are proven to be a better approaches. Here, in our work we have adopted an improved Map- Reduce programming model to combine the Canopy clustering and K-means clustering algorithms to process the Microarray data with the available commodity hardware with an aim to achieve better performance. The results obtained from different scenarios shown that the proposed method was capable of improving computational speed significantly by increasing the nodes as required. In this research, this paper explored the efficiency by evaluating and implementing the proposed Improved k-means with Improved Map Reduce algorithm which runs on Hadoop Frame work using Microarray dataset along with different datasets. [ABSTRACT FROM AUTHOR]

In order to explore potential value of explosively growing data in thermal power unit, this paper proposes a big data mining method based on Hadoop-based Spark cluster-computing framework and algorithms. Firstly, positive and negative balance methods are used to accurately obtain actual net coal consumption, and maximum information coefficient method is used to select all parameters related to optimization objectives. Then, Spark-based Mini-Batch K-means algorithm and Elbow method are constructed to divide whole operating modes. After that, all data are discretized and mapped to corresponding intervals by using Spark-based Elbow method and Mini-Batch K-means algorithm. Finally, Spark-based parallel FP-growth algorithm is used to deeply mine the potential relationships and laws. To verify the proposed method, a 350-MW thermal power unit is taken as a study case. The important conclusions are as follows: (1) the proposed Spark-based Mini-Batch K-means algorithm reduces the calculation time by 57.11% compared with Mini-Batch K-means algorithm, and 85.61% calculation time compared with K-means algorithm. The proposed Spark-based FP-growth algorithm reduces computational time by 32.8% compared with FP-growth algorithm. (2) Strong association rules of whole operating modes are mined, and operating optimization guidance schemes for important parameters are obtained. Take operating mode 1 as an example: if the optimal result can be reasonably applied, it can save 2.942 g coal per kilowatt hour. (3) Besides, we have found out some other potential relationships among parameters, which have important reference value for on-site operators to analyze economy of the thermal power unit. [ABSTRACT FROM AUTHOR]

In this paper, the clustering algorithm of big data fuzzy K-means is used to research and analyze the teaching management system. Since the values of the indicator variables of bad units are usually significantly different in smooth areas and discontinuous areas, we cluster the indicators of local areas through K-means clustering, so that only good units or bad units are included in each class. The server is deployed on the campus and is networked with the teaching computers of each classroom. Teachers log in to the class rate management system before or during class, log in to their respective users, select the corresponding class, and make corresponding attendance records according to the attendance of students in the class. The system automatically counts the number of expected arrivals, attendance, late arrivals, leave requests, and absenteeism for the class. Teachers can also fill in information feedback for this class: a list of absenteeism, classroom discipline, student learning status, teaching suggestions, and teaching testimonials. The development environment built by this system is a combination of PHP + MYSQL, which reasonably planned according to the overall needs. The development mode adopts the top-down model, which runs through the whole process from system development to testing to application. The most used floating point number encoding is the decimal floating point number encoding. The relationship of each module of the virtual reality teaching management system is sorted out, the overall framework of the virtual reality teaching management system is established, and the design of the business flow diagram of the management system is completed. This paper also uses Oracle database technology to further explain the design process of designing the system database. Finally, combined with the system design and implementation, the system test is completed from two aspects: system function test and performance test. [ABSTRACT FROM AUTHOR]

With the advancement of science and technology as well as the continual improvement of big data analysis technology, the accuracy of traditional data information classification has declined, making it impossible to assess English ability effectively. A competency evaluation model for college English teaching vacancies is built using this information and the big data architecture. The ability of the big data information model is evaluated and feature information of ability constraints is extracted using the predefined constraint parameter index analysis model. Simultaneously, the K-means clustering algorithm is used to cluster and integrate a series of index parameters of English ability using big data, and the English teaching resource allocation plan is completed in accordance with this, allowing for the scientific evaluation of English teaching ability. The results of the studies show that the clustering method utilized in the context of big data can aid in the evaluation of English competence. In the experiment, four test cycles of English teaching skills were set up, and the effectiveness of the English evaluation techniques described in this paper and two classical cluster evaluation methods were compared and tested. The research shows that using the method described in this paper to evaluate English teaching skills can significantly improve the full utilization of data. [ABSTRACT FROM AUTHOR]

One of the problems in implementing DRPs is the lack of sufficient understanding of consumers' behaviour when implementing DRPs. This paper compares consumers' load patterns annually by the improved Weighted Fuzzy Average (WFA) K‐means clustering method. According to the collected data, DRPs are discussed annually using a combined Convolutional Neural Network (CNN) and Long Short‐Term Memory (LSTM). To make CNN‐LSTM a practical algorithm for executing DRPs, Time Series Prediction (TSP) operations must be performed, and input data must be memorized. Finally, according to the TSP done for the obtained data, the implementation of DRPs in practice is facilitated. Then, the DRPs executed with the Deep Learning (DL) model and the power model will be prioritized by Technique for Order of Preference by Similarity to the Ideal Solution (TOPSIS) method, and decision indicators will be determined and weighted using the Shannon entropy method. The numerical studies section shows that the CNN‐LSTM algorithm is able to simulate DRPs with the Mean Absolute Percentage Error (MAPE) almost below 1% in the residential cluster and the highest MAPE in the commercial cluster of 15%. Nevertheless, this algorithm could easily give the same answer as the DRPs power model in the optimal selection of programs. [ABSTRACT FROM AUTHOR]

In the big data era, traditional data mining technology cannot meet the requirements of massive data processing with the background of intelligent manufacturing. Aiming at insufficient computing power and low efficiency in mining process, this paper proposes a improved K-means clustering algorithm based on the concept of distributed clustering in cloud computing environment. The improved algorithm (T.K-means) is combined with MapReduce computing framework of Hadoop platform to realize parallel computing, so as to perform processing tasks of massive data. In order to verify the practical performance of T.K-means algorithm, taking machining data of milling Ti-6Al-4V alloy as the mining object. The mapping relationship among cutting parameters, surface roughness, and material removal rate is mined, and the optimized value for cutting parameters is obtained. The results show that T.K-means algorithm can be used to mine the optimal cutting parameters, so that the best surface roughness can be obtained in milling Ti-6Al-4V titanium alloy. [ABSTRACT FROM AUTHOR]

In the era of big data, text as an information reserve database is very important, in all walks of life. From humanities research to government decision-making, from precision medicine to quantitative finance, from customer management to marketing, massive text, as one of the most important information carriers, plays an important role everywhere. The text data generated in these practical problems of humanities research, financial industry, marketing, and other fields often has obvious domain characteristics, often containing the professional vocabulary and unique language patterns in these fields and often accompanied by a variety of "noise." Dealing with such texts is a great challenge for the current technical conditions, especially for Chinese texts. A clustering algorithm provides a better solution for text big data information processing. Clustering algorithm is the main body of cluster analysis, K-means algorithm with its implementation principle is simple, low time complexity is widely used in the field of cluster analysis, but its K value needs to be preset, initial clustering center random selection into local optimal solution, other clustering algorithm, such as mean drift clustering, K-means clustering in mining text big data. In view of the problems of the above algorithm, this paper first extracts and analyzes the text big data and then does experiments with the clustering algorithm. Experimental conclusion: by analyzing large-scale text data limited to large-scale and simple data set, the traditional K-means algorithm has low efficiency and reduced accuracy, and the K-means algorithm is susceptible to the influence of initial center and abnormal data. According to the above problems, the K-means cluster analysis algorithm for data sets with large data volumes is analyzed and improved to improve its execution efficiency and accuracy on data sets with large data volume set. Mean shift clustering can be regarded as making many random centers move towards the direction of maximum density gradually, that is, moving their mean centroid continuously according to the probability density of data and finally obtaining multiple maximum density centers. It can also be said that mean shift clustering is a kernel density estimation algorithm. [ABSTRACT FROM AUTHOR]