Search

Considering that in the process of job scheduling, the cluster load should be prebalanced rather than remedied when the load is seriously unbalanced; therefore, in this paper, the task scheduling flow of the Hadoop cluster is analyzed deeply. On the Hadoop platform, a self-dividing algorithm is proposed for load balancing. An intelligent optimization algorithm is used to solve load balance. A dynamic feedback load balancing scheduling method is proposed from the point of view of task scheduling. In order to solve the shortcoming of the fair scheduling algorithm, this paper proposes two ways to improve the resource utilization and overall performance of Hadoop. When the mapping task is completed and the tasks to be reduced are assigned, the task assignment is based on the performance of the nodes to be reduced. It gives full play to the advantages of the ant colony algorithm and the hive colony algorithm so that the fusion algorithm can better deal with load balance. Then, three existing scheduling algorithms are introduced in detail: single queue scheduling, capacity scheduling, and fair scheduling. On this basis, an improved task scheduling strategy based on genetic algorithm is proposed to allocate and execute application tasks to reduce task completion time. The experiment verifies the effectiveness of the algorithm. The LBNP algorithm greatly improves the efficiency of reducing task execution and job execution. The delay capacity scheduling algorithm can ensure that most tasks can achieve localization scheduling, improve resource utilization, improve load balance, and speed up job completion time. [ABSTRACT FROM AUTHOR]

Nowadays, with the constant change of public aesthetic standards, a large number of new types and themes of film programs have emerged. For this reason, this paper proposes an improved neural network optimized by mutation ant colony algorithm for automatic acquisition of film labels, which not only overcomes the disadvantages of traditional neural network, such as difficulty in determining weights, slow convergence speed, and easiness to fall into local minimum, but also makes up for the shortcomings faced by using ant colony algorithm alone through the gradient information of quantum genetic algorithm neural network. The results show that the user similarity judgment is added in the process of calculating the user rating deviation between movies, and the neighbor chooses to add the movie tag weight and rating similarity as the basis for the neighbor selection of the target movie in the process of predicting the target movie rating. Experiments show the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]