Search

Purpose: To examine the contextual and personal factors that influence teachers' reported adoption of one or more instructional models (IMs). Methods: Participants (n = 25) were interviewed that reported adopting Sport Education; Teaching Games for Understanding; Teaching Personal and Social Responsibility; and/or Sports, Play, and Active Recreation for Kids in their K-12 curriculum. Results: Viewed through the lens of occupational socialization theory and utilizing constant comparison methodology, the interview data were analyzed and three major themes, each with related subthemes, were developed: (a) IM adoption depends on context and socialization, (b) socializing agents influence IM adherence, and (c) IM implementation is selective. Discussion/Conclusions: Teachers vary in their rationale for adopting one or more IMs, and adherence is frequently a result of principal and student encouragement. Future efforts to disseminate IMs should consider pathways to adoption identified in this study, including teaching conventions, department supervisors, and teaching colleagues. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Changzhou University (Natural Science Edition) / Changzhou Daxue Xuebao (Ziran Kexue Ban) is the property of Editorial Department of Journal of Changzhou University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In recent years, cloud computing applications have facilitated the distribution of heterogeneous, unstructured digitized data among users' social networks of varying opinions. Text processing at a large scale requires high-precision computational techniques, which increases the computational burden. The advent of big data analytics along with Natural Language Processing is a powerful factor in improving the efficiency of processing large-scale text data, using the kernel of the MapReduce big data analytics frameworks that allow parallelization of large computational operations. In this paper, we propose an intelligent sentiment prediction approach based on deep learning, batch, and streaming big data analytics. In fact, our main objective is to take advantages of the powerful tools provided by the distributed platforms, such as, Hadoop and Spark to preprocess streaming data. This involves various tasks such as cleaning the data, reducing its size, minimizing access time, and decreasing storage volume. This step prepares the big streaming data to be fully exploited by Deep Learning models. This work includes a research study on processing big data related to short volume scripts based on batch and streaming distributed frameworks as well as deep learning approaches in Natural Language Processing. We detail our idea for analyzing short texts to determine their semantic context and categorize them into pros and cons poles. There were different stages in building this model, the first involving data reduction and refinement using selected features and big data analysis tools. In the second stage, words are embedded by global vector to be computed in layers of convolutional and recurrent neural networks. The experimental study and the analysis of the results confirm the usefulness of our proposed model and its superiority over the main approaches studied in the literature. Our model achieved a performance of 96% in terms of accuracy. [ABSTRACT FROM AUTHOR]

In this study, the duplex stainless steel is machined using a wire-cut electrical discharge machining process to overcome the issues of poor finishing, lower dimensional accuracy, and rapid wear of cutting tools in conventional machining processes. The higher rate of material removal and superior finish on the machine surface is dependent on kerf loss, which is expressed as the mean kerf width assessed at certain locations. Nevertheless, this approach is typically not effective in explicating the intricacies of true kerf loss due to the random selection of distinctive points. This work proposes an area-based measuring approach to quantify the kerf loss by taking into account any imperfections on the kerf wall. At the pulse on time of 20 μs and current of 3 A, the machining appears not stable. The pulse on time, current, and voltage of 12 µs, 1 A, and 80 V proved suitable for obtaining excellent dimensional precision. [ABSTRACT FROM AUTHOR]

Early detection and diagnosis of breast cancer are critical for saving lives. This paper addresses two major challenges associated with this task: the vast amount of data processing involved and the need for early detection of breast cancer. To tackle these issues, we developed thirty hybrid architectures by combining five deep learning techniques (Xception, Inception-V3, ResNet50, VGG16, VGG19) as feature extractors and six classifiers (random forest, logistic regression, naive Bayes, gradient-boosted tree, decision tree, and support vector machine) implemented on the Spark framework. We evaluated the performance of these architectures using four classification criteria. The results, analyzed using Scott Knott's statistical test, demonstrated the effectiveness of merging deep learning feature extraction techniques with traditional classifiers for classifying breast cancer into malignant and benign tumors. Notably, the hybrid architecture using logistic regression as the classifier and ResNet50 for feature extraction (RESLR) emerged as the top performer. It achieved impressive accuracy scores of 98.20%, 96.59%, 96.64%, and 94.84% across the Break-His dataset at different magnifications (40X, 100X, 200X, and 400X) respectively. Additionally, RESLR achieved an accuracy of 97.05% on the ICIAR dataset and a remarkable accuracy of 95.31% on the FNAC dataset. [ABSTRACT FROM AUTHOR]

Copyright of Retos: Nuevas Perspectivas de Educación Física, Deporte y Recreación is the property of Federacion Espanola de Asociaciones de Docentes de Educacion Fisica and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The commonly used method for network intrusion is based on pattern matching systems, but these systems do not have high detection accuracy when facing large-scale high-speed network traffic environments. In addition, when facing a wide variety of network attacks, relying solely on a certain network security technology is difficult to ensure network security. Therefore, a distributed network intrusion prevention system based on the Spark framework and dynamic information security theory model was innovatively proposed to improve the detection efficiency of network intrusion and solve these issues. The architecture and functional structure of the network intrusion prevention system were constructed by improving the random forest algorithm and Spark. In addition, the dynamic network intrusion prevention system was designed on the basis of the Policy Protection Detection Response (P2DR) model and the Protection Detection Reaction Recovery (PDRR) model to cope with the diverse network attacks and make up for the lack of dynamic defense based on improved forest algorithm and Spark. The test results showed that the network intrusion prevention system based on the improved random forest algorithm and Spark had a maximum detection time of 13,593 ms, a minimum detection time of 13,318 ms, and an average detection time of 13,468 ms when the data were 6000 pieces. The average success rate of the dynamic network intrusion prevention system based on the dynamic information security theory model was 87.72%, the average detection rate was 71.68%, and the average false alarm rate was 17.23%. The F1 values corresponding to the improved random forest algorithm under 7 different attack types of data were 0.985, 0.983, 0.876, 0.843, 0.797, 0.983, and 0.890, respectively, which were significantly better than the comparison algorithm. It can be seen that the dynamic network intrusion prevention system based on the improved random forest algorithm, Spark, and the dynamic information security theoretical model, has good performance and can effectively detect network intrusions, providing technical support for solving network intrusion problems in reality. The contribution of the research is reflected in the improvement of the detection performance of network intrusion detection systems and the reduction of detection time and false alarm rates. [ABSTRACT FROM AUTHOR]

The performance optimization of Apache Spark, a widely used distributed computing framework, is crucial for the efficient execution of data-intensive workloads. However, the automatic tuning of around 180 internal configuration features of Spark for getting optimum performance can be a complex, expensive, and time-consuming task. This research paper aims to address these challenges along with the task of reducing execution time and tuning cost for benchmark Spark MLlib workloads on a benchmark dataset by automating the tuning process. A grid search based novel approach Reduced Internal Configuration Grid (RICG) is proposed that reduces the Spark's configuration feature values grid levaraging exponential projective sampling. A significant reduction in Spark workload execution time is obtained at a low cost without compromising the performance of the CPU, memory and network resources of the system. Extensive experimentation and evaluation demonstrate the effectiveness of the proposed approach, showcasing notable reductions of around 26–30% in execution time for benchmark workloads. This reduction is achieved by cutting the features tuning cost by approximately 58 to 60% compared to the prior best grid search based approach. The proposed RICG model outperforms existing statistical trial-test approaches, striking a favorable balance between performance optimization and cost optimization. Lastly, the logs generated by the RICG model are used to predict the performance of Spark application workloads by using the boosting algorithms with around 90% accuracy. The findings of this research highlight the potential of automated tuning techniques to enhance the performance of Apache Spark-based applications with cost efficiency. [ABSTRACT FROM AUTHOR]

A threat to governments and the medical community globally are infectious illness outbreaks such as COVID-19, Spanish flu, and Ebola, which kill millions of people. Furthermore, because people are afraid to go to their employment, infectious diseases hinder economic progress. Therefore, it is imperative to implement interventions to lessen the spread of infectious diseases. From this angle, the practice of tracking contacts might be viewed as a mitigation strategy to stop the spread of contagious diseases. As this is going on, users' trace data sets are growing exponentially larger over time, making it difficult to conduct contact tracing queries over them. In this paper, a novel Spark contact-tracing query processing method is proposed. This method determines if users are suspected cases by analyzing their paths based on two variables: nearby social distance and exposure time. Additionally, the developed method makes full use of the Spark framework to address scalability issues and effectively respond to contact tracing inquiries over a wide range of trajectories. [ABSTRACT FROM AUTHOR]

Credit fraud is a common financial crime that causes significant economic losses to financial institutions. To address this issue, researchers have proposed various fraud detection methods. Recently, research on deep forests has opened up a new path for exploring deep models beyond neural networks. It combines the features of neural networks and ensemble learning, and has achieved good results in various fields. This paper mainly studies the application of deep forests to the field of fraud detection and proposes a distributed dense rotation deep forest algorithm (DRDF-spark) based on the improved RotBoost. The model has three main characteristics: firstly, it solves the problem of multi-granularity scanning due to the lack of spatial correlation in the data by introducing RotBoost. Secondly, Spark is used for parallel construction to improve the processing speed and efficiency of data. Thirdly, a pre-aggregation mechanism is added to the distributed algorithm to locally aggregate the statistical results of sub-forests in the same node in advance to improve communication efficiency. The experiments show that DRDF-spark performs better than deep forests and some mainstream ensemble learning algorithms on the fraud dataset in this paper, and the training speed is up to 3.53 times faster. Furthermore, if the number of nodes is further increased, the speedup ratio will continue to increase. [ABSTRACT FROM AUTHOR]

The MeanShift algorithm is a nonparametric method based on gradient ascent and it can effectively handle complex variations in lychee orchard scenes as well as changes in lychee tree canopies due to its adaptability, multi-scale analysis capabilities, and robustness, making it widely used in the segmentation processing of drone-based remote sensing images of lychee orchards. However, due to the high computational complexity of the MeanShift algorithm, its performance in processing large-scale drone remote sensing images of lychee tree canopies is not highly efficient, leading to low segmentation efficiency, which hampers its broader application. To address these issues, this study proposes high-speed MeanShift parallel segmentation algorithms for drone remote sensing images of lychee tree canopies based on MapReduce and Spark distributed computing frameworks. In this study, a cluster consisting of four nodes with Hadoop and Spark was set up, and 4000 drone remote sensing images were used as test data to evaluate the algorithm. Experimental results show that, the MeanShift algorithm based on MapReduce reduced the task execution time by 86.1% compared to the traditional MeanShift algorithm, while the MeanShift algorithm based on Spark reduced the task execution time by 88.0%, without compromising segmentation accuracy. The MeanShift parallel segmentation algorithm based on Hadoop and Spark platform can overcome the bottleneck of task execution efficiency and significantly enhance computational speed on a single machine. [ABSTRACT FROM AUTHOR]

Wavelet neural networks (WNN) have been applied in many fields to solve regression as well as classification problems. After the advent of big data, as data gets generated at a brisk pace, it is imperative to analyze it as soon as it is generated owing to the fact that the nature of the data may change dramatically in short time intervals. This is necessitated by the fact that big data is all pervasive and throws computational challenges for data scientists. Therefore, in this paper, we built an efficient Scalable, Parallelized Wavelet Neural Network (SPWNN) which employs the parallel stochastic gradient algorithm (SGD) algorithm. SPWNN is designed and developed under both static and streaming environments in the horizontal parallelization framework. SPWNN is implemented by using Morlet and Gaussian functions as activation functions. This study is conducted on big datasets like gas sensor data which has more than 4 million samples and medical research data which has more than 10,000 features, which are high dimensional in nature. The experimental analysis indicates that in the static environment, SPWNN with Morlet activation function outperformed SPWNN with Gaussian on the classification datasets. However, in the case of regression, there is no clear trend was observed. In contrast, in the streaming environment i.e., Gaussian outperformed Morlet on the classification and Morlet outperformed Gaussian on the regression datasets. Overall, the proposed SPWNN architecture achieved a speedup of 1.22 - 1.78. [ABSTRACT FROM AUTHOR]

In today's era, digital data is being created and transmitted majorly in the form of images and videos. Storing such a huge number of images and transmitting them requires a lot of computer resources. Instead of storing the image data as is, if we compress and store it, it saves a lot of resources. Image compression is the act of removing the maximum possible redundant data from an image and maintaining only the non-redundant data. In this paper, to compress and decompress such big image data, a distributed environment with a map-reduce paradigm using Hadoop Distributed File System (HDFS) and Spark is used. In addition to these, Microsoft Azure cloud environment is also used. Various setups like a single system, 1 + 4, 1 + 15, and 1 + 18 node clusters are used to show the time comparisons among these setups with the self-created large image dataset. On these four self-made clusters, more than 200 million (219,340,800) images are compressed and decompressed; the execution times are compared with two of the traditional image compression methods: JPEG and CCITT Group-3. To evaluate the efficiency of these two compression methods: Compression Ratio, Root Mean Square Error (RMSE), and Peak Signal to Noise Ratios (PSNR) are used. [ABSTRACT FROM AUTHOR]

Developing and executing large-scale data analysis applications in parallel and distributed environments can be a complex and time-consuming task. Developers often find themselves diverted from their application logic to handle technical details about the underlying runtime and related issues. To simplify this process, ParSoDA, a Java library, has been proposed to facilitate the development of parallel data mining applications executed on HPC systems. It simplifies the process by providing built-in scalability mechanisms relying on the Hadoop and Spark frameworks. This paper presents ParSoDA-Py, the Python version of the ParSoDA library, which allows for further support of commonly used runtimes and libraries for big data analysis. After a complete library redesign, ParSoDA can be now easily integrated with other Python-based distributed runtimes for HPC systems, such as COMPSs and Apache Spark, and with the large ecosystem of Python-based data processing libraries. The paper discusses the adaptation process, which takes into consideration the new technical requirements, and evaluates both usability and scalability through some case study applications. [ABSTRACT FROM AUTHOR]

Typhoon trajectory related data involve many factors, such as atmospheric factors, oceanic factors, and physical factors. It has the characteristics of high dimension, strong spatio-temporal correlation, and nonlinear correlation, which increases the difficulty of typhoon trajectory prediction. Using feature selection approaches to select appropriate prediction factors becomes an important means to reduce the dimension of typhoon trajectory related data and improve the performance and accuracy of typhoon trajectory prediction methods. However, the existing feature selection methods based on linear correlation analysis cannot well depict the nonlinear correlation between data features, which results in low accuracy of feature selection. The feature selection methods based on nonlinear correlation analysis are computationally expensive, which affects the timeliness of feature selection. To solve the problem, we propose a parallel feature selection method NX-Spark-DC based on the Spark platform for typhoon trajectory related data. The method firstly filters out the redundant features of typhoon related data by normalized mutual information (NMI) method, subsequently eliminates the useless features by XGBoost machine learning model, and thus reducing the dimension of typhoon related data. On this basis, an improved Spark-based parallel distance correlation algorithm (Spark-DC) is proposed to select the feature combinations with strong correlation. A series of experimental results show that NX-Spark-DC method has high execution efficiency and accuracy, which is significantly better than the existing methods. [ABSTRACT FROM AUTHOR]

In the realm of distributed computing frameworks, such as Apache Spark and MapReduce Hadoop, the efficacy of these frameworks varies across diverse applications and algorithms contingent upon distinctive evaluation metrics and critical parameters. This research paper diligently scrutinizes the extant body of research that compares these two frameworks concerning said evaluation metrics and parameters. Subsequently, it conducts empirical investigations to authenticate the performance of these frameworks in the context of an iterative Gradient Boosting Tree Regression (GBTR) algorithm. Remarkably, the comparative analyses in previous studies encompass a spectrum of iterative machine learning regression and classification techniques, batch processing, SQL, and Graph processing algorithms. Furthermore, numerous investigations have explored the application of machine learning algorithms encompassing logistic regression, Page Rank, K-Means, KNN, and the HiBench suite. This paper presents the comparison between the two distributed computing platforms on iterative GBTR for classification task on the HIGGS dataset from the physics domain and for the regression task on the Covid-19 dataset from the healthcare domain. The empirical findings corroborate that Apache Spark exhibits superior execution speed in iterative tasks when the available physical memory significantly exceeds the dataset size. Conversely, Hadoop outperforms Spark when dealing with substantial datasets or constrained physical memory resources. [ABSTRACT FROM AUTHOR]

Spark, a distributed computing platform, has rapidly developed in the field of big data. Its in-memory computing feature reduces disk read overhead and shortens data processing time, making it have broad application prospects in large-scale computing applications such as machine learning and image processing.However, the performance of the Spark platform still needs to be improved.When a large number of tasks are processed simultaneously, Spark's cache replacement mechanism cannot identify high-value data partitions, resulting inmemory resources not being fully utilized and affecting the performance of the Spark platform. To address the problem that Spark's default cache replacement algorithm cannot accurately evaluate high-value data partitions, firstly the weight influence factors of data partitions are modeled and evaluated. Then, based on this weighted model, a cache replacement algorithm based on dynamic weighted data value is proposed, which takes into account hit rate and data difference. Better integration and usage strategies are implemented based on LRU(LeastRecentlyUsed). Theweight update algorithm updates the weight value when the data partition information changes, accurately measuring the importance of the partition in the current job; the cache removal algorithm clears partitions without useful values in the cache to releasememory resources; the weight replacement algorithm combines partition weights and partition information to replace RDD partitions when memory remaining space is insufficient. Finally, by setting up a Spark cluster environment, the algorithm proposed in this paper is experimentally verified. Experiments have shown that this algorithmcan effectively improve cache hit rate, enhance the performance of the platform, and reduce job execution time by 7.61% compared to existing improved algorithms. [ABSTRACT FROM AUTHOR]

Copyright of Application Research of Computers / Jisuanji Yingyong Yanjiu is the property of Application Research of Computers Edition and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Big Data Research (2096-0271) is the property of Beijing Xintong Media Co., Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Systems Engineering & Electronics is the property of Journal of Systems Engineering & Electronics Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In recent years, various studies have been conducted on SVMs and their applications in different area. They have been developed significantly in many areas. SVM is one of the most robust classification and regression algorithms that plays a significant role in pattern recognition. However, SVM has not been developed significantly in some areas like large-scale datasets, unbalanced datasets, and multiclass classification. Efficient SVM training in large-scale datasets is of great importance in the big data era. However, as the number of samples increases, the time and memory required to train SVM increase, making SVM impractical even for a medium-sized problem. With the emergence of big data, this problem becomes more significant. This paper presents a novel distributed method for SVM training in which a very small subset of training samples is used for classification, which reduces the problem size and thus the required memory and computational resources. The solution of this problem almost converges to standard SVM. This method includes three steps: first, detecting a subset of distributed training samples, second, creating local models of SVM and obtaining partial vectors, and finally combining the partial vectors and obtaining the global vector and the final model. In addition, the datasets which suffer from unbalanced number of samples and tend to the majority class, the proposed method balances the samples of the two classes and it can be used in unbalanced datasets. The empirical results show that using this method is efficient for large-scale problems. [ABSTRACT FROM AUTHOR]

The cooperative coevolution framework was introduced to address the shortcomings of metaheuristic algorithms in solving continuous large-scale global optimization problems. By dividing the problem into subcomponents which can be optimized separately, the framework can improve on both the solution's quality as well as the computational speed by exposing a degree of parallelism. Distributed computing platforms, such as Apache Spark, have long been used to improve the speed of different algorithms in solving computational problems. This work proposes a distributed implementation of the cooperative coevolution framework for solving large-scale global optimization problems on the Apache Spark distributed computing platform. By using a formerly outlined distributed variant of the cooperative coevolution framework, features of the Spark platform are utilized to enhance the computational speed of the algorithm while maintaining comparable search quality to other works in the literature. To test for the proposed implementation's improvement in computational speed, the CEC 2010 large-scale global optimization benchmark functions are used due to the diversity they offer in terms of complexity, separability and modality. Results of the proposed distributed implementation suggest that a speedup of up to ×3.36 is possible on large-scale global optimization benchmarks using the Apache Spark platform. [ABSTRACT FROM AUTHOR]

The large amount of data generated every day makes necessary the re-implementation of new methods capable of handle with massive data efficiently. This is the case of Association Rules, an unsupervised data mining tool capable of extracting information in the form of IF-THEN patterns. Although several methods have been proposed for the extraction of frequent itemsets (previous phase before mining association rules) in very large databases, the high computational cost and lack of memory remains a major problem to be solved when processing large data. Therefore, the aim of this paper is three fold: (1) to review existent algorithms for frequent itemset and association rule mining, (2)to develop new efficient frequent itemset Big Data algorithms using distributive computation, as well as a new association rule mining algorithm in Spark, and (3) to compare the proposed algorithms with the existent proposals varying the number of transactions and the number of items. To this purpose, we have used the Spark platform which has been demonstrated to outperform existing distributive algorithmic implementations. [ABSTRACT FROM AUTHOR]

Background: Fire simulators are increasingly used to predict fire spread. Australian fire agencies have been concerned at not having an objective basis to choose simulators for this purpose. Aims: We evaluated wildland fire simulators currently used in Australia: Australis, Phoenix, Prometheus and Spark. The evaluation results are outlined here, together with the evaluation framework. Methods: Spatial metrics and visual aids were designed in consultation with simulator end-users to assess simulator performance. Simulations were compared against observations of fire progression data from 10 Australian historical fire case studies. For each case, baseline simulations were produced using as inputs fire ignition and fuel data together with gridded weather forecasts available at the time of the fire. Perturbed simulations supplemented baseline simulations to explore simulator sensitivity to input uncertainty. Key results: Each simulator showed strengths and weaknesses. Some simulators displayed greater sensitivity to different parameters under certain conditions. Conclusions: No simulator was clearly superior to others. The evaluation framework developed can facilitate future assessment of Australian fire simulators. Implications: Collection of fire behaviour observations for routine simulator evaluation using this framework would benefit future simulator development. Australian fire and land management agencies use fire simulators to predict characteristics of fires in the landscape, and require evaluation of simulator accuracy. Simulations of fire spread were assessed against observations. Inputs were perturbed to evaluate simulator sensitivity. Against the evaluation measures, no simulators proved universally superior to the others. [ABSTRACT FROM AUTHOR]

In recent years, the surge in continuously accelerating data generation has given rise to the prominence of big data technology. The MapReduce architecture, situated at the core of this technology, provides a robust parallel environment. Spark, a leading framework in the big data landscape, extends the capabilities of the traditional MapReduce model. Coping with big data, especially in the realm of clustering, requires more efficient techniques. Meta-heuristic-based clustering, known for offering global solutions within reasonable time frames, emerges as a promising approach. This paper introduces a parallel-distributed clustering algorithm for big data within the Spark Framework, named Spark, chaotic improved PSO (S-CIPSO). Centered on particle swarm optimization (PSO), the proposed algorithm is enhanced with a chaotic map and an efficient procedure. Test results, conducted on both real and artificial datasets, establish the superior performance and quality of clustering results achieved by the proposed approach. Additionally, the scalability and robustness of S-CIPSO are validated, demonstrating its effectiveness in handling large-scale datasets. [ABSTRACT FROM AUTHOR]

Big data analytics is an emerging topic in academic and industrial engineering fields, where the large-scale data issue is the most attractive challenge. It is crucial to design an effective large-scale data processing model to handle big data. In this paper, we aim to improve the accuracy of the classification task and reduce the execution time for large-scale data within a small cluster. In order to overcome these challenges, this paper presents a novel ensemble-based paradigm that consists of the procedure of splitting large-scale data files and developing ensemble models. Two different splitting methods are first developed to partition large-scale data into small data blocks without overlapping. Then we propose two ensemble-based methods with high accuracy and less execution time: bagging-based and boosting-based methods. Finally, the proposed paradigm can be implemented by four predictive models, which are combinations of two splitting methods and two ensemble-based methods. A series of persuasive experiments was conducted to evaluate the effectiveness of the proposed paradigm with four different combinations. Overall, the proposed paradigm with boosting-based is the best in terms of the accuracy metric compared with existing methods. In addition, boosting-based methods achieve 91.6% accuracy compared with 52% accuracy of base line model for a big data file with 10 million samples. However, the paradigm with bagging-based takes the least execution time to yield results. This paper also reveals the effectiveness of the computing Spark cluster for large-scale data and points out the weakness of RDD (Resilient Distributed dataset). [ABSTRACT FROM AUTHOR]

Spark is one of the most popular big data analytical platforms. To save time, achieve high resource utilization, and remain cost-effective for Spark jobs, it is challenging but imperative for data scientists to configure suitable resource portions.In this paper, we investigate the proper parameter values that meet workloads' performance requirements with minimized resource cost and resource utilization time. We propose SimCost, a simulation-based cost model, to predict the performance of jobs accurately. We achieve low-cost training by taking advantage of simulation framework, i.e., Monte Carlo simulation, which uses a small amount of data and resources to make a reliable prediction for larger datasets and clusters. Our method's salient feature is that it allows us to invest low training costs while obtaining an accurate prediction. Through empirical experiments with 12 benchmark workloads, we show that the cost model yields less than 5% error on average prediction accuracy, and the recommendation achieves up to 6x resource cost saving. [ABSTRACT FROM AUTHOR]

The present work concerns microchannel fabrication using magnetic-assisted electrochemical discharge machining on silica glass. The magnetic field regulates the process by controlling bubble formation and separation adjacent to the tool edge. Due to the applied magnetic and electric flux on the electrolyte and electrode interface, the Lorentz force impacts the bubble establishment and film quality. The fractional bubble coverage decreases with the magnetic field, which increases the current density and affects the discharge frequency. The analytical model is explored here to investigate the magnetic field influences on departure radius. A novel approach has been adopted to couple the Laplace and Navier stokes equations for studying the magnetohydrodynamic effect. The experimental and analytical results reveal that magnetic flux decreases the bubble departure radius. The regression model explains the impact of controllable factors on surface roughness, material removal rate, and width overcut. The multi-response optimization with the hybridization of response surface methodology and grey relation analysis technique provides the optimum value of material removal rate, width overcut and surface roughness as 1.2 mg/min, 0.046 mm, and 1.58 µm, respectively. [ABSTRACT FROM AUTHOR]

Autism is heterogeneous at many levels, including clinical symptoms and etiology. A key strategy in studying heterogeneous conditions is having large enough sample sizes to stratify into smaller groups that are more homogeneous. SPARK and Simons Searchlight are large and growing research cohorts of individuals with autism in the United States and individuals with genetically defined neurodevelopmental conditions around the world, respectively. They both provide freely available phenotypic and genotypic data with the ability to re-contact participants through the research match program. Deep dives into each gene in Searchlight provide comprehensive natural history data to understand the differing clinical courses to inform proper clinical care, and work toward treatment for each condition. Moreover, pilots of genetically based newborn screening programs for neurogenetic disorders can provide opportunities for equitable and early diagnosis to try to improve outcomes with earlier interventions. [ABSTRACT FROM AUTHOR]

The Spark computing framework provides an efficient solution to address the major requirements of big data processing, but data partitioning and job scheduling in the Spark framework are the two major bottlenecks that limit Spark's performance. In the Spark Shuffle phase, the data skewing problem caused by unbalanced data partitioning leads to the problem of increased job completion time. In response to the above problems, a balanced partitioning strategy for intermediate data is proposed in this article, which considers the characteristics of intermediate data, establishes a data skewing model and proposes a dynamic partitioning algorithm. In Spark heterogeneous clusters, because of the differences in node performance and task requirements, the default task scheduling algorithm cannot complete scheduling efficiently, which leads to low system task processing efficiency. In order to deal with the above problems, an efficient job scheduling strategy is proposed in this article, which integrates node performance and task requirements, and proposes a task scheduling algorithm using greedy strategy. The experimental results prove that the dynamic partitioning algorithm for intermediate data proposed in this article effectively alleviates the problem that data skew leads to the decrease of system task processing efficiency and shortens the overall task completion time. The efficient job scheduling strategy proposed in this article can efficiently complete the job scheduling tasks under heterogeneous clusters, allocate jobs to nodes in a balanced manner, decrease the overall job completion time and increase the system resource utilization. [ABSTRACT FROM AUTHOR]

Processing extensive and diverse data in real-time is a significant challenge in the context of smart cities. Timely access to information and efficient analytics is essential for smart city services to make data-driven decisions and enhance urban living. Scheduling algorithms play a crucial role in ensuring the prompt delivery of services and efficient task completion. This paper explores various scheduling techniques, including static, dynamic, and hybrid schedulers, and compares their objectives and performance. Additionally, the study examines two prominent data processing frameworks, Hadoop and Spark, and compares their capabilities in handling big data in smart cities. With its ability to process large amounts of data quickly and efficiently, Spark has shown superiority over Hadoop in realtime data processing and performance optimization. The paper concludes by highlighting the strengths and limitations of each framework. It discusses the need for further research in optimizing scheduling techniques and exploring hybrid artificial intelligence scheduling for Spark. Overall, the findings contribute to a better understanding of data processing in real-time and provide insights for researchers and practitioners in smart cities. [ABSTRACT FROM AUTHOR]

With the use of cloud computing, which provides the infrastructure necessary for the efficient delivery of smart city services to every citizen over the internet, intelligent systems may be readily integrated into smart cities and communicate with one another. Any smart system at home, in a car, or in the workplace can be remotely controlled and directed by the individual at any time. Continuous cloud service availability is becoming a critical subscriber requirement within smart cities. However, these cost-cutting measures and service improvements will make smart city cloud networks more vulnerable and at risk. The primary function of Intrusion Detection Systems (IDS) has gotten increasingly challenging due to the enormous proliferation of data created in cloud networks of smart cities. To alleviate these concerns, we provide a framework for automatic, reliable, and uninterrupted cloud availability of services for the network data security of intelligent connected devices. This framework enables IDS to defend against security threats and to provide services that meet the users' Quality of Service (QoS) expectations. This study's intrusion detection solution for cloud network data from smart cities employed Spark and Waikato Environment for Knowledge Analysis (WEKA). WEKA and Spark are linked and made scalable and distributed. The Hadoop Distributed File System (HDFS) storage advantages are combined with WEKA's Knowledge flow for processing cloud network data for smart cities. Utilizing HDFS components, WEKA's machine learning algorithms receive cloud network data from smart cities. This research utilizes the wrapper-based Feature Selection (FS) approach for IDS, employing both the Pigeon Inspired Optimizer (PIO) and the Particle Swarm Optimization (PSO). For classifying the cloud network traffic of smart cities, the tree-based Stacking Ensemble Method (SEM) of J48, Random Forest (RF), and eXtreme Gradient Boosting (XGBoost) are applied. Performance evaluations of our system were conducted using the UNSW-NB15 and NSL-KDD datasets. Our technique is superior to previous works in terms of sensitivity, specificity, precision, false positive rate (FPR), accuracy, F1 Score, and Matthews correlation coefficient (MCC). [ABSTRACT FROM AUTHOR]

In the dynamic realm of big data processing, conquering the challenges imposed by highdimensional datasets is imperative. This paper introduces a groundbreaking advancement in dimensionality reduction, employing Variational Auto-Encoder (VAE) within the Spark distributed framework. The deliberate selection of the "TLC" dataset, representative of New York City taxi trips with inherent high dimensionality, highlights the practicality of our approach. Our research showcases the virtuoso performance of VAE, achieving an impressive 95.12% reduction ratio and 89.26% accuracy. This highlights VAE's ability to elegantly distill essential information while discarding superfluous dimensions, achieving a harmonious balance between reduction and accuracy. Furthermore, building on the demonstrated superiority of Spark over Hadoop in prior successes, our adoption of VAE aligns with the overarching goal of enhancing big data processing. Spark's consistent advantage as a distributed framework reaffirms its reliability in handling diverse machine learning algorithms. This paper not only contributes to the advancement of machine learning in big data processing but also underscores the adaptability, versatility, and consistent performance of our approach across various methodologies and frameworks. The success of VAE in reducing dimensionality, coupled with Spark's inherent advantages, positions this research as a valuable contribution to the exploration of advanced techniques in distributed big data processing. [ABSTRACT FROM AUTHOR]

Research in movie recommendation systems addresses several specific challenges to enhance the accuracy, relevance, and user satisfaction of the recommendations. Some of the key challenges include sparsity of data and cold start for new movies. Integrating collaborative filtering and content-based filtering in a seamless and effective manner also poses another key challenge. This article explores strategies for optimizing hybrid recommendation systems to leverage the strengths of each approach. By addressing these three key challenges, we aim to advance the state of movie recommendation systems, providing more effective and user-centric solutions that cater to the diverse preferences and needs of movie enthusiasts. Similarity-based models (memory-based) combined with matrix factorization-based models (model-based) were used to generate an optimal hybrid movie recommendation model. We evaluated the models using a large dataset of movies and we showed that the proposed model is both efficient and scalable. [ABSTRACT FROM AUTHOR]

This study uses the Random Forest algorithm to evaluate the efficiency of Hadoop, and Spark distributed computing systems for intrusion detection, highlighting the growing importance of efficient distributed systems in handling big data. This research aims to assess and compare the performance of Hadoop and Spark in the context of an intelligent intrusion detection system. We use the Random Forest machine learning algorithm to train and test the system. The methods developed an intrusion detection system using Hadoop and Spark frameworks, followed by a thorough performance assessment using a real-world dataset. The problem this study tackles is the ever-increasing demand for processing data swiftly and accurately in a distributed fashion. We aim to identify the strengths and weaknesses of Hadoop and Spark in the context of machine learning-based intrusion detection. The “intelligent network detection system for intrusions” in this study uses a sophisticated security system using machine learning algorithms to detect potential intrusions, assessing Hadoop and Spark's performance in realworld scenarios and handling large-scale data processing. The findings provide insightful information about the efficacy and efficiency of distributed systems in machine learning activities, which can help select big data application frameworks. [ABSTRACT FROM AUTHOR]

When there is a need to make an ultimate decision about the unique features of big data platforms, one should note that they have configurable parameters. Apache Spark is an open-source big data processing platform that can process real-time data, and it requires an advanced central processing unit and high memory capacity. Therefore, it gives us a great number of configurable parameters such as the number of cores and driver memory that are tuned during the execution. Different from the preceding works, in this study, a Kriging-based multi-objective optimization method is developed. Kriging-based means executing a surrogate model to create a response surface by providing a set of optimal solutions. The most important advantage of the proposed method over the alternatives is that it consists of three fitness functions. The method is evaluated on the MLlib library and the benchmarks of Hibench. MLlib provides various machine learning algorithms that are suitable to execute on resilient distributed data sets. The experimental results show that the proposed method outperformed the alternatives in hypervolume improvement and reducing uncertainty. Further, the results support the hypothesis that focusing on the parameters associated with data compression and memory usage improves the effectiveness of multi-objective optimization methods developed for Spark. Multi-objective optimization leads to an inevitable complexity in Spark due to the dimensionality of objective functions. Despite the fact that simplifying the setup and steps of optimization has proven to be the most effective way to reduce that complexity, it is not very effective to avoid ambiguity of the Pareto front. While the proposed method achieved 1.93x speedup in benchmark experiments, there is a remarkable difference (0.63 of speedup) between the speedup of our method and that of the closest competitor. Increasing the number of cores in multi-objective optimization does not contribute to speedup; rather, it leads to waste of CPU sources. Instead, the optimal number of cores should be determined by checking the changes of speedup with varying Spark configurations. [ABSTRACT FROM AUTHOR]

In recent years, technological advancements have been replicated in various industries, including sports medicine. Recent developments, such as big data analytics and data mining, which have revolutionized medical services in sports, are apparent in this transformation. This technological shift is motivated by the need to enhance athletic performance, prevent injuries, and offer individualized health advice. Modern lifestyles have simultaneously increased people's attention to their health, creating a demand for better medical services. However, China's ability to provide superior medical care needs to be improved due to a lack of medical resources and an ever-increasing patient population. To address these challenges, this research paper presents an integrated framework that leverages Spark-based big data analytics and the XGBoost algorithm. The framework aims to provide a robust sports medical service encompassing real-time health monitoring and data-driven insights. Powered by the formidable distributed computing platform Spark, it adeptly manages extensive sports data generated during training and events, facilitating instant health evaluations. Incorporating the XGBoost algorithm for data mining amplifies health prediction and recommendation capabilities. Renowned for its predictive prowess, XGBoost excels in discerning intricate sports data patterns and trends. Its proficiency in tackling intricates feature selection and modeling tasks ensures precision and actionable insights. Empirical findings underscore substantial enhancements in sports medical services. When applied to chronic disease datasets, the XGBoost algorithm garnered an impressive 93% trust rate. In contrast to conventional methods like K-Nearest Neighbors (KNN), Random Forest (RF), Decision Trees (DT), Support Vector Machines (SVM), Naïve Bayes (NB), and Logistic Regression (LR), the proposed framework consistently outperforms these established techniques. This remarkable performance underscores the transformative potential of the integrated framework in revolutionizing sports medical services. [ABSTRACT FROM AUTHOR]

Distributed deep learning is a promising approach for training and deploying large and complex deep learning models. This paper presents a comprehensive workflow for deploying and optimizing the YOLACT instance segmentation model as on big data clusters. OpenVINO, a toolkit known for its high-speed data processing and ability to optimize deep learning models for deployment on a variety of devices, was used to optimize the YOLACT model. The model is then run on a big data cluster using BigDL, a distributed deep learning library for Apache Spark. BigDL provides a high-level programming interface for defining and training deep neural networks, making it suitable for large-scale deep learning applications. In distributed deep learning, input data is divided and distributed across multiple machines for parallel processing. This approach offers several advantages, including the ability to handle very large data that can be stored in a distributed manner, scalability to decrease processing time by increasing the number of workers, and fault tolerance. The proposed workflow was evaluated on virtual machines and Azure Databricks, a cloud-based platform for big data analytics. The results indicated that the workflow can scale to large datasets and deliver high performance on Azure Databricks. This study explores the benefits and challenges of using distributed deep learning on big data clusters for instance segmentation. Popular distributed deep learning frameworks are discussed, and BigDL is chosen. Overall, this study highlights the practicality of distributed deep learning for deploying and scaling sophisticated deep learning models on big data clusters. [ABSTRACT FROM AUTHOR]

Spatial database is a spatial information database and is the core component of geographic information systems (GIS). Aiming at the problem that time complexity of k-nearest neighbor (kNN) querying algorithms are proportionate to scale of training samples, an efficient query method for spatial database based on the Spark framework and the reversed k-nearest neighbor (RkNN) is proposed. Firstly, based on the Spark framework, a two-layer indexing structure based on grid and Voronoi diagram is constructed, and an efficient filtering and a refining processing algorithm are proposed. Secondly, the filtering step of proposed algorithm is used to obtain the candidates, and the refining step is used to remove the candidates. Finally, the candidate sets from different regions are merged to get the final result. Results of experiments on real-world datasets validate that the proposed method has better query performance and better stability and significantly improves the processing speed. [ABSTRACT FROM AUTHOR]

This study introduces UWF-ZeekDataFall22, a newly created dataset labeled using the MITRE ATT&CK framework. Although the focus of this research is on classifying the never-before classified resource development tactic, the reconnaissance and discovery tactics were also classified. The results were also compared to a similarly created dataset, UWF-ZeekData22, created in 2022. Both of these datasets, UWF-ZeekDataFall22 and UWF-ZeekData22, created using Zeek Conn logs, were stored in a Big Data Framework, Hadoop. For machine learning classification, Apache Spark was used in the Big Data Framework. To summarize, the uniqueness of this work is its focus on classifying attack tactics. For UWF-ZeekdataFall22, the binary as well as the multinomial classifier results were compared, and overall, the results of the binary classifier were better than the multinomial classifier. In the binary classification, the tree-based classifiers performed better than the other classifiers, although the decision tree and random forest algorithms performed almost equally well in the multinomial classification too. Taking training time into consideration, decision trees can be considered the most efficient classifier. [ABSTRACT FROM AUTHOR]

The microwave diagnostics of discharges occurring in an artificial cloud of charged water droplets created in an open air simulating the environment of thunderclouds is implemented. An artificial cloud with a droplet size of about 1 µm is opaque in the visible range, so intra-cloud discharges are not available for investigation by traditional methods in the spark discharge physics based on the registration of visible discharge radiation. Microwaves pass through such a cloud without noticeable attenuation, they interact only with the plasma of discharges occurring in the cloud. The probing microwave radiation had a wavelength of 8 mm. The attenuation of microwaves passed through the cloud was measured with temporary resolution of about 10 ns. The temporal characteristics of intracloud discharges were investigated. [ABSTRACT FROM AUTHOR]

We consider the formation of an electric spark from a point anode in atmospheric air when the primary spark channel from a very early stage of its evolution immediately begins to develop in the form of a cluster of highly ionized plasma microchannels. Using laser probing at a wavelength of 1064 nm, spark microchannels are resolved in the near-anode region of the discharge, and the data on the magnitude and distribution of the plasma electron density of single microchannels are obtained. It is shown that during the first nanoseconds after the discharge gap breakdown, the electron density in individual microchannels is distributed nonuniformly and ranges from 1018 to 5 × 1019 cm–3, with characteristic microchannel diameters being of 20–50 μm. [ABSTRACT FROM AUTHOR]

High altitude relight is a critical aspect of the aeronautical engine certification and may be addressed with the numerical simulation of two-phase ignition. However, such configurations are stiff and combined with local evaporation may lead to numerical issues. This paper provides several methods to perform two-phase ignition simulations using analytically reduced chemistry in the context of unstructured large Eddy simulation and Euler–Lagrange formalism. Firstly, an exponential formulation combined with a local and dynamic sub-cycling of the stiff chemistry is demonstrated to allow stable integration at the flow time step. Secondly, a particle-bursting method is applied to limit the impact of stiffness induced by the Lagrangian point-source approach in fine meshes. These methods are then applied in the simulation of ignition of a mono-disperse, multi-component kerosene spray in air. The use of the analytically reduced chemistry model enables us to describe in detail the chemical structure of the flame kernel during its formation. Moreover, local increase of fuel concentration is observed as the ignition proceeds which has a large influence on the combustion processes and the flame kernel development. [ABSTRACT FROM AUTHOR]

Twitter has become a key element of political discourse in candidates’ campaigns. The political polarization on Twitter is vital to politicians as it is a popular public medium to analyze and predict public opinion concerning political events. The analysis of the sentiment of political tweet contents mainly depends on the quality of sentiment lexicons. Therefore, it is crucial to create sentiment lexicons of the highest quality. In the proposed system, the domain-specific of the political lexicon is constructed by using the supervised approach to extract extreme political opinions words, and features in tweets. Political multi-class sentiment analysis (PMSA) system on the big data platform is developed to predict the inclination of tweets to infer the results of the elections by conducting the analysis on different political datasets: including the Trump election dataset and the BBC News politics. The comparative analysis is the experimental results which are better political text classification by using the three different models (multinomial naïve Bayes (MNB), decision tree (DT), linear support vector classification (SVC)). In the comparison of three different models, linear SVC has the better performance than the other two techniques. The analytical evaluation results show that the proposed system can be performed with 98% accuracy in linear SVC. [ABSTRACT FROM AUTHOR]

Apache Spark is a high-speed computing engine for processing massive data. With its widespread adoption, there is a growing need to analyze its correctness and temporal properties. However, there is scarce research focused on the verification of temporal properties in Spark programs. To address this gap, we employ the code-level runtime verification tool UMC4M based on the Modeling, Simulation, and Verification Language (MSVL). To this end, a Spark program S has to be translated into an MSVL program M, and the negation of the property P specified by a Propositional Projection Temporal Logic (PPTL) formula that needs to be verified is also translated to an MSVL program M1; then, a new MSVL program "M and M1" can be compiled and executed. Whether program S violates the property P is determined by the existence of an acceptable execution of "M and M1". Thus, the key issue lies in how to formalize model Spark programs using MSVL programs. We previously proposed a solution to this problem—using the MSVL functions to perform Resilient Distributed Datasets (RDD) operations and converting the Spark program into an MSVL program based on the Directed Acyclic Graph (DAG) of the Spark program. However, we only proposed this idea. Building upon this foundation, we implement the conversion from RDD operations to MSVL functions and propose, as well as implement, the rules for translating Spark programs to MSVL programs based on DAG. We confirm the feasibility of this approach and provide a viable method for verifying the temporal properties of Spark programs. Additionally, an automatic translation tool, S2M, is developed. Finally, a case study is presented to demonstrate this conversion process. [ABSTRACT FROM AUTHOR]

The stochastic nature of environmental factors that govern the behavior of fire, such as wind and fuel, exposes wildfire modeling to a degree of uncertainty. In order to produce more realistic wildfire predictions, it is, therefore, necessary to incorporate these uncertainties within wildfire models in a way that reflects the influence of environmental stochasticity on wildfire propagation. Otherwise, the risks of the potential danger of a given wildfire may be under-represented. Specifically, environmental stochasticity in the form of wind variability results in considerable uncertainty in the output of fire spread models. Here, we consider two stochastic wind models and their implementation in the spark fire simulator framework to capture the environmental uncertainty related to wind variability. The results are compared with the output from purely deterministic wildfire spread models and are discussed in the context of the potential ramifications for wildfire risk management. [ABSTRACT FROM AUTHOR]

Miller and Urey applied electric sparks to a reducive mixture of CH4, NH3, and water to obtain a complex organic mixture including biomolecules. In this study, we examined the impact of temperature, initial pressure, ammonia concentration, and the spark generator on the chemical profile of a Miller–Urey-type prebiotic broth. We analyzed the broth composition using Gas Chromatography combined with Mass Spectroscopy (GC/MS). The results point towards strong compositional changes with the nature of the spark. Ammonia exhibited catalytic properties even with non-nitrogen-containing compounds. A more elevated temperature led to a higher variety of substances. We conclude that to reproduce such a broth as well as possible, all the studied parameters need to be tightly controlled, the most difficult and important being spark generation. [ABSTRACT FROM AUTHOR]

The purpose of this study was to determine how the research intervention called Healthy Eaters, Lifelong Movers (HELM) and associated San Luis Valley Physical Education Academy (SLVPEA) influenced teachers' beliefs about physical education and the extent to which they sustained pedagogical changes over time. Seventeen physical educators who completed the 2-year intervention were interviewed 3 years later, and data collected during HELM/SLVPEA using the System for Observing Fitness Instruction Time were analyzed to create an individual change profile. Mean difference of System for Observing Fitness Instruction Time variables at baseline and postintervention was analyzed using dependent, paired-samples t tests, treating each participant as a separate case. Qualitative data were analyzed using a standard interpretive approach and constant comparison methodology. Teachers made significant changes during HELM/SLVPEA and maintained these changes 3 years later. Their beliefs about physical education were altered, and many reported feeling less marginalized. The provision of resources along with ongoing site support facilitated changes in beliefs and practice. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Fuzhou University is the property of Journal of Fuzhou University, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

An improved Hybrid Collaborative Filtering algorithm (H-CF) is proposed, addressing the issues of data sparsity, low recommendation accuracy, and poor scalability present in traditional collaborative filtering algorithms. The core of H-CF is a linear weighted hybrid algorithm based on the Latent Factor Model (LFM) and the Improved Item Clustering and Similarity Calculation Collaborative Filtering Algorithm (ITCSCF). To begin with, the items are clustered based on their attribute dimension, which accelerates the computation of the nearest neighbor set. Subsequently, H-CF enhances the formula for scoring similarity by penalizing popular items and optimizing unpopular items. This improvement enhances the rationality of scoring similarity and reduces the impact of data sparseness. Furthermore, a weighting function is employed to combine the various improved algorithms. The balance factor of the weighting function is dynamically adjusted to attain the optimal recommendation list. To address the real-time and scalability concerns, the algorithm leverages the Spark big data distributed cluster computing framework. Experiments were conducted using the public dataset MovieLens, where the improved algorithm's performance was compared against the algorithm before enhancement and the algorithm running on a single machine. The experimental results demonstrate that the improved algorithm outperforms in terms of data sparsity, recommendation personalization, accuracy, recall, and efficiency. [ABSTRACT FROM AUTHOR]