Search

Algorithm is the core category and basic methods of the digital age, and advanced technologies such as big data, artificial intelligence, and blockchain all need to rely on various algorithm designs or take the algorithm as the underlying principle. However, due to the characteristics of algorithm design, application, and technology itself, there are also hidden worries such as algorithm black-box, algorithm discrimination, and difficulty in accountability in the operation process to varying degrees. This paper summarizes these problems into three aspects: unexplainable, self-reinforcing and autonomous. Facing the opportunities and risks generated by the application of the algorithm in national governance, while actively promoting the development of algorithm technology to continuously promote the modernization process of national governance, it is also necessary to increase the governance of the algorithm. The practice has proved that enhancing the interpretability of algorithm, optimizing algorithm design, and adopting legal regulatory algorithm are the basic approaches to effective regulatory algorithm in the era of intelligent governance. [ABSTRACT FROM AUTHOR]

With the rapid development of AI and big data mining technologies, computerized medical decision-making has become increasingly prominent. The aim of high-utility pattern mining (HUPM) is to discover meaningful patterns in medical databases that contribute to maximizing the utility from the perspective of diagnosis. However, HUPM pays less attention to the interpretability and explainability of these patterns in medical decision-making scenarios. This paper proposes a novel algorithm called the Improved fuzzy high-utility pattern mining (IF-HUPM) to address this problem. First, the paper applies a fuzzy preprocessing method to divide the fuzzy intervals of a medical quantitative data set, which enhances the fuzziness and interpretability of the data. Next, in the process of IF-HUPM, both fuzzy tree and list structures are employed to calculate fuzzy high-utility values. By combining the characteristics of the one-stage and two-stage algorithms of HUPM, an adaptive-phase Fuzzy HUPM hybrid frame is proposed. The experimental results demonstrate that the proposed IF-HUPM algorithm enhances both accuracy and efficiency and the mining process requires less time and space on average. [ABSTRACT FROM AUTHOR]

The location planning of relief distribution centres (DCs) is crucial in humanitarian logistics as it directly influences the disaster response and service to the affected victims. In light of the critical role of facility location in humanitarian logistics planning, the study proposes a two-stage relief distribution location problem. The first stage of the model determines the minimum number of relief DCs, and the second stage find the optimal location of these DCs to minimize the total cost. To address a more realistic situation, restrictions are imposed on the coverage area and capacity of each DCs. In addition, for optimally solving this complex NP-hard problem, a novel two-phase algorithm with exploration and exploitation phase is developed in the paper. The first phase of the algorithm i.e., exploration phase identifies a near-optimal solution while the second phase i.e. exploitation phase enhances the solution quality through a close circular proximity investigation. Furthermore, the comparative analysis of the proposed algorithm with other well-known algorithms such as genetic algorithm, pattern search, fmincon, multistart and hybrid heuristics is also reported and computationally tested from small to large data sets. The results reveal that the proposed two-phase algorithm is more efficient and effective when compared to the conventional metaheuristic methods. [ABSTRACT FROM AUTHOR]

The large amount of data generated every day makes necessary the re-implementation of new methods capable of handle with massive data efficiently. This is the case of Association Rules, an unsupervised data mining tool capable of extracting information in the form of IF-THEN patterns. Although several methods have been proposed for the extraction of frequent itemsets (previous phase before mining association rules) in very large databases, the high computational cost and lack of memory remains a major problem to be solved when processing large data. Therefore, the aim of this paper is three fold: (1) to review existent algorithms for frequent itemset and association rule mining, (2)to develop new efficient frequent itemset Big Data algorithms using distributive computation, as well as a new association rule mining algorithm in Spark, and (3) to compare the proposed algorithms with the existent proposals varying the number of transactions and the number of items. To this purpose, we have used the Spark platform which has been demonstrated to outperform existing distributive algorithmic implementations. [ABSTRACT FROM AUTHOR]

Data mining has reached a state that is difficult to break through, while the scale of data is growing rapidly, due to the lack of traditional computing power and limited data storage space. Efficient and accurate extraction of valuable information from massive data has become a challenge. Researchers have combined quantum computing with data mining to address this problem, hence the concept of quantum data mining has emerged. The fundamental tenets of quantum physics are adhered to for information transmission and computing operations in quantum data mining, which use the states of minuscule particles to represent and process information. Quantum data mining are based on the characteristics of quantum computing, such as superposition and entanglement, which make the ability of computational and information extraction effectively improved. The paper discusses and summarizes the relevant literature on quantum data mining in recent 3 years. After introducing relevant basic concepts of quantum computing, quantum data mining is presented in five aspects: quantum data classification, quantum data clustering, quantum dimensionality reduction, quantum association rules, quantum linear regression, and quantum causal analysis. These approaches, based on quantum computing, offer new perspectives and tools for handling complex data mining tasks. In conclusion, the development of quantum data mining is promising and crucial to overcome the difficulties associated with large-scale data mining. [ABSTRACT FROM AUTHOR]

In this paper, the issues on current optimization methods for identifying modern digital camera photosensors are presented. Methods for improving the reliability of identification of digital cameras are discussed, i.e., determining the digital camera used to take a particular photograph. Homogeneous images were optically recorded to form a noise signature, and sets of amateur images were tested for identification of three types of cameras from the images. Digital image filtering and image identity metric were optimized. An optimal digital filter was selected to evaluate smoothed images to obtain the noise signatures of the identified cameras. An optimal identity criterion was obtained by comparison of camera noise signatures. Using an optimal filter and identity criterion allowed increasing, on average, the reliability of identifying the camera used to take a particular photograph by > 60 times. This work provides reference to solve image problems in image registration and processing, security, forensics, and big data analysis. [ABSTRACT FROM AUTHOR]

Collecting and analyzing data from all devices to improve the efficiency of business processes is an important task of Industrial Internet of Things (IIoT). In the age of data explosion, extensive text data generated by the IIoT have given birth to a variety of text representation methods. The task of text representation is to convert the natural language to a form that computer can understand with retaining the original semantics. However, these methods are difficult to effectively extract the semantic features among words and distinguish polysemy in natural language. Combining the advantages of convolutional neural network (CNN) and variational autoencoder (VAE), this paper proposes an intelligent CNN-VAE text representation algorithm as an advanced learning method for social big data within next-generation IIoT, which help users identify the information collected by sensors and perform further processing. This method employs the convolution layer to capture the local features of the context and uses the variational technique to reconstruct feature space to make it conform to the normal distribution. In addition, the improved word2vec model based on topical word embedding (TWE) is utilized to add topical information to word vectors to distinguish polysemy. This paper takes the social big data as an example to illustrate the way of the proposed algorithm applied in the next-generation IIoT and utilizes Cnews dataset to verify the performance of proposed method with four evaluating metrics (i.e., recall, accuracy, precision, and F1-score). Experimental results indicate that the proposed method outperforms word2vec-avg and CNN-AE in K-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM) classifiers and distinguishes polysemy effectively. [ABSTRACT FROM AUTHOR]

Data mining algorithms can process target data and extract useful hidden information, which is helpful for decision making. However, current mining algorithms have some shortcomings such as time-consuming processing of big data or inability to process massive data. Since data mining technology cannot be used in the traditional cloud platform environment, it is necessary to improve the algorithm to make it more suitable for the cloud platform environment. By analyzing the actual application process of BP classification algorithm, this paper expounds the practicability of BP classification algorithm, analyzes the data mining process based on Hadoop cloud platform, and expounds the development ideas of BP classification algorithm. The source of data mining algorithm supported by cloud computing is discussed. Finally, Hadoop cloud platform. This paper designed the corresponding system architecture and data interface, and established a suitable test environment for this system, and completed the simulation experiment test through the design. The calculation time of this algorithm is proportional to the amount of data, showing a linear relationship. In data mining, the optimized BP algorithm in this paper can significantly save resources in terms of spatial features. This paper designs an optimized operating system based on Hadoop platform through comprehensive analysis and improvement algorithm. [ABSTRACT FROM AUTHOR]

Online streaming feature selection (OSFS) methods are used to dynamically update the feature space as well as remove irrelevant and redundant features from the data. Since most Big Data in real-world applications are generated in the form of data streams, effective methods should be developed in this area. Further, methods with low computational complexity are required to make online decisions. In this paper, the OSFS process is modeled as a multi-objective optimization problem. To the best of our knowledge, this is the first time that the concept of Pareto dominance has been applied to find the optimal subset of features in OSFS. When a new feature arrives, it is evaluated in the multi-objective space. The non-dominated features are the optimal subset for each timestamp. We proposed an efficient and effective method which enhances the classification accuracy in OSFS by minimizing the number of features within a short time. In addition, the proposed method is insensitive to the feature streams. Experiments are conducted using two classifiers and seven OSFS methods, including OSFSMI, K-OFSD, OFS-A3M, OFS-Density, Alpha-Investing, SAOLA, and OFSS-FI. [ABSTRACT FROM AUTHOR]

This paper proposes a quantitative evaluation algorithm based on a comprehensive analysis of material's shape, texture, color, style, and other characteristics using indoor wireless networking design and cloud data to improve the application effectiveness of ecological environment materials. It also adopts a data-driven method to quantitatively evaluate the application effectiveness of ecological environment materials using indoor wireless network design based on preliminary feature information. Specifically, this work reconstructs the feature space of ecological environment materials by extracting dimensionality features related to the quantitative evaluation, and analyzes the mutual information features of big data fusion. Moreover, a quantitative regression analysis and statistical model construction is used to evaluate the application effectiveness by using preliminary feature information as a constraint condition. In addition, this study employs the spatial effect element feature analysis method to quantitatively analyze and rigorously test the application effectiveness assessment. The empirical results show that the proposed quantitative evaluation algorithm has good accuracy and high reliability, which can effectively improve the application quality of ecological environment materials in indoor wireless network design. [ABSTRACT FROM AUTHOR]

In order to explore potential value of explosively growing data in thermal power unit, this paper proposes a big data mining method based on Hadoop-based Spark cluster-computing framework and algorithms. Firstly, positive and negative balance methods are used to accurately obtain actual net coal consumption, and maximum information coefficient method is used to select all parameters related to optimization objectives. Then, Spark-based Mini-Batch K-means algorithm and Elbow method are constructed to divide whole operating modes. After that, all data are discretized and mapped to corresponding intervals by using Spark-based Elbow method and Mini-Batch K-means algorithm. Finally, Spark-based parallel FP-growth algorithm is used to deeply mine the potential relationships and laws. To verify the proposed method, a 350-MW thermal power unit is taken as a study case. The important conclusions are as follows: (1) the proposed Spark-based Mini-Batch K-means algorithm reduces the calculation time by 57.11% compared with Mini-Batch K-means algorithm, and 85.61% calculation time compared with K-means algorithm. The proposed Spark-based FP-growth algorithm reduces computational time by 32.8% compared with FP-growth algorithm. (2) Strong association rules of whole operating modes are mined, and operating optimization guidance schemes for important parameters are obtained. Take operating mode 1 as an example: if the optimal result can be reasonably applied, it can save 2.942 g coal per kilowatt hour. (3) Besides, we have found out some other potential relationships among parameters, which have important reference value for on-site operators to analyze economy of the thermal power unit. [ABSTRACT FROM AUTHOR]

Fuzzy C-Means is a well-known fuzzy clustering technique. Although FCM can cover the uncertainty problem by forming overlapping clusters, it involves issues such as sensitivity to noise and outliers, the fuzzification parameter m, and the initial guess of cluster centers. There are various improvements made to FCM to tackle these limitations. One such extension, FCOM, has proved to be an efficient method for handling noise and outliers. However, it still suffers from the last two ones. In the present paper, inspired by FCOM, a new model is designed to solve these issues. Whereas varying the degree of fuzziness m leads to different clusters, determining the appropriate value optimizes solutions. However, manually tuning this parameter can be time-consuming, especially when dealing with large data sets. To mitigate the dependence on this parameter, the proposed model utilizes the entropy theory to control the uncertainty associated with the input data. Extensive evaluations are conducted on benchmark datasets to analyze the impact of m on cluster formation and clustering results. The competitive results confirm the effectiveness of the proposed model for handling fuzziness degree and its capability to accelerate convergence to optimal solutions. Moreover, the results show that the proposed model discovers vague boundaries precisely. [ABSTRACT FROM AUTHOR]

In sports, advance sensing technologies generate massive amount of unstructured telemedicine data that need to be refined for accurate diagnosis of underlying diseases. For accurate prediction of diseases and classification of athletes' data, deep learning algorithms are frequently used at the cloud. However, the transmission of raw data of athletes to the cloud faces numerous challenges. Among them, security and privacy are a major challenge in view of the sensitive and personal information present within the unstructured data. In this paper, first we present a data block scrambling algorithm (without key management) for secured transmission and storage of ECG (electrocardiogram) data of table tennis players at the cloud. A small piece of original data stored at the cloud is used for scrambling the massive amount of remaining ECG data. The secured telemedicine data is then imported into Hadoop Distributed File System for data management, which is read by Spark framework to form Resilient Distributed Datasets. Finally, a deep learning approach is used that extracts useful features, learns the related information, and weights and sums the feature vectors at different layers for classification. Theoretical analysis proves that our proposed approach is highly robust and resilient to brute force attacks and at the same time has a much better accuracy, sensitivity, and specificity as compared to the existing approaches. [ABSTRACT FROM AUTHOR]

With the development of the big data era, the manufacturing industry is focusing on integrating the technologies related to the evolving intelligent industry. Intelligent manufacturing is the latest form of advanced manufacturing development. In order to pay attention to the design and evaluation status of manufacturing system, we understand the connotation of manufacturing system from the perspective of actual manufacturing production and sales. In this paper, the evaluation model and algorithm of Intelligent Manufacturing System Based on pattern recognition and big data are proposed. In the process of studying the information transmission of intelligent manufacturing system, the evaluation hierarchy algorithm of intelligent control system is improved with reference to several modeling methods. Finally, an intelligent manufacturing system suitable for processing big data is designed. In the experiment in workshop A, with the help of computer simulation of artificial intelligent activities, the intelligent manufacturing system analyzes the processing and production data, obtains the dynamic requirements of products in the machining workshop, and provides judgment basis for the construction of dynamic units in enterprises. The experimental results show that after the change of the production organization model using the seasonal change forecasting method, the introduction of the smart manufacturing system, the sales production of this shop has been increasing, and in the 20th month has increased seven times more than the initial, while the overall situation of the predicted sales and actual sales and costs, the superiority of using the smart manufacturing evaluation model is derived. [ABSTRACT FROM AUTHOR]

The aim of this paper is to identify the co-expressed potential genes that may serve for the development of the portions of normal or tumor. This paper differentiates the co-expressed genes into normal samples and tumor samples from gene expression dataset GSE25066. Since the dataset has vague boundaries and having common characteristics between the clusters, identifying the subgroups contain similar gene expression is really a tricky task one. Therefore, this paper introduces an effective fuzzy iterative clustering algorithm by incorporating kernel function, possibilistic c-means, fuzzy memberships, neighborhood information, median of neighboring objects and penalty term. The performances of the proposed clustering techniques have been shown through the succession experimental works on GSE25066. The effects of clustering results have been proved through comparing the resulted classes with ground truth. [ABSTRACT FROM AUTHOR]

The incredible increase in the volume of remote sensing data has made the concept of Remote Sensing as Big Data reality with recent technological developments. Remote sensing image processing is characterized with features of massive data processing and intensive computation, which makes the processes difficult. To optimize the remote sensing image processing for GPU, compute unified device architecture (CUDA) is widely used to implement remote sensing algorithms. However, the usage of GPU in remote sensing image processing has been constrained by the complexity of its implementation and configuration. Therefore, how to take full advantage of the parallel organization of GPU architecture is awfully challenging. In this paper, a dynamic adaptive acceleration (DAA) method is proposed to determine calculation parameters of GPU adaptively and preprocess the input remote sensing images on host dynamically. By this method, we determine calculation parameters according to the hardware parameters of GPU firstly. And then, the input remote sensing images are reconstructed based on the calculation parameters. Finally, the preprocessed image blocks are arranged to stream tasks and executed on GPU respectively. The effectiveness of the proposed DAA method in accelerating remote sensing algorithm with point operations was verified by experiments in this paper, and the experimental results indicated that the DAA method can obtain better performance than traditional methods. [ABSTRACT FROM AUTHOR]

In the big data era, traditional data mining technology cannot meet the requirements of massive data processing with the background of intelligent manufacturing. Aiming at insufficient computing power and low efficiency in mining process, this paper proposes a improved K-means clustering algorithm based on the concept of distributed clustering in cloud computing environment. The improved algorithm (T.K-means) is combined with MapReduce computing framework of Hadoop platform to realize parallel computing, so as to perform processing tasks of massive data. In order to verify the practical performance of T.K-means algorithm, taking machining data of milling Ti-6Al-4V alloy as the mining object. The mapping relationship among cutting parameters, surface roughness, and material removal rate is mined, and the optimized value for cutting parameters is obtained. The results show that T.K-means algorithm can be used to mine the optimal cutting parameters, so that the best surface roughness can be obtained in milling Ti-6Al-4V titanium alloy. [ABSTRACT FROM AUTHOR]

The growing data have brought tremendous pressure for query processing and storage, so there are many studies that focus on using GPU to accelerate join operation, which is one of the most important operations in modern database systems. However, existing GPU acceleration join operation researches are not very suitable for the join operation on big data. Based on this, this paper speeds up nested loop join, hash join and theta join, combining Hadoop with GPU, which is also the first to use GPU to accelerate theta join. At the same time, after the data pre-filtering and pre-processing, using MapReduce and HDFS in Hadoop proposed in this paper, the larger data table can be handled, compared to existing GPU acceleration methods. Also with MapReduce in Hadoop, the algorithm proposed in this paper can estimate the number of results more accurately and allocate the appropriate storage space without unnecessary costs, making it more efficient. Experimental results show that comparing with GPU-based approach without Hadoop, our approach increases the speed by 1.5–2 times, and comparing with the Hadoop-based approaches without GPU, our approach increases the speed by 1.3–2 times. [ABSTRACT FROM AUTHOR]

This paper deals with the current Blockchain adopted in the various types of trading markets supporting smart contracts that process text-based transaction information only when exchanging cryptocurrencies. Even though the Blockchain itself is decentralized, services can still be provided through a centralized system in order to provide adequate services to the users. It also allows the Autochain to apply Blockchain technology to existing businesses conducted under the Fourth Industrial Revolution to realize higher productivity or competitiveness and improve profitability to a remarkable extent. The Autochain is adaptable to all Blockchain technologies, companies, or research works around the world to overcome the limitations of the existing Blockchain technology. It creates the Blockchain-platform ecology chain and ecosystem, making our lives more convenient while revolutionizing every industry. In addition, it automates all Blockchain-based systems and offers a base for constructing a system wherein everyone can easily provide, operate, and maintain Blockchain services. This paper discusses a number of Blockchain-related technologies and presents a decentralized Application (dApp) exclusively designed for mobile use, taking house rental service as an example. It also presents the existing Blockchain and the Blockchain database storage method and proposes the dApp based on it. [ABSTRACT FROM AUTHOR]

Big data analytics in cloud environments introduces challenges such as real-time load balancing besides security, privacy, and energy efficiency. This paper proposes a novel load balancing algorithm in cloud environments that performs resource allocation and task scheduling efficiently. The proposed load balancer reduces the execution response time in big data applications performed on clouds. Scheduling, in general, is an NP-hard problem. Our proposed algorithm provides solutions to reduce the search area that leads to reduced complexity of the load balancing. We recommend two mathematical optimization models to perform dynamic resource allocation to virtual machines and task scheduling. The provided solution is based on the hill-climbing algorithm to minimize response time. We evaluate the performance of proposed algorithms in terms of response time, turnaround time, throughput metrics, and request distribution with some of the existing algorithms that show significant improvements. [ABSTRACT FROM AUTHOR]

Recent trends in big data have shown that the amount of data continues to increase at an exponential rate. This trend has inspired many researchers over the past few years to explore new research direction of studies related to multiple areas of big data. The widespread popularity of big data processing platforms using MapReduce framework is the growing demand to further optimize their performance for various purposes. In particular, enhancing resources and jobs scheduling are becoming critical since they fundamentally determine whether the applications can achieve the performance goals in different use cases. Scheduling plays an important role in big data, mainly in reducing the execution time and cost of processing. This paper aims to survey the research undertaken in the field of scheduling in big data platforms. Moreover, this paper analyzed scheduling in MapReduce on two aspects: taxonomy and performance evaluation. The research progress in MapReduce scheduling algorithms is also discussed. The limitations of existing MapReduce scheduling algorithms and exploit future research opportunities are pointed out in the paper for easy identification by researchers. Our study can serve as the benchmark to expert researchers for proposing a novel MapReduce scheduling algorithm. However, for novice researchers, the study can be used as a starting point. [ABSTRACT FROM AUTHOR]

The maximum independent set algorithm for large-scale data semi-existing data is studied and the solving method of the largest independent set problem in large-map data is mainly analyzed in this paper. The specific research contents are mainly divided into semi-external map algorithm based on Greedy heuristic strategy, semi-external map algorithm based on swap and design, and implementation of semi-external graph algorithm processing function library. Experiments on a large number of real and artificially generated data sets show that the algorithm in this paper is very efficient both in time and in space. The largest independent set obtained by the algorithm can reach more than 96% of its theoretical upper bound for most of the data. [ABSTRACT FROM AUTHOR]

Algorithmic vision, the computational process of making meaning from digital images or visual information, has changed the relationship between the image and the human subject. In this paper, I explicate on the role of algorithmic vision as a technique of algorithmic governance, the organization of a population by algorithmic means. With its roots in the United States post-war cybernetic sciences, the ontological status of the computational image undergoes a shift, giving way to the hegemonic use of automated facial recognition technologies towards predatory policing and profiling practices. By way of example, I argue that algorithmic vision reconfigures the philosophical links between vision, image, and truth, paradigmatically changing the way a human subject is represented through imagistic data. With algorithmic vision, the relationship between subject and representation challenges the humanistic discourse around images, calling for a critical displacement of the human subject from the center of an analysis of how computational images make meaning. I will explore the relationship between the operative image, the image that acts but is not seen by human eyes, and what Louise Amoore calls an "emergent subject," a subject that is made visible through algorithmic techniques (2013). Algorithmic vision reveals subjects to power in a mode that requires a new approach towards analyzing the entanglement and invisiblization of the human in automated decision-making systems. [ABSTRACT FROM AUTHOR]

With the development of big data technologies, green cloud data centers have become a key factor in academia and industry. An energy-efficient cloud data center can save costs for cloud computing users. However, the problem of virtual machine mapping has always been a core problem. In the most existing research, the energy consumption generated by cloud data centers has become an important bottleneck restricting the technology of cloud computing. This paper establishes a consumption model of cloud data center energy and a virtual machine mapping rule. Based on this, a multi-dimensional double descending maximum padding priority (MD3MP2) virtual machine mapping algorithm is proposed. The algorithm can not only solve the one-dimensional virtual machine mapping problem of homogeneous data centers, but also successfully solve the multi-dimensional virtual machine mapping problem of homogeneous data centers. Finally, the algorithm is compared with four other algorithms. The experimental results show that the MD3MP2 algorithm is better than the compared algorithms. [ABSTRACT FROM AUTHOR]

Big data (BD) is attaining major attention in the information field due to the eruption of data in the preceding decade. Philosophical techniques of "query optimization (QO)" have an essential function in data retrieval as of a BD environment. Numerous cloud-centered distributed data processing platforms were developed to render effective as well as lucrative solutions for BD query optimization. Nevertheless, most techniques brought about higher "energy consumptions (EC)" along with low accuracy level because of the lack of deliberation of energy issues as well as query characteristics, correspondingly. To tackle the issues of query optimization process, this paper proposes an "efficient query optimization (EQO)" utilizing σ ANFIS load balancer in addition to the CaM-BW optimizer. The proposed technique comprises '2′ phases: (1) BD arrangement and (2) QO. In the initial phase, the BD is arranged by utilizing preprocessing, feature extraction, together with clustering. The MCoV-FCM algorithm takes care of the clustering. In the second phase, the σ ANFIS load balancer in addition to the CaM-BW optimizer concentrates on disregarding the energy-efficient query plans and enhancing the general query processing performance. Lastly, numerical simulation outcomes are rendered to display the proposed method's effectiveness. [ABSTRACT FROM AUTHOR]

Our experience of the present is defined by numbers, graphs and, increasingly, an algorithmically calculated future, based on the mathematical and formal reasoning that began with the rise of modern science in the sixteenth and seventeenth centuries. Today, this reasoning is further modified and extended in the form of computer-executed, algorithmic reasoning. Instead of fallible human reasoning, algorithms—based on mining databases for 'information'—are seen to provide more efficient processes, offering fast solutions. In this paper, then, I will follow Jan Patočka, who suggests that we live in an age of supercivilisation, one in which human reasoning has become self-sufficient, ceasing to depend on the supernatural or cultural traditions that previously guided human lives. My argument is that Patočka's analysis of supercivilisation can open up a different way to reflect on the 'spiritual foundation of our times'. As Patočka says, to reflect on our situation does not mean that we can change it, but reflection can give us a new understanding that will open up different ways to think about our human future. [ABSTRACT FROM AUTHOR]

With the development of quantum computation theory, some researchers further apply it to improve the efficiency of classical machine learning algorithms. Based on physical graph state, an efficient quantum version of classifier is proposed in this paper. Different from existing classical methods, realizing the proposed scheme is an entangling-subgraphs process of physical system to build the classifier by using the graph states. Resort to the efficiency of graph state, the quantum algorithm is more efficient to big data than the classical ones. [ABSTRACT FROM AUTHOR]

Consensus clustering can produce novel, stable, and robust clustering results. Consensus clustering intends to merge a few existing basic segments into a coordinated one, and this has been broadly perceived as a promising solution for heterogeneous data clustering for big data. Even though many clustering algorithms have been proposed, getting a decent quality segment with high effectiveness is still not yet decided. In this paper, we propose a scalable incremental fuzzy consensus clustering (SIFCC) algorithm for a big data framework. It has been implemented on Apache Spark cluster framework, a distributed data stream environment for handling big data by considering the data as a set of data subsets that are processed incrementally. Sparks work great for iterative algorithms by supporting in-memory calculations, scalability, etc. SIFCC not only facilitates efficient big data clustering, but also improves the quality of clusters, performs storage space optimization, and time complexity during clustering. To establish the comparison, we designed and implemented the scalable model of existing fuzzy consensus clustering (FCC) on Apache Spark cluster, named as a scalable fuzzy consensus clustering (SFCC). Extensive experiments on real-world datasets show that the SIFCC algorithm achieves the better potential for clustering of Big Data in comparison with SFCC. [ABSTRACT FROM AUTHOR]

Clustering is a fundamental tool for analyzing large data sets. A rich body of work has been devoted to designing data-stream algorithms for the relevant optimization problems such as k-center, k-median, and k-means. Such algorithms need to be both time and and space efficient. In this paper, we address the problem of correlation clustering in the dynamic data stream model. The stream consists of updates to the edge weights of a graph on n nodes and the goal is to find a node-partition such that the end-points of negative-weight edges are typically in different clusters whereas the end-points of positive-weight edges are typically in the same cluster. We present polynomial-time, O (n · polylog n) -space approximation algorithms for natural problems that arise. We first develop data structures based on linear sketches that allow the "quality" of a given node-partition to be measured. We then combine these data structures with convex programming and sampling techniques to solve the relevant approximation problem. Unfortunately, the standard LP and SDP formulations are not obviously solvable in O (n · polylog n) -space. Our work presents space-efficient algorithms for the convex programming required, as well as approaches to reduce the adaptivity of the sampling. [ABSTRACT FROM AUTHOR]

In this paper, considering year 2020 and Covid-19, we analyze medical imaging tools and their performance scores in accordance with the dataset size and their complexity. For this, we mainly consider AI-driven tools that employ two different types of image data, namely chest Computed Tomography (CT) and X-ray. We elaborate on their strengths and weaknesses by taking the following important factors into account: i) dataset size; ii) model fitting criteria (over-fitting and under-fitting); iii) transfer learning in the deep learning era; and iv) data augmentation. Medical imaging tools do not explicitly analyze model fitting. Also, using transfer learning, with fewer data, one could possibly build Covid-19 deep learning model but they are limited to education and training. We observe that, in both image modalities, neither the dataset size nor does data augmentation work well for Covid-19 screening purposes because a large dataset does not guarantee all possible Covid-19 manifestations and data augmentation does not create new Covid-19 cases. [ABSTRACT FROM AUTHOR]

With the introduction of wearable network technologies, smart healthcare is gradually being realized. Real-time health care monitoring system should be safe, effective, and patient-centered. This paper aims to analyze the opinions and emotions of medical and health products, and to design and develop monitoring systems by crawling multiple websites and reviews in a variety of formats. Data preprocessing is used to preprocess the quantized image, and finally, it is recommended in combination with the analysis results. This system combines a large amount of products' comment data, combined with MATLAB and other tools to achieve the visualization of big data analysis, by the use of the maximum entropy algorithm for comprehensive analysis and comparison. Support vector machine (SVM) algorithm is used to analyze and compare various parts of the medical image in detail. It is suitable for current medical applications. It can provide advice for consumers to purchase drugs and achieve real-time health monitoring objectives. [ABSTRACT FROM AUTHOR]

Osmotic Computing represents a glue solution able to manage the deployment and orchestration of interconnected microelements across heterogeneous physical and virtual infrastructures (e.g., IoT, Edge and Cloud nodes) according to the behavior of hardware and software components during the time. The adoption of Osmotic Computing is challenging, but addressing networking issues is a key research topic due to the emergence of new problems in terms of QoS requirements. In this paper, we analyze how to exploit well-known networking solutions, such as the Dijkstra's algorithm, and Big Data oriented technologies, such as the Hadoop and MapReduce, to provide efficient newtorking functionalities in Osmotic Computing. In particular, our objective is to minimize the routing path computation time in the software defined network (SDN) at the basis of microelement networking, as well as to ensure a global view and a high level of dynamism of our network topology. To accomplish this task, we process routing tables through a MapReduce based implementation of the Dijkstra's algorithm whenever a topology change occurs, and we export routing results into the SDN. Our experimental results show that our networking strategy drastically reduces the best path computation time whenever the network of microelements is very large. [ABSTRACT FROM AUTHOR]

The biggest feature of the era of big data is that people can easily generate, access, and make use of massive data resources. As one of the most important and popular kind of big data, location big data and its application technology provide users with convenient services. However, improper collection, analysis and publishing of location big data also brings huge crisis of personal privacy disclosure. Spatial decomposition is one of the effective ways to achieve the statistics publication of location big data. In order to make full use of the redundant characteristics of location big data in spatial and temporal distribution, a hierarchical differential privacy hybrid decomposition algorithm is proposed in this paper. In the first layer of decomposition, an adaptive density grid structure is used to cluster the location big data, which not only reduces the uniform assumption errors but also avoids noise errors caused by large number of empty nodes. In order to guide the reasonable decomposition for skewed grids in the second layer, a heuristic quad-tree decomposition algorithm based on regional uniformity is designed, which solved the difficult problem for determining stop condition of the top-down decomposition of two-dimensional space. Comparative experiments show that the hierarchical differential privacy hybrid decomposition algorithm proposed in this paper has good effect in improving the accuracy of regional counting queries. The proposed algorithm has low computational complexity and obvious advantages in the publishing environment of big data. [ABSTRACT FROM AUTHOR]

In the big data age, the traditional parallel collaborative filtering algorithm cannot meet the needs of data analysis in the efficiency and accuracy of data processing. Therefore, this paper improves the traditional parallel collaborative filtering algorithm, analyzes the execution flow of the collaborative filtering algorithm, discusses the shortcomings of the traditional parallel collaborative filtering algorithm, and then describes in detail the steps of improved the collaborative filtering algorithm from generating nodes scoring vectors, obtaining neighboring nodes and forming recommendation information. Finally, the improved parallel collaborative filtering algorithm is verified through three aspects of running time, speedup and recommended accuracy. Experimental results show that the improved parallel collaborative filtering algorithm proposed in this paper has better running efficiency and higher recommendation accuracy than traditional parallel algorithm based on co-occurrence matrix. [ABSTRACT FROM AUTHOR]

Clustering as a major task in data mining is responsible for discovering hidden patterns in unlabeled datasets. Finding the best clustering is also considered as one of the most challenging problems in data mining. Due to the problem complexity and the weaknesses of primary clustering algorithm, a large part of research has been directed toward ensemble clustering methods. Ensemble clustering aggregates a pool of base clusterings and produces an output clustering that is also named consensus clustering. The consensus clustering is usually better clustering than the output clusterings of the basic clustering algorithms. However, lack of quality in base clusterings makes their consensus clustering weak. In spite of some researches in selection of a subset of high quality base clusterings based on a clustering assessment metric, cluster-level selection has been always ignored. In this paper, a new clustering ensemble framework has been proposed based on cluster-level weighting. The certainty amount that the given ensemble has about a cluster is considered as the reliability of that cluster. The certainty amount that the given ensemble has about a cluster is computed by the accretion amount of that cluster by the ensemble. Then by selecting the best clusters and assigning a weight to each selected cluster based on its reliability, the final ensemble is created. After that, the paper proposes cluster-level weighting co-association matrix instead of traditional co-association matrix. Then, two consensus functions have been introduced and used for production of the consensus partition. The proposed framework completely overshadows the state-of-the-art clustering ensemble methods experimentally. [ABSTRACT FROM AUTHOR]

In this paper, we discuss a family of robust, high-dimensional regression models for quantile and composite quantile regression, both with and without an adaptive lasso penalty for variable selection. We reformulate these quantile regression problems and obtain estimators by applying the alternating direction method of multipliers (ADMM), majorize-minimization (MM), and coordinate descent (CD) algorithms. Our new approaches address the lack of publicly available methods for (composite) quantile regression, especially for high-dimensional data, both with and without regularization. Through simulation studies, we demonstrate the need for different algorithms applicable to a variety of data settings, which we implement in the cqrReg package for R. For comparison, we also introduce the widely used interior point (IP) formulation and test our methods against the IP algorithms in the existing quantreg package. Our simulation studies show that each of our methods, particularly MM and CD, excel in different settings such as with large or high-dimensional data sets, respectively, and outperform the methods currently implemented in quantreg. The ADMM approach offers specific promise for future developments in its amenability to parallelization and scalability. [ABSTRACT FROM AUTHOR]

The rise of high-resolution and high-throughput sequencing technologies has driven the emergence of such new fields of application as precision medicine. However, this has also led to an increase in the storage and processing requirements for the bioinformatics tools, which can only be provided by high-performance and massive data processing infrastructures. Such technologies allow the development of scalable, efficient and reliable bioinformatics tools. In this paper, a new implementation of the Basic Local Alignment Search Tool algorithm is presented. Our proposal, named Sparky-Blast, utilizes Cassandra database to store the different reference datasets and the Apache Spark processing framework to calculate the indexes and process the queries. This successful approach avoids the bottleneck that suffers the original BLAST version that is limited to the resources of a single machine. Sparky-Blast is capable of using the distributed resources of a Big-Data Cluster to process queries in parallel, thus, improving both the response time and the system throughput. At the same time, the use of a distributed architecture like Hadoop provides unlimited scalability from the point of view of both the hardware infrastructure and performance. [ABSTRACT FROM AUTHOR]

Single Linkage algorithm is a hierarchical clustering method which is most unsuitable for large dataset because of its high convergence time. The paper proposes an efficient accelerated technique for the algorithm for clustering univariate data with a merging threshold. It is a two-stage algorithm with the first one as an incremental pre-clustering step that uses the farthest neighbour principle to partially cluster the database by scanning it only once. The algorithm uses the Segment Addition Postulate as a major tool for accelerating the pre-clustering stage. The incremental approach makes it suitable for partial clustering of streaming data while collecting it. The Second stage merges these pre-clusters to produce the final set of Single Linkage clusters by comparing the biggest and the smallest data of each pre-cluster and thereby converging faster in comparison to those methods where all the members of the clusters are used for a clustering action. The algorithm is also suitable for fast-changing dynamic databases as it can cluster a newly added data without using all the data of the database. Experiments are conducted with various datasets and the result confirms that the proposed algorithm outperforms its well-known variants. [ABSTRACT FROM AUTHOR]

Due to the huge amount of data being generating from different sources, the analyzing and extracting of useful information from these data becomes a very complex task. The difficulty of dealing with big data optimization problems comes from many factors such as the high number of features, and the existing of lost data. The feature selection process becomes an important step in many data mining and machine learning algorithms to reduce the dimensionality of the optimization problems and increase the performance of the classification or clustering algorithms. In this paper, a set of hybrid and efficient genetic algorithms are proposed to solve feature selection problem, when the handled data has a large feature size. The proposed algorithms use a new gene-weighted mechanism that can adaptively classify the features into strong relative features, weak or redundant features, and unstable features during the evolution of the algorithm. Based on this classification, the proposed algorithm gives the strong features high priority and the weak features less priority when generating new candidate solutions. In the same time, the proposed algorithm tries to more concentrate on unstable features that sometimes appear and sometimes disappear from the best solutions of the population. The performance of proposed algorithms is investigated by using different datasets and feature selection algorithms. The results show that our proposed algorithms can outperform the other feature selection algorithms and effectively enhance the classification performance over the tested datasets. [ABSTRACT FROM AUTHOR]

Tremendous amounts of data are generated by sensors and connected devices with high velocity in a variety of forms and large volumes. These characteristics, defined as big data, need new models and methods to be processed in near real-time. The nature of decentralized large-scale data sources requires distributed algorithms in which it is assumed that the data sources are capable of processing their own data and collaborating with neighbor sources. The network objective is to make an optimal decision, while the data are processed in a distributed manner. New technologies, like next generation of wireless communication and 5G, introduce practical issues such as imperfect communication that should be addressed. In this paper, we study a generalized form of distributed algorithms for decision-making over decentralized data sources. We propose an optimal algorithm that uses optimal weighting to combine the resource of neighbors. We define an optimization problem and find the solution by applying the proposed algorithm. We evaluate the performance of the developed algorithm by using both mathematical methods and computer simulations. We introduce the conditions in which the convergence of proposed algorithm is guaranteed and prove that the network error decreases considerably in comparison with some of the known modern methods. [ABSTRACT FROM AUTHOR]

Workload scheduling in cloud computing is currently an active research field. Scheduling plays an important role in cloud computing performance, especially when the platform is used for big data analysis and as less predictable workloads dynamically enter the clouds. Finding the optimized scheduling solution with different parameters in different environments is still a challenging issue. In dynamic environments such as cloud, scheduling strategies should feature rapid altering to be able to adapt more easily to the changes in input workloads. However, achieving an optimized solution is an important issue, which has a trade-off with the speed of finding the solution. In this article, an ordinal optimization method is proposed that considers the volume of workloads, load balancing and the volume of exchanged messages among virtual clusters, considering the replications. The algorithm in the present paper is based on ordinal optimization (OO) and evolutionary OO. In any time periods, a criterion is calculated to determine the similarity of workloads in two-consequence time periods, which is appropriate for timely changes in the scheduling procedure. In this paper, considering more than one parameter, a proper scheduling would be created for each time period. This scheduler is an organization for the number of virtual machines for each virtual cluster, but if there is a desirable similarity between workloads of two-consequence time periods, this procedure would be ignored. The results show that a more optimized solution is obtained in comparison with the rated methods, such as blind pink, OO, Monte Carlo and eOO in a reasonable time. The suggested method is flexible and it is possible to change the weight ratio of the proposed criteria in different environments to be consistent with different environmental conditions. The results show that proposed method achieved up to 28% performance improvement in comparison with eOO. [ABSTRACT FROM AUTHOR]

Finding frequent itemsets in a continuous streaming data is an important data mining task which is widely used in network monitoring, Internet of Things data analysis and so on. In the era of big data, it is necessary to develop a distributed frequent itemset mining algorithm to meet the needs of massive streaming data processing. Apache Spark is a unified analytic engine for massive data processing which has been successfully used in many data mining fields. In this paper, we propose a distributed algorithm for mining frequent itemsets over massive streaming data named SWEclat. The algorithm uses sliding window to process streaming data and uses vertical data structure to store the dataset in the sliding window. This algorithm is implemented by Apache Spark and uses Spark RDD to store streaming data and dataset in vertical data format, so as to divide these RDDs into partitions for distributed processing. Experimental results show that SWEclat algorithm has good acceleration, parallel scalability and load balancing. [ABSTRACT FROM AUTHOR]

Evolutionary community discovery is a hot research topic related to the dynamic or temporal social networks. The communities detected in a dynamic network should get reasonable partition for the current network and do not deviate drastically from the previous ones. This paper is an extended version of our previous work in Gao et al. (in: Proceedings of the 2016 international conference on big data and smart computing (BigComp), pp 53–60, 2016). First, an evolutionary community discovery algorithm named EvoLeaders, which is inspired by TopLeaders algorithm, is proposed. Second, based on TopLeaders, an improved TopLeaders algorithm (i.e., AutoLeaders) is proposed. Experiments on three classic data sets are conducted, and experimental results show that the AutoLeaders can correctly find the number of communities and at the same time can discover reasonable communities. Third, the EvoAutoLeaders algorithm is proposed for detecting the communities in a dynamic network. Compared with the TopLeaders algorithm and EvoLeaders, experimental results over two real-world data sets demonstrate that the EvoAutoLeaders is more suitable for dynamic scenarios. [ABSTRACT FROM AUTHOR]

Despite the rapid improvement in dental health over the last few decades, a significant portion of our population continue seek dental care every year. Estimates show that 13% of adults seek dental care for dental infection or fatigue within four years. The Social and individual burden of this disease can be reduced by its early detection. However, the symptoms of teeth infection in the early stages are not clear, hence, it would be relatively difficult to predict teeth infections based solely on human skills and experience. Big Data (BD) technologies have a great potential in transforming dental care, as they have revolutionized other industries. In addition to reducing cost, they could save millions of lives and improve patient outcomes. This paper proposes a novel integrated prediction model that extracts hidden knowledge from radiographic datasets containing a large volume of dental X-ray images and utilizes this knowledge to predict dental infections. Initially, preprocessing techniques using morphological skeleton and mean approach is applied to eliminate noise and enhance the images. Next, Multi Scale Segmented Region (MSR) approach, Watershed Approach (WA), Sobel edge Detection (SD), Histogram based Segmentation (HS), Trainable Segmentation (TS), Dual Clustering (DC), and Fuzzy C-Means clustering (FCM) are examined for image segmentation and feature extraction. Among these methods, MSR was selected for feature extraction since it outperformed other methods in terms of accuracy, specificity, precision, recall and F1-score. Then, a set of neural network classifiers are trained to identify patterns in the extracted optimized features and predict dental infections. For this purpose, we have examined Bacterial Optimized Recurrent Neural Networks (BORNN), Deep Learning Neural Networks (DANN), Genetic Optimized Neural Networks (GONN) and Adaptive Neural Networks Algorithm (ADNN). BORNN have shown maximum accuracy and Roc value (98.1% and 0.92 respectively), and minimum error values (MSE = 0.189, MAE = 0.143). The output of the proposed integrated prediction model is fed into a dental robot who proceeds with the treatment process with high accuracy and minimum delay. The proposed prediction model was implemented using a big data analytics tool called Apache SAMOA and experimental results showed its correctness and effectiveness. [ABSTRACT FROM AUTHOR]

Although Durr and Hoyer have proposed state-of-the-art quantum algorithm (DHA) for searching minimum value, the lower limit of DHA's successful probability is 1/2. Also, DHA requires approximately (log 2 N) 2 copies of the initial state. In this paper, we propose a new quantum maximum or minimum searching algorithm (QUMMSA). In big data scenarios, according to sparse sampling with different densities, we can estimate the corresponding precision parameters. QUMMSA can improve the successful probability close to 100 % . Furthermore, with the quantum exact search algorithm, QUMMSA only requires approximately log 2 N copies of the initial state to solve this problem. Since preparing an arbitrary quantum state is a problem with exponential complexity, our algorithm has a greater advantage with the increasing database size. In addition, we first propose a general method for circuits construction, which can be used in any database. An experiment implemented in an IBM superconducting processor and a numerical simulation of a 6-qubit system to solve a real issue indicate the feasibility and efficiency of QUMMSA. QUMMSA can serve as a subroutine in various quantum algorithms which involves searching maximum or minimum. [ABSTRACT FROM AUTHOR]

Data assimilation is an analysis technique which combines observations and the numerical results from theoretical models to deduce more realistic and accurate data. It is widely used in investigations of the atmosphere, ocean and land surface. Due to the complicated data structure of the inputs from dynamical models and the increase of the amount of model data, the parallelization of data assimilation suffers from high overhead on file reading and data communication. In this paper, we propose a flexible parallel data access approach for reading a large number of data from disks firstly. Using this approach, the data access conflict is avoided successfully, and the frequency of disk addressing operations is also decreased significantly. Next, we design a communication-avoiding strategy to reduce the communication volume at the cost of some additional computations. Furthermore, we present a "pipe-flow" scheme for data exchange to conduct conflict-free message passing. Consequently, a fast data-obtaining algorithm is developed for the data assimilation. Our experiments show that the fast data-obtaining algorithm gains a performance of 5 × speedup compared with the baseline, which is excellent at data-obtaining for the parallel data assimilation. Due to the reduction of disk addressing operations, the new approach achieves 6 × speedup on average for the file reading process. Since a large amount of data movement can be avoided, the new approach achieves 2.7 × speedup on average for the communication between processors. [ABSTRACT FROM AUTHOR]

In this paper, a doppler distortion compensation method for underwater transmission of OFDM (Orthogonal Frequency Division Multiplexing) signals is proposed, which is used to realize real-time underwater acoustic communication between high-speed UUV (Unmanned Underwater Vehicle) and intelligent nodes. The method decomposes channel variations based on a functions with a set of known equations. OFDM symbol reconstruction is performed after the Taylor(T) FFT (Fast Fourier Transformation) process. We design an adaptive stochastic gradient descent algorithm based on MMSE criterion to learn the combiner weights for differentially coherent detection, thereby achieving adaptive channel equalization in the underwater acoustic channel. Comprehensive data and experimental data from the recent mobile acoustic communication delta are used to demonstrate the feasibility of this approach. The method can realize high-speed underwater acoustic communication, provide data sample basis for long-term big data analysis, and realize real-time communication result analysis. [ABSTRACT FROM AUTHOR]

Dimension reduction tools offer a popular approach to analysis of high-dimensional big data. In this paper, we propose an algorithm for sparse Principal Component Analysis for non-Gaussian data. Since our interest for the algorithm stems from applications in text data analysis we focus on the Poisson distribution which has been used extensively in analysing text data. In addition to sparsity our algorithm is able to effectively determine the desired number of principal components in the model (order determination). The good performance of our proposal is demonstrated with both synthetic and real data examples. [ABSTRACT FROM AUTHOR]

Energy consumption is explosive increasing with the fast growth of big data applications. High carbon emissions from big data platforms have serious impacts on environment. In this paper, we propose an energy-aware scheduling algorithm for Spark (EASAS) to reduce energy consumption while satisfying the service level agreement (SLA). First, we present a new energy consumption model based on Spark framework. Then a strategy table for the relationship between tasks and executors is designed to record the execution time and energy consumption of tasks. The task scheduling in Spark is conducted and optimized based on the strategy table. The proposed strategy overcomes the defect of the default scheduling strategy FIFO and FAIR which cannot perceive energy consumption with the characteristics of energy consumption perception and dynamic optimization scheduling. Compared against FIFO and FAIR, Our EASAS effectively reduces on average about 25–40% of the total energy consumption of Spark applications under deadline constrains. [ABSTRACT FROM AUTHOR]

The k-nearest neighbors outlier detection is a simple yet effective widely renowned method in data mining. The actual application of this model in the big data domain is not feasible due to time and memory restrictions. Several distributed alternatives based on MapReduce have been proposed to enable this method to handle large-scale data. However, their performance can be further improved with new designs that fit with newly arising technologies. Furthermore, it gives to each attribute the same importance to outlier. There are several approaches to enhance its precision, with the entropy-based outlier detection being among the most successful ones. Entropy-based outlier detection computes attribute entropy of the data set to weighted distance formula for the outlier detection. Apart from the existing the k-nearest neighbors outlier detection to handle big datasets, there is not an entropy-based outlier detection to manage that volume of data. In this paper, we propose an entropy-based outlier detection based on Spark. It presents three separately stages. The first stage computes attribute entropy. The second stage finds the k nearest neighbors and calculates the degrees of outliers using the attribute entropy computed previously. The third stage ranks each point on the degrees of outliers and declares the top n points in this ranking to be outliers. Extensive experimental results show the advantages of the proposed method. This algorithm can improve the outlier detection precision, reduce the runtime and realize the effective large scale dataset outlier detection. [ABSTRACT FROM AUTHOR]