Search

In this article, a scalable autonomous separation assurance framework is proposed for high-density en route airspace sectors with heterogeneous aircraft objectives. To handle the complex dynamic decision making under uncertainty, multi-agent reinforcement learning is used in a decentralized approach with each aircraft being represented as an agent. Based on this, each agent locally solves the separation assurance problem, allowing the framework to scale to a large number of aircraft. In addition, each agent has the ability to learn the intention of the intruder aircraft, which is essential in environments with heterogeneous agents. Numerical experiments are performed in a real-time air traffic simulator. The results demonstrate that the proposed framework is able to effectively ensure the safe separation of heterogeneous agents, while also optimizing the intrinsic agent objectives in high-density en route airspace sectors. In addition, the efficiency of the proposed framework is demonstrated and shown to provide real-time decision making for separation assurance. Note to Practitioners—In commercial aviation, the workload of human air traffic controllers increases with the growth of air traffic density. Robust decision making systems to augment human air traffic controllers allows for increased air traffic without increased workload, resulting in a safer airspace environment. In addition, advanced air mobility (AAM) is concerned with low-altitude airspace operations with both human and autonomous pilots. In this environment, autonomous real-time separation assurance systems are required. Most traditional separation assurance approaches fail to handle stochastic and high-density environments, rendering them inapplicable to future high-density traditional airspace and the envisioned low-altitude AAM operations. Therefore, it is important to study decentralized approaches that can place the separation assurance problem as an intrinsic objective of each aircraft to ensure cooperation in high-density airspace. With this consideration, a scalable, decentralized autonomous separation assurance framework capable of handling heterogeneous agents is proposed in this article. This framework is able to perceive the current air traffic environment and select speed advisories to ensure safe separation requirements, while balancing intrinsic objectives such as minimizing delay. While one limitation of multi-agent reinforcement learning is the long training time, this article demonstrates how the framework also can leverage modern computing clusters to significantly reduce training time without sacrificing performance. [ABSTRACT FROM AUTHOR]

This report describes The University of Texas Student Cluster Competition team’s effort to reproduce the results of “MemXCT: memory-centric X-ray CT reconstruction with massive parallelization” (Hidayetoğlu et al., 2019). The article details a new memory-centric approach that reconstructs X-ray computed tomography (XCT) from noisy raw data. In our reproduction experiments, we utilized Microsoft Azure’s CycleCloud tool to provision, orchestrate, and manage our computing cluster in the cloud. In particular, we scheduled and benchmarked reconstruction workloads using Azure’s CPU-based HC44rs and GPU-based NC12s v2 virtual machine (VM) types to evaluate the scalability properties of the reconstruction approach and the performance differences between architectures. The HC44rs VMs contained 44 Intel Xeon Platinum cores, while the NC12s v2 VM was equipped with two NVIDIA P100 GPUs. We used a recent version of Intel’s compiler stack with the MKL library for our CPU code along with CUDA 11.1 on GPUs. Overall, our results confirm the findings of the original article, demonstrating similar acceleration on GPUs and scalability properties on CPUs. Digital artifacts from these experiments are available at: 10.5281/zenodo.5598108 [ABSTRACT FROM AUTHOR]

Recently, with the large-scale outbreak of the global financial crisis and public safety incidents (such as COVID-19), high-performance computing has been widely applied to risk prediction, vaccine development, and other fields. In scenarios where high-performance computing infrastructure responds to the instantaneous explosion of computing demands, a crucial issue is to provide large-scale flexible allocation and adjustment of computing capability by rapidly constructing computing clusters. Existing large-scale computing cluster deployment solutions usually utilize source code deployment or other deployment tools. The great challenge of existing deployment methods is to reduce excessive image distribution time and refrain from configuration defects. In this article, we design an intelligent distributed registry deployment (IDRD) architecture based on the OpenStack cloud platform, which adaptively places distributed image repositories using the containerized deployment of multiple registries. We propose a server load priority algorithm to solve multiple registries placement problems in IDRD. Furthermore, we devise a clustering algorithm based on demand density that can optimize the global performance of IDRD and improve large-scale cluster load balancing capabilities, which has been implemented in the TianHe Supercomputing environment. Extensive experimental results demonstrate that IDRD can effectively reduce 30% − 50% of the distribution time of component images and significantly improve the efficiency of large-scale cluster deployment. [ABSTRACT FROM AUTHOR]

For the rapid growth computation requirements in big data and artificial intelligence area, CPU-GPU heterogeneous clusters can provide more powerful computing capacity compared to CPU clusters. The high parallel computing capabilities of GPUs greatly accelerate computation-intensive applications. And the number of GPUs on single computing node is scalable, which greatly improves the computing capacity of the cluster under the condition of limited cluster size. However, there is a lack of the effective load-balancing scheduling model in multi-GPU hardware environment. This article proposes AEML, an acceleration engine for multi-GPU load-balancing in distributed heterogeneous environment. AEML can effectively integrate GPUs into distributed processing framework and achieve great load-balance among multiple heterogeneous GPUs. We propose a heterogeneous task execution model based on multiple GPUs and multiple streams (MGMS), which can effectively balance the workload of multiple GPUs. MGMS model utilizes four core techniques: a fine-grained task mapping mechanism, a device resource unified management scheme, a novel resource-aware GPU task scheduling strategy, and a feedback-based streams adjustment scheme. The implementation of AEML system is based on Spark 3.0.0 and NVIDIA CUDA 10.0. We comprehensively evaluate the performance of AEML with multiple typical benchmarks. Experimental results show that AEML can fully exploit the computing power of GPUs and achieve great load-balance among multiple heterogeneous GPUs. [ABSTRACT FROM AUTHOR]

In this communication, a novel message passing interface (MPI) parallel algorithm for nodal discontinuous Galerkin time-domain (NDGTD) method has been developed. A unified MPI + MPI technique has been introduced for extreme parallelism on a large-scale computer cluster. Through the data transmission between CPU nodes using MPI persistent nonblocking two-side communication and the direct data connection between processors in the same node via MPI shared memory windows, a two-layered parallel architecture is implemented to minimize the communication. To further accelerate the solution of the multiscale problems, the local time stepping (LTS) technique has been employed in the NDGTD method. A fast time step estimation method has been presented in this communication. With high overlap between the information transmission and the data calculation, the proposed MPI + MPI scheme overcomes the degradation of the parallel efficiency of the pure MPI technique in the scenario of the LTS technique and the large-scale CPU cores. Up to 94% parallel efficiency in 6400 CPU cores is achieved for the average single-core loading about 1700 finite elements, and 18 times acceleration for time step estimation can be obtained with the fourth-order basis function. Three practical complex examples are given to demonstrate a good performance of the proposed method. [ABSTRACT FROM AUTHOR]

Big data frameworks such as Apache Spark is becoming prominent to perform large-scale data analytics jobs in various domains. However, due to limited resource availability, the local or on-premise computing resources are often not sufficient to run these jobs. Therefore, public cloud resources can be hired on a pay-per-use basis from the cloud service providers to deploy a Spark cluster entirely on the cloud. Nevertheless, using only cloud resources can be costly. Hence, both local and cloud resources nowadays are used together to deploy a hybrid cloud computing cluster. However, scheduling jobs in a cluster deployed on hybrid clouds is challenging in the presence of various Service-Level Agreement (SLA) demands such as cost minimization and job deadline guarantee. Most of the existing works either consider a public or a locally deployed cluster and mainly focus on improving job performance in the cluster. In this article, we propose efficient scheduling algorithms that leverage from different VM instance pricing in a hybrid cloud deployed cluster to optimize the Virtual Machine (VM) usage cost for both local and cloud resources and maximize the job deadline met percentage. We have conducted extensive simulation-based experiments to compare our proposed algorithms with the baseline approaches. In addition, we have developed a prototype system on top of Apache Mesos cluster manager and performed real experiments to evaluate the applicability of our proposed approaches in a real platform with benchmark applications. The results show that our proposed algorithms are highly scalable and reduce the cost of VM usage of a hybrid cluster for up to 20 percent. [ABSTRACT FROM AUTHOR]

Recent applications in low-power (1-20 mW) near-sensor computing require the adoption of floating-point arithmetic to reconcile high precision results with a wide dynamic range. In this article, we propose a low-power multi-core computing cluster that leverages the fined-grained tunable principles of transprecision computing to provide support to near-sensor applications at a minimum power budget. Our solution – based on the open-source RISC-V architecture – combines parallelization and sub-word vectorization with a dedicated interconnect design capable of sharing floating-point units (FPUs) among the cores. On top of this architecture, we provide a full-fledged software stack support, including a parallel low-level runtime, a compilation toolchain, and a high-level programming model, with the aim to support the development of end-to-end applications. We performed an exhaustive exploration of the design space of the transprecision cluster on a cycle-accurate FPGA emulator, varying the number of cores and FPUs to maximize performance. Orthogonally, we performed a vertical exploration to identify the most efficient solutions in terms of non-functional requirements (operating frequency, power, and area). We conducted an experimental assessment on a set of benchmarks representative of the near-sensor processing domain, complementing the timing results with a post place-&-route analysis of the power consumption. A comparison with the state-of-the-art shows that our solution outperforms the competitors in energy efficiency, reaching a peak of 97 Gflop/s/W on single-precision scalars and 162 Gflop/s/W on half-precision vectors. Finally, a real-life use case demonstrates the effectiveness of our approach in fulfilling accuracy constraints. [ABSTRACT FROM AUTHOR]

Power and energy is the first-class design constraint for multi-core processors and is a limiting factor for future-generation supercomputers. While modern processor design provides a wide range of mechanisms for power and energy optimization, it remains unclear how software can make the best use of them. This article presents a novel approach for runtime power optimization on modern multi-core systems. Our policy combines power capping and uncore frequency scaling to match the hardware power profile to the dynamically changing program behavior at runtime. We achieve this by employing reinforcement learning (RL) to automatically explore the energy-performance optimization space from training programs, learning the subtle relationships between the hardware power profile, the program characteristics, power consumption and program running times. Our RL framework then uses the learned knowledge to adapt the chip's power budget and uncore frequency to match the changing program phases for any new, previously unseen program. We evaluate our approach on two computing clusters by applying our techniques to 11 parallel programs that were not seen by our RL framework at the training stage. Experimental results show that our approach can reduce the system-level energy consumption by 12 percent, on average, with less than 3 percent of slowdown on the application performance. By lowering the uncore frequency to leave more energy budget to allow the processor cores to run at a higher frequency, our approach can reduce the energy consumption by up to 17 percent while improving the application performance by 5 percent for specific workloads. [ABSTRACT FROM AUTHOR]

Noisy gate-model quantum processing units (QPUs) are currently available from vendors over the cloud, and digital quantum programming approaches exist to run low-depth circuits on physical hardware. These digital representations are ultimately lowered to pulse-level instructions by vendor quantum control systems to affect unitary evolution representative of the submitted digital circuit. Vendors are beginning to open this pulse-level control system to the public via specified interfaces. Robust programming methodologies, software frameworks, and backend simulation technologies for this analog model of quantum computation will prove critical to advancing pulse-level control research and development. Prototypical use cases for this include error mitigation, optimal pulse control, and physics-inspired pulse construction. Here we present an extension to the XACC quantum-classical software framework that enables pulse-level programming for superconducting, gate-model quantum computers, and a novel, general, and extensible pulse-level simulation backend for XACC that scales on classical compute clusters via MPI. Our work enables custom backend Hamiltonian definitions and gate-level compilation to available pulses with a focus on performance and scalability. We end with a demonstration of this capability, and show how to use XACC for pertinent pulse-level programming tasks. [ABSTRACT FROM AUTHOR]

For many data-parallel computing systems like Spark, a job usually consists of multiple computation stages and inter-stage communication (i.e., coflows). Many efforts have been done to schedule coflows and jobs independently. The simple combination of coflow scheduling and job scheduling, however, would prolong the average job completion time (JCT) due to the conflict. For this reason, we propose a new abstraction of scheduling unit, named coBranch, which takes the dependency between computation stages and coflows into consideration, to schedule coflows and jobs jointly. Besides, mainstream coflow schedulers are order-preserving, i.e., all coflows of a high-priority job are prioritized than those of a low-priority job. We observe that the order-preserving constraint incurs low inter-job parallelism. To overcome the problem, we employ an urgency-based mechanism to schedule coBranches, which aims to decrease the average JCT by enhancing the inter-job parallelism. We implement the urgency-based coBranch Scheduling (BS) method on Apache Spark, conduct prototype-based experiments, and evaluate the performance of our method against the shortest-job-first critical-path method and the FIFO method. Results show that our method achieves around 10 and 15 percent reduction in the average JCT, respectively. Large-scale simulations based on the Google trace show that our method performs better and reduces JCT by 23 and 35 percent, respectively. [ABSTRACT FROM AUTHOR]

Distributed synchronous stochastic gradient descent has been widely used to train deep neural networks (DNNs) on computer clusters. With the increase of computational power, network communications generally limit the system scalability. Wait-free backpropagation (WFBP) is a popular solution to overlap communications with computations during the training process. In this article, we observe that many DNNs have a large number of layers with only a small amount of data to be communicated at each layer in distributed training, which could make WFBP inefficient. Based on the fact that merging some short communication tasks into a single one can reduce the overall communication time, we formulate an optimization problem to minimize the training time in pipelining communications and computations. We derive an optimal solution that can be solved efficiently without affecting the training performance. We then apply the solution to propose a distributed training algorithm named merged-gradient WFBP (MG-WFBP) and implement it in two platforms Caffe and PyTorch. Extensive experiments in three GPU clusters are conducted to verify the effectiveness of MG-WFBP. We further exploit trace-based simulations of 4 to 2048 GPUs to explore the potential scaling efficiency of MG-WFBP. Experimental results show that MG-WFBP achieves much better scaling performance than existing methods. [ABSTRACT FROM AUTHOR]

A state-of-the-art real-time rain detection and wiper control method is proposed in this article. Currently, commercial models adopt electronic sensors that can only sample the humidity of a small region of the windshield. The existing computer vision methods primarily focus on the detection and counting of raindrops and provide a recall rate of less than 70%. Here we adopted a holistic-view deep learning approach to build a visual classifier that is robust to large varieties of background scenes, illumination, and water forms. Specifically, Deep Residual Network (ResNet) was adopted as the visual classifier that distinguishes between rainy and fair street scenes and controls the wipers accordingly. To verify the practicality of the proposed deep learning framework, we tested the network on various embedded computing systems, including an embedded computing cluster. The results show that the deep learning rain detector outperforms previous state-of-the-art methods with higher rain recall and precision. It was also found that with the help of some graphic computation-enhancing components, commonly available embedded systems in the market can provide comparable performance to personal computers. While using the enhanced embedded systems to build a cluster, a performance superior to PC was witnessed. As the embedded system is cost-effective, small, and lighter, normalized performances for various aspects clearly show the competitive edge of the embedded systems and confirm the practicality of the proposed system. We also release the dataset of 160 k images used for training the visual rain detector. [ABSTRACT FROM AUTHOR]

Many Data-Intensive Scalable Computing (DISC) Systems provide easy-to-use functional APIs, and efficient scheduling and execution strategies allowing users to build concise data-parallel programs. In these systems, data transformations are concealed by exposed APIs, and intermediate execution states are masked under dataflow transitions. Consequently, many crucial features and optimizations (e.g., debugging, data provenance, runtime skew detection), which require runtime datafow states, are not well-supported. Inspired by our experience in implementing features and optimizations over DISC systems, we present $\mathsf {SEIZE}$ SEIZE , a unified framework that enables dataflow inspection—wiretapping the data-path with listening logic—in MapReduce-style programming model. We generalize our lessons learned by providing a set of primitives defining dataflow inspection, orchestration options for different inspection granularities, and operator decomposition and dataflow punctuation strategy for dataflow intervention. We demonstrate the generality and flexibility of the approach by deploying $\mathsf {SEIZE}$ SEIZE in both Apache Spark and Apache Flink, and by implementing a prototype runtime query optimizer for Spark. Our experiments show that, the overhead introduced by the inspection logic is most of the time negligible (less than 5 percent in Spark and 10 percent in Flink). [ABSTRACT FROM AUTHOR]

Enabling data processing at the network edge, as close to the actual source of data as possible, is a challenging, yet realistic goal to be achieved by the Internet of Things (IoT), which still primarily relies on the Cloud for data processing. By further extending the Fog and Edge computing principles, recent research advancements enabled aggregation of computing resources from multiple edge devices to support data-intensive task processing using Big Data clustering middleware. The use of these existing solutions, however, is hindered by the heterogeneous, dynamic, mobile, resource-constrained, and time-critical nature of IoT ecosystems. More specifically, a particularly challenging goal is to discover, select, and cluster suitable edge devices – on the one hand, and decompose and allocate data-intensive tasks with respect to discovered resources – on the other. To address this challenge, this paper introduces a novel decentralized architecture for clustering heterogeneous edge devices and executing data-intensive IoT workflows. The proposed approach first breaks down a complex workflow into simpler tasks, then discovers and selects suitable edge devices, and finally allocates the tasks to the selected nodes, connecting them to recompose the original workflow. The proposed approach benefits from an intelligent mapping algorithm that takes into account available cluster resources and processing demands to efficiently allocate fine-grained tasks to selected nodes. To support the clusterisation process, the proposed solution relies on a unified semantic knowledge base that provides a common vocabulary of terms for modelling task requirements and edge device properties, as well as enables automated task grouping and match-making for device discovery and selection, using built-in reasoning capabilities. [ABSTRACT FROM AUTHOR]

Big data classification problems have drawn great attention from diverse fields, and many classifiers have been developed. Among those classifiers, the extended belief rule-based system (EBRBS) has shown its potential in both big data and multiclass situations, while the time complexity and computing efficiency are two challenging issues to be handled in EBRBS. As such, three improvements of EBRBS are proposed first in this paper to decrease the time complexity and computing efficiency of EBRBS for multiclass classification under the assumption of large amount of data, including the strategy to skip rule weight calculation, a simplified evidential reasoning algorithm, and the domain division-based rule reduction method. This turns out to be a micro version of the EBRBS, called Micro-EBRBS. Moreover, one of commonly used cluster computing, named Apache Spark, is then applied to implement the parallel rule generation and inference schemes of the Micro-EBRBS for big data multiclass classification problems. The comparative analyses of experimental studies demonstrate that the Micro-EBRBS not only can obtain a desired accuracy but also has the comparatively better time complexity and computing efficiency than some popular classifiers, especially for multiclass classification problems. [ABSTRACT FROM AUTHOR]

Datacenters are evolving to host heterogeneous workloads on shared clusters to reduce the operational cost and achieve higher resource utilization. However, it is challenging to schedule heterogeneous workloads with diverse resource requirements and QoS constraints. On one hand, latency-critical jobs need to be scheduled as soon as they are submitted to avoid any queuing delays. On the other hand, best-effort long jobs should be allowed to occupy the cluster when there are idle resources to improve cluster utilization. The challenge lies in how to minimize the queuing delays of short jobs while maximizing cluster utilization. In this article, we propose and develop BIG-C, a container-based resource management framework for data-intensive cluster computing. The key design is to leverage lightweight virtualization, a.k.a, containers, to make tasks preemptable in cluster scheduling. We devise two types of preemption strategies: immediate and graceful preemptions and show their effectiveness and tradeoffs with loosely-coupled MapReduce workloads as well as iterative, in-memory Spark workloads. Based on the mechanisms for task preemption, we further develop job-level and task-level preemptive policies as well as a preemptive fair share cluster scheduler. Our implementation on Yarn and evaluation with synthetic and production workloads show that low job latency and high resource utilization can be both attained when scheduling heterogeneous workloads on a contended cluster. [ABSTRACT FROM AUTHOR]

Many existing clustering methods usually compute clusters from the reduced data sets obtained by summarizing the original very large data sets. BIRCH is a popular summary-based clustering method that first builds a CF tree, and then performs a global clustering using the leaf entries of the tree. However, to the best of our knowledge, no prior studies have proposed a global clustering method that uses the structure of a CF tree. Therefore, we propose a novel global clustering method ERC (effective multiple range queries-based clustering), which takes advantage of the structure of a CF tree. We further propose a CF $^+$ + tree, which optimizes the node split scheme used in the CF tree. As a result, the CF $^+$ + -ERC (CF $^+$ + tree-based ERC) method effectively computes clusters over large data sets. Furthermore, it does not require a predefined number of clusters to compute the clusters. We present in-depth theoretical and experimental analyses of our method. Experimental results on very large synthetic data sets demonstrate that the proposed approach is effective in terms of cluster quality and robustness and is significantly faster than existing clustering methods. In addition, we apply our clustering method to real data sets and achieve promising results. [ABSTRACT FROM AUTHOR]

STN has been considered a novel network architecture to accommodate a variety of services and applications in future networks. Being a promising paradigm, MEC has been regarded as a key technology-enabler to offer further service innovation and business agility in STN. However, most of the existing research in MEC enabled STN regards a satellite network as a relay network, and the feasibility of tasks processing directly on the satellites is largely ignored. Moreover, the problem of multi-layer edge computing architecture design and heterogeneous edge computing resource co-scheduling, have not been fully considered. Therefore, different from previous works, in this article, we propose a novel architecture named STECN, in which computing resources exist in multi-layer heterogeneous edge computing clusters. The detailed functional components of the proposed STECN are discussed, and we present the promising technical challenges, including meeting QoE requirements, cooperative computation offloading, multi-node task scheduling, mobility management and fault/failure recovery. Finally, some potential research issues for future research are highlighted. [ABSTRACT FROM AUTHOR]

Deep neural networks enable state-of-the-art accuracy on visual recognition tasks such as image classification and object detection. However, modern networks contain millions of learned connections, and the current trend is towards deeper and more densely connected architectures. This poses a challenge to the deployment of state-of-the-art networks on resource-constrained systems, such as smartphones or mobile robots. In general, a more efficient utilization of computation resources would assist in deployment scenarios from embedded platforms to computing clusters running ensembles of networks. In this paper, we propose a deep network compression algorithm that performs weight pruning and quantization jointly, and in parallel with fine-tuning. Our approach takes advantage of the complementary nature of pruning and quantization and recovers from premature pruning errors, which is not possible with two-stage approaches. In experiments on ImageNet, CLIP-Q (Compression Learning by In-Parallel Pruning-Quantization) improves the state-of-the-art in network compression on AlexNet, VGGNet, GoogLeNet, and ResNet. We additionally demonstrate that CLIP-Q is complementary to efficient network architecture design by compressing MobileNet and ShuffleNet, and that CLIP-Q generalizes beyond convolutional networks by compressing a memory network for visual question answering. [ABSTRACT FROM AUTHOR]

Rotating machinery is widely applied in industrial fields. However, it generally operates under tough working conditions, which leads to the weak fault features and renders fault diagnosis more difficult. In this case, an emerging method called sparse representation classification (SRC) is proposed to enhance the fault features and identify the fault status. However, the typical SRC theory fails to consider the locality of the test sample and training sample, and the training set generally contains much redundant information, which may reduce the fault recognition accuracy. Moreover, the time-shift deviation of vibration signal cannot be avoided effectively using a typical SRC model. To overcome the above-mentioned problems, a novel SRC model, i.e., weighted SRC based on dictionary learning (DL-WSRC), is proposed. For the training set, different fault signals are learned based on the improved K-singular value decomposition (K-SVD) algorithm, which can not only adaptively update the whole training set but also reduce redundant information so as to enhance the sample fault features. For the test sample, DL-WSRC selects an accurate time-domain parameter using the K-means clustering algorithm and computes the weighted coefficients according to the parameter distance between the test sample and the training samples. Then, it sparsely represents the test sample by solving a weighted $l_{0}$ -norm problem. The goal of weighting is to pay more attention to the locality of the sample so as to improve the recognition accuracy. Finally, according to the results of sparse representation, the fault status can be identified through the correlation analysis, which can effectively solve the time-shift deviation problem. The effectiveness of the proposed method is validated by the experiments of rotating machinery, and the results indicate that the proposed method realizes fault classification with a high accuracy. [ABSTRACT FROM AUTHOR]

Network function virtualization (NFV) has emerged as a new technology to reduce the cost of hardware deployment. It is an architecture that using virtualized functions run on the virtual machine to achieve services instead of using specific hardware. Although NFV brings more opportunities to enhance the flexibility and efficiency of the network, resource allocation problems should be well taken into consideration. In this paper, we investigate the virtual network function (VNF) resource allocation problem to minimize the network operation cost for different services. Both setting the VNF instances for each virtual machine and allocating the traffic volume in the network are considered. The problem is formulated as a mixed integer programming problem. Although it can be solved in a centralized fashion which requires a central controller to collect information from all virtual machines, it is not practical for large-scale networks. Thus, we propose a distributed iteration algorithm to achieve the optimal solution. The proposed algorithm framework is developed based on the joint Benders decomposition and alternating direction method of multipliers (ADMM), which allows us to deal with integer variables and decompose the original problem into multiple subproblems for each virtual machine. Furthermore, we describe the detail implementation of our algorithm to run on a computer cluster using the Hadoop MapReduce software framework. Finally, the simulation results indicate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

Coupling regular topologies with optimized routing algorithms is key in pushing the performance of interconnection networks of supercomputers. In this article, we present Dmodc, a fast deterministic routing algorithm for parallel generalized fat trees (PGFTs), which minimizes congestion risk even under massive network degradation caused by equipment failure. Dmodc computes forwarding tables with a closed-form arithmetic formula by relying on a fast preprocessing phase. This allows complete rerouting of networks with tens of thousands of nodes in less than a second. In turn, this greatly helps centralized fabric management react to faults with high-quality routing tables and has no impact on running applications in current and future very large scale high-performance computing clusters. [ABSTRACT FROM AUTHOR]

This paper analyzes the performance of a clustered small cell network, where the base stations (BSs) are equipped with multiple antennas and the network topology is modeled as Matern cluster process (MCP). For such a clustered multi-antenna network, serious intra-cluster interference is a main bottleneck of the system performance. Utilizing the intra-cluster coordinated beamforming scheme is an effective approach for the intra-cluster interference cancellation. From this, we firstly derive an approximate expression for the users’ average data rate based on stochastic geometry, and then we discuss the optimal number of the maximum degrees of freedom (DoF) used for coordinated beamforming at the BS, which maximize the average data rate. Simulation results confirm the accuracy of our derivations and also present the performance with the intra-cluster interference cancellation is much better than that with no intra-cluster interference cancellation. [ABSTRACT FROM AUTHOR]

Word2vec is a widely used algorithm for extracting low-dimensional vector representations of words. State-of-the-art algorithms including those by Mikolov et al. [1] , [2] have been parallelized for multi-core CPU architectures, but are based on vector-vector operations with "Hogwild" updates that are memory-bandwidth intensive and do not efficiently use computational resources. In this paper, we propose "HogBatch" by improving reuse of various data structures in the algorithm through the use of minibatching and negative sample sharing, hence allowing us to express the problem using matrix multiply operations. We also explore different techniques to distribute word2vec computation across nodes in a computer cluster, and demonstrate good strong scalability up to 32 nodes. The new algorithm is particularly suitable for modern multi-core/many-core architectures, especially Intel's latest Knights Landing processors, and allows us to scale up the computation near linearly across cores and nodes, and process hundreds of millions of words per second, which is the fastest word2vec implementation to the best of our knowledge. We released the source code for reproducible research and general usage. [ABSTRACT FROM AUTHOR]

Modern communication and computation systems often consist of large networks of unreliable nodes. Still, it is well known that such systems can provide aggregate reliability via redundancy. While duplication may increase the load on a system, it can lead to significant performance improvement when combined with the judicious management of extra system resources. Prime examples of this abstract paradigm include multi-path routing across communication networks, content access from multiple caches in delivery networks, and master/slave computations on compute clusters. Several recent contributions in the area establish bounds on the performance of redundant systems, characterizing the latency-redundancy tradeoff under specific load profiles. Following a similar line of research, this paper introduces new analytical bounds and approximation techniques for the latency-redundancy tradeoff for a range of system loads and a class of symmetric redundancy schemes, under the assumption of Poisson arrivals, exponential service-rates, and fork-join scheduling policy. The proposed approach can be employed to efficiently approximate the latency distribution of a queueing system at equilibrium. Various metrics can subsequently be derived for this system, including the mean and variance of the sojourn time, and the tail decay rate of the stationary distribution. This paper also establishes the stability region in terms of arrival rates for redundant systems with certain symmetries. Finally, it offers selection guidelines for design parameters to provide latency guarantees based on the proposed approximations. Findings are substantiated by numerical results. [ABSTRACT FROM AUTHOR]

In large-scale distributed computing clusters, such as Amazon EC2, there are several types of “system noise” that can result in major degradation of performance: system failures, bottlenecks due to limited communication bandwidth, latency due to straggler nodes, and so on. There have been recent results that demonstrate the impact of coding for efficient utilization of computation and storage redundancy to alleviate the effect of stragglers and communication bottlenecks in homogeneous clusters. In this paper, we focus on general heterogeneous distributed computing clusters consist of a variety of computing machines with different capabilities. We propose a coding framework for speeding up distributed computing in heterogeneous clusters by trading redundancy for reducing the latency of computation. In particular, we propose heterogeneous coded matrix multiplication (HCMM) algorithm for performing distributed matrix multiplication over heterogeneous clusters that are provably asymptotically optimal for a broad class of processing time distributions. Moreover, we show that HCMM is unboundedly faster than any uncoded scheme that partitions the total workload among the workers. To demonstrate how the proposed HCMM scheme can be applied in practice, we provide results from numerical studies and Amazon EC2 experiments comparing HCMM with three benchmark load allocation schemes—uniform uncoded, load-balanced uncoded, and uniform coded. In particular, in our numerical studies, HCMM achieves speedups of up to 73%, 56%, and 42%, respectively, over the three benchmark schemes mentioned earlier. Furthermore, we carry out experiments over Amazon EC2 clusters and demonstrate how HCMM can be combined with rateless codes with nearly linear decoding complexity. In particular, we show that HCMM combined with the Luby transform codes can significantly reduce the overall execution time. HCMM is found to be up to 61%, 46%, and 36% faster than the aforementioned three benchmark schemes, respectively. Additionally, we provide a generalization to the problem of optimal load allocation in heterogeneous settings, where we take into account the monetary costs associated with distributed computing clusters. We argue that HCMM is asymptotically optimal for budget-constrained scenarios as well. In particular, we characterize the minimum possible expected cost associated with a computation task over a given cluster of machines. Furthermore, we develop a heuristic algorithm for (HCMM) load allocation for the distributed implementation of budget-limited computation tasks. [ABSTRACT FROM AUTHOR]

Trace signal selection is of great importance for post-silicon debug. Debuggers traditionally use state restoration to improve the observability of the trace data, and state restoration ratio (SRR) is computed after state restoration. In this paper, we exploit the combination of snapshot states and trace states to improve the observability. First, we propose a novel state restoration method, called cluster restoration. It uses both the snapshot states of flip-flop clusters at the beginning of tracing, and the tracing states of the clusters’ inputs during the tracing window to deterministically restore all states of these clusters during the tracing window. We also present a cluster restoration-based trace signal selection method to select clusters instead of trace signals directly, which includes two stages: 1) cluster generation and 2) cluster evaluation. For cluster generation, feedback loop-based cluster generation and backward tracing-based cluster generation techniques are proposed. For cluster evaluation, a new metric, called the global state restoration improvement is proposed to evaluate the candidate clusters. The experimental results show that in comparison to prior trace signal selection methods, our method can improve the SRR and reduce the runtime of trace signal selection as well. [ABSTRACT FROM AUTHOR]

A revised method based on the standard Epstein test is proposed to improve the accuracy of the estimation of the core losses of induction machines (IMs). Since the magnetic field strength patterns at the four corners of the standard test samples are non-uniform, high-order excess losses at the corners would deteriorate the accuracy of the model for core losses. To address this issue, test samples were designed which have an appropriate geometry for the measurement of core losses under the test conditions. The excess losses could be eliminated by introducing an air-gap magnetic reluctance at the four corners so that the core-loss data from the tests consist of only hysteresis and eddy current parts. In experimental validations, three 5-HP IMs were bench-tested for the verification of the core-loss estimation. The use of commercial computer-aided engineering software packages allowed for the estimated core losses to be recorded and compared. The results indicate that high-order excess losses could account for 7% of the total core losses. [ABSTRACT FROM AUTHOR]

In the era of big data, high performance data analytics applications are frequently executed on large-scale cluster architectures to accomplish massive data-parallel computations. Often, these applications involve iterative machine learning algorithms to extract information and make predictions from large data sets. Multicast data dissemination is one of the major performance bottlenecks for such data analytics applications in cluster computing, as terabytes of data need to be distributed frequently from a single data source to hundreds of computing nodes. To overcome this bottleneck for big data applications, we propose BiGNoC, a manycore chip platform with a novel application-specific photonic network-on-chip (PNoC) fabric. BiGNoC is designed for big data computing and exploits multicasting in photonic waveguides. For high performance data analytics applications, BiGNoC improves throughput by up to ${{9.9}}\times$ while reducing latency by up to 88 percent and energy-per-bit by up to 98 percent over two state-of-the-art PNoC architectures as well as a broadcast-optimized electrical mesh NoC architecture, and a traditional electrical mesh NoC architecture. [ABSTRACT FROM AUTHOR]

Cluster computing applications, such as MapReduce and Spark, have been widely deployed in data centers to support commercial applications and scientific research. These applications often involve a collection of parallel flows generated by two groups of machines, and the slowest flow will determine the completion of applications. However, existing network-level optimizations are agnostic to the special communication pattern of the applications. Most of them only focus on improving the completion time of an individual flow instead of a collection of parallel flows. The recently proposed coflow abstraction exactly expresses the requirements of cluster computing applications and creates new opportunities to reduce the completion time of jobs in cluster computing. Due to the wide variations in coflow characteristics, coflow scheduling faces great challenges in data center networks. Much work has been proposed to solve one or some of the various challenges. Therefore, in this article, we survey the latest development in coflow scheduling for data center networks. We hope that this article will help readers quickly understand the causes of each problem and learn about current research progress, so as to serve as a guide and motivation toward further research in this area. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the dynamic small cell (SC) clustering strategy and their precoding design problem for interference coordination in two-tier heterogeneous networks (HetNets) with massive MIMO (mMIMO). To reduce interference among different SCs, an interference graph-based dynamic SC clustering scheme is proposed. Based on this, we formulate an optimization problem as design precoding weights at macro base station (MBS) and clustered SCs for maximizing the downlink sum rate of SC users (SUs) subject to the power constraint of each SC BS (SBS), while mitigating inter-cluster, eliminating inter-tier, intra-cluster and multi-macro user (MU) interference. To eliminate the inter-tier and multi-MU interference simultaneously, we propose a clustered SC block diagonalization precoding scheme for the MBS. Next, each SU’s precoding vector at clustered SCs is designed as the product of the following two parts. The first part is designed with singular value decomposition to remove the intra-cluster interference. The second part is designed to coordinate the inter-cluster interference for maximizing the downlink sum rate of SUs, which is a non-convex optimization problem and difficult to solve directly. A non-cooperative game-based distributed algorithm is proposed to obtain a suboptimal solution. Meanwhile, we prove the existence and uniqueness of Nash equilibrium for the formed game. Finally, simulation results verify the effectiveness of our proposed schemes. [ABSTRACT FROM PUBLISHER]

The current and envisaged increase of cellular traffic poses new challenges to mobile network operators (MNO), who must densify their radio access networks (RAN) while maintaining low capital expenditure and operational expenditure to ensure long-term sustainability. In this context, this paper analyzes optimal clustering solutions based on device-to-device communications to mitigate partially or completely the need for MNOs to carry out extremely dense RAN deployments. Specifically, a low-complexity algorithm that enables the creation of spectral efficient clusters among users from different cells, denoted as enhanced clustering optimization for resources’ efficiency is presented. Due to the imbalance between uplink and downlink traffic, a complementary algorithm, known as clustering algorithm for load balancing, is also proposed to create nonspectral efficient clusters when they result in a capacity increase. Finally, in order to alleviate the energy overconsumption suffered by cluster heads, the clustering energy efficient algorithm (CEEa) is also designed to manage the tradeoff between the capacity enhancement and the early battery drain of some users. Results show that the proposed algorithms increase the network capacity and outperform existing solutions, while, at the same time, CEEa is able to handle the cluster heads energy overconsumption. [ABSTRACT FROM PUBLISHER]

Bloom filters have been used to reduce the delay in networking and computing applications when a set membership check is to be applied. Error sources can affect the behavior of Bloom filters resulting in a wrong outcome of this membership test and a possible effect in the system's output. Single event transients are a type of temporary errors altering the operation of combinational logic. A single event transient affecting the hash generation logic of a hardware-implemented Bloom filter can produce errors such as false negatives. This paper presents different approaches to build Bloom filters that are tolerant to single event transients occurring in the hash generation circuitry. They are compared to the use of traditional Modular Redundancy approaches. The results show that the new schemes can reduce significantly the circuit area needed to implement the Bloom filter. [ABSTRACT FROM AUTHOR]

Data transfers happen frequently in server clusters for software and application deployment, and in parallel computing clusters to transmit intermediate results in batches among servers between computation stages. This paper presents Cooper, an optimized prototype system to speedup multi-batch data transfers among a cluster of servers, leveraging a theoretically proven optimal algorithm called “coded permutation gossip,” which employs a simple random topology control scheme to best utilize bandwidth and decentralized random linear network coding to maximize the useful information transmitted. On a process-level coding-transfer pipeline, we investigate the best block division, batch division and inter-batch scheduling strategies to minimize the broadcast finish time in a realistic setting. For batch-based transfers, we propose a scheduling algorithm with low overhead that overlaps the transfers of consecutive batches and temporarily prioritizes later batches, to further reduce the broadcast finish time. We describe an asynchronous and distributed implementation of Cooper and have deployed it on Amazon EC2 for evaluation. Based on results from real experiments, we show that Cooper can almost double the speed of data transfers in computing clusters, as compared to state-of-the-art content distribution tools like BitTorrent, at a low CPU overhead. [ABSTRACT FROM AUTHOR]

Load imbalance is a major source of overhead in parallel programs such as MapReduce. Due to the uneven distribution of input data, tasks with more data become stragglers and delay the overall job completion. Running Hadoop in a private cloud opens up opportunities for expediting stragglers with more resources but also introduces problems that often outweigh the performance gain: (1) performance interference from co-running jobs may create new stragglers; (2) there exists a semantic gap between the Hadoop task management and resource pool-based virtual cluster management preventing tasks from using resources efficiently. In this paper, we strive to make Hadoop more resilient to data skew and more efficient in cloud environments. We present FlexSlot, a user-transparent task slot management scheme that automatically identifies map stragglers and resizes their slots accordingly to accelerate task execution. FlexSlot adaptively changes the number of slots on each virtual node to balance the resource usage so that the pool of resources can be efficiently utilized. FlexSlot further improves mitigation of data skew with an adaptive speculative execution strategy. Experimental results show that FlexSlot effectively reduces job completion time up to $47.2$ percent compared to stock Hadoop and two recently proposed skew mitigation and speculative execution approaches. [ABSTRACT FROM PUBLISHER]

A new deadlock-free unicast-based broadcast scheme is proposed based on a new routing scheme called minus-first routing. Minus-first routing is a partially adaptive routing scheme in dragonfly networks without any virtual channels. The main goals of the broadcast schemes are to minimize the total delivery time, and any router does not receive any message more than once. No channel competition is introduced. Two different broadcast schemes are proposed: (1) the group-first, and (2) the router-first. It is shown that unicast-based broadcast schemes are necessary to avoid deadlocks at the consumption channels. The group-first broadcast scheme delivers a message to all groups as early as possible; and the router-first scheme minimizes the number of unicast steps to traverse global links. To our knowledge, the method in this paper is the first collective communication work for dragonfly networks in the literature. Simulation results are presented to evaluate the proposed unicast-based broadcast schemes. [ABSTRACT FROM PUBLISHER]

A large portion of existing multithreaded embedded sofware has been programmed according to symmetric shared memory platforms where a monolithic memory block is shared by all cores. Such platforms accommodate popular parallel programming models such as POSIX threads and OpenMP. However with the growing number of cores in modern manycore embedded architectures, they present a bottleneck related to their centralized memory accesses. This paper proposes a solution tailored for an efficient execution of applications defined with shared-memory programming models onto on-chip distributed-memory multicore architectures. It shows how performance, area and energy consumption are significantly improved thanks to the scalability of these architectures. This is illustrated in an open-source realistic design framework, including tools from ASIC to microkernel. [ABSTRACT FROM PUBLISHER]

This paper presents the use of a computer cluster with heterogeneous computing components to provide concurrency and multi-level parallelism at coarse grain and massive fine-grain for multiview video coding (MVC) applications. MVC involves coding of multiple video sequences that are taken from the same scene but different perspective. In addition to motion estimation (ME) used in conventional video coding for single view video for exploiting inter-frame temporal similarities, MVC adopts disparity estimation (DE) to further increase compression. To overcome the huge computational cost associated with ME and by extension with DE, attention has been mainly focused on developing fast ME/DE algorithms. Although fast ME/DE algorithms bring substantial speedup, to achieve realtime MVC encoding, it requires further acceleration of the coding process at higher levels. Towards this end, this paper proposes a multiple-view-parallel, multiple-interleaved group of pictures (multiple-IGOP) scheduling scheme for MVC. When evaluated over eight views, with no loss in rate distortion (RD) performance, the proposed scheme outperforms view-sequential coding by a factor of up to 12.4 and 12.3, respectively, for two popular prediction structures, IBP and IPP. [ABSTRACT FROM AUTHOR]

Microscopes can now cover large spatial areas and capture stem cell behavior over time. However, without discovering statistically reliable quantitative stem cell quality measures, products cannot be released to market. A Web-based measurement system overcomes desktop limitations by leveraging cloud and cluster computing for offline computations and by using Deep Zoom extensions for interactive viewing and measurement. [ABSTRACT FROM AUTHOR]

We use randomized network coding (RNC) with simple connection topology control to approach the theoretical limit on finish time of disseminating $k$ blocks in a server cluster of $n$ nodes. Unlike prior gossip literature which relies on completely random contact, we prove that with RNC, any receiver selection following a simple permutation rule can achieve a broadcast completion time of $k+n$ and that a time-varying random ring topology achieves a completion time of $k+o(k)+O(\log \,n)$ , both with high probability. Since the theoretical limit on finish time is $k+\lceil \log _2 \,n\rceil$ , our simple permutation algorithms achieve absolutely optimal (not only order-optimal) block pipelining for the $k$ blocks. Our results hold for both one-to-all (broadcast) and all-to-all transfers. We demonstrate the usefulness of the proposed organized network coded gossip with an application to content distribution in cluster computing systems like MapReduce, and discuss practical block dividing strategies to hide the negative effect of computation overhead of network coding. [ABSTRACT FROM PUBLISHER]

Remote sensing applications in Digital Earth are overwhelmed with vast quantities of remote sensing (RS) image data. The intolerable I/O burden introduced by the massive amounts of RS data and the irregular RS data access patterns has made the traditional cluster based parallel I/O systems no longer applicable. We propose a RS data object-based parallel file system for remote sensing applications and implement it with the OrangeFS file system. It provides application-aware data layout policies, together with RS data object based data I/O interfaces, for efficient support of various data access patterns of RS applications from the server side. With the prior knowledge of the desired RS data access patterns, HPGFS could offer relevant space-filling curves to organize the sliced 3-D data bricks and distribute them over I/O servers. In this way, data layouts consistent with expected data access patterns could be created to explore data locality and achieve performance improvement. Moreover, the multi-band RS data with complex structured geographical metadata could be accessed and managed as a single data object. Through experiments on remote sensing applications with different access patterns, we have achieved performance improvement of about 30 percent for I/O and 20 percent overall. [ABSTRACT FROM PUBLISHER]

Efficient file query is important to the overall performance of peer-to-peer (P2P) file sharing systems. Clustering peers by their common interests can significantly enhance the efficiency of file query. Clustering peers by their physical proximity can also improve file query performance. However, few current works are able to cluster peers based on both peer interest and physical proximity. Although structured P2Ps provide higher file query efficiency than unstructured P2Ps, it is difficult to realize it due to their strictly defined topologies. In this work, we introduce a Proximity-Aware and Interest-clustered P2P file sharing System (PAIS) based on a structured P2P, which forms physically-close nodes into a cluster and further groups physically-close and common-interest nodes into a sub-cluster based on a hierarchical topology. PAIS uses an intelligent file replication algorithm to further enhance file query efficiency. It creates replicas of files that are frequently requested by a group of physically close nodes in their location. Moreover, PAIS enhances the intra-sub-cluster file searching through several approaches. First, it further classifies the interest of a sub-cluster to a number of sub-interests, and clusters common-sub-interest nodes into a group for file sharing. Second, PAIS builds an overlay for each group that connects lower capacity nodes to higher capacity nodes for distributed file querying while avoiding node overload. Third, to reduce file searching delay, PAIS uses proactive file information collection so that a file requester can know if its requested file is in its nearby nodes. Fourth, to reduce the overhead of the file information collection, PAIS uses bloom filter based file information collection and corresponding distributed file searching. Fifth, to improve the file sharing efficiency, PAIS ranks the bloom filter results in order. Sixth, considering that a recently visited file tends to be visited again, the bloom filter based approach is enhanced by only checking the newly added bloom filter information to reduce file searching delay. Trace-driven experimental results from the real-world PlanetLab testbed demonstrate that PAIS dramatically reduces overhead and enhances the efficiency of file sharing with and without churn. Further, the experimental results show the high effectiveness of the intra-sub-cluster file searching approaches in improving file searching efficiency. [ABSTRACT FROM PUBLISHER]

Infrastructure-as-a-Service clouds offer diverse pricing options, including on-demand and reserved instances with various discounts to attract different cloud users. A practical problem facing cloud users is how to minimize their costs by choosing among different pricing options based on their own demands. In this paper, we propose a new cloud brokerage service that reserves a large pool of instances from cloud providers and serves users with price discounts. The broker optimally exploits both pricing benefits of long-term instance reservations and multiplexing gains. We propose dynamic strategies for the broker to make instance reservations with the objective of minimizing its service cost. These strategies leverage dynamic programming and approximation algorithms to rapidly handle large volumes of demand. Our extensive simulations driven by large-scale Google cluster-usage traces have shown that significant price discounts can be realized via the broker. [ABSTRACT FROM PUBLISHER]

The development of efficient parallel algorithms based on mixed mode programming represents one of the most popular lines of research in current bioinformatics. By exploiting hardware resources at inter-node/intra-node level, we can address grand computational challenges which involve the optimization of multiple objective functions simultaneously. In this sense, the inference of evolutionary trees represents one of the most difficult NP-hard problems in the field. Tackling such a problem requires efficient parallel designs to take advantage of the characteristics of modern multicore clusters. In this paper, we aim to solve the phylogenetic inference problem by applying MPI/OpenMP schemes to two multiobjective metaheuristics: fast non-dominated sorting genetic algorithm and multiobjective firefly algorithm. In order to assess the performance achieved by these proposals under different system and problem sizes, we have conducted experiments on six real nucleotide data sets according to a statistical methodology. Our parallel and multiobjective metrics point out the relevance of combining hybrid programming and novel bioinspired designs with regard to other parallel and biological approaches from the literature. [ABSTRACT FROM PUBLISHER]

Over the past few decades, IO workload characterization has been a critical issue for operating system and storage community. Even so, the issue still deserves investigation because of the continued introduction of novel storage devices such as solid-state drives (SSDs), which have different characteristics from traditional hard disks. We propose novel IO workload characterization and classification schemes, aiming at addressing three major issues: (i) deciding right mining algorithms for IO traffic analysis, (ii) determining a feature set to properly characterize IO workloads, and (iii) defining essential IO traffic classes state-of-the-art storage devices can exploit in their internal management. The proposed characterization scheme extracts basic attributes that can effectively represent the characteristics of IO workloads and, based on the attributes, finds representative access patterns in general workloads using various clustering algorithms. The proposed classification scheme finds a small number of representative patterns of a given workload that can be exploited for optimization either in the storage stack of the operating system or inside the storage device. [ABSTRACT FROM AUTHOR]

The scheduling of tasks with deadlines on clusters is a key issue for offering quality-of-service (QoS) assurance. A critical challenge in real-time task scheduling is to handle various types of applications. This paper investigates the scheduling problem for processing a set of tasks comprising both divisible and indivisible real-time tasks on cluster systems. Indivisible tasks are characterized by the property that they need to be processed on their entirety on a single processor while divisible tasks can be distributed across several processing nodes by exploiting the underlying data parallelism. We propose a dynamic (on-line) real-time scheduling algorithm referred to as Hybrid Loads Push-Pull Scheduling (HLPPS) algorithm for handling a set of tasks comprising both divisible and indivisible real-time tasks on cluster systems. HLPPS is shown to efficiently exploit the parallelism in divisible tasks without undermining the schedulability of indivisible tasks and thereby optimize the overall performance. We consider two distinct network platforms - tightly coupled and loosely coupled clusters in designing the strategy. We conduct extensive performance evaluation studies to quantify the performance of the proposed algorithm under a variety of scenarios. [ABSTRACT FROM AUTHOR]

In this paper, we first propose a very interesting and practical problem, pattern matching in a distributed mobile environment. Pattern matching is a well-known problem and extensive research has been conducted for performing effective and efficient search. However, previous proposed approaches assume that data are centrally stored, which is not the case in a mobile environment (e.g., mobile phone networks), where one person's pattern could be separately stored in a number of different stations, and such a local pattern is incomplete compared with the global pattern. A simple solution to pattern matching over a mobile environment is to collect all the data distributed in base stations to a data center and conduct pattern matching at the data center afterwards. Clearly, such a simple solution will raise huge amount of communication traffic, which could cause the communication bottleneck brought by the limited wireless bandwidth to be even worse. Therefore, a communication efficient and search effective solution is necessary. In our work, we present a novel solution which is based on our well-designed weighted bloom filter (WBF), called, D istributed Incomplete pattern matching ( DI-matching), to find target patterns over a distributed mobile environment. Specifically, to save communication cost and ensure pattern matching in distributed incomplete patterns, we use WBF to encode a query pattern and disseminate the encoded data to each base station. Each base station conducts a local pattern search according to the received WBF. Only qualified IDs and corresponding weights in each base station are sent to the data center for aggregation and verification. Through non-trivial theoretical analysis and extensive empirical experiments on a real city-scale mobile networks data set, we demonstrate the effectiveness and efficiency of our proposed solutions. [ABSTRACT FROM PUBLISHER]

A new step-by-step comprehensive MR physics simulator (MRISIMUL) of the Bloch equations is presented. The aim was to develop a magnetic resonance imaging (MRI) simulator that makes no assumptions with respect to the underlying pulse sequence and also allows for complex large-scale analysis on a single computer without requiring simplifications of the MRI model. We hypothesized that such a simulation platform could be developed with parallel acceleration of the executable core within the graphic processing unit (GPU) environment. MRISIMUL integrates realistic aspects of the MRI experiment from signal generation to image formation and solves the entire complex problem for densely spaced isochromats and for a densely spaced time axis. The simulation platform was developed in MATLAB whereas the computationally demanding core services were developed in CUDA-C. The MRISIMUL simulator imaged three different computer models: a user-defined phantom, a human brain model and a human heart model. The high computational power of GPU-based simulations was compared against other computer configurations. A speedup of about 228 times was achieved when compared to serially executed C-code on the CPU whereas a speedup between 31 to 115 times was achieved when compared to the OpenMP parallel executed C-code on the CPU, depending on the number of threads used in multithreading (2–8 threads). The high performance of MRISIMUL allows its application in large-scale analysis and can bring the computational power of a supercomputer or a large computer cluster to a single GPU personal computer. [ABSTRACT FROM PUBLISHER]