Search

Recently, research on six-degree-of-freedom virtual reality (VR) systems based on a light field (LF) has been actively conducted. The LF-based approach is photorealistic and less prone to errors than the existing three-dimensional (3D) modeling-based approach. On the other hand, because the amount of data is very large for LF-based immersive virtual reality, it is important to manage the data efficiently. This paper makes two major contributions to data management for rendering acceleration in this context. First, the GPUs commonly used for high-speed rendering require optimization of the data transfer process between the host memory and the device memory. This paper proposes a simple but effective way to maximize data reuse by finding the proper size of a transfer unit depending on the target applications and the system capability constraints. Also, view synthesis and data transfer are performed in a pipelined manner to hide the data movement time. Second, for the LF rendering of a wide space, data management in the storage-host-GPU hierarchy is attempted for the first time. As the size of the virtual space increases, the required amount of LF data grows rapidly and exceeds the allocatable host memory capacity. Because the loading of data stored in storage is a very slow process, a system-level LF rendering structure that carefully considers the characteristics of the storage-host-GPU memory hierarchy should be designed. In this paper, progressive LF updating is proposed, and it prevents rendering stalls caused by slow storage data loading. Experimental results show that a 360-degree view with a $9000 \times 2048$ resolution can be rendered in an average of 2.57 ms from LF data with a $4096 \times 2048$ resolution, achieving performance close to 400 frames per second. [ABSTRACT FROM AUTHOR]

So-called spatial web queries retrieve web content representing points of interest, such that the points of interest have descriptions that are relevant to query keywords and are located close to a query location. Two broad categories of such queries exist. The first encompasses queries that retrieve single spatial web objects that each satisfy the query arguments. Most proposals belong to this category. The second category, to which this paper's proposal belongs, encompasses queries that support exploratory user behavior and retrieve sets of objects that represent regions of space that may be of interest to the user. Specifically, the paper proposes a new type of query, the top- $k$ k spatial textual cluster retrieval ($k$ k -STC) query that returns the top- $k$ k clusters that (i) are located close to a query location, (ii) contain objects that are relevant with regard to given query keywords, and (iii) have an object density that exceeds a given threshold. To compute this query, we propose a DBSCAN-based approach and an OPTICS-based approach that rely on on-line density-based clustering and that exploit early stop conditions. Empirical studies on real data sets offer evidence that the paper's proposals can find good quality clusters and are capable of excellent performance. [ABSTRACT FROM AUTHOR]

Visual exploration has become an integral part of big spatial data management. With the increase in volume and number of spatial datasets, several specialized mechanisms have been proposed to speed up the exploration of these datasets. However, the existing techniques have major limitations which make them incapable of providing visual exploration for hundreds of thousands of big datasets on a single machine. This paper introduces a new index structure, termed AID*, that facilitates the visual exploration of an arbitrarily large number of big spatial datasets on a single machine. The AID* index defines multi-resolution fixed-size tiles on the input and classifies them as image, data, shallow, or empty tiles, based on their processing cost. Then, it uses this classification to build an index with a minimal index size and construction time, while supporting the desired real-time exploration interface. The index is constructed in parallel, using Hadoop or Spark, and is accessible to end users through a standard web interface similar to Google Maps. The small size of the index allows a single-machine server to host arbitrarily many datasets. Our experiments, on up-to 1 TB of data and 27 billion records, show that the construction of the proposed index is up-to an order of magnitude faster than the baselines without compromising the end-user interactivity. [ABSTRACT FROM AUTHOR]

High voltage circuit breakers (HVCBs) play a critical role on providing the desired reliability, and resiliency in power systems. In order to extend their lifetime and predict the failures, various maintenance policies could be applied on these critical components. Amongst these strategies, condition-based maintenance (CBM) provides a satisfactory agreement with future smart environment. This paper aims to provide an insight into the relevant developments in this subject and to explore the viable visions compatible with future research stream. Accordingly, three directions, i.e., diagnostic signals, intelligent modelling and using monitoring data in asset management have been addressed in this paper. It presents challenges dealing with real-time assessment of the diagnostic signals relating to measurements, and analyses. Subsequently, the issues associated with using artificial intelligent (AI) and Machine learning for providing intelligent algorithms have been discussed. Finally, the connection between the monitoring data and the asset management approach is investigated. The latter is looking for the subjects including remaining lifetime estimation, prioritization, and health index definitions. This paper has attempted to make a bridge from past to future research trends in the failure diagnosis of HVCBs. [ABSTRACT FROM AUTHOR]

Efficient real-time analytics are an integral part of an increasing number of data management applications, such as computational targeted advertising, algorithmic trading, and Internet of Things. In this paper, we focus primarily on accelerating stream joins, which are arguably one of the most commonly used and resource-intensive operators in stream processing. We propose a scalable circular pipeline design ($\sf{ Circular\text{-}MJ}$ Circular - MJ ) in hardware to orchestrate a multiway join while minimizing data flow disruption. In this circular design, each new tuple (given its origin stream) starts its processing from a specific join core and passes through all respective join cores in a pipeline sequence to produce the final results. We also present a novel two-stage pipeline stream join ($\sf{ Stashed\text{-}MJ}$ Stashed - MJ ) that uses a best-effort buffering technique (referred to as stash) to maintain intermediate results. If an overwrite is detected in the stash, our design automatically resorts to recomputing intermediate results. Finally, we present a parallelized version of our multiway stream join by integrating our proposed pipelines into a parallel unidirectional flow-based architecture ($\sf{ Parallel\text{-}MJ}$ Parallel - MJ ). Our experimental results demonstrate a linear throughput scaling with respect to the numbers of streams and processing cores. [ABSTRACT FROM AUTHOR]

Erasure codes are being extensively deployed in distributed storage systems instead of replication to achieve fault tolerance in a storage efficient manner. While traditional erasure codes are storage efficient, they can result in a significant increase in the amount of data access and downloaded during rebuilding of failed or otherwise unavailable nodes. In this paper, we present a new framework, which we call piggybacking, for constructing distributed storage codes that are efficient in the amount of data read and downloaded during rebuilding, while meeting requirements arising out of system considerations in data centers—maximum-distance-separability (MDS), high-rate, and a small number of so-called substripes. Under this setting, to the best of our knowledge, piggyback codes achieve the minimum average amount of data access and downloaded during rebuilding among all existing explicit solutions. The piggybacking framework also offers a rich design space for constructing codes for a variety of other settings. In particular, we construct codes that require minimum amount of data access and downloaded for rebuilding among all existing solutions for: 1) binary MDS array codes with more than two parities and 2) MDS codes with the smallest locality during rebuilding. In addition, we show how piggybacking can be employed to enable efficient repair of parity nodes in codes that address the rebuilding of only systematic nodes. The basic idea behind the piggybacking framework is to take multiple instances of existing codes and add carefully designed functions of the data from one instance to the others. This framework provides 25% to 50% savings in the average amount of data access and downloaded during rebuilding depending on the choice of the code parameters. [ABSTRACT FROM AUTHOR]

In a power system, unlike some critical and standalone assets that are equipped with condition monitoring devices, the conditions of most regular in-group assets are acquired through periodic inspection work. Due to their large quantities, significant amount of manual inspection effort and sometimes data management issues, it is not uncommon to see the asset condition data in a target study area becomes unavailable or incomplete. Lack of asset condition data undermines the reliability assessment work. This paper tackles this important problem from an unconventional way – it explores how to generate numerical and non-numerical asset condition data based on condition degradation, condition correlation and categorical distribution models. Empirical knowledge from human experts can also be incorporated in the modeling process. Also, a probabilistic diversification step can be taken to make the generated numerical condition data probabilistic. This method can generate close-to-real asset condition data and has been validated systematically based on two public datasets. An area reliability assessment example based on cables is given to demonstrate the usefulness of this method and its generated data. This method can also be used to conveniently generate hypothetical asset condition data for research purposes. [ABSTRACT FROM AUTHOR]

With 5G on the verge of being adopted as the next mobile network, there is a need to analyze its impact on the landscape of computing and data management. In this paper, we analyze the impact of 5G on both traditional and emerging technologies and project our view on future research challenges and opportunities. With a predicted increase of 10-100x in bandwidth and 5-10x decrease in latency, 5G is expected to be the main enabler for smart cities, smart IoT and efficient healthcare, where machine learning is conducted at the edge. In this context, we investigate how 5G can help the development of federated learning. Network slicing, another key feature of 5G, allows running multiple isolated networks on the same physical infrastructure. However, security remains the main concern in the context of virtualization, multi-tenancy and high device density. Formal verification of 5G networks can be applied to detect security issues in massive virtualized environments. In summary, 5G will make the world even more densely and closely connected. What we have experienced in 4G connectivity will pale in comparison to the vast amounts of possibilities engendered by 5G. [ABSTRACT FROM AUTHOR]

This paper presents a distributed synchrophasor data management (DSDM) technique for wide-area monitoring systems. The proposed technique focuses on managing voltage/current phasor data measured from a specific target bus by defining neighboring phasor measurement units (PMUs). We developed an algorithm for searching neighboring PMUs by interpreting the power network topology as a graph structure. By exploiting the data of neighboring PMUs, we constructed a multi-channel structure to recover low-quality entries by using the parallel implementation of long short-term memory networks. A successive output decision-making rule automatically determines the final output, considering the occurrence of defective input, data quality of the measured data, and operational conditions. The proposed DSDM can complement existing centralized management techniques by preserving information on local dynamics. In particular, because of the multi-channel structure, DSDM can provide robust responses to defective data in neighboring PMUs, such as an unintended change in a network topology. We verified the effectiveness of the DSDM with real-world synchrophasor data as well as simulated data from the IEEE 68-bus test system. [ABSTRACT FROM AUTHOR]

Data quality plays a key role in big data management today. With the explosive growth of data from a variety of sources, the quality of data is faced with multiple problems. Motivated by this, we study the multiple data cleaning on incompleteness and inconsistency with currency reasoning and determination in this paper. We introduce a 4-step framework, named ${\sf Imp3C}$ Imp 3 C , for errors detection and quality improvement in incomplete and inconsistent data without timestamps. We achieve an integrated currency determining method to compute the currency orders among tuples, according to currency constraints. Thus, the inconsistent data and missing values are repaired effectively considering the temporal impact. For both effectiveness and efficiency consideration, we carry out inconsistency repair ahead of incompleteness repair. A currency-related consistency distance metric is defined to measure the similarity between dirty tuples and clean ones more accurately. In addition, currency orders are treated as an important feature in the missing imputation training process. The solution algorithms are introduced in detail with case studies. A thorough experiment on three real-life datasets verifies our method ${\sf Imp3C}$ Imp 3 C improves the performance of data repairing with multiple quality problems. ${\sf Imp3C}$ Imp 3 C outperforms the existing advanced methods, especially in the datasets with complex currency orders. [ABSTRACT FROM AUTHOR]

In mobile communication networks, when a user roams to and accesses a foreign network, the foreign operator needs to request the user’s subscription data from a centralized authentication server, which is managed by the user’s home operator. However, centralized authentication introduces single point of failure. Meanwhile, the real-time participation of the home operator and the trust relationship between the foreign operator and the home operator are difficult to guarantee. In this paper, by adopting blockchain and smart contracts, we propose a secure and efficient access control scheme of user subscription data in roaming scenarios. We further design a flexible user authentication scheme, which utilizes derivable tokens based on the proposed access control scheme. By using blockchain to store and manage user subscription data, access control can be decentralized without any trusted third party. Besides, by implementing automatic verification of access privilege through smart contracts, the limitation of the home operators’ real-time participation is eliminated. In addition, to further improve security and reduce the on-chain storage overhead, we optimize the data encryption and storage scheme utilizing threshold secret sharing. Our security and performance analysis show that the proposed user subscription data access control scheme for roaming service provides high-level security while causing acceptable time and storage overhead. [ABSTRACT FROM AUTHOR]

The important topic of global climate change builds on a huge collection of scientific research. It is common for agencies releasing climate change information to be served with requests for all supporting materials resulting in a particular conclusion. Capturing and presenting global change provenance, linking to the research papers, data sets, models, analyses, observations, satellites, etc., that support the key research findings in this domain can increase understanding and aid in reproducibility of results and conclusions. The U.S. Global Change Research Program is now coordinating the production of a national climate assessment (NCA) that presents our best understanding of global change. We are now developing a global change information system that will present the content of that report and its provenance, including the scientific support for the findings of the assessment. We are using an approach that will present this information both through a human accessible Web site as well as a machine-readable interface for automated mining of the provenance graph. We plan to use the developing World Wide Web Consortium (W3C) PROV data model and ontology for this system. This paper will describe an overview of the process of developing the NCA and how the provenance trail of the report and each of the technical inputs can be captured and represented using the W3C PROV ontology. This will improve the visibility into the assessment process, increase understanding and possibility of reproducibility, and ultimately increase the credibility and trust of the resulting report. [ABSTRACT FROM AUTHOR]

Electric power systems are going through a major upgrade with the integration of advanced technologies to enable the smarter electric grid (SEG). The SEG will use information and communications technology to have enhanced controllability and will become more interactive. This ongoing change also necessitates educating professionals and future generation of engineers to manage evolving complexity of the electric grid. This paper presents experiences in the design and teaching of a unique multidisciplinary team-taught course on smart grid cyber infrastructure to provide a new generation of engineers with a solid foundation of smart grid concepts and their associated challenges. This paper identifies the course topics covered, learning objectives, and assessment activities for the class as well as lessons learned based on course evaluations obtained from multiple offerings. All of the course material is available in public domain and can be easily adopted at another educational institution. [ABSTRACT FROM AUTHOR]

In Australia and many other countries, distribution network service providers (DNSPs) have an obligation to their customers to provide electrical power that is reliable and of high quality. Failure to do so may have significant implications ranging from financial penalties theoretically through to the loss of a license to distribute electricity. In order to ensure the reliability and quality of supply are met, DNSPs engage in monitoring and reporting practice. This paper provides an overview of a large long-running power-quality monitoring project that has involved most of Australia's DNSPs at one time or another. This paper describes the challenges associated with conducting the project as well as some of the important outcomes and lessons learned. A number of novel reporting techniques that have been developed as part of the monitoring project are also presented. A discussion about large-volume data management, and issues related to reporting requirements in future distribution networks is included. [ABSTRACT FROM PUBLISHER]

This paper reports the results and post-challenge analyses of ChaLearn’s AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) that had been introduced in a variety of settings, but lacked fair comparisons. All input data modalities (time series, images, videos, text, tabular) were formatted as tensors and all tasks were multi-label classification problems. Code submissions were executed on hidden tasks, with limited time and computational resources, pushing solutions that get results quickly. In this setting, DL methods dominated, though popular Neural Architecture Search (NAS) was impractical. Solutions relied on fine-tuned pre-trained networks, with architectures matching data modality. Post-challenge tests did not reveal improvements beyond the imposed time limit. While no component is particularly original or novel, a high level modular organization emerged featuring a “meta-learner”, “data ingestor”, “model selector”, “model/learner”, and “evaluator”. This modularity enabled ablation studies, which revealed the importance of (off-platform) meta-learning, ensembling, and efficient data management. Experiments on heterogeneous module combinations further confirm the (local) optimality of the winning solutions. Our challenge legacy includes an ever-lasting benchmark (http://autodl.chalearn.org), the open-sourced code of the winners, and a free “AutoDL self-service.” [ABSTRACT FROM AUTHOR]

Due to the growing popularity of indoor location-based services, indoor data management has received significant research attention in the past few years. However, we observe that the existing indexing and query processing techniques for the indoor space do not fully exploit the properties of the indoor space. Consequently, they provide below par performance which makes them unsuitable for large indoor venues with high query workloads. In this paper, we first propose two novel indexes called Indoor Partitioning Tree (IP-Tree) and Vivid IP-Tree (VIP-Tree) that are carefully designed by utilizing the properties of indoor venues. The proposed indexes are lightweight, have small pre-processing cost and provide near-optimal performance for shortest distance and shortest path queries. We are also the first to study spatial keyword queries in indoor venues. We propose a novel data structure called Keyword Partitioning Tree (KP-Tree) that indexes objects in an indoor partition. We propose an efficient algorithm based on VIP-Tree and KP-Trees to efficiently answer spatial keyword queries. Our extensive experimental study on real and synthetic data sets demonstrates that our proposed indexes outperform the existing solutions by several orders of magnitude. [ABSTRACT FROM AUTHOR]

In this paper, we investigate threshold effects associated with the swapping of signal and noise subspaces in estimating signal parameters from compressed noisy data. The term threshold effect refers to a sharp departure of mean-squared error from the Cramér–Rao bound when the signal-to-noise ratio falls below a threshold SNR. In many cases, the threshold effect is caused by a subspace swap event, when the measured data (or its sample covariance) is better approximated by a subset of components of an orthogonal subspace than by the components of a signal subspace. We derive analytical lower bounds on the probability of a subspace swap in compressively measured noisy data in two canonical models: a first-order model and a second-order model. In the first-order model, the parameters to be estimated modulate the mean of a complex multivariate normal set of measurements. In the second-order model, the parameters modulate the covariance of complex multivariate measurements. In both cases, the probability bounds are tail probabilities of F-distributions, and they apply to any linear compression scheme. These lower bounds guide our understanding of threshold effects and performance breakdowns for parameter estimation using compression. In particular, they can be used to quantify the increase in threshold SNR as a function of a compression ratio C. We demonstrate numerically that this increase in threshold SNR is roughly 10\log10 \ C dB, which is consistent with the performance loss that one would expect when measurements in Gaussian noise are compressed by a factor C. [ABSTRACT FROM PUBLISHER]

Wind turbine blades have been constantly increasing since wind energy becomes a popular renewable energy source to generate electricity. Therefore, the wind sector requires a more efficient and representative characterization of vertical wind speed profiles to assess the potential for a wind power plant site. This paper proposes an alternative characterization of vertical wind speed profiles based on Ward's agglomerative clustering algorithm, including both wind speed module and direction data. This approach gives a more accurate incoming wind speed variation around the rotor swept area, and subsequently, provides a more realistic and complete wind speed vector characterization for vertical profiles. Real wind database collected for 2018 in the Forschungsplattformen in Nordund Ostsee (FINO) research platform is used to assess the methodology. A preliminary pre-processing stage is proposed to select the appropriated number of heights and remove missing or incomplete data. Finally, two locations and four heights are selected, and 561588 wind data are characterized. Results and discussion are also included in this paper. The methodology can be applied to other wind database and locations to characterize vertical wind speed profiles and identify the most likely wind data vector patterns.

Mining digitalisation have been receiving significant attention due to the utilisation of advanced technologies, such as IoT, automation, and sensing. However, maximising the potential value of collected data in the mining industry remains a challenge. Therefore, this paper aims to review timely concern topics to facilitate the fusion implementation in mining engineering. Specifically, this review covers recent popular topics, such as, data visualisation, data management, data analytics, data fusion, visual analytics, and mining digital twin construction. In this paper, we aim to draw a comprehensive picture about the fusion of data visualisation and analytics in the big data context, by examining the recent academic research related to these topics. Therefore, this paper reviews the visualisation domain by conventional classification, including scientific visualisation, information visualisation, and visual analytics, associated with the analysis of current digital twin development. Next, according to the challenges and issues related to visualisation development, this paper reviews the data management and data analytics domains as well. Incorporating with the fusion concept, machine learning-oriented fusion applications and potential scenarios in the mining industry have been discussed. In addition, based on the observation across various domains, this paper presents challenges and future potentials of data fusion in mining.

Random projection has been widely used in data classification. It maps high-dimensional data into a low-dimensional subspace in order to reduce the computational cost in solving the related optimization problem. While previous studies are focused on analyzing the classification performance in the low-dimensional space, in this paper, we consider the recovery problem, i.e., how to accurately recover the optimal solution to the original high-dimensional optimization problem based on the solution learned after random projection. We present a simple algorithm, termed dual random projection, which uses the dual solution of the low-dimensional optimization problem to recover the optimal solution to the original problem. Our theoretical analysis shows that with a high probability, the proposed algorithm is able to accurately recover the optimal solution to the original problem, provided that the data matrix is (approximately) low-rank and/or optimal solution is (approximately) sparse. We further show that the proposed algorithm can be applied iteratively to reducing the recovery error exponentially. [ABSTRACT FROM PUBLISHER]

For over many decades, relational database model has been considered as the leading model for data storage and management. However, as the Big Data explosion has generated a large volume of data, alternative models like NoSQL and NewSQL have emerged. With the advancement of communication technology, these database systems have given the potential to change the existing architecture from centralized mechanism to distributed in nature, to deploy as cloud-based solutions. Though all of these evolving technologies mostly focus on performance guarantees, it is still being a major concern how these systems can ensure the security and privacy of the information they handle. Different datastores support different types of integrated security mechanisms, however, most of the non-relational database systems have overlooked the security requirements of modern Big Data applications. This paper reviews security implementations in today's leading database models giving more emphasis on security and privacy attributes. A set of standard security mechanisms have been identified and evaluated based on different security classifications. Further, it provides a thorough review and a comprehensive analysis on maturity of security and privacy implementations in these database models along with future directions/enhancements so that data owners can decide on most appropriate datastore for their data-driven Big Data applications. [ABSTRACT FROM AUTHOR]

Emergency Medical Services (EMS) are of great importance to saving people's lives from emergent accidents and diseases by efficiently picking up patients using ambulances. The transporting capability of an EMS system (e.g., defined as the average pickup time of patients) significantly depends on the real-time redeployment strategy of ambulances. That is, which station should an ambulance be redeployed to, after it becomes available (after it transports a patient to a hospital or after it finishes the in-site treatment for a patient)? However, it is a challenging task concerning with the multiple data D1-D5 as detailed in Introduction. To this end, in this paper, we propose a data-driven real-time ambulance redeployment approach that redeploys an ambulance to a proper station after it becomes available, so as to optimize the transporting capability of an EMS system, considering the aforementioned multiple data D1-D5. Specifically, the proposed approach is comprised of two stages to well consider the D1-D5. First, we propose a method (a safety time-based urgency index) to incorporate D1, D2, and D3 into each ambulance station's urgency degree (D*). Second, we propose an optimal matching algorithm to combine D*, D4, and D5 into the redeployment of the current available ambulance. Experimental results using data collected in real world demonstrate the significant advantages of our approach over many baselines. Comparing with baselines, our approach can save $\sim$ ∼ 4 minutes ($\sim$ ∼ 35 percent) of the average pickup time for each patient, improve the ratio of patients picked up within 10 minutes from 0.684 and 0.803 ($\sim$ ∼ 17 percent), and largely enhance the survival rate of patients ($\sim$ ∼ 12 percent for patients in category A1 and $\sim$ ∼ 17 percent for patients in A2). [ABSTRACT FROM AUTHOR]

Load balancing is a widely accepted technique for performance optimization of scientific applications on parallel architectures. Indeed, balanced applications do not waste processor cycles on waiting at points of synchronization and data exchange, maximizing this way the utilization of processors. In this paper, we challenge the universality of the load-balancing approach to optimization of the performance of parallel applications. First, we formulate conditions that should be satisfied by the performance profile of an application in order for the application to achieve its best performance via load balancing. Then we use a real-life scientific application, EULAG MPDATA kernel, to demonstrate that its performance profile on a modern parallel architecture, Intel Xeon Phi, significantly deviates from these conditions. Based on this observation, we propose a method of performance optimization of scientific applications through load imbalancing. In the case of data parallel application, the method uses functional performance models of the application to find partitioning that minimizes its computation time but not necessarily balances the load of processors. We apply this method to optimization of MPDATA on Intel Xeon Phi. Experimental results demonstrate that the performance of this carefully optimized load-balanced application can be further improved by 15percent using the proposed load-imbalancing technique. [ABSTRACT FROM PUBLISHER]

With the development of the economy, products are significantly enriched, and uncertainty has been their inherent quality. The probabilistic dynamic skyline (PDS) query is a powerful tool for customers to use in selecting products according to their preferences. However, this query suffers several limitations: it requires the specification of a probabilistic threshold, which reports undesirable results and disregards important results; it only focuses on the objects that have large dynamic skyline probabilities; and, additionally, the results are not stable. To address this concern, in this paper, we formulate an uncertain dynamic skyline (UDS) query over a probabilistic product set. Furthermore, we propose effective pruning strategies for the UDS query, and integrate them into effective algorithms. In addition, a novel query type, namely the top κ favorite probabilistic products (TFPP) query, is presented. The TFPP query is utilized to select κ products which can meet the needs of a customer set at the maximum level. To tackle the TFPP query, we propose a TFPP algorithm and its efficient parallelization. Extensive experiments with a variety of experimental settings illustrate the efficiency and effectiveness of our proposed algorithms. [ABSTRACT FROM PUBLISHER]

Software-Defined Datacenters (SDDC) have been widely used for load-aware data management for different applications across the globe. Due to its centralized architecture, the issues of scalability along with resilience (to overcome the failure of single or multiple controllers) are still challenging because of an exponential increase in the data generated from different smart devices. Most of the solutions reported in the literature for this problem use a single controller which may not address the scalability issues. However, the issues as mentioned above of scalability and resilience in SDDC can be solved by deploying multiple distributed controllers at the control plane. However, the primary concern in a network having various controllers is the optimal Controller Placement Problem (CPP) to resolve the issues of fault-tolerance, latency among controllers, availability, and placement. Hence, to resolve the issues described above, in this paper, we propose Placement Availability Resilient Controller (PARC) scheme. The PARC scheme works in the following four phases: (i) stable network partitioning (ii) localization of controllers using the cooperative game theory (iii) computation of an optimal number of multiple controllers and (iv) computation of minimal extra backup controllers to improve the overall network cost. The numerical results of the PARC scheme are evaluated on Internet2 OS3E topology using POCO-toolset simulated in Matlab. The experimental results demonstrated that the cost of deploying the number of controllers using the PARC scheme has reduced to 12.98%, 8.16%, and 6.25% as compared to the POCO-SA, POCO-MOALO, and CNCP schemes respectively. Moreover, the PARC scheme outperforms the existing state-of-the-art schemes (POCO-SA, POCO-MOALO, and CNCP) for inter-controller as well as switch-to-controller latency. [ABSTRACT FROM AUTHOR]

There has been an increasing trend of moving computing activities closer to the edge of the network, particularly in smart city applications (e.g., vehicle-to-everything – V2X). Such a paradigm allows the end user’s requests to be handled/processed by nodes at the edge of the network; thus, reducing latency, and preserving privacy of user data/activities. However, there are a number of challenges in such an edge computing ecosystem. Examples include (1) potential inappropriate utilization of resources at the edge nodes, (2) operational challenges in cache management and data integrity due to data migration between edge nodes, particularly when dealing with vehicular mobility in a V2X application, and (3) high energy consumption due to continuous link breakage and subsequent reestablishment of link(s). Therefore in this paper, we design a blockchain-based secure data processing framework for an edge envisioned V2X environment (hereafter referred to as BloCkEd). Specifically, a multi-layered edge-enabled V2X system model for BloCkEd is presented, which includes the formulation of a multi-objective optimization problem. In addition, BloCkEd comprises an optimal container-based data processing scheme, and a blockchain-based data integrity management scheme, designed to minimize link breakage and reducing latency. Using Chandigarh City, India, as the scenario, we implement and evaluate the proposed approach in terms of its latency, energy consumption, and service level agreement compliance. [ABSTRACT FROM AUTHOR]

Melt pool monitoring in selective laser melting (SLM) is a key challenge to enhance the understanding of the basic process physics, as well as the in situ identification of drifts. The main issues are related to the molten pool’s fast dynamics and small spatial extent, which require high-performance monitoring systems in terms of temporal and spatial resolutions, often resulting in unmanageable data rates. Furthermore, the broadband thermal emission from the process can be viewed in different spectral ranges depending on the sensor and optical system employed. Accordingly, a wide range of sensing configurations is possible. The choice of monitoring parameters is crucial to achieve a solution capable of describing process dynamics with industrial applicability in terms of data transport and analysis. This paper discusses the design of a coaxial monitoring module to assess melt pool dynamics in SLM using temporal and spatial cues related to the process physics. Restrictions regarding data management were considered to ensure future inline process control. The system was tested with an external illuminator to reveal the actual melt pool geometry employing an acquisition rate of 1200 frames per second, 4.3 mm $\times $ 4.3 mm field of view, and $14~\mu \text{m}$ /pixel spatial resolution. The process emission in visible (640 nm) and near-infrared regions (850–1000 nm) was also acquired and the band choice discussed. The proposed solution captured successfully the melting conditions from both spatial and temporal viewpoints. The monitoring system depicted the melt pool shape variations when processing different geometries using both modulated and continuous wave laser emissions. [ABSTRACT FROM AUTHOR]

Data caching at mobile clients is an important technique for improving the performance of wireless data dissemination systems. However, variable data sizes, data updates, limited client resources, and frequent client disconnections make cache management a challenge. In this paper, we propose a gain-based cache replacement policy, Min-SAUD, for wireless data dissemination when cache consistency must be enforced before a cached item is used. Min-SAUD considers several factors that affect cache performance, namely, access probability, update frequency, data size, retrieval delay, and cache validation cost. This paper employs stretch as the major performance metric since it accounts for the data service time and, thus, is fair when items have different sizes. We prove that Min-SAUD achieves optimal stretch under some standard assumptions. Moreover, a series of simulation experiments have been conducted to thoroughly evaluate the performance of Min-SAUD under various system configurations. The simulation results show that, in most cases, the Min-SAUD replacement policy substantially outperforms two existing policies, namely, LRU and SAIU. [ABSTRACT FROM AUTHOR]

Query processing over uncertain data has gained growing attention, because it is necessary to deal with uncertain data in many real-life applications. In this paper, we investigate skyline queries over uncertain data in distributed environments (DSUD query) whose research is only in an early stage. The state-of-the-art algorithm, called e-DSUD algorithm, is designed for processing this query. It has the desirable characteristics of progressiveness and minimum bandwidth consumption. However, it still needs to be perfected in three aspects. (1) Progressiveness. Each time it only returns one query result at most. (2) Efficiency. There are a significant amount of redundant I/O cost and numerous iterations which causes a long total query time. (3) Universality. It is restricted to the case where local skyline tuples are incomparability. To address these concerns, we first present a detailed analysis of the e-DSUD algorithm and then develop an improved framework for the DSUD query, namely IDSUD. Based on the new framework, we propose an adaptive algorithm, called ADSUD, for the DSUD query. In the algorithm, we redefine the approximate global skyline probability and choose local representative tuples due to minimum probabilistic bounding rectangle adaptively. Furthermore, we design a progressive pruning method and apply the reuse mechanism to improve its efficiency. The results of extensive experiments verify the better overall performance of our algorithm than the e-DSUD algorithm. [ABSTRACT FROM AUTHOR]

Group skyline query is a powerful tool for optimal group analysis. Most of the existing group skyline queries select optimal groups by comparing the dominance relationship between aggregate-based points; such feature creates difficulties for users to specify an appropriate aggregate function. Besides, many significant groups that have great attractions to users in practice may be overlooked. To address these issues, the group skyline (GSky) query is formulated on the basis of a general definition of group dominance operator. While the existing GSky query algorithms are effective, there is still room for improvement in terms of progressiveness and efficiency. In this paper, we propose some new lemmas which facilitate direct generation of the GSky query results. Consecutively, we design a layered unit-based (LU) algorithm that applies a layered optimum strategy. Additionally, for the GSky query over the data that are dynamically produced and cannot be indexed, we propose a novel index-independent algorithm, called sorted-based progressive (SP) algorithm. The experimental results demonstrate the effectiveness, efficiency, and progressiveness of the proposed algorithms. By comparing with the state-of-the-art algorithm for the GSky query, our LU algorithm is more scalable and two orders of magnitude faster. [ABSTRACT FROM AUTHOR]

The inclusion of various intelligent electronic devices such as smart meters for AMI is expected to result in intermittent or frequent communications network congestion if additional network infrastructure investments are not made. One approach to deal with such a data volume challenge in smart grids without additional investments to increase network capacity is to aggregate data streams within the network whenever network congestion happens, but this needs to be done carefully so as not to significantly impact the applications that need the data. This paper proposes a novel approach to manage AMI data traffic volume in multi-level data collection trees through data aggregation that estimates the expected network delays messages would suffer and dynamically determines an aggregation policy to be applied at forwarding nodes of the tree to reduce the network delays with which the electric utility can get the necessary information. The proposed algorithm is evaluated for different network congestion scenarios using the NS-3 simulator. The results illustrate that the algorithm is immensely effective in controlling the increase in network latencies as congestion levels increase in AMI networks. In addition, it does well in satisfying quality-of-service (QoS) requirements in terms of the data granularity required by smart grid applications. [ABSTRACT FROM AUTHOR]

The papers in this special section focus on advanced techniques for reliable cloud storage. Cloud computing has emerged as a disruptive technology and new computing paradigm in which IT resources (hardware, software, applications, data storage, computing capacity, etc.) are provided as services over the Internet, through "everything-as-a-service". It builds on the foundations of distributed computing, grid computing, virtualization, service orientation, etc. Cloud storage, which is one of the most attractive cloud services, offers numerous benefits from both the technology and functionality perspectives such as increased availability, flexibility, functionality and costs. However, Cloud environment needs to manage data center virtualization, lower cost and boost reliability by consolidating systems in the cloud. [ABSTRACT FROM AUTHOR]

Hardware transactional memory (HTM) is an emerging hardware feature. HTM simplifies the programming model of concurrent programs while preserving high and scalable performance. With the commercial availability of HTM-capable processors, HTM has recently been adopted to construct efficient concurrent index structures. However, with the expansion of data volume and user amount, data management systems have to process workloads exhibiting high contention; meanwhile, according to our experiments, the conventional HTM-base concurrent index structures fail to provide scalable performance under highly-contented workloads. Such performance pathology strictly constrains the usage of HTM on data management systems. In this paper, we first conduct a thorough analysis on HTM-based concurrent index structures, and uncover several reasons for excessive HTM aborts incurred by both false and true conflicts under contention. Based on the analysis, we advocate Eunomia, a design pattern for HTM-based concurrent index structure which contains several principles to improve HTM performance, including splitting HTM regions with version-based concurrency control to reduce HTM working sets, partitioned data layout to reduce false conflicts, proactively detecting and avoiding conflicting requests, and adaptive concurrency control strategy. To validate their effectiveness, we apply such design principles to construct a scalable concurrent B+Tree and a skip list using HTM. Evaluation using key-value store and database benchmarks on a 20-core HTM-capable multi-core machine shows that Eunomia leads to substantial speedup under high contention, while incurring small overhead under low contention. [ABSTRACT FROM AUTHOR]

Distributed stream processing frameworks are designed to perform continuous computation on possibly unbounded data streams whose rates can change over time. Devising solutions to make such systems elastically scale is a fundamental goal to achieve desired performance and cut costs caused by resource over-provisioning. These systems can be scaled along two dimensions: the operator parallelism and the number of resources. In this paper, we show how these two dimensions, as two symbiotic entities, are independent but must mutually interact for the global benefit of the system. On the basis of this observation, we propose a fine-grained model for estimating the resource utilization of a stream processing application that enables the independent scaling of operators and resources. A simple, yet effective, combined management of the two dimensions allows us to propose ELYSIUM, a novel elastic scaling approach that provides efficient resource utilization. We implemented the proposed approach within Apache Storm and tested it by running two real-world applications with different input load curves. The outcomes backup our claims showing that the proposed symbiotic management outperforms elastic scaling strategies where operators and resources are jointly scaled. [ABSTRACT FROM AUTHOR]

This paper considers the problem of linear parameter-varying (LPV) dual-rate system identification with slow-rate output data corrupted by random measurement delays. The local approach is used, and the LPV model is built through convex combination of several local linear models with time-varying weights. The dual-rate sampled data often exist in industry, and the measured output data are inevitably corrupted by random measurement delays. In order to handle random measurement delays and dual-rate sampled data in LPV system identification, the statistical description of the identification problem is established based on the presented output-interpolated LPV dual-rate model, and then, the proposed algorithm to estimate all the unknown model parameters, real output data, and measurement delays are derived in the generalized expectation–maximization algorithm scheme. One simulation example and a practical chemical process are used to verify the efficiency of the proposed method. [ABSTRACT FROM PUBLISHER]

Energy consumption is one of the prominent design constraints of multi-core embedded systems. Since the memory subsystem is responsible for a considerable portion of energy consumption of embedded systems, Non-Volatile Memories (NVMs) have been proposed as a candidate for replacing conventional memories such as SRAM and DRAM. The advantages of NVMs compared to conventional memories are that they consume less leakage power and provide higher density. However, these memories suffer from increased overhead of write operations and limited lifetime. In order to address these issues, researchers have proposed NVM-aware memory management techniques that consider the characteristics of the memories of the system when deciding on the placement of the application data. In systems equipped with memory management unit (MMU), the application data is partitioned into pages during the compile phase and the data is managed at page level during the runtime phase. In this paper we present an NVM-aware data partitioning and mapping technique for multi-core embedded systems equipped with MMU that specifies the placement of the application data based on access pattern of the data and characteristics of the memories. The experimental results show that the proposed technique improves the energy consumption of the system by 28.10 percent on average. [ABSTRACT FROM AUTHOR]

Distributed systems generate a large amount of monitoring data such as log files to track their operational status. However, it is hard to correlate such monitoring data effectively across distributed systems and along observation time for system management. In previous work, we proposed a concept named flow intensity to measure the intensity with which internal monitoring data reacts to the volume of user requests. We calculated flow intensity measurements from monitoring data and proposed an algorithm to automatically search constant relationships between flow intensities measured at various points across distributed systems. If such relationships hold all the time, we regard them as invariants of the underlying systems. Invariants can be used to characterize complex systems and suppoil various system management tasks. However, the computational complexity of the previous invariant search algorithm is high so that it may not scale well in large systems with thousands of measurements. In this paper, we propose two efficient but approximate algorithms for inferring invariants in large-scale systems. The computational complexity of new randomized algorithms is significantly reduced, and experimental results from a real system are also included to demonstrate the accuracy and efficiency of our new algorithms. [ABSTRACT FROM AUTHOR]

The Ozone Monitoring Instrument (OMI) was launched on July 15, 2004 on the National Aeronautics and Space Administration's Earth Observing System Aura satellite. OMI is an ultraviolet-visible imaging spectrograph providing daily global coverage with high spatial resolution. This paper discusses the ground data processing software used for Level 0 to Level lb processing of OMI data. In addition, the OMI operations scenario is described together with the data processing concept. This paper is intended to serve as a reference guide for users of OMI (Level 1b) data. [ABSTRACT FROM AUTHOR]

Peer-to-peer (P2P) online communities are commonly perceived as an environment offering both opportunities and threats. One way to minimize threats in such communities is to use community-based reputations to help estimate the trustworthiness of peers. This paper presents PeerTrust-a reputation-based trust supporting framework, which includes a coherent adaptive trust model for quantifying and comparing the trustworthiness of peers based on a transaction-based feedback system, and a decentralized implementation of such a model over a structured P2P network. PeerTrust model has two main features. First, we introduce three basic trust parameters and two adaptive factors in computing trustworthiness of peers, namely, feedback a peer receives from other peers, the total number of transactions a peer performs, the credibility of the feedback sources, transaction context factor, and the community context factor. Second, we define a general trust metric to combine these parameters. Other contributions of the paper include strategies used for implementing the trust model in a decentralized P2P environment, evaluation mechanisms to validate the effectiveness and cost of PeerTrust model, and a set of experiments that show the feasibility and benefit of our approach. [ABSTRACT FROM AUTHOR]

In the era of rapid data expansion and computer technology development, discrete storage, multiband push and fuzzy query remote sensing data management methods are no longer suitable for the data analysis needs of users, including the needs for long time series, global regions and multidata fusion. After analyzing traditional data management techniques, this paper discusses the existing achievements and development trends of current technologies. This paper aims to solve the problem of data sharing difficulties and organizational inconsistency caused by the use of different formats for the same spatial object. Based on a discrete global grid, this paper studies the blocky division method and coding specification of Google S2 and then accomplishes the layered storage of remote sensing data in HBase. Finally, Kylin is used to build a cube model to discuss the information mining analysis changes in the new data management model. Experiments show that the blocky and layered management schema (BLMS) can realize the unified management of global remote sensing data with multisource, heterogeneous, multiscale, and long-term characteristics and provide accurate data services on demand.

The widespread diffusion of distributed and cloud storage solutions has changed dramatically the way users, system designers, and service providers manage their data. Outsourcing data on remote storage provides indeed many advantages in terms of both capital and operational costs. The security of data outsourced to the cloud, however, still represents one of the major concerns for all stakeholders. Pollution attacks, whereby a set of malicious entities attempt to corrupt stored data, are one of the many risks that affect cloud data security. In this paper we deal with pollution attacks in coding-based block-level cloud storage systems, i.e., systems that use linear codes to fragment, encode, and disperse virtual disk sectors across a set of storage nodes to achieve desired levels of redundancy, and to improve reliability and availability without sacrificing performance. Unfortunately, the effects of a pollution attack on linear coding can be disastrous, since a single polluted fragment can propagate pervasively in the decoding phase, thus hampering the whole sector. In this work we show that, using rateless codes, we can design an early pollution detection algorithm able to spot the presence of an attack while fetching the data from cloud storage during the normal disk reading operations. The alarm triggers a procedure that locates the polluting nodes using the proposed detection mechanism along with statistical inference. The performance of the proposed solution is analyzed under several aspects using both analytical modelling and accurate simulation using real disk traces. Our results show that the proposed approach is very robust and is able to effectively isolate the polluters, even in harsh conditions, provided that enough data redundancy is used. [ABSTRACT FROM AUTHOR]

Many of the words in a given document either deliver facts (objective) or express opinions ( subjective), respectively, depending on the topics they are involved in. For example, given a bunch of documents, the word “bug” assigned to the topic “order Hemiptera” apparently remarks one object (i.e., one kind of insects), while the same word assigned to the topic “software ” probably conveys a negative opinion. Motivated by the intuitive assumption that different words have varying degrees of discriminative power in delivering the objective sense or the subjective sense with respect to their assigned topics, a model named as discriminatively objective- subjective LDA (dosLDA) is proposed in this paper. The essential idea underlying the proposed dosLDA is that a pair of objective and subjective selection variables are explicitly employed to encode the interplay between topics and discriminative power for the words in documents in a supervised manner. As a result, each document is appropriately represented as “bag-of-discriminative-words” (BoDW). The experiments reported on documents and images demonstrate that dosLDA not only performs competitively over traditional approaches in terms of topic modeling and document classification, but also has the ability to discern the discriminative power of each word in terms of its objective or subjective sense with respect to its assigned topic. [ABSTRACT FROM AUTHOR]

Discovering evolving communities in dynamic networks is essential to important applications such as analysis for dynamic web content and disease progression. Evolutionary clustering uses the temporal smoothness framework that simultaneously maximizes the clustering accuracy at the current time step and minimizes the clustering drift between two successive time steps. In this paper, we propose two evolutionary nonnegative matrix factorization (ENMF) frameworks for detecting dynamic communities. To address the theoretical relationship among evolutionary clustering algorithms, we first prove the equivalence relationship between ENMF and optimization of evolutionary modularity density. Then, we extend the theory by proving the equivalence between evolutionary spectral clustering and ENMF, which serves as the theoretical foundation for hybrid algorithms. Based on the equivalence, we propose a semi-supervised ENMF (sE-NMF) by incorporating a priori information into ENMF. Unlike the traditional semi-supervised algorithms, a priori information is integrated into the objective function of the algorithm. The main advantage of the proposed algorithm is to escape the local optimal solution without increasing time complexity. The experimental results over a number of artificial and real world dynamic networks illustrate that the proposed method is not only more accurate but also more robust than the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The semantic web has enabled the creation of a growing number of knowledge bases (KBs), which are designed independently using different techniques. Integration of KBs has attracted much attention as different KBs usually contain overlapping and complementary information. Automatic techniques for KB integration have been improved but far from perfect. Therefore, in this paper, we study the problem of knowledge base semantic integration using crowd intelligence. There are both classes and instances in a KB, in our work, we propose a novel hybrid framework for KB semantic integration considering the semantic heterogeneity of KB class structures. We first perform semantic integration of the class structures via crowdsourcing, then apply the blocking-based instance matching approach according to the integrated class structure. For class structure (taxonomy) semantic integration, the crowd is leveraged to help identifying the semantic relationships between classes to handle the semantic heterogeneity problem. Under the conditions of both large scale KBs and limited monetary budget for crowdsourcing, we formalize the class structure (taxonomy) semantic integration problem as a Local Tree Based Query Selection (LTQS) problem. We show that the LTQS problem is NP-hard and propose two greedy-based algorithms, i.e., static query selection and adaptive query selection. Furthermore, the KBs are usually of large scales and have millions of instances, direct pairwise-based instance matching is inefficient. Therefore, we adopt the blocking-based strategy for instance matching, taking advantage of the class structure (taxonomy) integration result. The experiments on real large scale KBs verify the effectiveness and efficiency of the proposed approaches. [ABSTRACT FROM AUTHOR]

Efficient processing of skyline queries with partially ordered domains has been intensively addressed in recent years. To further reduce the query processing time to support high-responsive applications, the skyline queries that were previously processed with user preferences similar to those of the new query contribute useful candidate result points. Hence, the answered queries can be cached with both their results and the user preferences such that the query processor can rapidly retrieve the result for a new query only from the result sets of cached queries with compatible user preferences. When caching a significant number of queries accumulated over time, it is essential to adopt effective access methods to index the cached queries to retrieve a set of relevant cached queries for facilitating the cache-based skyline query computations. In this paper, we propose an extended depth-first search indexing method (e-DFS for short) for accessing user preference profiles represented by directed acyclic graphs (DAGs), and emphasize the design of the e-DFS encoding that effectively encodes a user preference profile into a low-dimensional feature point which is eventually indexed by an R-tree. We obtain one or more traversal orders for each node in a DAG by traversing it through a modified version of the depth-first search which is utilized to examine the topology structure and dominance relations to measure closeness or similarity. As a result, e-DFS which combines the criteria of similarity evaluation is able to greatly reduce the search space by filtering out most of the irrelevant cached queries such that the query processor can avoid accessing the entire data set to compute the query results. Extensive experiments are presented to demonstrate the performance and utility of our indexing method, which outperforms the baseline planning techniques by reducing 37 percent of the computational time on average. [ABSTRACT FROM AUTHOR]

Any important data management and analytics tasks cannot be completely addressed by automated processes. These tasks, such as entity resolution, sentiment analysis, and image recognition can be enhanced through the use of human cognitive ability. Crowdsouring platforms are an effective way to harness the capabilities of people (i.e., the crowd) to apply human computation for such tasks. Thus, crowdsourced data management has become an area of increasing interest in research and industry. We identify three important problems in crowdsourced data management. (1) Quality Control: Workers may return noisy or incorrect results so effective techniques are required to achieve high quality; (2) Cost Control: The crowd is not free, and cost control aims to reduce the monetary cost; (3) Latency Control: The human workers can be slow, particularly compared to automated computing time scales, so latency-control techniques are required. There has been significant work addressing these three factors for designing crowdsourced tasks, developing crowdsourced data manipulation operators, and optimizing plans consisting of multiple operators. In this paper, we survey and synthesize a wide spectrum of existing studies on crowdsourced data management. Based on this analysis we then outline key factors that need to be considered to improve crowdsourced data management. [ABSTRACT FROM PUBLISHER]

The mining and metal processing industries are undergoing a transformation through digitization, with sensors and data analysis playing a crucial role in modernization and increased efficiency. Vibration sensors are particularly important in monitoring production infrastructure in metal processing plants. This paper presents the installation of vibration sensors in an actual industrial environment and the results of spectral vibration data analysis. The study demonstrates that vibration sensors can be installed in challenging environments such as metal processing plants and that analyzing vibration patterns can provide valuable insights into predicting machine failures and different machine states. By utilizing dimensionality reduction and dominant frequency observation, we analyzed vibration data and identified patterns that are indicative of potential machine states and critical events that reduce production throughput. This information can be used to improve maintenance, minimize downtime, and ultimately enhance the production process’s overall efficiency. This study highlights the importance of digitization and data analysis in the mining and metal processing industries, particularly the capability not only to predict critical events before they impact production throughput and take action accordingly but also to identify machine states for legacy equipment and be part of retrofitting strategies.