Search

Relational Keyword Search (R-KwS) systems enable naive/informal users to explore and retrieve information from relational databases without knowing schema details or query languages. They take a keyword query, locate their corresponding elements in the target database, and connect them using information on PK/FK constraints. Although there are many such systems in the literature, most of them only support queries with keywords referring to the contents of the database and just very few support queries with keywords refering the database schema. We propose Lathe, a novel R-KwS that supports such queries. To this end, we first generalize the well-known concepts of Candidate Joining Networks (CJNs) and Query Matches (QMs) to handle keywords referring to schema elements and propose new algorithms to generate them. Then, we introduce two major innovations: a ranking algorithm for selecting better QMs, yielding the generation of fewer but better CJNs, and an eager evaluation strategy for pruning void useless CJNs. We present experiments performed with query sets and datasets previously experimented with state-of-theart R-KwS systems. Our results indicate that Lathe can handle a wider variety of queries while remaining highly effective, even for databases with intricate schemas.

The spectral information system SPECCHIO was updated to support the generic handling of uncertainty information in the form of uncertainty tree diagrams. The updates involve changes to the relations database model as well as dedicated methods provided by the SPECCHIO application programming interface. A case study selected from classic field spectroscopy demonstrates the use of the functionality. In conclusion, a database-centric automated uncertainty propagation in combination with measurement protocol standardization will provide a crucial step toward spectroscopy data accompanied by propagated, traceable, uncertainty information.

In many application domains, position information is of fundamental importance. However, unlike the case of outdoor positioning, producing an accurate position estimation in the indoor setting turns out to be quite difficult. One of the most common localisation strategies makes use of fingerprinting. Research in this area has been faced with a number of challenges, leading to the proposal of a number of localisation algorithms, sampling strategies, benchmark datasets, and representations of building information. This proliferation made the modeling of the indoor positioning domain quite hard from both a theoretical and a practical point of view. In this paper, we propose a general and extensible framework, based on a relational database, that pairs fingerprints with building information. We show how the proposed system successfully deals with a number of problems that affect indoor positioning, supporting a large set of relevant tasks. The source code of the framework is available online, as well as an implementation of it, that provides an interactive open repository of indoor positioning data.

Classical data cleaning performs a minimal set of operations on the data to satisfy the given integrity constraints. Often, this minimization is equivalent to vertex cover, for example when tuples can be removed due to the violation of functional dependencies. Classically, the uncertainty of tuples and constraints is ignored. We propose not to view data as dirty but the uncertainty information about data. Since probabilities are often unavailable and their treatment is limited due to correlations in the data, we investigate a qualitative approach to uncertainty. Tuples are assigned degrees of possibility with which they occur, and constraints are assigned degrees of certainty that say to which tuples they apply. Our approach is non-invasive to the data as we lower the possibility degree of tuples as little as possible. The new resulting qualitative version of vertex cover remains NP-hard. We establish an algorithm that is fixed-parameter tractable in the size of the qualitative vertex cover. Experiments with synthetic and real-world data show that our algorithm outperforms the classical algorithm proportionally to the available number of uncertainty degrees. By mining the certainty degrees with which constraints hold, our framework becomes applicable even when uncertainty information is unavailable. [ABSTRACT FROM AUTHOR]

Engineering design has a major impact on the downstream performance, but is often guided, in both procurement and solution type, by intuition rather than a solid science base. Engineering is also increasingly collaborative, crossing organizational boundaries. However, the long-term close partnerships, as often proposed, are difficult to achieve in the context of individual engineering projects. Building on the concepts related to investments in specific assets, in this article, we describe the underlying characteristics of engineering design investments and then prescribes the relational investment and contract forms to select them, thereby contributing to our understanding of the engineering design and relational investments that can and should be made. We undertook multiple case study research articles, which are presented in two phases. The descriptive phase identifies the positive outcomes for solutions based on off-the-shelf designs with fixed price mechanisms, for solutions based on the adaptive designs with target cost mechanisms, and for bespoke solutions contracted either on cost-plus mechanisms or, if in bite-sized pieces, on fixed price mechanisms. Negative performance outcomes were found for adaptive solutions with fixed price mechanisms. A prescriptive phase, yielding a visualized model, then offers guidance for relational investments and contract mechanisms that are suitable for different engineering designs. Applicability zones and three potential transitions to challenge and guide current practice are developed to inform decision making. [ABSTRACT FROM AUTHOR]

Querying and reporting from large volumes of structured, semistructured, and unstructured data often requires some flexibility. This flexibility provided by fuzzy sets allows for categorization of the surrounding world in a flexible, human-mind-like manner. Apache Hive is a data warehousing framework working on top of the Hadoop platform for big data processing. Hive allows executing queries and aggregating and analyzing data stored in Hadoop distributed file system and other repositories. Hive responds to the current needs for efficient big data warehousing, which is impossible with traditional data warehouses due to their rigid nature. This article presents the FuzzyHive library that extends the Hive framework with fuzzy sets based techniques for querying, analyzing, and reporting on big data warehouses. We formalize the fuzzy techniques used while operating on Hive-based data warehouses (including fuzzy filtering on dimensional attributes, projection with fuzzy transformation, fuzzy grouping, and joining). We also show how we embedded these operations in Hive query language, which was not studied so far. Such extensions make big data warehousing more flexible and contribute to the portfolio of tools used by the community of people working with fuzzy sets and data analysis. The FuzzyHive library complements the spectrum of available solutions for fuzzy data processing and querying in large datasets. We investigate Hive fuzzy querying performance, effectiveness, and scalability for various data storage formats (text, Avro, and Parquet). Our experiments demonstrate that the proposed extensions introduce more elasticity and are also efficient for big data warehousing, which is the first such kind of solution for this environment. [ABSTRACT FROM AUTHOR]

Several systems proposed for processing keyword queries over relational databases rely on the generation and evaluation of Candidate Networks (CNs), i.e., networks of joined database relations that when processed as SQL queries, provide a relevant answer to the input keyword query. Although the evaluation of CNs has been extensively addressed in the literature, the problem of generating CNs efficiently and effectively has received much less attention. This challenging problem consists of automatically locating relations in the database that may contain relevant pieces of information, given a handful of keywords, and determining suitable ways of joining these relations to satisfy the implicit information needs expressed by a user while formulating his/her query. In this paper, we propose a novel approach for generating CNs, wherein the possible matches for the query in the database are efficiently enumerated at first. These query matches are then used to guide the CN generation process, avoiding the exhaustive search procedure used by the current state-of-art approaches. We show that our approach allows the generation of a compact set of CNs that leads to superior quality answers, and demands less resources in terms of processing time and memory. These claims are supported by a comprehensive set of experiments that we carried out using several query sets and datasets used in previous related works and whose results we report and analyze here. [ABSTRACT FROM AUTHOR]

With the advent of the Internet and Internet-connected devices, modern business applications can experience rapid increases as well as variability in transactional workloads. Database replication has been employed to scale performance and improve availability of relational databases but past approaches have suffered from various issues including limited scalability, performance versus consistency tradeoffs, and requirements for database or application modifications. This paper presents Hihooi, a replication-based middleware system that is able to achieve workload scalability, strong consistency guarantees, and elasticity for existing transactional databases at a low cost. A novel replication algorithm enables Hihooi to propagate database modifications asynchronously to all replicas at high speeds, while ensuring that all replicas are consistent. At the same time, a fine-grained routing algorithm is used to load balance incoming transactions to available replicas in a consistent way. Our thorough experimental evaluation with several well-established benchmarks shows how Hihooi is able to achieve almost linear workload scalability for transactional databases. [ABSTRACT FROM AUTHOR]

With the emergence of big data era, NoSQL key-value database is considered as a promising candidate for replacing relational database management system (RDBMS). As cost per GB of flash memory is getting closer to HDD, high performance solid state drive (SSD) is regarded as the best substitute of HDD. However, there are still some issues that need to be taken into consideration. et al. have reported that applying key-value store to SSD with conventional FTL would incur internal fragmentation and further cause the degradation of device lifespan. Although they proposed KVFTL to deal with the above issues, their work gave rise to the read amplification problem. In this article, we investigate the root cause of the read amplification problem and propose elegantly measured tanner FTL (EMT-FTL) to mitigate internal fragmentation and read amplification with acceptable memory overhead. Different from KVFTL that slices a variable-sized value into multiple partitions (up to 20) of different sizes and manages them as a linked chain, EMT-FTL slices a value into 16 KiB full partitions with its remaining bytes treated as a fragment partition and appended to the fragment buffer. For a 64 GiB SSD with 16 KiB pages, the overall memory usage of EMT-FTL is only 8.81% of KVFTL. The experiments showed that EMT-FTL achieved almost the optimal space utilization as KVFTL did under most of traces and averagely improved the get performance by 78.57% (compared with KVFTL) under the traces with request sizes ranging from 1 byte to 16 KiB. [ABSTRACT FROM AUTHOR]

Spectral Information Systems provide a framework to assemble, curate, and serve spectral data and their associated metadata. This article documents the evolution of the SPECCHIO system, devised to enable long-term usability and data-sharing of field spectroradiometer data. The new capabilities include a modern, web-based client-server architecture, a flexible metadata storage scheme for generic metadata handling, and a rich application programming interface, enabling scientists to directly access spectral data and metadata from their programming environment of choice. The SPECCHIO system source code has been moved into the open source domain to stimulate contributions from the spectroscopy community while binary distributions, including the SPECCHIO virtual machine, simplify the installation and use of the system for the end-users.

In an era of super computing, data is increasing exponentially requiring more proficiency from the available technologies of data storage, data processing, and analysis. Such continuous massive growth of structured and unstructured data is referred to as a “Big data”. The processing and storage of big data through a conventional technique is not possible. Due to improved proficiency of Big Data solution in handling data, such as NoSQL caused the developers in the previous decade to start preferring big data databases, such as Apache Cassandra, Oracle, and NoSQL. NoSQL is a modern database technology that is designed to provide scalability to support voluminous data, leading to the rise of NoSQL as the most viable database solution. These modern databases aim to overcome the limitations of relational databases such as unlimited scalability, high performance, data modeling, data distribution, and continuous availability. These days, the larger enterprises need to shift NoSQL databases due to their more flexible models. It is a great challenge for business organizations and enterprises to transform their existing databases to NoSQL databases considering heterogeneity and complexity in relational data. In addition, with the emergence of big data, data cleansing has become a great challenge. In this paper, we proposed an approach that has two modules: data transformation and data cleansing module. The first phase is the transformation of a relational database to Oracle NoSQL database through model transformation. The second phase provides data cleansing ability to improve data quality and prepare it for big data analytics. The experiments show the proposed approach successfully transforms the relational database to a big data database and improve data quality.

This article presents a framework, Genesis (genome analysis), to efficiently and flexibly accelerate generic data manipulation operations that have become performance bottlenecks in the genomic data processing pipeline utilizing FPGAs-as-a-service. Genesis conceptualizes genomic data as a very large relational database and uses extended SQL as a domain-specific language to construct data manipulation queries. To accelerate the queries, we designed a Genesis hardware library of efficient coarse-grained primitives that can be composed into a specialized dataflow architecture. This approach explores a systematic and scalable methodology to expedite domain-specific end-to-end accelerated system development and deployment. [ABSTRACT FROM AUTHOR]

For over many decades, relational database model has been considered as the leading model for data storage and management. However, as the Big Data explosion has generated a large volume of data, alternative models like NoSQL and NewSQL have emerged. With the advancement of communication technology, these database systems have given the potential to change the existing architecture from centralized mechanism to distributed in nature, to deploy as cloud-based solutions. Though all of these evolving technologies mostly focus on performance guarantees, it is still being a major concern how these systems can ensure the security and privacy of the information they handle. Different datastores support different types of integrated security mechanisms, however, most of the non-relational database systems have overlooked the security requirements of modern Big Data applications. This paper reviews security implementations in today's leading database models giving more emphasis on security and privacy attributes. A set of standard security mechanisms have been identified and evaluated based on different security classifications. Further, it provides a thorough review and a comprehensive analysis on maturity of security and privacy implementations in these database models along with future directions/enhancements so that data owners can decide on most appropriate datastore for their data-driven Big Data applications. [ABSTRACT FROM AUTHOR]

E-health systems have witnessed widespread usage in the last few years, mainly due to advancements in health monitoring hardware and the availability of remote diagnostic services. Given the numerous benefits, there are two main challenges: management of the e-health ecosystem and security and privacy of sensitive data. The former results from nonunified, redundant, and often replicated information across numerous independent e-health service providers. While the later stems from sensitive information stored in centralized systems that can be compromised. Blockchain has emerged as a promising technology, which can be used to secure access and privacy of data and provide an umbrella management solution to large-scale distributed and decentralized enterprise systems. In this article, we present a unified system for migrating independent conventional e-health systems to a single blockchain-based ecosystem. More specifically, we address the issues of difference in data structures for conventional relational databases and blockchain file databases. The solution describes the conversion process and synchronization of information in a unified system for large-scale e-health data. The implementation and analysis show that significant improvements in data storage, access control, and seamless migration can be achieved. [ABSTRACT FROM AUTHOR]

In the sciences and elsewhere, the use of relational databases has become ubiquitous. An important challenge is finding hot spots of user interests. In principle, one can discover user interests by clustering the queries in the query log. Such a clustering requires a notion of query similarity. This, in turn, raises the question of what features of SQL queries are meaningful. We have studied the query representations proposed in the literature and corresponding similarity functions and have identified shortcomings of all of them. To overcome these limitations, we propose new similarity functions for SQL queries. They rely on the so-called access area of a query and, more specifically, on the overlap and the closeness of the access areas. We have carried out experiments systematically to compare the various similarity functions described in this article. The first series of experiments measures the quality of clustering and compares it to a ground truth. In the second series, we focus on the query log from the well-known SkyServer database. Here, a domain expert has interpreted various clusters by hand. We conclude that clusters obtained with our new measures of similarity seem to be good indicators of user interests. [ABSTRACT FROM AUTHOR]

Conceptual and relational data models of online transaction processing (OLTP) applications are usually created and maintained following the principle of normalization, which implies avoidance of redundancy. Data retrieval from a disk-based normalized relational database often requires complex and inefficient queries that may cause noticeable performance issues when executed on larger volumes of data. Computer professionals sometimes intentionally trade off the strict normal form to optimize data retrieval queries through error-prone manual tuning and denormalization. We propose a fully automatic optimization approach, based on data redundancy, that relies on a formal cost-benefit model. We prove that finding the optimal level of data redundancy, for given workload statistics, is an NP-Complete optimization problem. A detailed reduction of the problem to binary linear programming is presented in the paper. The proposed optimization approach was evaluated using the TPCE benchmark for OLTP systems. The evaluation has shown that the proposed optimization approach is highly scalable, and that it can be efficiently applied to real-life relational data models. [ABSTRACT FROM AUTHOR]

The typical five-minute computational latency of current Energy Management Systems (EMSs) can leave and has left system operators blind to the system state created by events that lead to cascading outages, system separation and/or system collapse. In this paper, we present a graph-computing based EMS that allows the EMS computational cycle time to be reduced to the SCADA sampling cycle, about five seconds. This is achieved in part by taking advantage of both nodal and hierarchical parallelism, to achieve high-speed graph-based state estimation, power flow, and contingency analysis applications. A production-mode case study on an existing large-scale system verifies the advantages and performance of graph computing, demonstrating that the graph-based EMS is more than 20 times faster than a contemporary commercial EMS. [ABSTRACT FROM AUTHOR]

Exploration of various characteristics of 3D protein structures through querying relational databases storing the structures can be challenging due to the necessity to conform to a particular database schema. However, this also brings several advantages, like the ability to perform extensive database searches with declarative SQL language, protect data against hardware damages through regular backup mechanisms, and secure data against unauthorized access. Since relational databases do not provide exploration methods specific for protein data and its biological semantics, like searches on the basis of protein structural patterns, the use of relational databases in this domain is still rare and requires the development of dedicated methods to increase the speed of data exploration techniques. In this paper, we show a novel data partitioning scheme for distributing data across database clusters that can be used for performing sophisticated explorations of 3D protein structures. The data partitioning scheme relies on protein construction, which requires data preprocessing but results in shorter exploration times through querying federated databases. We solve the problem of finding proteins in Oracle relational database on the basis of the similarity of 3D protein structures with the use of distributed PAR-P3D-SQL queries. Since 3D protein structure similarity searching is one of the most time-consuming exploration processes that can be performed for protein data, we make use of a distributed environment of Oracle federated databases, distributed query processing, and dedicated load balancing methods to accelerate the exploration. Results of performed tests confirm that we are able to significantly increase the speed of the exploration process, proportionally to the number of database nodes in the federated environment. [ABSTRACT FROM AUTHOR]

As data analytics has become an important application for modern data management systems, a new category of data management system has appeared recently: the scalable linear algebra system. In this paper, we argue that a parallel or distributed database system is actually an excellent platform upon which to build such functionality. Most relational systems already have support for cost-based optimization—which is vital to scaling linear algebra computations—and it is well-known how to make relational systems scale. We show that by making just a few changes to a parallel/distributed relational database system, such a system can be a competitive platform for scalable linear algebra. Taken together, our results should at least raise the possibility that brand new systems designed from the ground up to support scalable linear algebra are not absolutely necessary, and that such systems could instead be built on top of existing relational technology. Our results also suggest that if scalable linear algebra is to be added to a modern dataflow platform such as Spark, they should be added on top of the system's more structured (relational) data abstractions, rather than being constructed directly on top of the system's raw dataflow operators. [ABSTRACT FROM AUTHOR]

Entity Resolution (ER) is the task of finding entity profiles that correspond to the same real-world entity. Progressive ER aims to efficiently resolve large datasets when limited time and/or computational resources are available. In practice, its goal is to provide the best possible partial solution by approximating the optimal comparison order of the entity profiles. So far, Progressive ER has only been examined in the context of structured (relational) data sources, as the existing methods rely on schema knowledge to save unnecessary comparisons: they restrict their search space to similar entities with the help of schema-based blocking keys (i.e., signatures that represent the entity profiles). As a result, these solutions are not applicable in Big Data integration applications, which involve large and heterogeneous datasets, such as relational and RDF databases, JSON files, Web corpus etc. To cover this gap, we propose a family of schema-agnostic Progressive ER methods, which do not require schema information, thus applying to heterogeneous data sources of any schema variety. First, we introduce two naïve schema-agnostic methods, showing that straightforward solutions exhibit a poor performance that does not scale well to large volumes of data. Then, we propose four different advanced methods. Through an extensive experimental evaluation over 7 real-world, established datasets, we show that all the advanced methods outperform to a significant extent both the naïve and the state-of-the-art schema-based ones. We also investigate the relative performance of the advanced methods, providing guidelines on the method selection. [ABSTRACT FROM AUTHOR]

In software-defined networking (SDN), high-level abstractions typically offer a useful means to avoid writing network applications and policies on lower levels. However, abstractions are typically developed for a specific use case, which in turn results in an abundance of existing abstractions for different networking tasks. As a consequence, orchestrating these abstractions to implement a common network policy becomes an arduous task. To address this challenge, plain data representations of the network and its control infrastructure have been proposed recently, which offer programmable ad-hoc abstractions to administrators. However, these frameworks suffer from quite complex programming requirements and impractical performance in terms of latency, as they are based on relational database engines. In this paper, we address these shortcomings by introducing Gavel, an SDN controller that at its heart facilitates a plain data representation based on a graph database. By exploiting the native graph support of the database engine, Gavel significantly eases application and policy writing. Additionally, we show by experimental evaluation of several typical applications on multiple different topologies that Gavel offers significant performance improvements over the state-of-the-art solutions. [ABSTRACT FROM AUTHOR]

Hardware acceleration throughfield programmable gate arrays (FPGAs)has recently become a technique of growing interest for many data-intensive applications. Join query is one of the most fundamental database query types useful in relational database management systems. However, the available solutions so far have been beset by higher costs in comparison to other query types. In this paper, we develop a novel solution to accelerate the processing of sort-merge join queries with low match rates. Specifically, our solution makes use of hierarchical indexes to identify result-yielding regions in the solution space in order to take advantage of result sparseness. Further, in addition to one-dimensionalequi-joinquery processing, our solution supports processing of multidimensional similarity join queries. Experimental results show that our solution is superior to the best existing method in a low match rate setting; the method achieves a speedup factor of 4.8 for join queries with a match rate of 5 percent. [ABSTRACT FROM AUTHOR]

Heterogeneous networks are networks consisting of different types of objects and links. They can be found in several fields, ranging from the Internet to social sciences, biology, epidemiology, geography, finance, and many others. In the literature, several methods have been proposed for the analysis of network data, but they usually focus on homogeneous networks, where all the objects are of the same type, and links among them describe a single type of relationship. More recently, the complexity of real scenarios has impelled researchers to design methods for the analysis of heterogeneous networks, especially focused on classification and clustering tasks. However, they often make assumptions on the structure of the network that are too restrictive or do not fully exploit different forms of network correlation and autocorrelation. Moreover, when nodes which are the main subject of the classification task are linked to several nodes of the network having missing values, standard methods can lead to either building incomplete classification models or to discarding possibly relevant dependencies (correlation or autocorrelation). In this paper, we propose an ensemble learning approach for multi-type classification. We adopt the system Mr-SBC, which is originally able to analyze heterogeneous networks of arbitrary structure, within an ensemble learning approach. The ensemble allows us to improve the classification accuracy of Mr-SBC by exploiting i) the possible presence of correlation and autocorrelation phenomena, and ii) the classification of instances (which contain missing values) of other node types in the network. As a beneficial side effect, we have also that the models are more stable in terms of standard deviation of the accuracy, over different samples used for training. Experiments performed on real-world datasets show that the proposed method is able to significantly outperform the standard implementation of Mr-SBC. Moreover, it gives Mr-SBC the advantage of outperforming four other well-known algorithms for the classification of data organized in a network. [ABSTRACT FROM AUTHOR]

When RDF datasets become too large to be managed by centralised systems, they are often distributed in a cluster of shared-nothing servers, and queries are answered using a distributed join algorithm. Although such solutions have been extensively studied in relational and RDF databases, we argue that existing approaches exhibit two drawbacks. First, they usually decidestatically(i.e., at query compile time) how to shuffle the data, which can lead to missed opportunities for local computation. Second, they often materialise large intermediate relations whose size is determined by the entire dataset (and not the data stored in each server), so these relations can easily exceed the memory of individual servers. As a possible remedy, we present a novel distributed join algorithm for RDF. Our approach decides when to shuffle datadynamically, which ensures that query answers that can be wholly produced within a server involve only local computation. It also uses a novel flow control mechanism to ensure that every query can be answered even if each server has a bounded amount of memory that is much smaller than the intermediate relations. We complement our algorithm with a new query planning approach that balances the cost of communication against the cost of local processing at each server. Moreover, as in several existing approaches, we distribute RDF data using graph partitioning so as to maximise local computation, but we refine the partitioning algorithm to produce more balanced partitions. We show empirically that our techniques can outperform the state of the art by orders of magnitude in terms of query evaluation times, network communication, and memory use. In particular, bounding the memory use in individual servers can mean the difference between success and failure for answering queries with large answer sets. [ABSTRACT FROM AUTHOR]

In recent years, many fields that experience a sudden proliferation of data, which increases the volume of data that must be processed and the variety of formats the data is stored in have been identified. This causes pressure on existing compute infrastructures and data analysis methods, as more and more data are considered as a useful source of information for making critical decisions in particular fields. Among these fields exist several areas related to human life, e.g., various branches of medicine, where the uncertainty of data complicates the data analysis, and where the inclusion of fuzzy expert knowledge in data processing brings many advantages. In this paper, we show how fuzzy techniques can be incorporated in big data analytics carried out with the declarative U-SQL language over a big data lake located on the cloud. We define the concept of big data lake together with the Extract, Process, and Store process performed while schematizing and processing data from the Data Lake, and while storing results of the processing. Our solution, developed as a Fuzzy Search Library for Data Lake, introduces the possibility of massively parallel, declarative querying of big data lake with simple and complex fuzzy search criteria, using fuzzy linguistic terms in various data transformations, and fuzzy grouping. Presented ideas are exemplified by a distributed analysis of large volumes of biomedical data on Microsoft Azure cloud. Results of performed tests confirm that the presented solution is highly scalable on the Cloud and is a successful step toward soft and declarative processing of data on a large scale. The solution presented in this paper directly addresses three characteristics of big data, i.e., volume, variety, and velocity, and indirectly addresses, veracity and value. [ABSTRACT FROM AUTHOR]

Automated service composition fulfills complex tasks by combining different existing web services together. Unfortunately, optimizing service composition is still a challenging area that needs to be addressed. In this article, we propose a novel relational database approach for automated service composition. All possible service combinations are generated beforehand and stored in a relational database. When a user request comes, our system composes SQL queries to search in the database and return the best Quality of Service (QoS) solutions. We test the performance of the proposed system with a web service challenge data set. Our experimental results demonstrate that this system can always find top-K valid solutions to satisfy user's functional and non-functional requirements. [ABSTRACT FROM AUTHOR]

Database design is critical for high performance in relational databases and a myriad of tools exist to aid application designers in selecting an appropriate schema. While the problem of schema optimization is also highly relevant for NoSQL databases, existing tools for relational databases are inadequate in that setting. Application designers wishing to use a NoSQL database instead rely on rules of thumb to select an appropriate schema. We present a system for recommending database schemas for NoSQL applications. Our cost-based approach uses a novel binary integer programming formulation to guide the mapping from the application's conceptual data model to a database schema. We implemented a prototype of this approach for the Cassandra extensible record store. Our prototype, the NoSQL Schema Evaluator (NoSE) is able to capture rules of thumb used by expert designers without explicitly encoding the rules. Automating the design process allows NoSE to produce efficient schemas and to examine more alternatives than would be possible with a manual rule-based approach. [ABSTRACT FROM PUBLISHER]

Battery energy storage effectively staBIlizes the electric grid and aids renewable integration by balancing supply and demand in real time. The importance of such storage is especially crucial in densely populated urban areas, where traditional storage techniques such as pumped hydroelectric energy storage and compressed-air energy storage are often not feasible. [ABSTRACT FROM PUBLISHER]

Since NoSQL databases use data models, access modes and query languages that differ from the relational databases, the testing NoSQL database applications raises new challenges. There has been much attention to testing relational database applications, however testing NoSQL database applications is an area that has been hardly explored. This paper describes a model-based approach that allows the tester to model the important features of a graph database application to be tested, called test views. These test views are used to derive the test requirements that guide the generation of test cases. The approach is illustrated with an application that represents a real-world problem related with access control. The results show that using this model-based approach leads to test cases that are able to reveal faults and failures in both system specification and implementation. [ABSTRACT FROM PUBLISHER]

Recently, field-programmable gate array (FPGA) vendors (such as Altera) have started to address the programmability issues of FPGAs via OpenCL SDKs. In this paper, we analyze the performance of relational database applications on FPGAs using OpenCL. In particular, we study how to improve the performance of data partitioning, which is a very important building block in relational database. Since the data partitioning causes random memory accesses, it is time-consuming, and then, it has been the major bottleneck for database operators, such as partitioned hash join. In particular, we import the state-of-the-art OpenCL implementation of data partitioning from OmniDB, which was originally designed and optimized for CPUs/GPUs, and we find that this implementation suffers from both lock overhead and memory bandwidth overhead. Accordingly, we present a multikernel approach to address the lock overhead by leveraging two emerging features (task kernel and channel) from Altera OpenCL software development kit. In order to reduce the memory bandwidth overhead, on-chip buckets are used to reduce the number of random global memory transactions. We further develop an FPGA-specific cost model to guide the parameter configuration. We evaluate the proposed design on a recent OpenCL-based FPGA. We have applied our optimized partitioning method to a number of data processing tasks, including hash join, histogram, and hash search. Our experimental results demonstrate that our cost model can accurately guide the user to determine the optimal parameter combination for data partitioning and the optimal parameter combination can achieve $16.6\times$ speedup over the default multithreaded implementation. [ABSTRACT FROM PUBLISHER]

Data-intensive systems in e-governance collect and process data to ensure conformance to a set of business rules. Testers meticulously verify data in test databases, extracted from different steps of a live production stream , for correct application of business rules. We simplify the process by allowing testers to model a test domain on a relational database and automatically generate test cases representing data interactions satisfying combinatorial interaction coverage criteria. This paper also introduces test cases with self-referential interactions, which is a necessity in real-world databases. We verify these test cases using our human-in-the-loop tool, Depict. Depict, with expert assistance, generates complex SQL queries for test cases and produces a visual report of test case satisfaction. We apply the approach to two scenarios: 1) simplify and optimize a periodic archiving operation and 2) verify fault codes within the testing environment of the Custom directorate's TVINN system. [ABSTRACT FROM AUTHOR]

In this work we address the problem of retrieving potentially interesting matrix views to support the exploration of networks. We introduce Matrix Diagnostics (or Magnostics), following in spirit related approaches for rating and ranking other visualization techniques, such as Scagnostics for scatter plots. Our approach ranks matrix views according to the appearance of specific visual patterns, such as blocks and lines, indicating the existence of topological motifs in the data, such as clusters, bi-graphs, or central nodes. Magnostics can be used to analyze, query, or search for visually similar matrices in large collections, or to assess the quality of matrix reordering algorithms. While many feature descriptors for image analyzes exist, there is no evidence how they perform for detecting patterns in matrices. In order to make an informed choice of feature descriptors for matrix diagnostics, we evaluate 30 feature descriptors—27 existing ones and three new descriptors that we designed specifically for MAGNOSTICS-with respect to four criteria: pattern response, pattern variability, pattern sensibility, and pattern discrimination. We conclude with an informed set of six descriptors as most appropriate for Magnostics and demonstrate their application in two scenarios; exploring a large collection of matrices and analyzing temporal networks. [ABSTRACT FROM AUTHOR]

Causality and responsibility is an essential tool in the database community for providing intuitive explanations for answers/non-answers to queries. Causality denotes the causes for the answers/non-answers to queries, and responsibility represents the degree of a cause which reflects its influence on the answers/non-answers to queries. In this paper, we study the causality and responsibility problem (CRP) for the non-answers to probabilistic reverse skyline queries (PRSQ). We first formalize CRP on PRSQ, and then, we propose an efficient algorithm termed as CP to compute the causality and responsibility for the non-answers to PRSQ. CP first finds candidate causes, and then, it performs verification to obtain actual causes with their responsibilities, during which several strategies are used to boost efficiency. Further, we explore the CRP for the non-answers to reverse skyline queries. Towards this, we extend CP to identify directly all the actual causes and their responsibilities for a non-answer to reverse skyline queries without additional verification. Extensive experiments using both real and synthetic data sets demonstrate the effectiveness and efficiency of our presented algorithms. [ABSTRACT FROM AUTHOR]

The large-scale aggregation and analysis of user opinions is becoming increasingly relevant to a variety of applications, from detecting social mood on some political topics to tracking their sentiment changes related to events. The analysis of diverse sentiments is another important application, which becomes possible based on the ability of modern methods to capture sentiment polarity on various topics with high precision and on the ever-growing scale. Therefore, there is a need for a scalable way of sentiment aggregation with respect to the time dimension, which stores enough information to preserve diversity, and which allows statistically accurate analysis of sentiment trends and opinion shifts. In this paper, we are focusing on the novel problem of aggregating diverse sentiments at a large scale, based on data sources that are continuously updated. First, we develop a theoretical framework that models sentiment diversity (contradiction) and defines two types of contradictions, depending on the distribution of sentiments over time. Second, we introduce novel measures that capture sentiment diversity from aggregated sentiment statistics. Third, we develop robust and scalable indexing and storage methods for diverse sentiments. Finally, we propose an adaptive approach for identifying contradictions at different time scales. The experimental evaluation demonstrates the effectiveness of the proposed method of capturing contradictions and its superiority over relational databases in real-world scenarios. [ABSTRACT FROM AUTHOR]

Now-related temporal data play an important role in many applications. Clifford et al.’s approach is a milestone to model the semantics of ‘now’ in temporal relational databases. Several relational representation models for now-related data have been presented; however, the semantics of such representations has not been explicitly studied. Additionally, the definition of a relational algebra to query now-related data is an open problem. We propose the first integrated approach that provides both a neat semantics for now-related data and a compact 1NF representation (data model and relational algebra) for them. Additionally, our approach also extends current approaches to consider (i) domains where it is not always possible to know when changes in the world are recorded in the database and (ii) now-related data with a bound on their persistency in the future. To do so, we explicitly model the notion of temporal indeterminacy in the future for now-related data. The properties of our approach are also analyzed both from a theoretical (semantic correctness and reducibility of the algebra) and from an experimental point of view. Experiments show that, despite the fact that our approach is a major extension to current temporal relational approaches, no significant overhead is added to deal with ‘now’. [ABSTRACT FROM PUBLISHER]

Despite the huge amount of work devoted to the treatment of time within the relational context, few relevant temporal phenomena still remain to be addressed. One of them is the treatment of “nearly periodic events ”, i.e., events/facts that occur in intervals of time which repeat periodically (e.g., a meeting occurring twice each Monday, possibly not at regular times). Nearly periodic events are quite frequent in everyday life, and thus in many applicative contexts. Their treatment within the relational model is quite challenging, since it involves the integrated treatment of three aspects: (i) the number of repetitions, (ii) their periodicity, and (iii) temporal indeterminacy. Coping with this problem requires an in-depth extension of current temporal relational database techniques. In this paper, we introduce a new data model, and new definitions of relational algebraic operators coping with the above issues. We ascertain the properties of the new model and algebra, with emphasis on the expressiveness of our representation model, on the reducibility property, and on the correctness of the algebraic operators. [ABSTRACT FROM PUBLISHER]

Operational NoSQL systems are relatively new in the data-management ecosystem, and there is much confusion about their capabilities and how they differ from traditional relational database systems. This summary of characteristics clearly distinguishes the two system classes and provides a glimpse into directions for future work. [ABSTRACT FROM AUTHOR]

Time is pervasive of reality, and many relational database approaches have been developed to cope with it. In practical applications, facts can repeat several times, and only the overall period of time containing all the repetitions may be known (consider, e.g., On January, John attended five meetings of the Bioinformatics project). While some temporal relational databases have faced facts repeated at (known) periodic time, or single facts occurred at temporally indeterminate time, the conjunction of non-periodic repetitions and temporal indeterminacy has not been faced yet. Coping with this problem requires an in-depth extension of current techniques. In this paper, we have introduced a new data model, and new definitions of relational algebraic operators coping with the above issues. We have studied the properties of the new model and algebra (with emphasis on the reducibility property), and how it can be integrated with other models in the literature. [ABSTRACT FROM PUBLISHER]

We propose a method for unsupervised many-to-many object matching from multiple networks, which is the task of finding correspondences between groups of nodes in different networks. For example, the proposed method can discover shared word groups from multi-lingual document-word networks without cross-language alignment information. We assume that multiple networks share groups, and each group has its own interaction pattern with other groups. Using infinite relational models with this assumption, objects in different networks are clustered into common groups depending on their interaction patterns, discovering a matching. The effectiveness of the proposed method is experimentally demonstrated by using synthetic and real relational data sets, which include applications to cross-domain recommendation without shared user/item identifiers and multi-lingual word clustering. [ABSTRACT FROM PUBLISHER]

The SQL group-by operator plays an important role in summarizing and aggregating large datasets in a data analytics stack. While the standard group-by operator, which is based on equality, is useful in several applications, allowing similarity aware grouping provides a more realistic view on real-world data that could lead to better insights. The Similarity SQL-based Group-By operator (SGB, for short) extends the semantics of the standard SQL Group-by by grouping data with similar but not necessarily equal values. While existing similarity-based grouping operators efficiently realize these approximate semantics, they primarily focus on one-dimensional attributes and treat multi-dimensional attributes independently. However, correlated attributes, such as in spatial data, are processed independently, and hence, groups in the multi-dimensional space are not detected properly. To address this problem, we introduce two new SGB operators for multi-dimensional data. The first operator is the clique (or distance-to-all) SGB, where all the tuples in a group are within some distance from each other. The second operator is the distance-to-any SGB, where a tuple belongs to a group if the tuple is within some distance from any other tuple in the group. Since a tuple may satisfy the membership criterion of multiple groups, we introduce three different semantics to deal with such a case: (i) eliminate the tuple, (ii) put the tuple in any one group, and (iii) create a new group for this tuple. We implement and test the new SGB operators and their algorithms inside PostgreSQL. The overhead introduced by these operators proves to be minimal and the execution times are comparable to those of the standard Group-by. The experimental study, based on TPC-H and a social check-in data, demonstrates that the proposed algorithms can achieve up to three orders of magnitude enhancement in performance over baseline methods developed to solve the same problem. [ABSTRACT FROM AUTHOR]

Internet of Things (IoT)-generated data are characterized by its continuous generation, large amount, and unstructured format. The existing relational database technologies are inadequate to handle such IoT-generated data due to the limited processing speed and the significant storage-expansion cost. Thus, big data processing technologies, which are normally based on distributed file systems, distributed database management, and parallel processing technologies, have arisen as a core technology to implement IoT-generated data repositories. In this paper, we propose a sensor-integrated radio frequency identification (RFID) data repository-implementation model using MongoDB, the most popular big data-savvy document-oriented database system now. First, we devise a data repository schema that can effectively integrate and store the heterogeneous IoT data sources, such as RFID, sensor, and GPS, by extending the event data types in electronic product code information services standard, a de facto standard for the information exchange services for RFID-based traceability. Second, we propose an effective shard key to maximize query speed and uniform data distribution over data servers. Last, through a series of experiments measuring query speed and the level of data distribution, we show that the proposed design strategy, which is based on horizontal data partitioning and a compound shard key, is effective and efficient for the IoT-generated RFID/sensor big data. [ABSTRACT FROM PUBLISHER]

A Structured Query Language extension uses an estimator module to evaluate quality profiles that rate the accuracy and completeness of query results. Users receive information that matches their defined quality constraints and better serves their data needs. [ABSTRACT FROM PUBLISHER]

Given a database table with records that can be ranked, an interesting problem is to identify selection conditions for the table, which are qualified by an input record and render its ranking as high as possible among the qualifying tuples. In this paper, we study this standing maximization problem, which finds application in object promotion and characterization. After showing the hardness of the problem, we propose greedy methods, which are experimentally shown to achieve high accuracy compared to exhaustive enumeration, while scaling very well to the problem input size. Our contributions include a linear-time algorithm for determining the optimal selection range for an ordinal attribute and techniques for choosing and prioritizing the most promising selection predicates to apply. Experiments on real datasets confirm the effectiveness and efficiency of our techniques. [ABSTRACT FROM AUTHOR]

Spreadsheets are among the most commonly used applications for data management and analysis. They combine data processing with very diverse supplementary features: statistics, visualization, reporting, linear programming solvers, Web queries periodically downloading data from external sources, etc. However, the spreadsheet paradigm of computation still lacks sufficient analysis. In this article, we demonstrate that a spreadsheet can implement all data transformations definable in SQL, merely by utilizing spreadsheet formulas. We provide a query compiler, which translates any given SQL query into a worksheet of the same semantics, including NULL values. Thereby, database operations become available to the users who do not want to migrate to a database. They can define their queries using a high-level language and then get their execution plans in a plain vanilla spreadsheet. The functions available in spreadsheets impose limitations on the algorithms one can implement. In this paper, we offer $O(n\,\log ^2\,n)$ sorting spreadsheet, using a non-constant number of rows, and, surprisingly, Depth-First-Search and Breadth-First-Search on graphs. [ABSTRACT FROM PUBLISHER]

Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing. [ABSTRACT FROM PUBLISHER]

Hybrid human/computer database systems promise to greatly expand the usefulness of query processing by incorporating the crowd for data gathering and other tasks. Such systems raise many implementation questions. Perhaps the most fundamental issue is that the closed world assumption underlying relational query semantics does not hold in such systems. As a consequence, the meaning of even simple queries can be called into question. Furthermore, query progress monitoring becomes difficult due to non-uniformities in the arrival of crowdsourced data and peculiarities of how people work in crowdsourcing systems. To address these issues, we develop statistical tools that enable users and systems developers to reason about query completeness. These tools can also help drive query execution and crowdsourcing strategies. We evaluate our techniques using experiments on a popular crowdsourcing platform. [ABSTRACT FROM PUBLISHER]

Database technology affects many disciplines beyond computer science and business. This paper describes two animations developed with images and color that visually and dynamically introduce fundamental relational database concepts and querying to students of many majors. The goal is for educators in diverse academic disciplines to incorporate the animations in their existing courses in order to meet their pedagogical needs. The introduction of the animations was assessed and evaluated within several contexts, including non-majors courses offered by computer scientists as well as a course on computational molecular biology. The evaluation indicated that student knowledge of fundamental database concepts improved significantly with the use of the courseware. The animations provide visual learners with an engaging method to learn the topics presented with the ability to replay the dynamic presentation. Overall, the evaluation indicates the accessibility of database topics across the disciplines as well as the specific concepts that need further elaboration. [ABSTRACT FROM AUTHOR]

In a relational database, tuples are called “duplicate” if they describe the same real-world entity. If such duplicate tuples are observed, it is recommended to remove them and to replace them with one tuple that represents the joint information of the duplicate tuples to a maximal extent. This remove-and-replace operation is called a fusion operation. Within the setting of a relational database management system, the removal of the original duplicate tuples can breach referential integrity. In this paper, a strategy is proposed to maintain referential integrity in a semantically correct manner, thereby optimizing the quality of relationships in the database. An algorithm is proposed that is able to propagate a fusion operation through the entire database. The algorithm is based on a framework of first and second order fusion functions on the one hand, and conflict resolution strategies on the other hand. It is shown how classical strategies for maintaining referential integrity, such as delete cascading, are highly specialized cases of the proposed framework. Experimental results are reported that (i) show the efficiency of the proposed algorithm and (ii) show the differences in quality between several second order fusion functions. It is shown that some strategies easily outperform delete cascading. [ABSTRACT FROM PUBLISHER]

Data Mining has wide applications in many areas such as banking, medicine, scientific research and among government agencies. Classification is one of the commonly used tasks in data mining applications. For the past decade, due to the rise of various privacy issues, many theoretical and practical solutions to the classification problem have been proposed under different security models. However, with the recent popularity of cloud computing, users now have the opportunity to outsource their data, in encrypted form, as well as the data mining tasks to the cloud. Since the data on the cloud is in encrypted form, existing privacy-preserving classification techniques are not applicable. In this paper, we focus on solving the classification problem over encrypted data. In particular, we propose a secure $k$ -NN classifier over encrypted data in the cloud. The proposed protocol protects the confidentiality of data, privacy of user’s input query, and hides the data access patterns. To the best of our knowledge, our work is the first to develop a secure $k$ -NN classifier over encrypted data under the semi-honest model. Also, we empirically analyze the efficiency of our proposed protocol using a real-world dataset under different parameter settings. [ABSTRACT FROM PUBLISHER]