Search

Many tools empower analysts and data scientists to consume analysis results in a visual interface, such as a dashboard. When the underlying data changes, these results need to be updated, but this update can take a long time -- all while the user continues to explore the results. In this context, tools can either (i) hide away results that haven't been updated, hindering exploration; (ii) make the updated results immediately available to the user (on the same screen as old results), leading to confusion and incorrect insights; or (iii) present old -- and therefore stale -- results to the user during the update. To help users reason about these options and others, and make appropriate trade-offs, we introduce Transactional Panorama, a formal framework that adopts transactions to jointly model the system refreshing the analysis results and the user interacting with them. We introduce three key properties that are important for user perception in this context, visibility (allowing users to continuously explore results), consistency (ensuring that results resented are from the same version of the data), and monotonicity (making sure that results don't "go back in time"). Within transactional panorama, we characterize all of the feasible property combinations, design new mechanisms (that we call lenses) for presenting analysis results to the user while preserving a given property combination, formally prove their relative orderings for various performance criteria and discuss their use cases. We propose novel algorithms to preserve each property combination and efficiently present fresh analysis results. We implement our transactional panorama framework in a popular, open-source BI tool, illustrate the relative performance implications of different lenses, demonstrate the benefits of the novel lenses, and outline the performance improvement by our optimizations.

Relatively recently, the field of join processing has been swayed by the discovery of a new class of multi-way join algorithms. The new algorithms join multiple relations simultaneously rather than perform a series of pairwise joins. The new join algorithms satisfy stronger worst-case runtime complexity guarantees than any of the existing approaches based on pairwise joins -- they are worst-case optimal in data complexity. These research efforts have resulted in a flurry of papers documenting theoretical and some practical contributions. However, there is still the quest of making the new worst-case optimal join algorithms truly practical in terms of (1) ease of implementation and (2) secondary index efficiency in terms of number of indexes created to answer a query. In this paper, we present a simple worst-case optimal multi-way join algorithm called the radix triejoin. Radix triejoin uses a binary encoding for reducing the domain of a database. Our main technical contribution is that domain reduction allows a bit-interleaving of attribute values that gives rise to a query-independent relation representation, permitting the computation of multiple queries over the same relations worst-case optimally without having to construct additional secondary indexes. We also generalise the core algorithm to conjunctive queries with inequality constraints and provide a new proof technique for the worst-case optimal join result., Comment: 12 pages, 4 figures

Concurrency control (CC) algorithms must trade off strictness for performance. Serializable CC schemes generally pay higher cost to prevent anomalies, both in runtime overhead and in efforts wasted by aborting transactions. We propose the serial safety net (SSN), a serializability-enforcing certifier which can be applied with minimal overhead on top of various CC schemes that offer higher performance but admit anomalies, such as snapshot isolation and read committed. The underlying CC retains control of scheduling and transactional accesses, while SSN tracks the resulting dependencies. At commit time, SSN performs an efficient validation test by examining only direct dependencies of the committing transaction to determine whether it can commit safely or must abort to avoid a potential dependency cycle. SSN performs robustly for various workloads. It maintains the characteristics of the underlying CC without biasing toward certain types of transactions, though the underlying CC might. Besides traditional OLTP workloads, SSN also allows efficient handling of heterogeneous workloads with long, read-mostly transactions. SSN can avoid tracking the majority of reads (thus reducing the overhead of serializability certification) and still produce serializable executions with little overhead. The dependency tracking and validation tests can be done efficiently, fully parallel and latch-free, for multi-version systems on modern hardware with substantial core count and large main memory. We demonstrate the efficiency, accuracy and robustness of SSN using extensive simulations and an implementation that overlays snapshot isolation in ERMIA, a memory-optimized OLTP engine that is capable of running different CC schemes. Evaluation results confirm that SSN is a promising approach to serializability with robust performance and low overhead for various workloads.

Minimizing coordination, or blocking communication between concurrently executing operations, is key to maximizing scalability, availability, and high performance in database systems. However, uninhibited coordination-free execution can compromise application correctness, or consistency. When is coordination necessary for correctness? The classic use of serializable transactions is sufficient to maintain correctness but is not necessary for all applications, sacrificing potential scalability. In this paper, we develop a formal framework, invariant confluence, that determines whether an application requires coordination for correct execution. By operating on application-level invariants over database states (e.g., integrity constraints), invariant confluence analysis provides a necessary and sufficient condition for safe, coordination-free execution. When programmers specify their application invariants, this analysis allows databases to coordinate only when anomalies that might violate invariants are possible. We analyze the invariant confluence of common invariants and operations from real-world database systems (i.e., integrity constraints) and applications and show that many are invariant confluent and therefore achievable without coordination. We apply these results to a proof-of-concept coordination-avoiding database prototype and demonstrate sizable performance gains compared to serializable execution, notably a 25-fold improvement over prior TPC-C New-Order performance on a 200 server cluster., Comment: Extended version of paper appearing in PVLDB Vol. 8, No. 3

To minimize network latency and remain online during server failures and network partitions, many modern distributed data storage systems eschew transactional functionality, which provides strong semantic guarantees for groups of multiple operations over multiple data items. In this work, we consider the problem of providing Highly Available Transactions (HATs): transactional guarantees that do not suffer unavailability during system partitions or incur high network latency. We introduce a taxonomy of highly available systems and analyze existing ACID isolation and distributed data consistency guarantees to identify which can and cannot be achieved in HAT systems. This unifies the literature on weak transactional isolation, replica consistency, and highly available systems. We analytically and experimentally quantify the availability and performance benefits of HATs--often two to three orders of magnitude over wide-area networks--and discuss their necessary semantic compromises., Comment: Extended version of "Highly Available Transactions: Virtues and Limitations" to appear in VLDB 2014

The traditional architecture for a DBMS engine has the recovery, concurrency control and access method code tightly bound together in a storage engine for records. We propose a different approach, where the storage engine is factored into two layers (each of which might have multiple heterogeneous instances). A Transactional Component (TC) works at a logical level only: it knows about transactions and their "logical" concurrency control and undo/redo recovery, but it does not know about page layout, B-trees etc. A Data Component (DC) knows about the physical storage structure. It supports a record oriented interface that provides atomic operations, but it does not know about transactions. Providing atomic record operations may itself involve DC-local concurrency control and recovery, which can be implemented using system transactions. The interaction of the mechanisms in TC and DC leads to multi-level redo (unlike the repeat history paradigm for redo in integrated engines). This refactoring of the system architecture could allow easier deployment of application-specific physical structures and may also be helpful to exploit multi-core hardware. Particularly promising is its potential to enable flexible transactions in cloud database deployments. We describe the necessary principles for unbundled recovery, and discuss implementation issues., Comment: CIDR 2009

In this paper, we propose an approach to providing the benefits of isolation in service-oriented applications where it is not feasible to hold traditional locks for ACID transactions. Our technique, called "Promises", provides an uniform view for clients which covers a wide range of implementation techniques on the service side, all allowing the client to check a condition and then later rely on that condition still holding., Comment: This article is published under a Creative Commons License Agreement (http://creativecommons.org/licenses/by/2.5/.) You may copy, distribute, display, and perform the work, make derivative works and make commercial use of the work, but, you must attribute the work to the author and CIDR 2007. 3rd Biennial Conference on Innovative Data Systems Research (CIDR) January 710, 2007, Asilomar, California, USA