Search

Log-structured merge trees (LSM trees) are increasingly used as part of the storage engine behind several data systems, and are frequently deployed in the cloud. As the number of applications relying on LSM-based storage backends increases, the problem of performance tuning of LSM trees receives increasing attention. We consider both nominal tunings—where workload and execution environment are accurately known a priori—and robust tunings—which consider uncertainty in the workload knowledge. This type of workload uncertainty is common in modern applications, notably in shared infrastructure environments like the public cloud. To address this problem, we introduce Endure, a new paradigm for tuning LSM trees in the presence of workload uncertainty. Specifically, we focus on the impact of the choice of compaction policy, size ratio, and memory allocation on the overall performance. Endure considers a robust formulation of the throughput maximization problem and recommends a tuning that offers near-optimal throughput when the executed workload is not the same, instead in a neighborhood of the expected workload. Additionally, we explore the robustness of flexible LSM designs by proposing a new unified design called K-LSM that encompasses existing designs. We deploy our robust tuning system, Endure, on a state-of-the-art key-value store, RocksDB, and demonstrate throughput improvements of up to 5 × in the presence of uncertainty. Our results indicate that the tunings obtained by Endure are more robust than tunings obtained under our expanded LSM design space. This indicates that robustness may not be inherent to a design, instead, it is an outcome of a tuning process that explicitly accounts for uncertainty. [ABSTRACT FROM AUTHOR]

Configuration tuning is vital to achieving high performance for a database management system (DBMS). Recently, automatic tuning methods using Reinforcement Learning (RL) have been explored to find better configurations compared with database administrators (DBAs) and heuristics. However, existing RL-based methods still have several limitations: (1) Excessive overhead due to reliance on cloned databases; (2) trial-and-error strategy may produce dangerous configurations that lead to database failure; (3) lack the ability to handle dynamic workload. To address the above challenges, a fast and stable RL-based database tuning system, FASTune, is proposed. A virtual environment is proposed to evaluate configurations which is an equivalent yet more efficient scheme than the cloned database. To ensure stability during tuning, FASTune adopts an environment proxy to avoid dangerous configurations. In addition, a Multi-State Soft Actor–Critic (MS-SAC) model is proposed to handle dynamic workloads, which utilizes the soft actor–critic network to tune the database according to workload and database states. The experimental results indicate that, compared with the state-of-the-art methods, FASTune can achieve improvements in performance while maintaining stability in the tuning. [ABSTRACT FROM AUTHOR]

The performance optimization of database systems has been widely studied for years. From the perspective of the operation and maintenance personnel, it mainly includes three topics: prediction, diagnosis, and tuning. The prediction of future performance can guide the adjustment of configurations and resources. The diagnosis of anomalies can determine the root cause of performance regression. Tuning operations improve performance by adjusting influencing factors, e.g., knobs, indexes, views, resources, and structured query language (SQL) design. In this review, we focus on the performance optimization of database systems and review notable research work on the topics of prediction, diagnosis, and tuning. For prediction, we summarize the techniques, strengths, and limitations of several proposed systems for single and concurrent queries. For diagnosis, we categorize the techniques by the input data, i.e., monitoring metrics, logs, or time metrics, and analyze their abilities. For tuning, we focus on the approaches commonly adopted by the operation and maintenance personnel, i.e., knob tuning, index selection, view materialization, elastic resource, storage management, and SQL antipattern detection. Finally, we discuss some challenges and future work. [ABSTRACT FROM AUTHOR]

Thanks to the rapid advances in artificial intelligence, a brand new venue for database performance optimization is through deep neural networks and the reinforcement learning paradigm. Alongside the long literature in this regime, an iconic and crucial problem is the index structure building. For this problem, the prior works have largely adopted a pure learning-based solution replacing the traditional methods such as a B-tree and Hashing. While this line of research has drawn much attention in the field, they ubiquitously abandon the semantic guarantees and also suffer from performance loss in certain scenarios. In this work, we propose the Neural Index Search (NIS) framework. The core to this framework is to train a search policy to find a near optimal combination plan over the existing index structures, together with the required configuration parameters associated with each index structure in the plan. We argue that compared against the pure learning approaches, NIS enjoys the advantages brought by the chosen conventional index structures and further robustly enhances the performance from any singular index structure. Extensive empirical results demonstrate that our framework achieves state-of-the-art performances on several benchmarks. [ABSTRACT FROM AUTHOR]

The need to maintain database performance in Silterra Malaysia is crucial as data produced from complex manufacturing processes must be recorded in timely manner for reporting purposes. Query rewriting using Oracle Materialized View (MV) is one form of corrective action adopted by Silterra in order to tune its database, which is usually affected by problematic SQLs. However, whether MVs are useful in most cases of query rewriting is an open problem. In this paper, the flow of SQL query rewriting process using MVs is presented. Steps to identify problematic SQLs and to rewrite them are given based on DBA's experience in dealing with database performance issue in this industry. The result of using MVs shown using real fabrication data in Silterra reveals that, even though most MVs perform better than queries without MVs, there are cases that require alternative for MVs. [ABSTRACT FROM AUTHOR]

In present scenario the manual work (Done by Human) cost more to an organization than the automatic work ( Done by Machine)and the ratio is increasing day by day as per the tremendous increment in Machine (Hardware + Software) Intelligence. We are moving towards the world where the Machines will be able to perform better than today by their own intelligence. They will adjust themselves as per the customer's performance need. But to make this dream true, lots of human efforts (Theoretical and Practical) are needed to increase the capability of Machines to take their own decision and make the future free from manual work and reduce the working cost. Our life is covered with the different types of systems working around. The information system is one of them. All businesses are having the base by this system. So there is the most preference job of the IT researcher to make the Information system self-Manageable. The Development of well-established frameworks are needed to made them Auto-tuned is the basic need of the current business.The DBMS vendors are also providing the Auto-Tune packages with their DBMS Application. But they charge for these Auto-Tune packages. This extra cost of packages can be eliminated by using some basic Operating system utilities (e.g. VB Script, Task Scheduler, Batch Files, and Graphical Utility etc.). We have designed a working framework for Automatic Tuning of DBMS by using the Basic Utilities of Operating System (e.g. Windows) .These utilities will collect the statistics of SGA dynamic Parameters. The Framework will automatically analyze these SGA Parameter statistics and give suggestions fordiagnose the problem. In this paper we have presented that framework with practical Implementation. [ABSTRACT FROM AUTHOR]

The index selection problem for a relational database is of considerable practical importance. In [4], a solution to this problem is offered by reducing it to a classical knapsack problem and then applying an approximation algorithm. Here we show that this reduction process does not work in general by providing a counter-example. We also briefly discuss the practical significance of this counter-example. It turns out that the main ideas in [4] need not be discarded. In particular, the approximation algorithm used can be adapted fairly easily to take care of the problems which were raised by the counter-example. In spite of its simplicity, this modification can lead to a reduced number of indices, which is rather attractive from a practical point of view. [ABSTRACT FROM AUTHOR]