Search

Configuration tuning is vital to achieving high performance for a database management system (DBMS). Recently, automatic tuning methods using Reinforcement Learning (RL) have been explored to find better configurations compared with database administrators (DBAs) and heuristics. However, existing RL-based methods still have several limitations: (1) Excessive overhead due to reliance on cloned databases; (2) trial-and-error strategy may produce dangerous configurations that lead to database failure; (3) lack the ability to handle dynamic workload. To address the above challenges, a fast and stable RL-based database tuning system, FASTune, is proposed. A virtual environment is proposed to evaluate configurations which is an equivalent yet more efficient scheme than the cloned database. To ensure stability during tuning, FASTune adopts an environment proxy to avoid dangerous configurations. In addition, a Multi-State Soft Actor–Critic (MS-SAC) model is proposed to handle dynamic workloads, which utilizes the soft actor–critic network to tune the database according to workload and database states. The experimental results indicate that, compared with the state-of-the-art methods, FASTune can achieve improvements in performance while maintaining stability in the tuning.

This research aims to further promote the popularization and application of agricultural Internet of Things (IoT) and solve the practical problems of low transmission efficiency in agricultural data transmission. The targeted study is conducted on the video coding in agricultural IoT system to further promote the development of agricultural IoT system and improve the application effect of video surveillance technology. Firstly, an intelligent decision management platform is designed based on agricultural IoT system, consisting of the remote control module and video surveillance module. Meanwhile, a three-dimensional High Efficiency Video Coding (3D-HEVC) fast algorithm based on Bayesian Decision Theory is designed aiming to optimize the 3D-HEVC technology. Experiments are also performed to investigate the performance of intelligent management platform and 3D-HEVC fast algorithm based on Bayesian Decision Theory. The experimental results show that the intelligent management platform has good application effects in the database performance test, website access concurrency test, and data reception performance test. Compared to the traditional 3D-HEVC algorithm, the proposed 3D-HEVC fast algorithm reduces the coding time of depth map and total coding time by 46.5% and 22.52%, respectively, in addition to a better application effect. Meanwhile, with the support of the 3d-HEVC fast algorithm, the agricultural intelligent management platform shows better video transmission performance and application effect meeting the actual needs at this stage. The experiment results have certain practical value, providing scientific and effective reference data for subsequent research.

Achieving optimal performance of a database can be crucial for many businesses, and tuning its configuration parameters is a necessary step in this process. Many existing tuning methods involve complex machine learning algorithms and require large amounts of historical data from the system being tuned. However, training machine learning models can be problematic if a considerable amount of computational resources and data storage is required. This paper investigates the possibility of using less complex search algorithms or evolutionary algorithms to tune database configuration parameters, and presents a framework that employs Hill Climbing and Particle Swarm Optimization. The performance of the algorithms are tested on a PostgreSQL database using read-only workloads. Particle Swarm Optimization displayed the largest improvement in query response time, improving it by 26.09% compared to using the configuration parameters' default values. Given the improvement shown by Particle Swarm Optimization, evolutionary algorithms may be promising in the field of database tuning.

With the development of big data technology, the data management of complex applications has become more and more resource intensive. In this paper, we propose an automatic approach (DRLISA) to achieve NoSQL database index selection. For different workloads, we automatically select its corresponding indexes and parameters which can totally improve the database performance. Our DRLISA establishes an optimal index by building a deep reinforcement learning model which is able to adapt the dynamic change of workloads. We conducted our experiments in five aspects (the impact of data manipulation, the impact of operation count, comparison with random selection, comparison with existing method and the robustness of DRLISA) using the open source benchmark, YCSB. The experimental results showed that DRLISA has a high efficient index recommendation under the dynamic workloads.

The aim of this thesis was to compare the performance of two relational database management systems PostgreSQL and MySQL. For the purpose of the study a relational database was designed and a piece of software was implemented to connect desktop application with the database system. This software shall also create entities and relations between them in desired empty schemes on servers for databases. There has also been implemented a desktop application in Java programming language, that allows browsing data stored in database and performing the tests of database performance. Tests addressed basic operations of adding, collecting, updating and deleting data. Thisthesis makes a hypothesis "PostgreSQL is more efficient for desktop application while loaded with small data, in that case 1000 of queries", that was confirmed by achieved results from tests performed.

A large body of knowledge on database tuning is available in the form of natural language text. We propose to leverage natural language processing (NLP) to make that knowledge accessible to automated tuning tools. We describe multiple avenues to exploit NLP for database tuning, and outline associated challenges and opportunities. As a proof of concept, we describe a simple prototype system that exploits recent NLP advances to mine tuning hints from Web documents. We show that mined tuning hints improve performance of MySQL and Postgres on TPC-H, compared to the default configuration.

Database tuning is crucial step to enhance the performance of the application software. There are many tools available in Microsoft to evaluate the performance of the stored procedures and identify them. This paper presents a comparative performance of actual and tuned sample stored procedures in SQL Server 2008 R2 of Application software (Microsoft). The results showed there is a marginal gain in the efficiency after the database tuning.

Managing large chain of Hotels and ERP database comprises of core areas such as HRMS & PIP.HRMS (Human Resource Management System), which further includes areas such as Soft Joining, Promotion, Transfer, Confirmation, Leave Attendance and Exit, etc. PIP (Payroll Information Portal), wherein employees can view their individual Salary details, submit investment declaration, Reimbursement claim & CTC structuring, etc. Management of Large Chain of Hotels and ERP Database in AWS Cloud involves continuous monitoring with regards to the areas such as Performance of resource usages and optimization techniques relating to the use of PL/SQL. High Availability (HA) of data is accomplished through the Backup and Recovery mechanism and security of the data by Encryption & Decryption mechanism.

Processing data in an institution or organization has become an activity or routine for each meeting. In processing data, of course there are various kinds of problems, ranging from missing data, either because of an error in entering data or the result of a system failure. To overcome these problems, in an agency or company must have data backup, so as to create problems that occur in the data, the institution still has data backup. To call, another method of database replication is used. From the test results obtained data on the value of RTA (Round Trip Average) that is the time required for one time sending and receiving data from the replication database is 5426.22 ms and for compilation service diagrams a query attack occurs with a percentage of 29% in the database applications that produce 1774.45 ms results and for service diagrams at a query attack percentage of 29% a single database estimates the server down .

One of the most important applications of the intelligent operation and maintenance of a cloud database is its trend prediction of key performance indicators (KPI), such as disk use, memory use, etc. We propose a method named AutoPA4DB (Auto Prophet and ARIMA for Database) to predict the trend of the KPIs of the cloud database based on the Prophet model and the ARIMA model. Our AutoPA4DB method includes data preprocessing, model building, parameter tuning and optimization. We employ the weighted MAPE coverage to measure its accuracy and use 6 industrial datasets including 10 KPIs to compare the AutoPA4DB method with other three time-series trend prediction algorithms. The experimental results show that our AutoPA4DB method performs best in predicting monotonic variation data, e.g.disk use trend prediction. But it is unstable in predicting oscillatory variation data; for example, it is acceptable in memory use trend prediction but has poor accuracy in predicting the number of database connection trends.

Oracle, MSSQL, MySQL and PostgreSQL are four of the most popular relational databases. They are often used in internet applications. This paper aims to compare the efficiency of these technologies in terms of speed using virtualization with Docker. No publications that include virtualization were found among previous papers. After review of the literature, it was hypothesized that the Oracle engine would be the fastest. During the research, a series of experiments was carried out using the application, in which tests for measuring the time of instruction execution were implemented. Each query was measured 100 times and the first measurement was rejected. The obtained results confirmed the hypothesis about the superiority of the Oracle database. As in previous studies, it proved to be the fastest, also using virtualization.

One of the common problems at the intersection of geographical information science and transportation science is the estimation of origin-destination (OD) matrices. The emergence of sensor technologies offers unprecedented opportunities in this regard since massive amounts of traffic data can be collected in an easy way. Researchers and practitioners need to choose a suitable DataBase Management System (DBMS) among alternatives, such that storing and analysing traffic data to estimate the OD matrix is feasible. The aim of this paper is to compare the performance of two such notable DBMSs, PostgreSQL and MongoDB, in the context of OD matrix estimation. The experiments are carried out on New York City’s openly available taxi data on two different polygon sets: taxi zones and census blocks. These polygon layers consist of 263 and 38794 features respectively. The results suggest that Postgres outperforms MongoDB by generating the OD matrix instantly. The run time of MongoDB varies depending on the analysed time interval and follows a trip demand curve. As there are more trips involved in the generation of the OD matrix, so does the execution time increases in MongoDB. On the other hand, the query results are the same. Finally, the origin points of the taxi trips are visualised in QGIS using the ‘TimeManager’ plugin, and results are presented through a web-interface.

Configuring databases for efficient querying is a complex task, often carried out by a database administrator. Solving the problem of building indexes that truly optimize database access requires a substantial amount of database and domain knowledge, the lack of which often results in wasted space and memory for irrelevant indexes, possibly jeopardizing database performance for querying and certainly degrading performance for updating. In this paper, we develop the SmartIX architecture to solve the problem of automatically indexing a database by using reinforcement learning to optimize queries by indexing data throughout the lifetime of a database. We train and evaluate SmartIX performance using TPC-H, a standard, and scalable database benchmark. Our empirical evaluation shows that SmartIX converges to indexing configurations with superior performance compared to standard baselines we define and other reinforcement learning methods used in related work.

Dissertação de mestrado integrado em Engenharia Informática, Query optimizers are considered one of the most relevant and sophisticated components in a database management system. However, despite currently producing nearly optimal results, optimizers rely on statistical estimates and heuristics to reduce the search space of alternative execution plans for a single query. As a result, for more complex queries, errors may grow exponentially, often translating into sub-optimal plans resulting in less than ideal performance. Recent advances in machine learning techniques have opened new opportunities for many of the existing problems related to system optimization. This document proposes a solution built on top of PostgreSQL that learns to select the most efficient set of optimizer strategy settings for a particular query. Instead of depending entirely on the optimizer’s estimates to compare different plans under different configurations, it relies on a greedy selection algorithm that supports several types of predictive modeling techniques, from more traditional modeling techniques to a deep learning approach. The system is evaluated experimentally with the standard TPC-H and Join Order ing Benchmark workloads to measure the cost and benefits of adding machine learning capabilities to traditional query optimizers., Os otimizadores de queries são considerados um dos componentes de maior relevância e complexidade num sistema de gestão de bases de dados. No entanto, apesar de atualmente produzirem resultados quase ótimos, os otimizadores dependem do uso de estimativas estatísticas e de heurísticas para reduzir o espaço de procura de planos de execução alternativos para uma determinada query. Como resultado, para queries mais complexas, os erros podem crescer exponencialmente, o que geralmente se traduz em planos sub-ótimos, resultando num desempenho inferior ao ideal. Os recentes avanços nas técnicas de aprendizagem automática abriram novas oportunidades para muitos dos problemas existentes relacionados com otimização de sistemas. Este documento propõe uma solução construída sobre o PostgreSQL que aprende a selecionar o conjunto mais eficiente de configurações do otimizador para uma determinada query. Em vez de depender inteiramente de estimativas do otimizador para comparar planos de configurações diferentes, a solução baseia-se num algoritmo de seleção greedy que suporta vários tipos de técnicas de modelagem preditiva, desde técnicas mais tradicionais a uma abordagem de deep learning. O sistema é avaliado experimentalmente com os workloads TPC-H e Join Ordering Benchmark para medir o custo e os benefícios de adicionar aprendizagem automática a otimizadores de queries tradicionais., This work is financed by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia, within project UIDB/50014/2020.

National Institute of Aeronautics and Space of Indonesia is obliged to organize remote sensing data storage and distribution through National Remote Sensing Data Bank (BDPJN). BDPJN is a node of the remote sensing data network in a spatial national data network system. BDPJN has the primary function of collecting, storing, distributing metadata and remote sensing data for Indonesia territory. Evaluation of data operation management is needed to guarantee the continuity of the remote sensing data service at BDPJN. It has to ensure that: the catalog can be accessed by users, the catalog is continuously updated with data stored in the BDPJN storage system, and the data is available when needed. This research is purposed to evaluate data operation management on BDPJN. It is implemented by measuring the maturity level of data operations management activities using the CMMI and DMBOK frameworks. The results indicated that BDPJN has ten activities that have met an expected level of maturity. However, five of fifteen activities have below maturity level 3. Three activities at maturity level 1 are Monitor and Tune Database Performance, Plan for Data Retention, Archive, Retain, and Purge Data. The two others are Define the Data Technology Architecture and Evaluate Data Technology, which is at maturity level 2. Recommendations have been arranged based on references from DMBOK for these five activities.

This paper presents the Outer-Tuning framework, which aims to support the (semi) automatic tuning of relational database systems through a domain-specific ontology. Ontologies have shown themselves to be increasingly promising, adding semantics and standardizing the different terms used in a domain. Thereby, our framework seeks to explain and make explicit the tuning heuristics reasoning while enabling the evaluation of new ontology-inferred methods. In this paper we focus on the main aspects of the Outer-Tuning component-based architecture. We also give an overview of our tool in practice. Finally, we show two useful extensions, concerning new DBMSs and a way of dockerizing into a container.

When writing today's distributed programs, which frequently span both devices and cloud services, programmers are faced with complex decisions and coding tasks around coping with failure, especially when these distributed components are stateful. If their application can be cast as pure data processing, they benefit from the past 40--50 years of work from the database community, which has shown how declarative database systems can completely isolate the developer from the possibility of failure in a performant manner. Unfortunately, while there have been some attempts at bringing similar functionality into the more general distributed programming space, a compelling general-purpose system must handle non-determinism, be performant, support a variety of machine types with varying resiliency goals, and be language agnostic, allowing distributed components written in different languages to communicate. This paper introduces Ambrosia, the first system to satisfy all these requirements. We coin the term "virtual resiliency", analogous to virtual memory, for the platform feature which allows failure oblivious code to run in a failure resilient manner. We also introduce novel programming language constructs for resiliently handling non-determinism. Of further interest is the effective reapplication of much database performance optimization technology to make Ambrosia more performant than many of today's non-resilient cloud solutions.

The impetus for this study is the limited amount of information on performance characteristics of food science-encompassing online bibliographic databases. Database usage is an important element in modern research because a comprehensive understanding of current knowledge is essential for effective, unbiased hypothesis formulation and testing. Six databases commonly recommended by academic libraries for information retrieval in the food sciences (Academic Search Premier [ASP], Agricultural Online Access [AGRICOLA], CAB Direct, Food Science and Technology Abstracts [FSTA], PubMed, and Web of Science [WoS]) were compared in a case study based on the research topic "in vitro bile acid binding properties of dietary lignin." A complex string of search terms was used for citation retrieval, and predetermined inclusion/exclusion criteria were used to determine the relevancy of retrieved articles. Searching WoS and CAB Direct returned the greatest number of relevant articles, followed closely by FSTA, PubMed, and AGRICOLA. FSTA and AGRICOLA returned the highest ratios of relevant-to-irrelevant articles, followed closely by CAB Direct. None of the databases, when used alone, recovered all of the relevant articles identified in the study; WoS indexed the highest percentage of relevant articles identified (WoS = 10; total = 19). WoS also had the highest number of relevant articles that were unique to any one database. The thoroughness of searching the complete group of databases was tested by comparison of retrieved citations with those found in relevant review articles, revealing the need for testing overall inclusivity. PRACTICAL APPLICATION: Comprehensive online information retrieval is the most efficient means of accessing current knowledge. Awareness of current knowledge is essential for effective, unbiased decision making in private sector and academic/government-based research endeavors. Thus, online database usage is an essential element of modern food science research. This paper provides tangible examples of the performance characteristics of online bibliographic databases commonly recommended for information retrieval in the food sciences. The paper is written so as to aid the reader in making decisions with respect to database usage for the recovery of topic-relevant peer-reviewed articles germane to their area of research.

Database knob tuning is important to achieve high performance (e.g., high throughput and low latency). However, knob tuning is an NP-hard problem and existing methods have several limitations. First, DBAs cannot tune a lot of database instances on different environments (e.g., different database vendors). Second, traditional machine-learning methods either cannot find good configurations or rely on a lot of high-quality training examples which are rather hard to obtain. Third, they only support coarse-grained tuning (e.g., workload-level tuning) but cannot provide fine-grained tuning (e.g., query-level tuning). To address these problems, we propose a query-aware database tuning system QTune with a deep reinforcement learning (DRL) model, which can efficiently and effectively tune the database configurations. QTune first featurizes the SQL queries by considering rich features of the SQL queries. Then QTune feeds the query features into the DRL model to choose suitable configurations. We propose a Double-State Deep Deterministic Policy Gradient (DS-DDPG) model to enable query-aware database configuration tuning, which utilizes the actor-critic networks to tune the database configurations based on both the query vector and database states. QTune provides three database tuning granularities: query-level, workload-level, and cluster-level tuning. We deployed our techniques onto three real database systems, and experimental results show that QTune achieves high performance and outperforms the state-of-the-art tuning methods.

This paper presents an experimental study evaluating the performance of NoSQL database management systems. The study compares two NoSQL database management systems (Cassandra and MongoDB) and considers the following parameters/factors: workload and degree of parallelism. Two different workloads (update heavy and mostly read) were used, and different numbers of threads. The measured results are related to average latency: update latency and read latency. Our study shows that with the only exception of 1000 operations, both latency indicators have a quasi-parabolic behavior, where the minimum (i.e. the best performance) depends mainly on the number of threads and slightly varies with the increase in the number of operations. In the case of 1000 operations, there is also a maximum point (i.e. worst performance) case, after which the latency decreases.

OBJECTIVE The main purpose of our study was to subjectively assess the quality of a paediatric intensive care unit (PICU) database according to the Directory of Clinical Databases (DoCDat) criteria. DESIGN AND SETTING A survey was conducted between April 1 and June 15, 2018, among the Sainte Justine PICU research group. POPULATION Every member of this group whose research activity required the use of the database and/or who was involved in the development/validation of the database. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS All 10 research team members (one Information Technology specialist, one junior medical student, and eight clinician researchers) who used the high-resolution database fulfilled the survey (100% response rate). The median quality level of the Sainte Justine PICU database across all the 10 criteria was 3 (2-4), rated on a 1 (worst) to 4 (best) numeric scale. When compared with previously assessed databases through the DoCDat criteria, we found that the Sainte Justine PICU database performance was similar. CONCLUSIONS The PICU high-resolution database appeared of good quality when subjectively assessed by the DoCDat criteria. Further validation procedures are mandatory. We suggest that data quality assessment and validation procedures should be reported when creating a new database.

Smart city is a connection of physical and social infrastructure together with the information technology to leverage the collective intelligence of the city. Smart cities depend on a great extent on wireless sensor network to manage and maintain their services. Advanced sensor technologies are used to acquire information and help dealing with issues like air pollution, waste management, traffic optimization, and energy efficiency. However, no matter how much smart city may focus on sensor technology, data that are produced from sensors do not organize themselves in a database. Such tasks require a sophisticated database structure to produce informative data output. Besides that, wireless sensor network requires a proper design to improve the energy efficiency. The design will aid to prolong the lifespan of wireless network efficiently. In this study, we proposed a new technique that will be used to organize the information of wireless sensor network in the spatial database. Specific algorithm which is 3D geo-clustering algorithm is used to tackle several issues of location of the sensor in three-dimensional urban area in smart city. The algorithm is designed to minimizing the overlap among group clusters. Overlap plays an important role for energy efficiency. Thus, detection of sensors in two or more group clusters will avoid it from transmitting the same signal to cluster head node. It is prove that this algorithm would only create 5% to 10% overlap among group clusters. Several experiments are performed in this study to evaluate the algorithm. Based on the simulation results indicate that this algorithm can balance nodes energy consumption and prolong the network’s life span. It also has good stability and extensibility. Several tests are performed to validate the efficiency of the technique to measure the database performance.

External sorting algorithm is an important operation in computer science and technology. External sorting algorithm plays a key role in big data processing and database performance. With more and more hardware resources integrated into SSDs, it makes them full of parallelism. This paper proposed a new external sorting algorithm called FSSort trying to exploiting the internal parallelism to accelerate the sorting process. FSSort can fully parallel the data loading and make the data writing in batchs in the run merge phase. Moreover, it makes the pipeline easier. Therefore, FSSort can make efficient use of the internal parallelism of SSDs. Experiments on unsorted data and partially sorted data on three different type of SSDs are conducted. Experimental results show that for unsorted data, the FSSort algorithm can achieve a speedup of 1.39~1.50 comparing to MONTRES, the previously proposed state art of sorting algorithm.

Introduction. We have seen significant advances in the state of the art in natural language processing (NLP) over the past few years [20]. These advances have been driven by new neural network architectures, in particular the Transformer model [19], as well as the successful application of transfer learning approaches to NLP [13]. Typically, training for specific NLP tasks starts from large language models that have been pre-trained on generic tasks (e.g., predicting obfuscated words in text [5]) for which large amounts of training data are available. Using such models as a starting point reduces task-specific training cost as well as the number of required training samples by orders of magnitude [7]. These advances motivate new use cases for NLP methods in the context of databases.

Optimal configuration is vital for a DataBase Management System (DBMS) to achieve high performance. There is no one-size-fits-all configuration that works for different workloads since each workload has varying patterns with different resource requirements. There is a relationship between configuration, workload, and system performance. If a configuration cannot adapt to the dynamic changes of a workload, there could be a significant degradation in the overall performance of DBMS unless a sophisticated administrator is continuously re-configuring the DBMS. In this tutorial, we focus on autonomous workload-aware performance tuning, which is expected to automatically and continuously tune the configuration as the workload changes. We survey three research directions, including 1) workload classification, 2) workload forecasting, and 3) workload-based tuning. While the first two topics address the issue of obtaining accurate workload information, the third one tackles the problem of how to properly use the workload information to optimize performance. We also identify research challenges and open problems, and give real-world examples about leveraging workload information for database tuning in commercial products (e.g., Amazon Redshift). We will demonstrate workload-aware performance tuning in Amazon Redshift in the presentation.

This paper aims to initiate a discussion around benchmarking data management systems with machine-learned components. Traditional benchmarks such as TPC or YCSB are insufficient to analyze and understand these learned systems because they evaluate the performance under a stable workload and data distribution. Learned systems automatically specialize and adapt database components to a changing workload, database, and execution environment, thereby making conventional metrics such as average throughput ill-suited to understand their performance fully. Moreover, the standard cost-per-performance metrics fail to account for essential trade-offs related to the training cost of models and the elimination of manual database tuning. We present several ideas for designing new benchmarks that are better suited to evaluate learned systems. The main challenges entail developing new metrics to capture the particularities of learned systems and ensuring that benchmark results remain comparable across many deployments with wide-ranging designs.

Write-ahead logging (WAL) has been widely used to provide transactional atomicity in databases, such as SQLite and MySQL/InnoDB. However, the WAL introduces duplicate writes, where changes are recorded in the WAL file and then written to the database file, called checkpointing writes. On the other hand, NAND flash-based SSDs, which have an inherent indirection software layer, called flash translation layer (FTL), become commonplace in modern storage systems. Innovative SSD designs have been proposed to eliminate the WAL overheads by exploiting the FTL, such as providing an atomic write interface or utilizing its address remapping. However, these designs introduce significant performance overheads of maintaining and persisting extra transactional information to guarantee the transactional atomicity or mapping consistency. In this paper, we propose single-write WAL (SW-WAL), a novel cross-layer design, to eliminate WAL-induced duplicate writes on SSDs with minimal overheads. The SSD exposes an address remapping interface to the host, through which the checkpointing writes can be completed without conducting real data writes. To ensure the transactional atomicity and mapping consistency, we make the SSD aware of the transactional writes to the WAL file. Specifically, when transactional data are written to the WAL file, both transactional and mapping semantics are delivered from the host to the SSD and persisted in relevant flash pages as housekeeping metadata without any extra overheads. We implement a prototype of SW-WAL, which runs a popular database SQLite on an emulated NVMe SSD. Experimental results show that SW-WAL improves the database performance by up to 62% compared with original SQLite that bears the WAL overheads and up to 32% compared with the state-of-the-art design that eliminates the WAL overheads.

Gone is the time when a Solid-State Drive (SSD) was just a fast drop-in replacement for a Hard-Disk Drive (HDD). Thanks to the NVMe ecosystem, nowadays, SSDs are accessed through specific interfaces and modern I/O frameworks. SSDs have also grown versatile with time and can now support various use cases ranging from cold, high-density storage to hot, low-latency ones. The body of knowledge about building such different devices is mostly available, but it is less than accessible to non-experts. Finding which device variation can better support a given workload also requires deep domain knowledge. This tutorial's first goal is to make these tasks--understanding the design of SSDs and pairing them with the data-intensive workloads they support well--more inviting. The tutorial goes further, however, in that it suggests that a new kind of SSD plays an essential role in post-Moore computer systems. These devices can be co-designed to align their capabilities to an application's requirements. A salient feature of these devices is that they can run application logic besides just storing data. They can thus gracefully scale processing capabilities with the volume of data stored. The tutorial's second goal is thus to establish the design space for co-designed SSDs and show the tools available to hardware, systems, and databases researchers that wish to explore this space.

The wide popularity and the maturity of cloud platform promote the development of Cloud Native database systems. On-demand resource configuration is an attractive feature of cloud platforms, but its complexity in resource management challenges the benchmarking of database performance, which is no longer in a stand-alone test environment. Sharing or contending of resources aggravates the dynamics of environment, which can influence database performance much. In order to expose the real performance in production environment, environment simulation is prerequisite for benchmarking databases. Although Docker Containers have been promoted to isolate resources, we still cannot achieve the true resource isolation. In this paper, we first define four kinds of workload generators corresponding to the key environmental dimensions, then builds a multi-order polynomial linear regression model to calculate the correlation among workloads, and simulates the dynamical changes of environment. It is the first work to provide a complete and dynamic simulation to environment. We conduct comprehensive experiments on the open-source DBMS by running the standard benchmarks to verify the effectiveness of our work.

Writing good SQL is really about database performance. Although tuning of the database can help with performance, bad SQL will impact performance in ways that tuning the database just cannot cure. When the database administrator(s) has done their job, it is time for the training of the users to begin. All relational databases have their quirks, and it is paramount for the users to be aware of these and have the knowledge to either bypass them or create SQL that takes advantage of the database design and features.

The traditional big data control system is limited by hardware and software resources, which leads to low efficiency of data access and calculation. Therefore, a big data control system for electrical automation is designed. The system is mainly designed from two aspects of hardware and software. The hardware mainly designs a digital acquisition circuit, selects the model of each part in the overall working module of the acquisition board, and designs a pseudo dual-port RAM to store data, which is convenient for the processing of massive data and Control; in software design, according to the needs of the system, light MySQL and distributed HBase are used to jointly design the ER diagram and database table of the database user database to make the database performance more optimized. In the control algorithm of the system, the butterfly operation is used to optimized calculate the speed, obtain the operation error through the operation and correct it to ensure the efficiency of the system operation. The experimental results show that the performance of the designed system is better than the traditional system in data insertion speed, access speed and calculation speed, which fully verifies the application value of the system.

The preceding chapters covered a lot of optimization techniques: not only different ways to optimize SQL statements but also how database design affects performance, the importance of working together with application developers, the use of functions, and many other aspects of database performance. This chapter organizes these techniques into a set of guidelines providing a systematic way to develop efficient declarative queries and/or tune/optimize them, as well as choose the right database design.

Nowadays, the widely used NoSQL databases play a fundamental role for big data storage, processing and analysis. However, NoSQL databases usually consume a large amount of computing resources and always run in a low performance state, which seriously impacts the end-to-end quality of service. Therefore, tuning the performance of NoSQL databases via benchmarking tools becomes a critical issue. Unfortunately, it is non-trivial to solve this problem because there exists a considerable number of performance-related configuration parameters from both the databases (application-specific) and the operating system kernel (kernel-specific), and manually tuning methods only consider a small subset of the whole candidate configuration parameters space, leading to a sub-optimal performance. To address this challenge, we design ConfAdvisor, an automatic configuration tuning framework for NoSQL database benchmarking. Specifically, ConfAdvisor treats the database performance as a black-box function of configuration parameters and leverage an online learning method to search the best configurations. Experimental results based on several popular NoSQL databases show that: 1) Through only tuning the kernel-specific parameters can improve database performance by up to 30%, which is often overlooked in previous studies. 2) ConfAdvisor is able to achieve well-tuned configurations with a very few trials, even in a high-dimension configuration space. And compared with default configurations, the well-tuned parameters can improve database performance by up to 88%.

Databases are all around us, and ensuring their efficiency is of great importance. Database optimisation has many parts and many methods, two of these parts are database tuning and database optimisation. These can then further be split into methods such as indexing. These indexing techniques have been studied and compared between Database Management Systems (DBMSs) to see how much they can improve the execution time for queries. And many guides have been written on how to implement query optimisation and indexes. In this thesis, the question "How does indexing and query optimisation affect response time in PostgreSQL?" is posed, and was answered by investigating these previous studies and theory to find different optimisation techniques and compare them to each other. The purpose of this research was to provide more information about how optimisation techniques can be implemented and map out when what method should be used. This was partly done to provide learning material for students, but also people who are starting to learn PostgreSQL. This was done through a literature study, and an experiment performed on a database with different table sizes to see how the optimisation scales to larger systems. What was found was that there are many use cases to optimisation that mainly depend on the query performed and the type of data. From both the literature study and the experiment, the main take-away points are that indexes can vastly improve performance, but if used incorrectly can also slow it. The main use cases for indexes are for short queries and also for queries using spatio-temporal data - although spatio-temporal data should be researched more. Using the DBMS optimiser did not show any difference in execution time for queries, while correctly implemented query tuning techniques also vastly improved execution time. The main use cases for query tuning are for long queries and nested queries. Although, most systems benefit from some sort of query tuning, as it does not have to cost much in terms of memory or CPU cycles, in comparison to how indexes add additional overhead and need some memory. Implementing proper optimisation techniques could improve both costs, and help with environmental sustainability by more effectively utilising resources. Databaser finns överallt omkring oss, och att ha effektiva databaser är mycket viktigt. Databasoptimering har många olika delar, varav två av dem är databasjustering och SQL optimering. Dessa två delar kan även delas upp i flera metoder, så som indexering. Indexeringsmetoder har studerats tidigare, och även jämförts mellan DBMS (Database Management System), för att se hur mycket ett index kan förbättra prestanda. Det har även skrivits många böcker om hur man kan implementera index och SQL optimering. I denna kandidatuppsats ställs frågan "Hur påverkar indexering och SQL optimering prestanda i PostgreSQL?". Detta besvaras genom att undersöka tidigare experiment och böcker, för att hitta olika optimeringstekniker och jämföra dem med varandra. Syftet med detta arbete var att implementera och kartlägga var och när dessa metoder kan användas, för att hjälpa studenter och folk som vill lära sig om PostgreSQL. Detta gjordes genom att utföra en litteraturstudie och ett experiment på en databas med olika tabell storlekar, för att kunna se hur dessa metoder skalas till större system. Resultatet visar att det finns många olika användingsområden för optimering, som beror på SQL-frågor och datatypen i databasen. Från både litteraturstudien och experimentet visade resultatet att indexering kan förbättra prestanda till olika grader, i vissa fall väldigt mycket. Men om de implementeras fel kan prestandan bli värre. De huvudsakliga användingsområdena för indexering är för korta SQL-frågor och för databaser som använder tid- och rum-data - dock bör tid- och rum-data undersökas mer. Att använda databassystemets optimerare visade ingen förbättring eller försämring, medan en korrekt omskrivning av en SQL fråga kunde förbättra prestandan mycket. The huvudsakliga användingsområdet för omskriving av SQL-frågor är för långa SQL-frågor och för nestlade SQL-frågor. Dock så kan många system ha nytta av att skriva om SQL-frågor för prestanda, eftersom att det kan kosta väldigt lite när det kommer till minne och CPU. Till skillnad från indexering som behöver mer minne och skapar så-kallad överhead". Att implementera optimeringstekniker kan förbättra både driftkostnad och hjälpa med hållbarhetsutveckling, genom att mer effektivt använda resuser.

Current relational database systems are forming the main part of the intelligent information and support system. Data inside are secured, protected by the relational integrity covered by the transactions. Access and management are done by the relational algebra. Significant aspects expressing the quality and performance are the data access techniques themselves. The index is one of the key features ensuring data retrieval performance, however, from the other operations point of view, it brings additional costs. If the suitable index for the query is not present, the whole table must be scanned sequentially resulting in poor performance. If the data fragmentation is present, the problem is even deeper. This paper aims to propose its own techniques based on relevant block identification objects stored in private or shared memory areas. It uses rebalancing methods on the extent granularity. If the tables are to be joined and both indexes are not present, usually, the Hash join method is used. The technique of two-dimensional index mapper is discussed in the paper, as well.

The root cause diagnosis of performance anomaly for online database anomalies is challenging due to diverse types of database engines, different operational modes, and variable anomaly patterns. To relieve database operators from manual anomaly diagnosis and alarm storm, we propose FluxInfer, a framework to accurately and rapidly localize root cause related KPIs for database performance anomaly. It first constructs a Weighted Undirected Dependency Graph (WUDG) to represent the dependency relationships of anomalous KPIs accurately, and then applies a weighted PageRank algorithm to localize root cause related KPIs. The testbed evaluation experiments show that the AC@3, AC@5, and Avg@5 of FluxInfer are 0.90, 0.95, and 0.77, outperforming nine baselines by 64%, 60%, and 53% on average, respectively.

Due to the continuous expansion of telecom operators (TOs) services and subscriber’s base, a number of objects in TO product catalogue (PC) traditionally developed employing some relational database (RD) rapidly increase. This can result in degraded PC database performance in terms of reduced object visibility or slow query response time, which mandates a search for new types of databases that can cope with such challenges. A possible solution to these challenges is the transformation of RD into some graph database (GD), with visualisation and organisation capabilities that can cope with the increasing complexity of PC RDs. Hence, in this paper, a novel implementation concerning the transformation of a set of PC data and functions of the TO RD into ArangoDB GD is proposed. The proposed approach allows the upgrading of PC RD into a higher presentation layer through the implementation of the GD, which enables advanced data analysis and visualization. The developed solution shows that GD with appropriately structured queries, data and links between them, improves the visibility and simplifies the entire search process. Obtained results show that GD outperforms RD in terms of search query response time for different functions executed on an equal amount of database objects.

Digitalization is currently the key factor for progress, with a rising need for storing, collecting, and processing large amounts of data. In this context, NoSQL databases have become a popular storage solution, each specialized on a specific type of data. Next to that, the multi-model approach is designed to combine benefits from different types of databases, supporting several models for data. Despite its versatility, a multi-model database might not always be the best option, due to the risk of worse performance comparing to the single-model variants. It is hence crucial for software engineers to have access to benchmarks comparing the performance of multi-model and single-model variants. Moreover, in the current Big Data era, it is important to have cluster infrastructure considered within the benchmarks. In this paper, we aim to examine how the multi-model approach performs compared to its single-model variants. To this end, we compare the OrientDB multi-model database with the Neo4j graph database and the MongoDB document store. We do so in the cluster setup, to enhance state of the art in database benchmarks, which is not yet giving much insight into cluster-operating database performance.

IoT devices have become necessary today. As technology grows, the need for reliable, secure, and efficient applications has become essential. To have an active IoT service, the performance of the backend databases is critical. Database plays a crucial role in storing and organizing data that supports the IoT applications. The current research will examine the performance of databases for IoT applications.

Due to the amount of computational data generated each year and most often without standardization, the need arose for the development of specific Big Data tools. Currently cloud computing has been chosen as it is widely used in conjunction with Big Data due to its reduced cost and elasticity. Utilization of cloud services has increased over the years and many companies have migrated their computing services to the cloud to reduce the operational costs of maintaining the technology infrastructure. Due to the number of cloud providers available, benchmarking is now considered important for your choice, especially the performance of the specific application in the cloud environment. This paper aims to evaluate MongoDB database performance using the Yahoo Cloud Serving Benchmark tool, with 3 cloud application profiles from Google and Microsoft.

Cloud vendors provide ready-to-use distributed DBMS solutions as a service. While the provisioning of a DBMS is usually fully automated, customers typically still have to make important design decisions which were traditionally made by the database administrator such as finding an optimal partitioning scheme for a given database schema and workload. In this paper, we introduce a new learned partitioning advisor based on Deep Reinforcement Learning (DRL) for OLAP-style workloads. The main idea is that a DRL agent learns the cost tradeoffs of different partitioning schemes and can thus automate the partitioning decision. In the evaluation, we show that our advisor is able to find non-trivial partitionings for a wide range of workloads and outperforms more classical approaches for automated partitioning design.

This study aims to compare Firebase Realtime Database and MySQL performance as Database Management System for our Toddlers Daily Nutritional Needs Mobile Application. The comparison is in terms of Database response time. Their performance is compared using a common CRUD operations. Each operations data is analyzed using Wilcoxon Signed-Rank test. Our comparison results confirm that Firebase Realtime Database is more suitable as our Daily Nutritional Needs Mobile Application Database Management System since it could perform better response time. The key features match our needs to provide Realtime Data and it is also help simplify the process of developing applications.

Index selection plays a substantial role in database performance by reducing the I/O cost. Existing index advisors apply different heuristic methods to search the large search space of possible attributes for indexing. These heuristic approaches do not have a mechanism to learn about the goodness of the recommended index set. Thus, they might choose the same index set with a low impact on I/O cost reduction. Learning from their decisions can improve the quality of the recommended index set. We believe that Deep Reinforcement Learning (DRL) is a solution to tackle this issue. Using DRL, an index advisor can improve its decision using the feedbacks of its decisions. In this paper, we propose a DRL-index advisor for a cluster database. We describe the major components such as agent, environment, set of actions, the reward function, and other modules. We conclude the paper with open challenges and possible future work.

Database access logs are the starting point for many forms of database administration, from database performance tuning, to security auditing, to benchmark design, and many more. Unfortunately, query logs are also large and unwieldy, and it can be difficult for an analyst to extract broad patterns from the set of queries found therein. Clustering is a natural first step towards understanding the massive query logs. However, many clustering methods rely on the notion of pairwise similarity, which is challenging to compute for SQL queries, especially when the underlying data and database schema is unavailable. We investigate the problem of computing similarity between queries, relying only on the query structure. We conduct a rigorous evaluation of three query similarity heuristics proposed in the literature applied to query clustering on multiple query log datasets, representing different types of query workloads. To improve the accuracy of the three heuristics, we propose a generic feature engineering strategy, using classical query rewrites to standardize query structure. The proposed strategy results in a significant improvement in the performance of all three similarity heuristics.

A new methodology is proposed to compare database performance for streams of patient respiratory data from patients in an intensive care unit. New metrics are proposed through which databases may be compared both for this and similar streaming applications in the domain of the Internet of Things. Studies are reported using simulated patient data for four freely available databases. The statistical technique of non-parametric bootstrapping is used to minimise the total running time of the tests. We report mean values and bias corrected and accelerated confidence intervals for each metric and use these to compare the databases. We find that, among the four databases tested, ScaleDB is an optimum database technology when handling between 200 and 800 patients in this application, while PostgreSQL performs best outside of this range. Comparing the non-parametric bootstrapping method to a complete set of tests shows that the two approaches give results differing by a few percent.

Database performance throughput is one of the main measurements of the application effectiveness or a system, in terms of speed and delivery. Higher application efficiency is always a goal for any organization; therefore, finding the way to improve the database performance has priority. There are two main factors that can contribute to performance gain or slowness. Firstly, how well designed the database structure is in terms of data types, referential integrity, and indexes. Secondly, the underlying hardware infrastructure that consists of a server, storage, and network. During the application design lifecycle, the database schema is one area that continuously evolves. There are numerous configurational parameters that need to be adjusted according to database schema design and underlying infrastructure, in which in-memory is an important feature. In-memory database stores all data in physical memory, because of keeping entire data and changes of data in physical memory, the in-memory database provides very short response time and transactional throughput as compared to the disk-based database. This study provides comparison between in-memory and disk-based performance with impact analysis of concurrent users and parallelism using TPC-C benchmarking. A comprehensive benchmark guideline benefits for specific database environment was provided in order to track performance changes while migrating to in-memory from disk-based database. These benchmarking statistics provided comparative standpoint that can be verified during performance bottleneck. This study shows that transitioning from disk-based database to in-memory database decreased the response time and reduced the lock contention, however, requires detailed review for index design changes. Key words: Microsoft Structured Query Language (SQL) Server, in-memory, disk-based, transaction processing performance council (TPC-C), benchmarking, database performance, performance comparison, response time, parallelism, concurrent users.

This paper describes an experimental study regarding NoSQL database performance. Two NoSQL databases were considered (MongoDB and Cassandra), two different workloads (update heavy and mostly read) and several degrees of parallelism. The results refer to throughput performance metric. Statistical analysis performed is referring to throughput results. Conclusions confirm that MongoDB performs better than Cassandra in the context of a mostly read workload, while Cassandra outperformed MongoDB in the context of an update heavy workload where the number of operations was high.

Many organizations today store their critical business information permanently in XML format. XML data can be managed using: XML-Enabled Database (XED) systems which convert and store XML files in traditional database systems; Native XML Database (NXD) systems which store XML data natively using three main storage technologies – text-based, model-based, and schema-based techniques; and Hybrid Database systems which are comprised of both XML-Enabled and Native XML database systems. NXDs are faster than other database technologies because there is no need to convert the format of the data prior to storage. No performance evaluation has been carried out to compare all three storage strategies, hence, this paper reports on the first attempt to evaluate all three storage strategies by using open source products to measure the response time taken for each of the database basic tasks such as database creation, dataset insertion, and data manipulation. The results of the evaluation show that the schema-based storage strategy: performs 3.5 times faster than the other two storage techniques in data insertion; shows very good performance in query processing on small and large datasets; performs 10.33 times faster than text-based, and 7.5 times faster than model-based storage techniques in query processing of large datasets.