Search

In the multidimensional databases, various storage methods for the main memory have been proposed for efficient data processing. On the other hand, storage devices such as SSD still leverage the conventional row-oriented storage method to avoid complicated operating system modification. Since there is a difference in storage methods between main memory and storage device, it is necessary to reorganize the data between them. However, data reorganization in the conventional CPU-based systems causes a long latency due to a large number of memory accesses. In addition, it also causes high dynamic power consumption of a CPU. In this paper, we propose an In-storage Data Reorganization Accelerator for multidimensional databases (IDRA). We place the IDRA in an SSD to reorganize data on the fly while loading it into the main memory without intervention of the CPU. In our evaluation on the off-the-shelf system (not simulation), the IDRA-based system improves the performance by 78.6% and reduces the system-wide energy by 30.3%, on average, compared to the conventional CPU-based system.

Materialized view selection (MVS) is the problem of selecting an appropriate set of views to be materialized to speed up analytical query processing of data warehouses. Online analytical processing (OLAP) of queries is an essential application of the MVS problem, in which, the response times of the queries are reduced by storing the selected views. Views are intermediate results of query processing and are selected in the MVS problem to be stored and will then be exploited in answering process of several queries. Views are usually organized as a view representation structure in the MVS problem. Multiple Views Processing Plan (MVPP) is a standard structure used for view representation in the MVS problem. Due to the tremendous amount of data, constructing the MVPP is a challenge in the big data applications. The MR-MVPP (Map-Reduce-based construction of the MVPP) is the proposed method of this paper to address this problem. The MR-MVPP performs a set similarity join (similarity-based join) on the base relations and views using the map-reduce model and the hashing technique. The MVPP construction time in the proposed method is reduced by avoiding redundant calculations in the process of creating the MVPP. The performance of the proposed method is empirically evaluated. According to the results of the experiments, the execution time of the MR-MVPP method is better than the other methods. The average time improvement is about 26.5 units. This improvement is better than the other similar researches in this area and is significant due to the high volume of data in real applications. Moreover, the proposed method works well in terms of the effectiveness of the created MVPP and has about a 50% coverage rate for view selection methods. Deterministic methods are more accurate than hashing methods and can be utilized for set similarity join as future work to probably improve the effectiveness of the constructed MVPP.

Online analytical processing (OLAP) engines display aggregated data to help business analysts compare data, observe trends, and make decisions. Issues of data quality and, in particular, issues with missing data impact the quality of the information. Key decision-makers who rely on these data typically make decisions based on what they assume to be all the available data. The authors investigate three approaches to dealing with missing data: 1) ignore missing data, 2) show missing data explicitly (e.g., as unknown data values), and 3) design mitigation algorithms for missing data (e.g., allocate missing data into known value categories). The authors evaluate the approach with focus groups and controlled experiments. When one tries to inform decision-makers using the approaches in the research, the authors find that they often alter their decisions and adjust their decision confidence: individual differences of tolerance for ambiguity and pre-existing omission bias in the decision context influence their decisions.

Data from multiple sources are loaded into the organization data warehouse for analysis. Since some OLAP queries are quite frequently fired on the warehouse data, their execution time is reduced by storing the queries and results in a relational database, referred as materialized query database (MQDB). If the tables, fields, functions, and criteria of input query and stored query are the same but the query criteria specified in WHERE or HAVING clause do not match, then they are considered non-synonymous to each other. In the present research, the results of non-synonymous queries are generated by reusing the existing stored results after applying UNION or MINUS operations on them. This will reduce the execution time of non-synonymous queries. For superset criteria values of input query, UNION operation is applied, and for subset values, MINUS operation is applied. Incremental result processing of existing stored results, if required, is performed using Data Marts.

A knowledge graph (KG) represents real-world entities and their relationships. The represented knowledge is often context-dependent, leading to the construction of contextualized KGs. The multidimensional and hierarchical nature of context invites comparison with the OLAP cube model from multidimensional data analysis. Traditional systems for online analytical processing (OLAP) employ multidimensional models to represent numeric values for further analysis using dedicated query operations. In this paper, along with an adaptation of the OLAP cube model for KGs, we introduce an adaptation of the traditional OLAP query operations for the purposes of performing analysis over KGs. In particular, we decompose the roll-up operation from traditional OLAP into a merge and an abstraction operation. The merge operation corresponds to the selection of knowledge from different contexts whereas abstraction replaces entities with more general entities. The result of such a query is a more abstract, high-level view – a management summary – of the knowledge.

Libraries are service units with high storage complexity as evidenced by more data being stored for each year. The data that is not integrated makes the complex problem because every year the process that is carried out continues to increase, especially for the circulation of loans. As the number of books increases, the circulation of borrowing increases every year. On the other hand, the library must know exactly what collection of books they have and the transactions it has made. A lot of data is owned by the library cannot be utilized optimally, so that the managerial is unable to make full use of the data. In University scale libraries, this problem increases when the data is not fully integrated. In this study, the implementation of a star schema was carried out to solve problems related to data integration using a nine-step methodology, which includes selection, item selection, process dimensions, fact selection, fact storage, ensuring dimension tables, selecting database duration, changing dimensions, determining priorities, and query models. The results of this study indicate that the star schema can be implemented in the case of libraries, data warehouses and OLAP to support decision making for adding books, and produced 3 dimensions of the 4 grains found.

Medicine is a fresh way to utilize for curing, analyzing and detecting the diseases through data clustering with OLAP (Online Analytical Processing). The large amount of multidimensional clinical data is reduced the efficiency of OLAP query processing by enhancing the query accessing time. Hence, the performance of OLAP model is improved by using data clustering in which huge data is divided into several groups (clusters) with cluster heads to achieve fast query processing in least time. In this paper, a Dragon Fly Optimization based Clustering (DFOC) approach is proposed to enhance the efficiency of data clustering by generating optimal clusters from multidimensional clinical data for OLAP. The results are evaluated on MATLAB 2019a tool and shown the better performance of DFOC against other clustering methods ACO, GA and K-Means in terms of intra-cluster distance, purity index, F-measure, and standard deviation

Food Station is a city-owned enterprise owned by the Jakarta Provincial Government whose tasks include monitoring and publication of supply data and rice price through Cipinang Rice Market so that it is useful as a guide for traders and policy makers in relevant government institutions. This study aims to design a dashboard to visualize the price and supply of rice through Cipinang Rice Market. The research stages consist of identification of information requirement, data warehouse design and On-line Analytical Processing (OLAP) database, transfer of operational data from excel file and web scrapping into OLAP database through Extract, Transform, Load (ETL) process and create dashboard using PowerBI application. The result of this research is a data warehouse that can facilitate the analysis with some fact table that is price, quantity of rice enter, quantity of rice out and stock and some dimension table that is place dimension (city of origin / destination), dimension time, dimension of rice type and dimension market and dashboard of various price and supply charts that can be selected based on available dimensions. Keywords: dashboard, data warehouse, rice price, rice supply, visualization

Data processing can be roughly divided into two categories, online transaction processing OLTP(on-line transaction processing) and online analytical processing OLAP(on-line analytical processing). OLTP is the main application of traditional relational databases, and it is some basic daily transaction processing, such as bank pipeline transactions and so on. OLAP is the main application of the data warehouse system, it supports some more complex data analysis operations, focuses on decision support, and provides popular and intuitive analysis results. As the amount of data processed by enterprises continues to increase, distributed databases have gradually replaced stand-alone databases and become the mainstream of applications. However, the current business supported by distributed databases is mainly based on OLTP applications, lacking OLAP implementation. This paper proposes an implementation method of HTAP for distributed database CBase, which provides an implementation method of OLAP analysis for CBase, and can easily deal with data analysis of large amounts of data.

NoSQL solutions are started to be increasingly used in modern days’ Data Warehouses (DW). However, business analysts face challenges when performing On Line Analytical Processing (OLAP) queries on these NoSQL systems. The lack of uniform representation of various OLAP operations over different types of NoSQL based DWs is one of them. In addition, deficiency of precise semantics in OLAP operations create obstacles to effective query interpretation over distinct types DWs. This paper is aiming to deal with aforementioned challenges. Formal and rigorous specification are represented in this paper for different kinds of OLAP operators and operations. These precise specifications are capable to analyse business queries. Further, the proposed formal specifications are implemented in a document-oriented database using a suitable case study. In addition, the proposed approach aids efficient visualization techniques of data cubes over NoSQL based DWs.

This paper presents an architecture for the data warehouse of outpatient healthcare (DWOP) as a data repository collects data from two different sources (Databases of outpatient healthcare and Excel files from hospitals) and provides storage, functionality and responsiveness to queries to meet decision makers requirements.Successfully supporting managerial decision-making is critically dependent upon the availability of integrated, high quality information organized and presented in a timely and easily understood manner. “On-Line Analytical Processing (OLAP) is utilized for decision support to get interesting information” from the data warehouse with a rapid execution time. OLAP is considered one of Business Intelligence tools.

As to the problems of low data mining efficiency, less dimensionality, and low accuracy of traditional multidimensional association rules in the university big data environment, an OLAP-based multi-dimensional association rule mining method is proposed, which combines hash function and marked transaction compression technology to solve the problem of excessive or redundant candidate sets in the Apriori algorithm, and uses On Line Analytical Processing to manage the intermediate data in the association mining process , in order to reduce the time overhead caused by repeated calculations. To verify the validity of the proposed method, a learning situation analysis system is constructed in the field of colleges and universities. The multi-dimensional association rules mining method is used to analyze more than 21,000 desensitized real data, in order to mine the key factors affecting students' academic performance. The experimental results show that the proposed multi-dimensional mining model has good mining results and significantly improves the time performance.

International audience; A decision support system is used by decision makers for a long time. But, in some cases, the originally designed multidimensional schema does not cover the entire needs of decision makers, which can change over time. One such unfulfilled needs, is using facts to describe dimension members. In this article, we propose a methodology to transform the constellation schema of a data warehouse by integrating factual data into a dimension. The proposed methodology and algorithms enrich a constellation multidimensional schema with new analytical possibilities for decision makers. This enrichment has repercussions for the entire multidimensional schema that are managed by multidimensional modeling, hierarchy calculation and the hierarchy version. In this article, we present a theoretical view of the proposed methodology supported by a case study, an implemented prototype and a complete evaluation based on a standard benchmark.

Presents an interview conducted with Thomas Richter, the founder of Swarm64. Discusses the Postgres open source database project. Host Adam Conrad spoke with Richter about an introduction to relational databases, comparisons of online analytical processing and online transaction-processing structures, performance considerations, and advanced features of the Postgres architecture. We provide summary excerpts below. To hear the full interview, visit http://www.se-radio.net or access our archives via Really Simple Syndication at http://feeds.feedburner.com/se-radio.

This research has focused on a radon measurement campaign that was carried out in two dwellings in a residential building located in Madrid. A new methodology has been used in this field, such as the use of cubes based on On-Line Analytical Processing in SQL Server Analysis Services. The application of this methodology can be of particular interest in analysing thousands of radon measurements and complementary variables, which are easily obtained in any radon measurement campaign.

Online analytical processing (OLAP) approach is widely used in business intelligence to cater the multidimensional queries for decades. In this era of cutting-edge technology and the internet, data generation rates have been rising exponentially. Internet of things sensors and social media platforms are some of the major contributors, leading toward the absolute data boom. Storage and speed are the crucial parameters and undoubtedly the burning issues in efficient data handling. The key idea here is to address these two challenges of big data computing in OLAP. In this article, the authors have proposed and implemented OLAP on Hadoop by Indexing (OOHI). OOHI offers a simplified multidimensional model that stores dimensions in the schema server and measures on the Hadoop cluster. Overall setup is divided into various modules, namely: data storage module (DSM), dimension encoding module (DEM), cube segmentation module, segment selection module (SSM), and block selection and process (BSAP) module. Serialization and deserialization concept applied by DSM for storage and retrieval of the data for efficient space utilization. Integer encoding adopted by DEM in dimension hierarchy is selected to escape sparsity problem in multidimensional big data. To reduce search space by chunks of the cube from the queried chunks, SSM plays an important role. Map reduce-based indexing approach and series of seek operations of BSAP module were integrated to achieve parallelism and fault tolerance. Real-time oceanography data and supermarket data sets are applied to demonstrate that OOHI model is data independent. Various test cases are designed to cover the scope of each dimension and volume of data set. Comparative results and performance analytics portray that OOHI outperforms in data storage, dice, slice, and roll-up operations compared with Hadoop based OLAP.

This special issue was open for submissions in the field of big data in business. Accordingly, this special issue includes five contributions to the fields of business process innovation in the big data era, unstructured big data analytical methods in firms, online analytical processing approach for business intelligence in big data, geospatial insights for retail recommendation using similarity measures, and big data and operational changes through interactive data visualization. A bibliometric approach is used to visualize and highlight the exciting literature on big data followed by highlighting the contribution of this special issue.

Penelitian dilakukan untuk Aplikasi hanya membahas tentang analisis data simpanan dan pinjaman anggota pada KSP RIAS Musi Rawas. Selain itu merancang dan mengimplementasikan aplikasi manajemen data OLAP untuk analisis data anggota untuk memudahkan pihak manajemen dalam menyediakan berbagai jenis informasi yang dapat mendukung dalam pengambilan keputusan strategi. Dan juga menganalisa dan mencoba membuat program manajemen data OLAP dengan menggunakan bahasa pemrograman PHP dan MYSQ. Pada penelitian ini bagian dari rancang bangun dibuat sesuai dengan pengunaan sistem secara online untuk melakukan transaksi simpan pinjam yang sesuai dengan kode atau urutan pada kartu anggota koperasi. Metode Clustering akan mempermudah nasabah dalam melihat saldo simpanan yang terdapat pada aplikasi OLAP sehingga dapat menaikkan simpanannya.

Biodiversity is one of emerging issue over decades; many have performed research to map and to document the data over the world. This issue is very important due to the event of extinction have been accelerating happening because of human extinction. Bengkulu, as one of the province lied in one of 19 hotspots in the world, Sundanese, has experienced the degradation of flora and fauna over the case of forest degradation and habitat loss. Although many application and software has been developed to solve the case, the existences of data standardization still become an issue over this problem. In this research, study of data integration had been developed to make the process of biodiversity data acquisition can be more effective and efficient. The system proposed the integration based on OLAP and OLTP that will be connected to IUCN, as one of the biggest center for monitoring the loss of biodiversity all around the world. This application had been built with web based using UML design and followed SDLC to provide the best fit of the need. This research had also succeeded to build the automated integration to show the record of dynamic number over biodiversity existences in Bengkulu. The application had been tested using black box and has the perfect performance (100%) over the testing that can help the monitoring process over biodiversity data.

In recent years, we have witnessed significant efforts to improve the performance of Online Analytical Processing (OLAP) on graphics processing units (GPUs). Most existing studies have focused on improving memory efficiency since memory stalls can play an essential role in query processing performance on GPUs. Motivated by the recent rise of just-in-time (JIT) compilation in query processing, we investigate whether and how we can further improve query processing performance on GPU. Specifically, we study the execution of state-of-the-art JIT compile-based query processing systems. We find that thanks to advanced techniques such as database compression and JIT compilation, memory stalls are no longer the most significant bottleneck. Instead, current JIT compile-based query processing encounters severe under-utilization of GPU hardware due to divergent execution and degraded parallelism arising from resource contention. To address these issues, we propose a JIT compile-based query engine named Pyper to improve GPU utilization during query execution. Specifically, Pyper has two new operators, Shuffle and Segment , for query plan transformation, which can be plugged into a physical query plan in order to reduce divergent execution and resolve resource contention, respectively. To determine the insertion points for these two operators, we present an analytical model that helps insert Shuffle and Segment operators into a query plan in a cost-based manner. Our experiments show that 1) the analytical analysis of divergent execution and resource contention helps to improve the accuracy of the cost model, 2) Pyper significantly outperforms other GPU query engines on TPC-H and SSB queries.

Both RDBMS and NoSQL database vendors have added varying degrees of support for storing and processing JSON data. Some vendors store JSON directly as text while others add new JSON type systems backed by binary encoding formats. The latter option is increasingly popular as it enables richer type systems and efficient query processing. In this paper, we present our new native JSON datatype and how it is fully integrated with the Oracle Database ecosystem to transform Oracle Database into a mature platform for serving both SQL and NoSQL style access paradigms. We show how our uniquely designed Oracle Binary JSON format (OSON) is able to speed up both OLAP and OLTP workloads over JSON documents.

In the field of Business Intelligence we require quick and efficient analysis of multidimensional data. Data warehouse (DW) and Online analytical processing (OLAP) approaches have been developed fo...

This article sets in context Data Warehouses (DWs) and Online Analytical Processing (OLAP) against the backdrop of databases and Big Data and shows how data warehouses and OLAP were incorporated in...

The effectiveness of a Business Intelligence System is hugely dependent on these three fundamental components, 1) Data Acquisition (ETL), 2) Data Storage (Data Warehouse), and 3) Data Analytics (OLAP). The predominant challenges with these fundamental components are Data Volume, Data Variety, Data Integration, Complex Analytics, Constant Business changes, Lack of skill sets, Compliance, Security, Data Quality, and Computing requirements. There is no comprehensive documentation that talks about guidelines for ETL, Data Warehouse and OLAP to include the recent trends such as Data Latency (to provide real-time data), BI flexibility (to accommodate changes with the explosion of data) and SelfService BI. This research paper attempts to fill this gap by analyzing existing scholarly articles in the last three to five years to compile guidelines for effective design, implementation, and assessment of DW, ETL, and OLAP in BI.

This research work results from the construction of an OLAP cube (Online Analytical Processing). The objective is to show how the construction of an OLAP cube is, using the tools of Microsoft Analysis Services and Microsoft Excel, from a database obtained from a survey conducted by INEGI in 2016, which It is taken as an example in this investigation; the model developed was the waterfall model, where the five stages of this model were implemented, which are the stage of communication, planning, modeling, construction and deployment; One of the main conclusions that were obtained with the accomplishment of this investigation was that the tools of Microsoft Analysis Services and Microsoft Excel, help in the realization of the processing and analysis of data of a database by means of dimensions and measures. Keywords: OLAP Cubes, Construction, Analysis Services, Excel, Database.

OLTP applications are usually executed by a high number of clients in parallel and are typically faced with high throughput demand as well as a constraint latency requirement for individual statements. Interestingly, OLTP workloads are often read-heavy and comprise similar query patterns, which provides a potential to share work of statements belonging to different transactions. Consequently, OLAP techniques for sharing work have started to be applied also to OLTP workloads, lately. In this paper, we present an approach for merging read statements within interactively submitted multi-statement transactions consisting of reads and writes. We first define a formal framework for merging transactions running under a given isolation level and provide insights into a prototypical implementation of merging within a commercial database system. In our experimental evaluation, we show that, depending on the isolation level, the load in the system and the read-share of the workload, an improvement of the transaction throughput by up to a factor of 2.5X is possible without compromising the transactional semantics.

TPC-H continues to be the most widely used benchmark for relational OLAP systems. It poses a number of challenges, also known as "choke points", which database systems have to solve in order to achieve good benchmark results. Examples include joins across multiple tables, correlated subqueries, and correlations within the TPC-H data set. Knowing the impact of such optimizations helps in developing optimizers as well as in interpreting TPC-H results across database systems. This paper provides a systematic analysis of choke points and their optimizations. It complements previous work on TPC-H choke points by providing a quantitative discussion of their relevance. It focuses on eleven choke points where the optimizations are beneficial independently of the database system. Of these, the flattening of subqueries and the placement of predicates have the biggest impact. Three queries (Q2, Q17, and Q21) are strongly influenced by the choice of an efficient query plan; three others (Q1, Q13, and Q18) are less influenced by plan optimizations and more dependent on an efficient execution engine.

Commercial OLAP systems are economically unavailable for organizations with limited financial capabilities. Analytical processing of large amounts of data in these organizations can be accomplished using open-source software systems on a cost-effective cluster platform. Previously created Clusterix-like DBMS using a regular query processing plan is not efficient enough. Therefore, research on such systems was developed with a focus on a full load of processor cores and using the GPU acceleration (systems Clusterix-N, N—from new) up to the development of a system comparable in efficiency to the open-source system Spark, which is currently considered the most promising. The development methodology was based on the constructive system modeling methodology.

Heterogeneous HybridTransactionalAnalytical Processing ($$\mathrm{H}^{2}$$H2TAP) database systems have been developed to match the requirements for low latency analysis of real-time operational data. Due to technical challenges, these systems are hard to architect, non-trivial to engineer, and complex to administrate. Current research has proposed excellent solutions to many of those challenges in isolation – a unified engine enabling to optimize performance by combining these solutions is still missing. In this concept paper, we suggest a highly flexible and adaptive data structure (called gridtable) to physically organize sparse but structured records in the context of $$\mathrm{H}^{2}$$H2TAP. For this, we focus on the design of an efficient highly-flexible storage layout that is built from scratch for mixed query workloads. The key challenges we address are: (1) partial storage in different memory locations, and (2) the ability to optimize for mixed OLTP-/OLAP access patterns. To guarantee safe and well-specified data definition or manipulation, as well as fast querying with no compromises on performance, we propose two dedicated access paths to the storage.In this paper, we explore the architecture and internals of gridtables showing design goals, concepts and trade-offs. We close this paper with open research questions and challenges that must be addressed in order to take advantage of the flexibility of our solution.

This article proposes an algorithm for solving the modeling problem with numerous variables in the MATLAB programming language. According to the developed algorithm, the MATLAB model calculates the values of solar insolation and ambient temperature of a particular location depending on five variables simultaneously: (1) time of day (τ), (2) day of the year (n), (3) tilt angle (β), (4) azimuth angle (γ) and (5) albedo (ρ). Meanwhile, the calculation results form a multidimensional database of solar insolation and ambient temperature of a given location for any values of the above variables and their combinations.An example of using the created database is given for calculating the annual total photovoltaic panel productivity and productivity dynamics on days of the year for latitude 41°2995′, which corresponds to a location in the city of Tashkent. The developed algorithm can be used to analyze/forecast the efficiency of solar plants using parameters from the database and other variables, such as temperature coefficient, dust content, nominal efficiency, wind speed, humidity, etc., as input data. The availability of data makes it possible to use data for both one-way and two-way reception of radiation as for bifacial photovoltaic panels. The database created based on the proposed program model can serve as a basis for comparison with other calculations or measurement databases and for studying or determining other parameters that affect possible differences in the compared data and creating advanced modeling algorithms for the solar irradiance.

Highlights Digital data collection and management system is developed for the USDA-AMS’s shell-egg grading program. Database system consisting of OLTP, data warehouse and OLAP databases enables online data entry and trend reporting. Data and information management is done through web application servers. Users access the databases via web browsers. Abstract . This paper is concerned with development of web-based online data entry and reporting system, capable of centralized data storage and analytics of egg grading records produced by USDA egg graders. The USDA egg grading records are currently managed in paper form. While there is useful information for data-driven knowledge discovery and decision making, the paper-based egg grading record system has fundamental limitations in effective and timely management of such information. Thus, there has been a demand to electronically and digitally store and manage the egg grading records in a database for data analytics and mining, such that the quality trends of eggs observed at various levels (e.g., nation or state) are readily available to decision makers. In this study, we report the design and implementation of a web-based online data entry and reporting information system (called USDA Egg Grading Information Management System, EGIMS), based on a data warehouse framework. The developed information system consisted of web applications for data entry and reporting, and internal databases for data storage, aggregation, and query processing. The internal databases consisted of online transaction processing (OLTP) database for data entry and retrieval, data warehouse (DW) for centralized data storage and online analytical processing (OLAP) database for multidimensional analytical queries. Thus, the key design goal of the system was to build a system platform that could provide the web-based data entry and reporting capabilities while rapidly updating the OLTP, DW and OLAP databases. The developed system was evaluated by a simulation study with statistically-modeled egg grading records of one hypothetical year. The study found that the EGIMS could handle approximately up to 600 concurrent users, 32 data entries per second and 164 report requests per second, on average. The study demonstrated the feasibility of an enterprise-level data warehouse system for the USDA and a potential to provide data analytics and data mining capabilities such that the queries about historical and current trends can be reported. Once fully implemented and tested in the field, the EGIMS is expected to provide a solution to modernize the egg grading practice of the USDA and produce the useful information for timely decisions and new knowledge discovery. Keywords: Data warehouse, Database, OLTP, OLAP, Egg grading, Information management, Web application, Information system, Data.

This paper presents CirroData, a high-performance SQL-on-Hadoop system designed for Big Data analytics workloads. As a home-grown enterprise-level online analytical processing (OLAP) system with more than seven-year research and development (R&D) experiences, we share our design details to the community about how to achieve high performance in CirroData. Multiple optimization techniques have been discussed in the paper. The effectiveness and the efficiency of all these techniques have been proved by our customers’ daily usage. Benchmark-level studies, as well as several real application case studies of CirroData, have been presented in this paper. Our evaluations show that CirroData can outperform various types of counterpart database systems in the community, such as “Spark+Hive”, “Spark+HBase”, Impala, DB-X/Y, Greenplum, HAWQ, and others. CirroData can achieve up to 4.99x speedup compared with Greenplum, HAWQ, and Spark in the standard TPC-H queries. Application-level evaluations demonstrate that CirroData outperforms “Spark+Hive” and “Spark+HBase” by up to 8.4x and 38.8x, respectively. In the meantime, CirroData achieves the performance speedups for some application workloads by up to 20x, 100x, 182.5x, 92.6x, and 55.5x as compared with Greenplum, DB-X, Impala, DB-Y, and HAWQ, respectively.

Today, the relational database is not suitable for data management due to the large variety and volume of data which are mostly untrusted. Therefore, NoSQL has attracted the attention of companies. Despite it being a proper choice for managing a variety of large volume data, there is a big challenge and difficulty in performing online analytical processing (OLAP) on NoSQL since it is schema-less. This article aims to introduce a model to overcome null value in converting document-oriented NoSQL databases into relational databases using parallel similarity techniques. The proposed model includes four phases, shingling, chunck, minhashing, and locality-sensitive hashing MapReduce (LSHMR). Each phase performs a proper process on input NoSQL databases. The main idea of LSHMR is based on the nature of both locality-sensitive hashing (LSH) and MapReduce (MR). In this article, the LSH similarity search technique is used on the MR framework to extract OLAP cubes. LSH is used to decrease the number of comparisons. Furthermore, MR enables efficient distributed and parallel computing. The proposed model is an efficient and suitable approach for extracting OLAP cubes from an NoSQL database.

In this paper, a neutrosophic fuzzy data warehouse modelling approach is presented to support the neutrosophic analysis of the publishing house for books which allows integration of neutrosophic concept in dimensions and facts without affecting the core of a crisp data warehouse. Also we describe a method is presented which includes guidelines that can be used to convert a crisp data warehouse into a neutrosophic fuzzy domain. Finally we have presented an OLAP system that implements a neutrosophic multidimensional model to represent imprecision using neutrosophic concept in hierarchies and facts and achieving knowledge discovery from imperfect data. The use of neutrosophic structures and the definition of the OLAP operations (roll-up, drill-down, slice, and dice) enable the model to manage the imprecision of data and hide the complexity of the model and provide the user with a more understandable result. &nbsp

In the Big Data warehouse context, a column-oriented NoSQL database system is considered as the storage model which is highly adapted to data warehouses and online analysis. Indeed, the use of NoSQL models allows data scalability easily and the columnar store is suitable for storing and managing massive data, especially for decisional queries. However, the column-oriented NoSQL DBMS do not offer online analysis operators (OLAP). To build OLAP cubes corresponding to the analysis contexts, the most common way is to integrate other software such as HIVE or Kylin which has a CUBE operator to build data cubes. By using that, the cube is built according to the row-oriented approach and does not allow to fully obtain the benefits of a column-oriented approach. In this article, the focus is to define a cube operator called MC-CUBE (MapReduce Columnar CUBE), which allows building columnar NoSQL cubes according to the columnar approach by taking into account the non-relational and distributed aspects when data warehouses are stored.

Multi-model DBMSs (MMDBMSs) have been recently introduced to store and seamlessly query heterogeneous data (structured, semi-structured, graph-based, etc.) in their native form, aimed at effectively preserving their variety. Unfortunately, when it comes to analyzing these data, traditional data warehouses (DWs) and OLAP systems fall short because they rely on relational DBMSs for storage and querying, thus constraining data variety into the rigidity of a structured, fixed schema. In this paper, we investigate the performances of an MMDBMS when used to store multidimensional data for OLAP analyses. A multi-model DW would store each of its elements according to its native model; among the benefits we envision for this solution, that of bridging the architectural gap between data lakes and DWs, that of reducing the cost for ETL, and that of ensuring better flexibility, extensibility, and evolvability thanks to the combined use of structured and schemaless data. To support our investigation we define a multidimensional schema for the UniBench benchmark dataset and an ad-hoc OLAP workload for it. Then we propose and compare three logical solutions implemented on the PostgreSQL multi-model DBMS: one that extends a star schema with JSON, XML, graph-based, and key–value data; one based on a classical (fully relational) star schema; and one where all data are kept in their native form (no relational data are introduced). As expected, the full-relational implementation generally performs better than the multi-model one, but this is balanced by the benefits of MMDBMSs in dealing with variety. Finally, we give our perspective view of the research on this topic.

This business development in the globalized era provoked fierce competition especially with the application of it to support his daily process especially in data management, therefore, the BAW Tour & Travel needs a system for data processing so that data can be stored and integrated with each other or what's called data warehouse and followed by other regulatory ETL, OLAP methods in analysis and quick decision-making. Using the data warehouse can figure out a spike in visitors' visits so it can predict people to be able to predict the number of guides during a month in case there's a spike that reduces anyone's denial of acceptance on the grounds of lack of a guide and to booked a motel because it's already full.

Maritime shipping is a complex process because it requires a continuous analysis of huge amounts of information. Much of this information has variable spatial and temporal properties. In addition, the analysis must be effective to help in making decisions in a timely manner. Generally the analysis is applied with Spatial On-Line Analytical Processing (SOLAP) basing on spatial data warehouses SDW. Data warehousing by current technologies with relational database management systems (RDBMS) cannot be applied effectively to manage the increasing complexity and volume of geo-referenced data. In order to increase the performance and availability of services, NoSQL databases technologies have appeared. This paper presents the modeling and the realization of a spatial decision support system (SDSS) for maritime navigation namely “Maritima”, basing on the integration of the ECDIS and OLAP systems. In fact, the spatio-multidimensional analysis of the stored data can benefit from the expressive power of the electronic navigation chart offered by the ECDIS system. In order to benefit from the NoSQL model advantages, we try to apply transformations for the conceptual model to create document NoSQL DW for maritime data. In addition to that, the multidimensional analysis at different dimensions and aggregation levels can generate beneficial indicators for the decision-aid and can be very useful in addition of other steering systems for secure maritime navigation. Finally, “Maritima” is tested with a set of spatial data related to two different harbors “Arzew” and “Oran” in western of Algeria.

The accessibility of devices that track the positions of moving objects has attracted many researchers in Mobility Online Analytical Processing (Mobility OLAP). Mobility OLAP makes use of trajectory data warehousing techniques, which typically include a path of moving objects at a particular point in time. The Semantic Web (SW) users have published a large number of moving object datasets that include spatial and non-spatial data. These data are available as open data and require advanced analysis to aid in decision making. However, current SW technologies support advanced analysis only for multidimensional data warehouses and Online Analytical Processing (OLAP) over static spatial and non-spatial SW data. The existing technology does not support the modeling of moving object facts, the creation of basic mobility analytical queries, or the definition of fundamental operators and functions for moving object types. This article introduces the QB4MobOLAP vocabulary, which enables the analysis of mobility data stored in RDF cubes. This article defines Mobility OLAP operators and SPARQL user-defined functions. As a result, QB4MobOLAP vocabulary and the Mobility OLAP operators are evaluated by applying them to a practical use case of transportation analysis involving 8826 triples consisting of approximately 7000 fact triples. Each triple contains nearly 1000 temporal data points (equivalent to 7 million records in conventional databases). The execution of six pertinent spatiotemporal analytics query samples results in a practical, simple model with expressive performance for the enabling of executive decisions on transportation analysis.

This study is devoted to the problem of the ability to represent multidimensional design of building territory based on Online Analytical Processing (OLAP) technology. Building territory is viewed as complex information system, which relate with other information systems of surrounding areas. Mathematical modeling of such information system for multidimensional design based on OLAP technology is proposed to solve this problem. It is proposed to use a Online Transaction Processing (OLTP) system for multidimensional analysis and design of the two subject areas. The first subject area contains background information about the topography, geodesy and geology of the building territory. The second subject area contains information about the surrounding areas to the building territory. OLAP cubes represent multidimensional information objects with associated properties. Each OLTP system is an implementation of a domain model and is designed by a multidimensional matrix, which is represented as an OLAP cube. The application of this concept in a Web-based environment is covered.

Many real systems produce network data or highly interconnected data, which can be called information networks. These information networks form a critical component in modern information infrastructure, constituting a large graph data volume. The analysis of information network data covers several technological areas, among them OLAP technologies. OLAP is a technology that enables multi-dimensional and multi-level analysis on a large volume of data, providing aggregated data visualizations with different perspectives. This article presents a literature review on the main applications of OLAP technology in the analysis of information network data. To achieve such goal, it shows a systematic review to list the works that apply OLAP technologies in graph data. It defines seven comparison criteria (Materialization, Network, Selection, Aggregation, Model, OLAP Operations, Analytics) to qualify the works found based on their functionalities. The works are analyzed according to each criterion and discussed to identify trends and challenges in the application of OLAP in the information network.

Статтю присвячено аналізу технології OLAP з точки зору багатовимірного аналізу даних реляційних баз даних. Описано основні об’єкти технології OLAP: гіперкуб, вимір, показник, таблиця вимірів і фактів; наведено опис моделей перетворення реляційної бази даних до багатовимірної моделі. Авторами запропоновано узагальнений алгоритм застосування технології OLAP для здійснення багатовимірного аналізу реляційних баз даних, який полягає у виконанні таких кроків, як: отримання вхідних даних, визначення показників і вимірів, формування багатовимірної моделі даних, формування структури запитів, обробка отриманих даних. Наведено результати проектування гіперкубу на основі статистичних даних соціально-економічного становища України, збережених у таблицях реляційної бази даних. Приклади застосування операції багатовимірного аналізу для розробленого гіперкубу представлено в графічній формі.

OLAP is a core functionality in database systems and the performance is crucial to enable on-time decisions. However, OLAP queries are rather time consuming, especially on large datasets, and traditional exact solutions usually cannot meet the high-performance requirement. Recently, approximate query processing (AQP) has been proposed to enable approximate OLAP. However, existing AQP methods have some limitations. First, they may involve unacceptable errors on skewed data (e.g., long-tail distribution). Second, they require to store large amount of data and have no significant performance improvement. Third, they only support a small subset of SQL aggregation queries. To overcome these limitations, we propose a bounded approximate query processing framework ${\mathtt {BAQ}}$ BAQ . Given a predefined error bound and a set of queries, ${\mathtt {BAQ}}$ BAQ judiciously selects high-quality samples from the data to generate a unified synopsis offline, and then uses the synopsis to answer online queries. Compared with existing methods, ${\mathtt {BAQ}}$ BAQ has the following salient features. (1) ${\mathtt {BAQ}}$ BAQ does not need to generate a synopsis for each query while it only generates a unified synopsis, and thus ${\mathtt {BAQ}}$ BAQ has much smaller synopsis. (2) ${\mathtt {BAQ}}$ BAQ achieves much smaller error than existing studies. Specifically, ${\mathtt {BAQ}}$ BAQ can provide deterministic approximate results (i.e., the estimated query results must be within the error bound with 100 percent confidence) for SQL aggregation queries that do not contain selection conditions on numerical columns. For queries with selection conditions on numerical columns, we propose effective grouping-based techniques and the estimated results are also within the error bound in practice. Experimental results on both real and synthetic datasets show that ${\mathtt {BAQ}}$ BAQ significantly outperforms state-of-the-art approaches. For example, on a Microsoft production dataset (a real dataset with synthetic queries), ${\mathtt {BAQ}}$ BAQ has 10-100× improvement on synopsis size and 10-100× improvement on the error compared with state-of-the-art algorithms.

Intra-query fault tolerance has increasingly been a concern for online analytical processing, as more and more enterprises migrate data analytical systems from mainframes to commodity computers. Most massive parallel processing (MPP) databases do not support intra-query fault tolerance. They may suffer from prolonged query latency when running on unreliable commodity clusters. While SQL-on-Hadoop systems can utilize the fault tolerance support of low-level frameworks, such as MapReduce and Spark, their cost-effectiveness is not always acceptable. In this paper, we propose a smart intra-query fault tolerance (SIFT) mechanism for MPP databases. SIFT achieves fault tolerance by performing checkpointing, i.e., materializing intermediate results of selected operators. Different from existing approaches, SIFT aims at promoting query success rate within a given time. To achieve its goal, it needs to: (1) minimize query rerunning time after encountering failures and (2) introduce as less checkpointing overhead as possible. To evaluate SIFT in real-world MPP database systems, we implemented it in Greenplum. The experimental results indicate that it can improve success rate of query processing effectively, especially when working with unreliable hardware.

The analysis of the semantic changes of Korean verbs is an important part of Korean culture research. It is helpful for Korean researchers to analyze the historical evolution of Korean language and to deduce the teaching, learning, and dissemination of Korean language by strengthening the analysis of the semantic changes of Korean verbs. On the difficulty of extracting and analyzing the semantic variation characteristics of Korean verbs, firstly, this paper is based on database technology, big data mining, big data analysis, online analytical processing technology, and verb feature analysis technology. Secondly, taking the semantic change characteristics of Korean verbs as the object of study. What’s more, the system frame design of Korean verb semantic change feature analysis system based on massive culture corpus data is established. Finally, it is shown that the designed feature analysis system has a good application effect through the analysis of the relevant use cases and user feedback information of the system in this year.

Big companies that have many branches in different locations often have difficulty with analyzing transaction processes from each branch. The problem experienced by the company management is the rapid delivery of massive data provided by the branch to the head office so that the analysis process of the company's performance becomes slow and inaccurate. The results of this process used as a consideration in decision making which produce the right information if the data is complete and relevant. The right method of massive data collection is using the data warehouse approach. Data warehouse is a relational database designed to optimize queries in Online Analytical Processing (OLAP) from the transaction process of various data sources that can record any changes in data that occur so that the data becomes more structured. In applying the data collection, data warehouse has extracted, transform, and load (ETL) steps to read data from the Online Transaction Processing (OLTP) system, change the form of data through uniform data structures, and save to the final location in the data warehouse. This study provides an overview of the solution for implementing ETL that can work automatically or manually according to needs using the Python programming language so that it can facilitate the ETL process and can adjust to the conditions of the database in the company system., Perusahaan besar yang memiliki banyak cabang dengan lokasi berbeda sering sekali mengalami kesulitan dalam menganalisis proses transaksi dari setiap cabang. Masalah yang dialami dari pihak manajemen perusahaan terletak pada kurang cepatnya penyampaian data yang massive diberikan oleh cabang ke kantor pusat sehingga proses analisa kinerja perusahaan menjadi lambat dan kurang akurat. Hasil proses analisa yang digunakan sebagai pendukung dalam pengambilan keputusan akan menghasilkan informasi yang tepat jika data yang tersedia lengkap dan relevan. Metode pengumpulan data massive yang baik dan handal digunakan adalah dengan menggunakan pendekatan teknologi data warehouse. Data warehouse merupakan basis data relasional yang didesain lebih kepada mengoptimalkan query dalam Online Analytical Processing (OLAP) dari proses transaksi berbagai macam sumber data dan dapat mencatat segala perubahan data yang terjadi sehingga menjadi lebih terstruktur. Dalam penerapan pengumpulan datanya, data warehouse mempunyai tahapan extract, transform, dan load (ETL) untuk dapat membaca data dari sistem Online Transaction Processing (OLTP), merubah bentuk data melalui penyamaan struktur data, dan menyimpan ke lokasi akhir di data warehouse. Penelitian ini memberikan gambaran solusi implementasi ETL yang bisa bekerja secara otomatis maupun manual sesuai dengan kebutuhan dengan menggunakan bahasa pemrograman Python sehingga bisa memudahkan dalam proses ETL dan bisa menyesuaikan dengan kondisi database di sistem perusahaan

With data explosion in scale and variety, OLAP databases play an increasingly important role in serving real-time analysis with low latency (e.g., hundreds of milliseconds), especially when incoming queries are complex and ad hoc in nature. Moreover, these systems are expected to provide high query concurrency and write throughput, and support queries over structured and complex data types (e.g., JSON, vector and texts). In this paper, we introduce AnalyticDB, a real-time OLAP database system developed at Alibaba. AnalyticDB maintains all-column indexes in an asynchronous manner with acceptable overhead, which provides low latency for complex ad-hoc queries. Its storage engine extends hybrid row-column layout for fast retrieval of both structured data and data of complex types. To handle large-scale data with high query concurrency and write throughput, AnalyticDB decouples read and write access paths. To further reduce query latency, novel storage-aware SQL optimizer and execution engine are developed to fully utilize the advantages of the underlying storage and indexes. AnalyticDB has been successfully deployed on Alibaba Cloud to serve numerous customers (both large and small). It is capable of holding 100 trillion rows of records, i.e., 10PB+ in size. At the same time, it is able to serve 10m+ writes and 100k+ queries per second, while completing complex queries within hundreds of milliseconds.

Cloud-native databases become increasingly important for the era of cloud computing, due to the needs for elasticity and on-demand usage by various applications. These challenges from cloud applications present new opportunities for cloud-native databases that cannot be fully addressed by traditional on-premise enterprise database systems. A cloud-native database leverages software-hardware co-design to explore accelerations offered by new hardware such as RDMA, NVM, kernel bypassing protocols such as DPDK. Meanwhile, new design architectures, such as shared storage, enable a cloud-native database to decouple computation from storage and provide excellent elasticity. For highly concurrent workloads that require horizontal scalability, a cloud-native database can leverage a shared-nothing layer to provide distributed query and transaction processing. Applications also require cloud-native databases to offer high availability through distributed consensus protocols. At Alibaba, we have explored a suite of technologies to design cloud-native database systems. Our storage engine, X-Engine and PolarFS, improves both write and read throughputs by using a LSM-tree design and self-adapted separation of hot and cold data records. Based on these efforts, we have designed and implemented POLARDB and its distributed version POLARDB-X, which has successfully supported the extreme transaction workloads during the 2018 Global Shopping Festival on November 11, 2018, and achieved commercial success on Alibaba Cloud. We have also designed an OLAP system called AnalyticDB (ADB in short) for enabling real-time interactive data analytics for big data. We have explored a self-driving database platform to achieve autoscaling and intelligent database management. We will report key technologies and lessons learned to highlight the technical challenges and opportunities for cloud-native database systems at Alibaba.