Search

Group decision making is traditionally a human-centered process, where communication, synchronization and agreement are driven by the stakeholders involved. In the area of multi-objective optimization (MOO), this becomes a challenge, because MOO usually produces a large amount of trade-off solutions that need to be analyzed and discussed by the stakeholders. Moreover, for transparent group decision making, it is important that each decision maker is able to trace the entire decision process – from associated data and models to problem formulation and solution generation, as well as to the preferences and analyses of other decision makers. A graph database is capable of capturing such diverse information in the form of a knowledge graph. It can be used to store and query all dependencies and hence can support complex decision-making tasks. Further advantages are the inherent suitability for visualization and the possibilities for pattern matching, graph analytics and, if semantically enriched, to infer new connections in the graph. In this paper, we show how such a knowledge graph can be used to support more transparent and traceable decision-making activities, particularly when multiple stakeholders with differing preferences or perspectives are involved., CC BY-NC 4.0 DEED© 2024 The AuthorsCorrespondence Address: L. Mittermeier; University of Skövde, School of Engineering Science, Sweden; email: ludwig.mittermeier@his.se

Group decision making is traditionally a human-centered process, where communication, synchronization and agreement are driven by the stakeholders involved. In the area of multi-objective optimization (MOO), this becomes a challenge, because MOO usually produces a large amount of trade-off solutions that need to be analyzed and discussed by the stakeholders. Moreover, for transparent group decision making, it is important that each decision maker is able to trace the entire decision process – from associated data and models to problem formulation and solution generation, as well as to the preferences and analyses of other decision makers. A graph database is capable of capturing such diverse information in the form of a knowledge graph. It can be used to store and query all dependencies and hence can support complex decision-making tasks. Further advantages are the inherent suitability for visualization and the possibilities for pattern matching, graph analytics and, if semantically enriched, to infer new connections in the graph. In this paper, we show how such a knowledge graph can be used to support more transparent and traceable decision-making activities, particularly when multiple stakeholders with differing preferences or perspectives are involved., CC BY-NC 4.0 DEED© 2024 The AuthorsCorrespondence Address: L. Mittermeier; University of Skövde, School of Engineering Science, Sweden; email: ludwig.mittermeier@his.se

Group decision making is traditionally a human-centered process, where communication, synchronization and agreement are driven by the stakeholders involved. In the area of multi-objective optimization (MOO), this becomes a challenge, because MOO usually produces a large amount of trade-off solutions that need to be analyzed and discussed by the stakeholders. Moreover, for transparent group decision making, it is important that each decision maker is able to trace the entire decision process – from associated data and models to problem formulation and solution generation, as well as to the preferences and analyses of other decision makers. A graph database is capable of capturing such diverse information in the form of a knowledge graph. It can be used to store and query all dependencies and hence can support complex decision-making tasks. Further advantages are the inherent suitability for visualization and the possibilities for pattern matching, graph analytics and, if semantically enriched, to infer new connections in the graph. In this paper, we show how such a knowledge graph can be used to support more transparent and traceable decision-making activities, particularly when multiple stakeholders with differing preferences or perspectives are involved., CC BY-NC 4.0 DEED© 2024 The AuthorsCorrespondence Address: L. Mittermeier; University of Skövde, School of Engineering Science, Sweden; email: ludwig.mittermeier@his.se

Group decision making is traditionally a human-centered process, where communication, synchronization and agreement are driven by the stakeholders involved. In the area of multi-objective optimization (MOO), this becomes a challenge, because MOO usually produces a large amount of trade-off solutions that need to be analyzed and discussed by the stakeholders. Moreover, for transparent group decision making, it is important that each decision maker is able to trace the entire decision process – from associated data and models to problem formulation and solution generation, as well as to the preferences and analyses of other decision makers. A graph database is capable of capturing such diverse information in the form of a knowledge graph. It can be used to store and query all dependencies and hence can support complex decision-making tasks. Further advantages are the inherent suitability for visualization and the possibilities for pattern matching, graph analytics and, if semantically enriched, to infer new connections in the graph. In this paper, we show how such a knowledge graph can be used to support more transparent and traceable decision-making activities, particularly when multiple stakeholders with differing preferences or perspectives are involved., CC BY-NC 4.0 DEED© 2024 The AuthorsCorrespondence Address: L. Mittermeier; University of Skövde, School of Engineering Science, Sweden; email: ludwig.mittermeier@his.se

Group decision making is traditionally a human-centered process, where communication, synchronization and agreement are driven by the stakeholders involved. In the area of multi-objective optimization (MOO), this becomes a challenge, because MOO usually produces a large amount of trade-off solutions that need to be analyzed and discussed by the stakeholders. Moreover, for transparent group decision making, it is important that each decision maker is able to trace the entire decision process – from associated data and models to problem formulation and solution generation, as well as to the preferences and analyses of other decision makers. A graph database is capable of capturing such diverse information in the form of a knowledge graph. It can be used to store and query all dependencies and hence can support complex decision-making tasks. Further advantages are the inherent suitability for visualization and the possibilities for pattern matching, graph analytics and, if semantically enriched, to infer new connections in the graph. In this paper, we show how such a knowledge graph can be used to support more transparent and traceable decision-making activities, particularly when multiple stakeholders with differing preferences or perspectives are involved., CC BY-NC 4.0 DEED© 2024 The AuthorsCorrespondence Address: L. Mittermeier; University of Skövde, School of Engineering Science, Sweden; email: ludwig.mittermeier@his.se

Query processing in the Hadoop ecosystem is a critical component for organizations leveraging big data to extract insights and drive data-driven decisions. This paper explores the tools and best practices associated with query processing in the Hadoop ecosystem. As the volume of data continues to grow exponentially, the need for efficient and scalable query processing solutions becomes increasingly important. In this study, we examine the key components of the Hadoop ecosystem, such as the Hadoop Distributed File System (HDFS) and the MapReduce programming model, which laid the foundation for big data processing. We delve into how these components have evolved and given rise to more advanced query processing tools, like Apache Hive, Apache Pig, Apache Spark, and Apache HBase. We discuss the advantages and limitations of each tool, allowing readers to make informed decisions when selecting the right tool for their specific use cases. Furthermore, we explore best practices for optimizing query performance, including data modeling, indexing, and query tuning. These practices can significantly impact the efficiency of query processing within the Hadoop ecosystem. The paper also addresses the challenges associated with query processing in this complex ecosystem, including data security, resource management, and handling real-time data streams. We provide insights into strategies for overcoming these challenges to ensure reliable and secure query processing.

The rapid growth of big data in recent years has ushered in a new era of data-driven decision-making and insights. As organizations grapple with increasingly large and complex datasets, the need for efficient and scalable query-processing solutions has never been greater. Our research focuses on addressing these challenges and presents innovative approaches to query processing, data storage, and analytics that promise to reshape the landscape of big data analytics. Key topics covered in this paper include: Distributed Query Processing, Query Optimization, In-Memory Data Processing, Machine Learning Integration, Data Compression and Encoding, Query Processing on Heterogeneous Data Sources, and Real-time and Stream Processing, By examining these critical areas, this paper aims to provide a comprehensive overview of the state-of-the-art in big data query processing. It highlights the importance of adopting next-generation solutions to meet the ever-growing demands of the big data landscape, enabling organizations to extract valuable insights faster and more efficiently. The presented research not only contributes to the ongoing evolution of big data analytics but also sets the stage for a new era of data-driven decision-making and innovation.

The rapid growth of big data in recent years has ushered in a new era of data-driven decision-making and insights. As organizations grapple with increasingly large and complex datasets, the need for efficient and scalable query-processing solutions has never been greater. Our research focuses on addressing these challenges and presents innovative approaches to query processing, data storage, and analytics that promise to reshape the landscape of big data analytics. Key topics covered in this paper include: Distributed Query Processing, Query Optimization, In-Memory Data Processing, Machine Learning Integration, Data Compression and Encoding, Query Processing on Heterogeneous Data Sources, and Real-time and Stream Processing, By examining these critical areas, this paper aims to provide a comprehensive overview of the state-of-the-art in big data query processing. It highlights the importance of adopting next-generation solutions to meet the ever-growing demands of the big data landscape, enabling organizations to extract valuable insights faster and more efficiently. The presented research not only contributes to the ongoing evolution of big data analytics but also sets the stage for a new era of data-driven decision-making and innovation.

Query processing in the Hadoop ecosystem is a critical component for organizations leveraging big data to extract insights and drive data-driven decisions. This paper explores the tools and best practices associated with query processing in the Hadoop ecosystem. As the volume of data continues to grow exponentially, the need for efficient and scalable query processing solutions becomes increasingly important. In this study, we examine the key components of the Hadoop ecosystem, such as the Hadoop Distributed File System (HDFS) and the MapReduce programming model, which laid the foundation for big data processing. We delve into how these components have evolved and given rise to more advanced query processing tools, like Apache Hive, Apache Pig, Apache Spark, and Apache HBase. We discuss the advantages and limitations of each tool, allowing readers to make informed decisions when selecting the right tool for their specific use cases. Furthermore, we explore best practices for optimizing query performance, including data modeling, indexing, and query tuning. These practices can significantly impact the efficiency of query processing within the Hadoop ecosystem. The paper also addresses the challenges associated with query processing in this complex ecosystem, including data security, resource management, and handling real-time data streams. We provide insights into strategies for overcoming these challenges to ensure reliable and secure query processing.

Today, data-driven analysis and applications exploit vast streams of data that are perpetually generated and collected from numerous data sources. Such a surge in data production, which has reached over 10,000 Exabytes, is driving transformative advancements in sectors such as transportation, emergency services, and health and wellness. Before data is queried and used by downstream data-driven analytical tasks, various computationally-intensive computations often need to be performed. Such tasks include data cleaning, data integration, and/or data enrichment operations that often execute expensively AI/ML models incur non-trivial costs. Such computationally expensive tasks can often not be performed at data ingestion time due to the rate at which data is produced. Periodic, offline computation is also infeasible due to the volume of data. Query processing in such a situation requires careful incorporation and co-optimization of computationally expensive operations into the query engine that can streamline query analysis and enhance execution efficiency in terms of time and resources. The goal of this thesis is to develop mechanisms to support computationally expensive operations (e.g., enrichment, imputations, information extraction, data interpretation) within data management systems in order to support interactiveanalysis. While the techniques developed in the thesis have wide applicability, our focus is on emerging smart space applications. Smart spaces consist of sensor-embedded physical spaces that capture and represent the dynamic state of the physical infrastructure, and that of people interacting with the physical infrastructure, and with each other. Data management in smart spaces opens several new challenges one of which is the ability to support interactive analytics on very large volumes of data being captured at large velocities. The problems studied in this thesis draw its motivation from such challenges. In Chapter 3 we develop a query-time missing va

We introduce a framework for cardinality estimation of query patterns over property graph databases. This framework makes it possible to analyze, compare and combine different cardinality estimation approaches. It consists of three phases: obtaining a set of estimates for some subqueries, extending this set and finally combining the set into a single cardinality estimate for the query. We show that (parts of) many existing cardinality estimation approaches can be used as techniques in one of the phases from our framework. The phases are loosely coupled, making it possible to combine (parts of) current cardinality estimation approaches. We created a graph version of the Join Order Benchmark to perform experiments with different combinations of techniques. The results showed that query patterns without property constraints can be accurately estimated using synopses for small patterns. Accurate estimation of query patterns with property constraints require new estimation techniques to be developed that capture correlations between the property constraints and the topology in graph databases.

Query, expressing the user need and requirement, has an important role, in an information retrieval system, for reaching a high accuracy search. In this paper, we present an overview of the different refinement operations that the query may undergo, in the sake to enhance performance of an information retrieval system, such as: automatic query formulation through words prevision, query reformulation, query expansion, and query optimization.

Query, expressing the user need and requirement, has an important role, in an information retrieval system, for reaching a high accuracy search. In this paper, we present an overview of the different refinement operations that the query may undergo, in the sake to enhance performance of an information retrieval system, such as: automatic query formulation through words prevision, query reformulation, query expansion, and query optimization.

This paper contributes to the discussion initiated in a recent SIGIR paper describing a gap in the information retrieval (IR) literature on query understanding–where they come from and whether they serve their purpose. Particularly the connection between query variability and search engines regarding consistent and equitable access to all users. We focus on a user group typically underserved: children. Using preliminary experiments (based on logs collected in the classroom context) and arguments grounded in children IR literature, we emphasize the importance of dedicating research efforts to interpreting queries formulated by children and the information needs they elicit. We also outline open problems and possible research directions to advance knowledge in this area, not just for children but also for other often-overlooked user groups and contexts., Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Web Information Systems

Shortest path computation is a fundamental operation in many application domains and is especially challenging in frequently evolving small-world networks (i.e., graphs in which many nodes can be reached from every other node by a small number of hops). Index-based methods, especially ones based on 2-hop labeling are often used for high query efficiency. However, the evolvements of small-world networks in many realistic scenarios pose the challenge of efficient maintenance of the shortest path index. In this work, we adopt the state-of-the-art Parallel Shortest-distance Labeling (PSL) as the underlying 2-hop labeling construction method, and design algorithms to support its efficient update given edge weight changes (increase and decrease). Specifically, we focus on weighted PSL (WPSL) and propose a propagation-based update mechanism for both synchronous and asynchronous propagation. We also identify the curse of pruning power in the edge weight increase case, and solve it with a balance between index size and effectiveness. Followed by, we extend the asynchronous propagation method to Pruned Landmark Labeling (PLL) for faster index maintenance and query processing with a smaller index size. Finally, we further optimize the index performance by reducing the index size through graph contraction and accelerating the index update through parallelized mix index update. Our experimental results on real-life and synthetic networks demonstrate the superiority of our algorithms over the relevant baselines on index maintenance. IEEE

With the proliferation of location-based services, geo-textual data is becoming ubiquitous. Objects involved in geo-textual data include geospatial locations, textual descriptions or keywords, and various attributes (e.g., a point-of-interest has its expenses and users' ratings). One prominent type of spatial keyword queries is to find, for a query consisting of a location and keywords, a set of objects that covers all the keywords and is of good quality according to some criteria. Existing studies define the criteria either based on the geospatial information of the objects solely or simply treat the geospatial and attribute information of the objects together without differentiation. As a result, they cannot provide users flexibility to express finer grained preferences on the objects. In this paper, we propose a new criterion which is to find a set of objects where the distance (defined based on the geospatial information) is at most a threshold specified by users and the cost (defined based on the attribute information) is optimized. We develop a suite of three algorithms including an exact algorithm and two approximation algorithms with provable guarantees for the problem. We conducted extensive experiments on real datasets which verified the efficiency and effectiveness of proposed algorithms.

Indexing is an effective way to support efficient query processing in large databases. Recently the concept of learned index, which replaces or complements traditional index structures with machine learning models, has been actively explored to reduce storage and search costs. However, accurate and efficient similarity query processing in high-dimensional metric spaces remains to be an open challenge. In this paper, we propose a novel indexing approach called LIMS that uses data clustering, pivot-based data transformation techniques and learned indexes to support efficient similarity query processing in metric spaces. In LIMS, the underlying data is partitioned into clusters such that each cluster follows a relatively uniform data distribution. Data redistribution is achieved by utilizing a small number of pivots for each cluster. Similar data are mapped into compact regions and the mapped values are totally ordinal. Machine learning models are developed to approximate the position of each data record on disk. Efficient algorithms are designed for processing range queries and nearest neighbor queries based on LIMS, and for index maintenance with dynamic updates. Extensive experiments on real-world and synthetic datasets demonstrate the superiority of LIMS compared with traditional indexes and state-of-the-art learned indexes. IEEE

Range aggregation is a primitive operation in spatial data applications and there is a growing demand to support such operations over a data federation, where the entire spatial data are separately held by multiple data providers (a.k.a., data silos). Data federations notably increase the amount of data available for data-intensive applications such as smart mobility planning and public health emergency responses. Yet they also challenge the conventional implementation of range aggregation queries because the raw data cannot be shared within the federation and the data partition at each data silo is fixed during query processing. These constraints limit the design space of distributed range aggregation query processing. In this work, we propose approximate algorithms for efficient range aggregation over spatial data federation. We devise novel single-silo sampling algorithms that process queries in parallel and design a level sampling based algorithm which reduces the time complexity of local queries at each data silo to O(log 1/), where is the approximation ratio of the accuracy guarantee. Extensive evaluations with real-world data show that compared with state-of-the-arts, our solutions reduce the time cost and communication cost by up to 85.1x and 5.5x respectively, with average approximate errors of below 2.8%. IEEE

Visual Question Answering (VQA) is a well-known problem for which deep-learning is key. This poses a challenge for explaining answers to questions, the more if advanced notions like contrastive explanations (CEs) should be provided. The latter explain why an answer has been reached in contrast to a different one and are attractive as they focus on reasons necessary to flip a query answer. We present a CE framework for VQA that uses a neurosymbolic VQA architecture which disentangles perception from reasoning. Once the reasoning part is provided as logical theory, we use answer-set programming, in which CE generation can be framed as an abduction problem. We validate our approach on the CLEVR dataset, which we extend by more sophisticated questions to further demonstrate the robustness of the modular architecture. While we achieve top performance compared to related approaches, we can also produce CEs for explanation, model debugging, and validation tasks, showing the versatility of the declarative approach to reasoning.

Computing the shortest path between two vertices is a fundamental problem in road networks. Most of the existing works assume that the edges in the road networks have no labels, but in many real applications, the edges have labels and label constraints may be placed on the edges appearing on a valid shortest path. Hence, we study the label-constrained shortest path queries in this paper. In order to process such queries efficiently, we adopt an index-based approach and propose a novel index structure, LSD-Index, based on tree decomposition. With LSD-Index, we design efficient query processing and index construction algorithms with good performance guarantees. Moreover, due to the dynamic properties of real-world networks, we also devise index maintenance algorithms that can maintain the index efficiently. To evaluate the performance of proposed methods, we conduct extensive experimental studies using large real road networks including the whole USA road network. Compared with the state-of-the-art approach, the experimental results demonstrate that our algorithm not only achieves up to two orders of magnitude speedup in query processing time but also consumes much less index space. Meanwhile, the experimental results also show that the index can also be efficiently constructed and maintained for dynamic graphs.

Query processing in the Hadoop ecosystem is a critical component for organizations leveraging big data to extract insights and drive data-driven decisions. This paper explores the tools and best practices associated with query processing in the Hadoop ecosystem. As the volume of data continues to grow exponentially, the need for efficient and scalable query processing solutions becomes increasingly important. In this study, we examine the key components of the Hadoop ecosystem, such as the Hadoop Distributed File System (HDFS) and the MapReduce programming model, which laid the foundation for big data processing. We delve into how these components have evolved and given rise to more advanced query processing tools, like Apache Hive, Apache Pig, Apache Spark, and Apache HBase. We discuss the advantages and limitations of each tool, allowing readers to make informed decisions when selecting the right tool for their specific use cases. Furthermore, we explore best practices for optimizing query performance, including data modeling, indexing, and query tuning. These practices can significantly impact the efficiency of query processing within the Hadoop ecosystem. The paper also addresses the challenges associated with query processing in this complex ecosystem, including data security, resource management, and handling real-time data streams. We provide insights into strategies for overcoming these challenges to ensure reliable and secure query processing.

The rapid growth of big data in recent years has ushered in a new era of data-driven decision-making and insights. As organizations grapple with increasingly large and complex datasets, the need for efficient and scalable query-processing solutions has never been greater. Our research focuses on addressing these challenges and presents innovative approaches to query processing, data storage, and analytics that promise to reshape the landscape of big data analytics. Key topics covered in this paper include: Distributed Query Processing, Query Optimization, In-Memory Data Processing, Machine Learning Integration, Data Compression and Encoding, Query Processing on Heterogeneous Data Sources, and Real-time and Stream Processing, By examining these critical areas, this paper aims to provide a comprehensive overview of the state-of-the-art in big data query processing. It highlights the importance of adopting next-generation solutions to meet the ever-growing demands of the big data landscape, enabling organizations to extract valuable insights faster and more efficiently. The presented research not only contributes to the ongoing evolution of big data analytics but also sets the stage for a new era of data-driven decision-making and innovation.

Data has been growing at an unprecedented rate in the past two decades. As a result, systems that store, process, and analyze data have become mission-critical. Crucial to the performance of data systems is the query optimizer, which translates high-level declarative queries (e.g., SQL) into efficient execution plans. However, query optimization is highly complex, leading to two key challenges. First, optimizers use a myriad of hand-designed heuristics to tame the complexity, but heuristics leave performance on the table. Second, optimizers are highly costly to develop, where human experts may spend months writing a first version and years refining it.This dissertation applies and enhances machine learning advances to tame the complexity in query optimization. First, we remove for the first time decades-old and accuracy-impacting heuristics in cardinality estimation—the Achilles’ heel of optimizers where heuristics particularly abound—thereby significantly improving estimation accuracy. We present Naru and NeuroCard, two cardinality estimators based on self-supervised learning advances that learn the joint data distribution of tables without any heuristic assumptions. Our estimators improve the accuracy of cardinality estimation by orders of magnitude compared to the prior state of the art. Second, we show that automatically learning to optimize SQL queries, without learning from an expert-designed optimizer, is both possible and efficient, thereby potentially alleviating the high development cost. We introduce Balsa, a deep reinforcement learning agent that automatically learns to optimize SQL queries by trial-and-error. Balsa can learn to outperform the optimizers of PostgreSQL—one of the most popular database systems—and a commercial database engine with a few hours of learning.Overall, by enhancing machine learning advances with new, carefully designed systems and ML techniques, this line of work improves existing query optimizers, while opening the possibility

Bayesian networks are popular probabilistic models that capture the conditional dependencies among a set of variables. Inference in Bayesian networks is a fundamental task for answering probabilistic queries over a subset of variables in the data. However, exact inference in Bayesian networks is NP-hard, which has prompted the development of many practical inference methods. In this paper, we focus on improving the performance of the junction-tree algorithm, a well-known method for exact inference in Bayesian networks. In particular, we seek to leverage information in the workload of probabilistic queries to obtain an optimal workload-aware materialization of junction trees, with the aim to accelerate the processing of inference queries. We devise an optimal pseudo-polynomial algorithm to tackle this problem and discuss approximation schemes. Compared to state-of-the-art approaches for efficient processing of inference queries via junction trees, our methods are the first to exploit the information provided in query workloads. Our experimentation on several real-world Bayesian networks confirms the effectiveness of our techniques in speeding-up query processing., QC 20221214

Distance oracles answer shortest-path queries between any pair of nodes in a graph. They are often built using succinct graph representations such as spanners, sketches, and compressors to minimize oracle size and query answering latency. Node embeddings, in particular, offer graph representations that place adjacent nodes nearby each other in a low-rank space. However, their use in the design of distance oracles has not been suffciently studied. In this paper, we empirically compare exact distance oracles constructed based on a variety of node embeddings and other succinct representations. We evaluate twelve such oracles along three measures of efficiency: construction time, memory requirements, and query-processing time over fourteen real datasets and four synthetic graphs. We show that distances between embedding vectors are excellent estimators of graph distances when graphs are well-structured, but less so for more unstructured graphs. Overall, our findings suggest that exact oracles based on embeddings can be constructed faster than multi-dimensional scaling (MDS) but slower than compressed adjacency indexes, require less memory than landmark oracles but more than sparsifiers or indexes, can answer queries faster than indexes but slower than MDS, and are exact more often with a smaller additive error than spanners (that have multiplicative error) while not being lossless like adjacency lists. Finally, while the exactness of such oracles is infeasible to maintain for huge graphs even under large amounts of resources, we empirically demonstrate that approximate oracles based on GOSH embeddings can efficiently scale to graphs of 100M+ nodes with only small additive errors in distance estimations., QC 20230607

Graphic processing units (GPUs) have been employed as hardware accelerators for online analytics. However, multi-way joins, which are common in analytic workloads, are inefficient on GPUs. Therefore, we propose to accelerate two representative multi-way join algorithms on the GPU: a multi-way hash join (MHJ) and the worst-case optimal Leapfrog Triejoin (LFTJ). Specifically, we design a warp-based parallelization strategy to reduce thread divergence and to facilitate coalesced memory access in parallel searches in a table. We further enhance our implementations with a set of GPU-friendly optimizations, including dynamic workload sharing among threads and elimination of the result counting phase. Additionally, we enable out-of-core multi-way joins with software pipelining. Our experiments show that our optimized MHJ and LFTJ outperform the state-of-the-art GPU algorithms by a factor of up to 67 on an NVIDIA V100 GPU. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

We study the problem of active pure exploration with fixed confidence in generic stochastic bandit environments. The goal of the learner is to answer a query about the environment with a given level of certainty while minimizing her sampling budget. For this problem, instance-specific lower bounds on the expected sample complexity reveal the optimal proportions of arm draws an Oracle algorithm would apply. These proportions solve an optimization problem whose tractability strongly depends on the structural properties of the environment, but may be instrumental in the design of efficient learning algorithms. We devise Frank-Wolfe-based Sampling (FWS), a simple algorithm whose sample complexity matches the lower bounds for a wide class of pure exploration problems. The algorithm is computationally efficient as, to learn and track the optimal proportion of arm draws, it relies on a single iteration of Frank-Wolfe algorithm applied to the lower-bound optimization problem. We apply FWS to various pure exploration tasks, including best arm identification in unstructured, thresholded, linear, and Lipschitz bandits. Despite its simplicity, FWS is competitive compared to state-of-art algorithms., Part of proceedings: ISBN 978-1-7138-4539-3QC 20230921

We study the problem of active pure exploration with fixed confidence in generic stochastic bandit environments. The goal of the learner is to answer a query about the environment with a given level of certainty while minimizing her sampling budget. For this problem, instance-specific lower bounds on the expected sample complexity reveal the optimal proportions of arm draws an Oracle algorithm would apply. These proportions solve an optimization problem whose tractability strongly depends on the structural properties of the environment, but may be instrumental in the design of efficient learning algorithms. We devise Frank-Wolfe-based Sampling (FWS), a simple algorithm whose sample complexity matches the lower bounds for a wide class of pure exploration problems. The algorithm is computationally efficient as, to learn and track the optimal proportion of arm draws, it relies on a single iteration of Frank-Wolfe algorithm applied to the lower-bound optimization problem. We apply FWS to various pure exploration tasks, including best arm identification in unstructured, thresholded, linear, and Lipschitz bandits. Despite its simplicity, FWS is competitive compared to state-of-art algorithms., Part of proceedings: ISBN 978-1-7138-4539-3QC 20230921

We study the problem of active pure exploration with fixed confidence in generic stochastic bandit environments. The goal of the learner is to answer a query about the environment with a given level of certainty while minimizing her sampling budget. For this problem, instance-specific lower bounds on the expected sample complexity reveal the optimal proportions of arm draws an Oracle algorithm would apply. These proportions solve an optimization problem whose tractability strongly depends on the structural properties of the environment, but may be instrumental in the design of efficient learning algorithms. We devise Frank-Wolfe-based Sampling (FWS), a simple algorithm whose sample complexity matches the lower bounds for a wide class of pure exploration problems. The algorithm is computationally efficient as, to learn and track the optimal proportion of arm draws, it relies on a single iteration of Frank-Wolfe algorithm applied to the lower-bound optimization problem. We apply FWS to various pure exploration tasks, including best arm identification in unstructured, thresholded, linear, and Lipschitz bandits. Despite its simplicity, FWS is competitive compared to state-of-art algorithms., Part of proceedings: ISBN 978-1-7138-4539-3QC 20230921

We study the problem of active pure exploration with fixed confidence in generic stochastic bandit environments. The goal of the learner is to answer a query about the environment with a given level of certainty while minimizing her sampling budget. For this problem, instance-specific lower bounds on the expected sample complexity reveal the optimal proportions of arm draws an Oracle algorithm would apply. These proportions solve an optimization problem whose tractability strongly depends on the structural properties of the environment, but may be instrumental in the design of efficient learning algorithms. We devise Frank-Wolfe-based Sampling (FWS), a simple algorithm whose sample complexity matches the lower bounds for a wide class of pure exploration problems. The algorithm is computationally efficient as, to learn and track the optimal proportion of arm draws, it relies on a single iteration of Frank-Wolfe algorithm applied to the lower-bound optimization problem. We apply FWS to various pure exploration tasks, including best arm identification in unstructured, thresholded, linear, and Lipschitz bandits. Despite its simplicity, FWS is competitive compared to state-of-art algorithms., Part of proceedings: ISBN 978-1-7138-4539-3QC 20230921

Range aggregation is a primitive operation in spatial data applications and there is a growing demand to support such operations over a data federation, where the entire spatial data are separately held by multiple data providers (a.k.a., data silos). Data federations notably increase the amount of data available for data-intensive applications such as smart mobility planning and public health emergency responses. Yet they also challenge the conventional implementation of range aggregation queries because the raw data cannot be shared within the federation and the data partition at each data silo is fixed during query processing. These constraints limit the design space of distributed range aggregation query processing. In this work, we propose approximate algorithms for efficient range aggregation over spatial data federation. We devise novel single-silo sampling algorithms that process queries in parallel and design a level sampling based algorithm which reduces the time complexity of local queries at each data silo to O(log 1/), where is the approximation ratio of the accuracy guarantee. Extensive evaluations with real-world data show that compared with state-of-the-arts, our solutions reduce the time cost and communication cost by up to 85.1x and 5.5x respectively, with average approximate errors of below 2.8%. IEEE

Graphic processing units (GPUs) have been employed as hardware accelerators for online analytics. However, multi-way joins, which are common in analytic workloads, are inefficient on GPUs. Therefore, we propose to accelerate two representative multi-way join algorithms on the GPU: a multi-way hash join (MHJ) and the worst-case optimal Leapfrog Triejoin (LFTJ). Specifically, we design a warp-based parallelization strategy to reduce thread divergence and to facilitate coalesced memory access in parallel searches in a table. We further enhance our implementations with a set of GPU-friendly optimizations, including dynamic workload sharing among threads and elimination of the result counting phase. Additionally, we enable out-of-core multi-way joins with software pipelining. Our experiments show that our optimized MHJ and LFTJ outperform the state-of-the-art GPU algorithms by a factor of up to 67 on an NVIDIA V100 GPU. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

In this paper, we describe a secure version of the classical Yannakakis algorithm for computing free-connex join-aggregate queries. This protocol can be used in the secure two-party computation model, where the parties would like to evaluate a query without revealing their own data. Our protocol presents a dramatic improvement over the state-of-the-art protocol based on Yao's garbled circuit. In theory, its cost (both running time and communication) is linear in data size and polynomial in query size, whereas that of the garbled circuit is polynomial in data size and exponential in query size. This translates to a reduction in running time in practice from years to minutes, as tested on a number of TPC-H queries of varying complexity. © 2021 ACM.

In this paper, we describe a secure version of the classical Yannakakis algorithm for computing free-connex join-aggregate queries. This protocol can be used in the secure two-party computation model, where the parties would like to evaluate a query without revealing their own data. Our protocol presents a dramatic improvement over the state-of-the-art protocol based on Yao's garbled circuit. In theory, its cost (both running time and communication) is linear in data size and polynomial in query size, whereas that of the garbled circuit is polynomial in data size and exponential in query size. This translates to a reduction in running time in practice from years to minutes, as tested on a number of TPC-H queries of varying complexity. © 2021 ACM.

Range aggregation is a primitive operation in spatial data applications and there is a growing demand to support such operations over a data federation, where the entire spatial data are separately held by multiple data providers (a.k.a., data silos). Data federations notably increase the amount of data available for data-intensive applications such as smart mobility planning and public health emergency responses. Yet they also challenge the conventional implementation of range aggregation queries because the raw data cannot be shared within the federation and the data partition at each data silo is fixed during query processing. These constraints limit the design space of distributed range aggregation query processing. In this work, we propose approximate algorithms for efficient range aggregation over spatial data federation. We devise novel single-silo sampling algorithms that process queries in parallel and design a level sampling based algorithm which reduces the time complexity of local queries at each data silo to O(log 1/), where is the approximation ratio of the accuracy guarantee. Extensive evaluations with real-world data show that compared with state-of-the-arts, our solutions reduce the time cost and communication cost by up to 85.1x and 5.5x respectively, with average approximate errors of below 2.8%. IEEE

Graphic processing units (GPUs) have been employed as hardware accelerators for online analytics. However, multi-way joins, which are common in analytic workloads, are inefficient on GPUs. Therefore, we propose to accelerate two representative multi-way join algorithms on the GPU: a multi-way hash join (MHJ) and the worst-case optimal Leapfrog Triejoin (LFTJ). Specifically, we design a warp-based parallelization strategy to reduce thread divergence and to facilitate coalesced memory access in parallel searches in a table. We further enhance our implementations with a set of GPU-friendly optimizations, including dynamic workload sharing among threads and elimination of the result counting phase. Additionally, we enable out-of-core multi-way joins with software pipelining. Our experiments show that our optimized MHJ and LFTJ outperform the state-of-the-art GPU algorithms by a factor of up to 67 on an NVIDIA V100 GPU. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

Knowledge graphs are a powerful concept for querying large amounts of data. These knowledge graphs are typically enormous and are often not easily accessible to end-users because they require specialized knowledge in query languages such as SPARQL. Moreover, end-users need a deep understanding of the structure of the underlying data models often based on the Resource Description Framework (RDF). This drawback has led to the development of Question-Answering (QA) systems that enable end-users to express their information needs in natural language. While existing systems simplify user access, there is still room for improvement in the accuracy of these systems. In this paper we propose a new QA system for translating natural language questions into SPARQL queries. The key idea is to break up the translation process into 5 smaller, more manageable sub-tasks and use ensemble machine learning methods as well as Tree-LSTM-based neural network models to automatically learn and translate a natural language question into a SPARQL query. The performance of our proposed QA system is empirically evaluated using the two renowned benchmarks-the 7th Question Answering over Linked Data Challenge (QALD-7) and the Large-Scale Complex Question Answering Dataset (LC-QuAD). Experimental results show that our QA system outperforms the state-of-art systems by 15% on the QALD-7 dataset and by 48% on the LC-QuAD dataset, respectively. In addition, we make our source code available.

A search query consists of several words. In a proximity full-text search, we want to find documents that contain these words near each other. This task requires much time when the query consists of high-frequently occurring words. If we cannot avoid this task by excluding high-frequently occurring words from consideration by declaring them as stop words, then we can optimize our solution by introducing additional indexes for faster execution. In a previous work, we discussed how to decrease the search time with multi-component key indexes. We had shown that additional indexes can be used to improve the average query execution time up to 130 times if queries consisted of high-frequently occurring words. In this paper, we present another search algorithm that overcomes some limitations of our previous algorithm and provides even more performance gain. © 2021, Springer Nature Switzerland AG.

We study the problem of active pure exploration with fixed confidence in generic stochastic bandit environments. The goal of the learner is to answer a query about the environment with a given level of certainty while minimizing her sampling budget. For this problem, instance-specific lower bounds on the expected sample complexity reveal the optimal proportions of arm draws an Oracle algorithm would apply. These proportions solve an optimization problem whose tractability strongly depends on the structural properties of the environment, but may be instrumental in the design of efficient learning algorithms. We devise Frank-Wolfe-based Sampling (FWS), a simple algorithm whose sample complexity matches the lower bounds for a wide class of pure exploration problems. The algorithm is computationally efficient as, to learn and track the optimal proportion of arm draws, it relies on a single iteration of Frank-Wolfe algorithm applied to the lower-bound optimization problem. We apply FWS to various pure exploration tasks, including best arm identification in unstructured, thresholded, linear, and Lipschitz bandits. Despite its simplicity, FWS is competitive compared to state-of-art algorithms., Part of proceedings: ISBN 978-1-7138-4539-3QC 20230921

Given two queries Qsuper and Qsub, query containment is the problem of determining if Qsub(D) ⊆ Qsuper(D) for all databases D. This problem has long been explored, but to our knowledge no one has empirically evaluated a straightforward application of finite state automata to the problem. We do so here, covering the case of conjunctive queries with limited set conditions. We evaluate an implementation of our approach against straightforward implementations of both the canonical database and theorem proving approaches. Our implementation outperforms theorem proving on a natural language interface corpus over a photo/video domain. It also outperforms the canonical database implementation on single relation queries with large set conditions., QC 20220602

In this paper, we describe a secure version of the classical Yannakakis algorithm for computing free-connex join-aggregate queries. This protocol can be used in the secure two-party computation model, where the parties would like to evaluate a query without revealing their own data. Our protocol presents a dramatic improvement over the state-of-the-art protocol based on Yao's garbled circuit. In theory, its cost (both running time and communication) is linear in data size and polynomial in query size, whereas that of the garbled circuit is polynomial in data size and exponential in query size. This translates to a reduction in running time in practice from years to minutes, as tested on a number of TPC-H queries of varying complexity. © 2021 ACM.

Knowledge graphs are a powerful concept for querying large amounts of data. These knowledge graphs are typically enormous and are often not easily accessible to end-users because they require specialized knowledge in query languages such as SPARQL. Moreover, end-users need a deep understanding of the structure of the underlying data models often based on the Resource Description Framework (RDF). This drawback has led to the development of Question-Answering (QA) systems that enable end-users to express their information needs in natural language. While existing systems simplify user access, there is still room for improvement in the accuracy of these systems. In this paper we propose a new QA system for translating natural language questions into SPARQL queries. The key idea is to break up the translation process into 5 smaller, more manageable sub-tasks and use ensemble machine learning methods as well as Tree-LSTM-based neural network models to automatically learn and translate a natural language question into a SPARQL query. The performance of our proposed QA system is empirically evaluated using the two renowned benchmarks-the 7th Question Answering over Linked Data Challenge (QALD-7) and the Large-Scale Complex Question Answering Dataset (LC-QuAD). Experimental results show that our QA system outperforms the state-of-art systems by 15% on the QALD-7 dataset and by 48% on the LC-QuAD dataset, respectively. In addition, we make our source code available.

[Abstract] Raster time series, a.k.a. temporal rasters, are collections of rasters covering the same region at consecutive timestamps. These data have been used in many different applications ranging from weather forecast systems to monitoring of forest degradation or soil contamination. Many different sensors are generating this type of data, which makes such analyses possible, but also challenges the technological capacity to store and retrieve the data. In this work, we propose a space-efficient representation of raster time series that is based on Compact Data Structures (CDS). Our method uses a strategy of snapshots and logs to represent the data, in which both components are represented using CDS. We study two variants of this strategy, one with regular sampling and another one based on a heuristic that determines at which timestamps should the snapshots be created to reduce the space redundancy. We perform a comprehensive experimental evaluation using real datasets. The results show that the proposed strategy is competitive in space with alternatives based on pure data compression, while providing much more efficient query times for different types of queries.

There has been increasing need for secure data sharing. In practice a group of data owners often adopt a heterogeneous security scheme under which each pair of parties decide their own protocol to share data with diverse levels of trust. The scheme also keeps track of how the data is used. This paper studies distributed SQL query answering in the heterogeneous security setting. We define query plans by incorporating toll functions determined by data sharing agreements and reflected in the use of various security facilities. We formalize query answering as a bi-criteria optimization problem, to minimize both data sharing toll and parallel query evaluation cost. We show that this problem is PSPACE-hard for SQL and Σ_3^p-hard for SPC, and it is in NEXPTIME. Despite the hardness, we develop a set of approximate algorithms to generate distributed query plans that minimize data sharing toll and reduce parallel evaluation cost. Using real-life and synthetic data, we empirically verify the effectiveness, scalability and efficiency of our algorithms.

Modern query engines rely heavily on hash tables for query processing. Overall query performance and memory footprint is often determined by how hash tables and the tuples within them are represented. In this work, we propose three complementary techniques to improve this representation: Domain-Guided Prefix Suppression bit-packs keys and values tightly to reduce hash table record width. Optimistic Splitting decomposes values (and operations on them) into (operations on) frequently-accessed and infrequently-accessed value slices. By removing the infrequently-accessed value slices from the hash table record, it improves cache locality. The Unique Strings Self-aligned Region (USSR) accelerates handling frequently-occurring strings, which are very common in real-world data sets, by creating an on-the-fly dictionary of the most frequent strings. This allows executing many string operations with integer logic and reduces memory pressure. We integrated these techniques into Vectorwise. On the TPC-H benchmark, our approach reduces peak memory consumption by 2-4× and improves performance by up to 1.5×. On a real-world BI workload, we measured a 2× improvement in performance and in micro-benchmarks we observed speedups of up to 25×.

Exploiting available condition monitoring data of industrial machines for intelligent maintenance purposes has been attracting attention in various application fields. Machine learning algorithms for fault detection, diagnosis and prognosis are popular and easily accessible. However, our experience in working at the intersection of academia and industry showed that the major challenges of building an end-to-end system in a real-world industrial setting go beyond the design of machine learning algorithms. One of the major challenges is the design of an end-to-end data management solution that is able to efficiently store and process large amounts of heterogeneous data streams resulting from a variety of physical machines. In this paper we present the design of an end-to-end Big Data architecture that enables intelligent maintenance in a real-world industrial setting. In particular, we will discuss various physical design choices for optimizing high-dimensional queries, such as partitioning and Z-ordering, that serve as the basis for health analytics. Finally, we describe a concrete fault detection use case with two different health monitoring algorithms based on machine learning and classical statistics and discuss their advantages and disadvantages. The paper covers some of the most important aspects of the practical implementation of such an end-to-end solution and demonstrates the challenges and their mitigation for the specific application of laser cutting machines.

The rapid growth of big spatial data urged the research community to develop several big spatial data systems. Regardless of their architecture, one of the fundamental requirements of all these systems is to spatially partition the data efficiently across machines. The core challenges of big spatial partitioning are building high spatial quality partitions while simultaneously taking advantages of distributed processing models by providing load balanced partitions. Previous works on big spatial partitioning are to reuse existing index search trees as-is, e.g., the R-tree family, STR, Kd-tree, and Quad-tree, by building a temporary tree for a sample of the input and use its leaf nodes as partition boundaries. However, we show in this paper that none of those techniques has addressed the mentioned challenges completely. This paper proposes a novel partitioning method, termed R*-Grove, which can partition very large spatial datasets into high quality partitions with excellent load balance and block utilization. This appealing property allows R*-Grove to outperform existing techniques in spatial query processing. R*-Grove can be easily integrated into any big data platforms such as Apache Spark or Apache Hadoop. Our experiments show that R*-Grove outperforms the existing partitioning techniques for big spatial data systems. With all the proposed work publicly available as open source, we envision that R*-Grove will be adopted by the community to better serve big spatial data research.

NOSQL database management systems adopt semi-structured data models, such as JSON, to easily accommodate schema evolution and overcome the overhead generated from transforming internal structures to tabular data (i.e., impedance mismatch). There exist multiple, and equivalent, ways to physically represent semi-structured data, but there is a lack of evidence about the potential impact on space and query performance. In this paper, we embark on the task of quantifying that, precisely for document stores. We empirically compare multiple ways of representing semi-structured data, which allows us to derive a set of guidelines for efficient physical database design considering both JSON and relational options in the same palette., Partly funded by the European Commission through the programme “EM IT4BI-DC”., Peer Reviewed, Postprint (author's final draft)