Search

Memory operations are critical to an application's reliability and performance. To reason about their correctness and track opportunities for optimisations, sophisticated instrumentation frameworks, such as Valgrind and Pin, have been developed. Both provide only limited facilities for analysing the collected data. This work presents a Valgrind's extension for examining a software applications' dynamic memory profile through live interactive analysis with SQL. The Pico COllections Query Library (pico ql) module maps Valgrind's data structures that contain the instrumented application's memory operations metadata to a relational interface. Queries are type‐safe and the module imposes only a trivial overhead when idle. The authors evaluate the proposed approach on ten applications and through a qualitative study. They find 900 kb of undefined bytes in bzip2 that account for 12% of its total memory use and a performance‐critical code execution path in the Unix commands sort and uniq. The referenced functions are part of glibc and have been independently modified to boost the library's performance. The qualitative study has users rate the usefulness, usability, effort, correctness, and expressiveness of pico ql queries compared to Python scripts. The findings indicate that querying with pico ql incurs lower user effort. [ABSTRACT FROM AUTHOR]

Memory operations are critical to an application's reliability and performance. To reason about their correctness and track opportunities for optimisations, sophisticated instrumentation frameworks, such as Valgrind and Pin, have been developed. Both provide only limited facilities for analysing the collected data. This work presents a Valgrind's extension for examining a software applications' dynamic memory profile through live interactive analysis with SQL. The Pico COllections Query Library (pico ql) module maps Valgrind's data structures that contain the instrumented application's memory operations metadata to a relational interface. Queries are type-safe and the module imposes only a trivial overhead when idle. The authors evaluate the proposed approach on ten applications and through a qualitative study. They find 900 kb of undefined bytes in bzip2 that account for 12% of its total memory use and a performance-critical code execution path in the Unix commands sort and uniq. The referenced functions are part of glibc and have been independently modified to boost the library's performance. The qualitative study has users rate the usefulness, usability, effort, correctness, and expressiveness of PICO QL queries compared to Python scripts. The findings indicate that querying with PICO QL incurs lower user effort. [ABSTRACT FROM AUTHOR]

The Unix shell dgsh provides an expressive way to construct sophisticated and efficient non-linear pipelines. Such pipelines can use standard Unix tools, as well as third-party and custom-built components. Dgsh allows the specification of pipelines that perform non-uniform non-linear processing. These form a directed acyclic process graph, which is typically executed by multiple processor cores, thus increasing the processing task’s throughput. A number of existing Unix tools have been adapted to take advantage of the new shell’s multiple pipe input/output capabilities. The shell supports visualization of the process graphs, which can also aid debugging. Dgsh was evaluated through a number of common data processing and domain-specific examples, and was found to offer an expressive way to specify processing topologies, while also generally increasing processing throughput. [ABSTRACT FROM PUBLISHER]

Query formalisms and facilities have received significant attention in the past decades resulting in the development of query languages with varying characteristics; many of them resemble sql. Query facilities typically ship as part of database management systems or, sometimes, bundled with programming languages. For applications written in imperative programming languages, database management systems impose an expensive model transformation. In-memory data structures can represent sophisticated relationships in a manner that is efficient in terms of storage and processing overhead, but most general purpose programming languages lack an interpreter and/or an expressive query language for manipulating interactive queries. Issuing interactive ad-hoc queries on program data structures is tough. This work presents a method and an implementation for representing an application's arbitrary imperative programming data model as a queryable relational one. The Pico COllections Query Library ( pi co ql) uses a domain specific language to define a relational representation of application data structures and an sql interface implementation. Queries are issued interactively and are type safe. We demonstrate our relational representation for objects and the library's usefulness on three large c++ projects. pi co ql enhances query expressiveness and boosts productivity compared to querying via traditional programming constructs. [ABSTRACT FROM AUTHOR]

Serverless computing is currently the fastest-growing cloud services segment. The most prominent serverless offering is Function-as-a-Service (FaaS), where users write functions and the cloud automates deployment, maintenance, and scalability. Although FaaS is a good fit for executing stateless functions, it does not adequately support stateful constructs like microservices and scalable, low-latency cloud applications. Recently, there have been multiple attempts to add first-class support for state in FaaS systems, such as Microsoft Orleans, Azure Durable Functions, or Beldi. These approaches execute business code inside stateless functions, handing over their state to an external database. In contrast, approaches such as Apache Flink's StateFun follow a different design: a dataflow system such as Apache Flink handles all state management, messaging, and checkpointing by executing a stateful dataflow graph providing exactly-once state processing guarantees. This design relieves programmers from having to "pollute" their business logic with distributed systems error checking, management, and mitigation. Although programmers can easily develop applications without worrying about messaging and state management, executing transactions across stateful functions remains an open problem. In this paper, we introduce a programming model and implementation for transaction orchestration of stateful serverless functions. Our programming model supports serializable distributed transactions with two-phase commit, as well as eventually consistent workflows with Sagas. We design and implement our programming model on Apache Flink StateFun to leverage Flink's exactly-once processing and state management guarantees. Our experiments show that the approach of building transactional systems on top of dataflow graphs can achieve very high throughput, but with latency overhead due to checkpointing mechanism guaranteeing the exactly-once processing. We compare our approach to Beldi that implements two-phase commit on AWS lambda functions backed by DynamoDB for state management, as well as an implementation of a system that makes use of CockroachDB as its backend. • Dataflow systems can serve as execution engines for cloud applications. • Transactional guarantees can be implemented on stateful functions. • Implementing transaction protocols is very simple under exactly-once processing guarantees. • Further work is required for dataflow systems to reach their full potential as backends for cloud applications. • The performance of transactions on stateful functions is comparable to the one of distributed databases. [ABSTRACT FROM AUTHOR]