Search

In spite of increased attention on explainable machine learning models, explaining multi-output predictions has not yet been extensively addressed. Methods that use Shapley values to attribute feature contributions to the decision making are one of the most popular approaches to explain local individual and global predictions. By considering each output separately in multi-output tasks, these methods fail to provide complete feature explanations. We propose Shapley Chains to overcome this issue by including label interdependencies in the explanation design process. Shapley Chains assign Shapley values as feature importance scores in multi-output classification using classifier chains, by separating the direct and indirect influence of these feature scores. Compared to existing methods, this approach allows to attribute a more complete feature contribution to the predictions of multi-output classification tasks. We provide a mechanism to distribute the hidden contributions of the outputs with respect to a given chaining order of these outputs. Moreover, we show how our approach can reveal indirect feature contributions missed by existing approaches. Shapley Chains help to emphasize the real learning factors in multi-output applications and allows a better understanding of the flow of information through output interdependencies in synthetic and real-world datasets., Comment: DS22

In $\mathbb R^d$, it is well-known that cumulants provide an alternative to moments that can achieve the same goals with numerous benefits such as lower variance estimators. In this paper we extend cumulants to reproducing kernel Hilbert spaces (RKHS) using tools from tensor algebras and show that they are computationally tractable by a kernel trick. These kernelized cumulants provide a new set of all-purpose statistics; the classical maximum mean discrepancy and Hilbert-Schmidt independence criterion arise as the degree one objects in our general construction. We argue both theoretically and empirically (on synthetic, environmental, and traffic data analysis) that going beyond degree one has several advantages and can be achieved with the same computational complexity and minimal overhead in our experiments., Comment: 19 pages, 8 figures

Many forecasts consist not of point predictions but concern the evolution of quantities. For example, a central bank might predict the interest rates during the next quarter, an epidemiologist might predict trajectories of infection rates, a clinician might predict the behaviour of medical markers over the next day, etc. The situation is further complicated since these forecasts sometimes only concern the approximate "shape of the future evolution" or "order of events". Formally, such forecasts can be seen as probability measures on spaces of equivalence classes of paths modulo time-parametrization. We leverage the statistical framework of proper scoring rules with classical mathematical results to derive a principled approach to decision making with such forecasts. In particular, we introduce notions of gradients, entropy, and divergence that are tailor-made to respect the underlying non-Euclidean structure.

Sequential data such as time series, video, or text can be challenging to analyse as the ordered structure gives rise to complex dependencies. At the heart of this is non-commutativity, in the sense that reordering the elements of a sequence can completely change its meaning. We use a classical mathematical object -- the tensor algebra -- to capture such dependencies. To address the innate computational complexity of high degree tensors, we use compositions of low-rank tensor projections. This yields modular and scalable building blocks for neural networks that give state-of-the-art performance on standard benchmarks such as multivariate time series classification and generative models for video., Comment: 37 pages, 6 figures, 8 tables

The topology of weak convergence does not account for the growth of information over time that is captured in the filtration of an adapted stochastic process. For example, two adapted stochastic processes can have very similar laws but give completely different results in applications such as optimal stopping, queuing theory, or stochastic programming. To address such discontinuities, Aldous introduced the extended weak topology, and subsequently, Hoover and Keisler showed that both, weak topology and extended weak topology, are just the first two topologies in a sequence of topologies that get increasingly finer. We use higher rank expected signatures to embed adapted processes into graded linear spaces and show that these embeddings induce the adapted topologies of Hoover--Keisler., Comment: 39 pages, 6 figures

The sequence of so-called signature moments describes the laws of many stochastic processes in analogy with how the sequence of moments describes the laws of vector-valued random variables. However, even for vector-valued random variables, the sequence of cumulants is much better suited for many tasks than the sequence of moments. This motivates us to study so-called signature cumulants. To do so, we develop an elementary combinatorial approach and show that in the same way that cumulants relate to the lattice of partitions, signature cumulants relate to the lattice of so-called "ordered partitions". We use this to give a new characterisation of independence of multivariate stochastic processes; finally we construct a family of unbiased minimum-variance estimators of signature cumulants., Comment: 31 pages, 2 figures

The signature is an infinite graded sequence of statistics known to characterise a stream of data up to a negligible equivalence class. It is a transform which has previously been treated as a fixed feature transformation, on top of which a model may be built. We propose a novel approach which combines the advantages of the signature transform with modern deep learning frameworks. By learning an augmentation of the stream prior to the signature transform, the terms of the signature may be selected in a data-dependent way. More generally, we describe how the signature transform may be used as a layer anywhere within a neural network. In this context it may be interpreted as a pooling operation. We present the results of empirical experiments to back up the theoretical justification. Code available at https://github.com/patrick-kidger/Deep-Signature-Transforms., Comment: Published at NeurIPS 2019

We study analytically and numerically a model of random sequential adsorption (RSA) of segments on a line, subject to some constraints suggested by two kinds of physical situations: - deposition of dimers on a lattice where the sites have a spatial extension; - deposition of extended particles which must overlap one (or several) adsorbing sites on the substrate. Both systems involve discrete and continuous degrees of freedom, and, in one dimension, are equivalent to our model, which depends on one length parameter. When this parameter is varied, the model interpolates between a variety of known situations : monomers on a lattice, "car-parking" problem, dimers on a lattice. An analysis of the long-time behaviour of the coverage as a function of the parameter exhibits an anomalous 1/t^2 approach to the jamming limit at the transition point between the fast exponential kinetics, characteristic of the lattice model, and the 1/t law of the continuous one., Comment: 14 pages (Latex) + 4 Postscript figures

We study the deposition of line segments on a two-dimensional square lattice. The estimates for the coverage at jamming obtained by Monte-Carlo simulations and by $7^{th}$-order time-series expansion are successfully compared. The non-trivial limit of adsorption of infinitely long segments is studied, and the lattice coverage is consistently obtained using these two approaches., Comment: 19 pages in Latex+5 postscript files sent upon request ; PTB93_8

We compute the time-dependent coverage in the random sequential adsorption of aligned d-dimensional cubes in $R^d$ using time-series expansions. The seventh-order series in 2, 3 and 4 dimensions is resummed in order to predict the coverage at jamming. The result is in agreement with Monte-Carlo simulations. A simple argument, based on a property of the perturbative expansion valid at arbitrary orders, allows us to analytically derive some generalizations of the Pal\'asti approximation., Comment: pure LaTeX, 8 pages, LPTB 93-3