Search

This research was supported by the King Abdullah University of Science and Technology (KAUST).

We propose new tools for visualizing large amounts of functional data in the form of smooth curves. The proposed tools include functional versions of the bagplot and boxplot, which make use of the first two robust principal component scores, Tukey’s data depth and highest density regions. By-products of our graphical displays are outlier detection methods for functional data. We compare these new outlier detection methods with existing methods for detecting outliers in functional data, and show that our methods are better able to identify outliers. An R-package containing computer code and datasets is available in the online supplements.

A heart-transplanted patient is at risk of developing several complications such as rejection, which is one of the leading causes of deaths in the first year after the transplant. The regional myocardial motion is known to be depressed early on during rejection before the reduction in global systolic function. Therefore, early detection of regional anomalies is crucial. We use a magnetic resonance (MR) imaging method called tissue phase mapping (TPM) to capture regional myocardial motion of heart-transplanted patients in a longitudinal study. We compare the individual scans of the longitudinal study to a cohort of healthy volunteers to detect anomalies. We use a spatio-temporal visualization based on a radial layout where myocardial regions are laid out in an angular pattern similar to the American Heart Association (AHA) model and where the temporal dimension increases with increasing radius. We compute nested envelopes of central regions for the time series of each region and each of the three velocity directions using the concept of functional boxplots. We propose visual encodings to analyze regional anomalies of a scan of an individual patient dataset and perform a qualitative user study with medical experts. We extend this layout to the visual analysis of longitudinal data to monitor changes in regional anomalies of a patient for multiple scans taken at different times. We apply our approach to data from a longitudinal study of patients under observation after a heart-transplant procedure and evaluate this mechanism with medical and non-medical experts.

Aims: The detection of an epidemic outbreak is possible only if the baseline incidence level of a given disease is well defined. The determination of the baseline is complicated by the presence of epidemic outbreaks in historical data. The aim of the paper is to provide a new way of determining the baseline. Methods: The analyzed data containing weekly records on the incidence of acute respiratory infections including influenza (ARI) in the Czech Republic and its regions are taken from the nationwide surveillance system; data on 15 seasons from 2001/02 to 2015/16 are included. Functional boxplots of the data are constructed and five distinct methods (componentwise mean, componentwise median, median, trimmed mean, and adjusted mean) were used for the computation of the baseline level function. Results: It was shown that the methods based on functional data analysis could successfully overcome the problems that arise when the conventional methods are used for the determination of the baseline function. Conclusion: The functional boxplot - a new statistical tool - can bring not only a transparent visualisation of comprehensive data, but can also help epidemiologists and other public health experts to determine the baseline incidence level of a given disease as well as to detect unusual epidemic seasons.

Statistical depth and related quantile functions, originally introduced for nonparametric description and analysis of multivariate data in a way sensitive to inherent geometry, are in active development for functional data and in this setting offer special options since the data may be visualized regardless of dimension. This paper provides depth-based methods for revealing the structure of a functional data set in terms of relevant sample quantile curves displayed at selected levels, and for constructing and displaying confidence bands for corresponding population versions. Also, the usual functional boxplot is enhanced, by adding inner fences to flag possible shape outliers, along with the outer fences that flag location outliers. This enables the boxplot to serve as a stand-alone tool for functional data, as with univariate and multivariate data. Further, the spatial depth approach, well-established for multivariate data, is investigated for nonparametric description of functional data along these lines. In comparison with four other commonly used depth approaches for functional data, over a range of actual and simulated data sets, the spatial depth approach is seen to offer a very competitive combination of robustness, efficiency, computational ease, simplicity, and versatility. Methods for displaying sample quantile curves for a functional data set.Methods for computing and displaying confidence bands for population quantile curves for a functional data model.Formulation of double-fence functional boxplot to better identify shape outliers.Validation of the spatial depth approach for nonparametric description of functional data.

We start with a simple introduction to topological data analysis where the most popular tool is called a persistent diagram. Briefly, a persistent diagram is a multiset of points in the plane describing the persistence of topological features of a compact set when a scale parameter varies. Since statistical methods are difficult to apply directly on persistence diagrams, various alternative functional summary statistics have been suggested, but either they do not contain the full information of the persistence diagram or they are two-dimensional functions. We suggest a new functional summary statistic that is one-dimensional and hence easier to handle, and which under mild conditions contains the full information of the persistence diagram. Its usefulness is illustrated in statistical settings concerned with point clouds and brain artery trees. The appendix includes additional methods and examples, together with technical details. The R-code used for all examples is available at http://people.math.aau.dk/~christophe/Rcode.zip., 45 pages, 15 figures, 2 tables

Graphical methods are intended to be introduced in hydrology for visualizing functional data and detecting outliers as smooth curves. These proposed methods comprise of a rainbow plot for visualization of data in large amount and bivariate and functional bagplot and boxplot for detection of outliers graphically. The bagplot and boxplot are composed by using first two score series of robust principal component following Tukey’s depth and regions of highest density. These proposed methods have the tendency to produce not only the graphical display of hydrological data but also the detected outliers. These outliers are intended to be compared with outliers obtained from several other existing nongraphical methods of outlier detection in functional context so that the superiority of the proposed graphical methods for identifying outliers can be legitimated. Hence present paper aims to demonstrate that the graphical methods for detection of outliers are authentic and reliable approaches compare to those methods of outlier detection that are nongraphical.

In many cases, protein reactions with other small molecules (ligands) occur in a deeply buried active site. When studying these types of reactions, it is crucial for biochemists to examine trajectories of ligand motion. These trajectories are predicted with in-silico methods that produce large ensembles of possible trajectories. In this paper, we propose a novel approach to the interactive visual exploration and analysis of large sets of ligand trajectories, enabling the domain experts to understand protein function based on the trajectory properties. The proposed solution is composed of multiple linked 2D and 3D views, enabling the interactive exploration and filtering of trajectories in an informed way. In the workflow, we focus on the practical aspects of the interactive visual analysis specific to ligand trajectories. We adapt the small multiples principle to resolve an overly large number of trajectories into smaller chunks that are easier to analyze. We describe how drill-down techniques can be used to create and store selections of the trajectories with desired properties, enabling the comparison of multiple datasets. In appropriately designed 2D and 3D views, biochemists can either observe individual trajectories or choose to aggregate the information into a functional boxplot or density visualization. Our solution is based on a tight collaboration with the domain experts, aiming to address their needs as much as possible. The usefulness of our novel approach is demonstrated by two case studies, conducted by the collaborating protein engineers. acceptedVersion

Process capability indices measure the ability of a process to provide products that meet certain specifications. Few references deal with the capability of a process characterized by a functional relationship between a response variable and one or more explanatory variables, which is called profile. Specifically, there is not any reference analysing the capability of processes characterized by multivariate nonlinear profiles. In this paper, we propose a method to measure the capability of these processes, based on principal components for multivariate functional data and the concept of functional depth. A simulation study is conducted to assess the performance of the proposed method. An example from the sugar production illustrates the applicability of this approach.

Functional boxplot is an attractive technique to visualize data that come from functions. We propose an alternative to the functional boxplot based on depth measures. Our proposal generalizes the usual construction of the box-plot in one dimension related to the down-upward orderings of the data by considering two intuitive pre-orders in the functional context. These orderings are based on the epigraphs and hypographs of the data that allow a new definition of functional quartiles which is more robust to shape outliers. Simulated and real examples show that this proposal provides a convenient visualization technique with a great potential for analyzing functional data and illustrate its usefulness to detect outliers that other procedures do not detect.

Functional data analysis can be seriously impaired by abnormal observations, which can be classified as either magnitude or shape outliers based on their way of deviating from the bulk of data. Identifying magnitude outliers is relatively easy, while detecting shape outliers is much more challenging. We propose turning the shape outliers into magnitude outliers through data transformation and detecting them using the functional boxplot. Besides easing the detection procedure, applying several transformations sequentially provides a reasonable taxonomy for the flagged outliers. A joint functional ranking, which consists of several transformations, is also defined here. Simulation studies are carried out to evaluate the performance of the proposed method using different functional depth notions. Interesting results are obtained in several practical applications., Comment: 32 pages, 9 figures, and 3 tables

Functional Magnetic Resonance Imaging (fMRI) has become one of the leading methods for brain mapping in neuroscience and it is an important tool in modern neuroscience investigation. Moreover, the recent advances in fMRI analysis are widely used to define the default state of brain activity, functional connectivity and basal activity. Signal processing schemes have been suggested to analyze the resting state Blood-Oxygenation-Level-Dependent (BOLD) signal from simple correlations to spectral decomposition. Our goal is to determine which brain areas behave similarly in the time domain. To address this question, we apply functional curve clustering methods. We carry out an exploratory study using classical functional clustering of fMRI time series. The analysis confirms the hypothesis of a possible spatial influence on the results and therefore suggests the development of spatial curve clustering methods for brain data.

This paper presents DivClusFD, a new divisive hierarchical method for the non-supervised classification of functional data. Data of this type present the peculiarity that the differences among clusters may be caused by changes as well in level as in shape. Different clusters can be separated in different subregion and there may be no subregion in which all clusters are separated. In each step of division, the DivClusFD method explores the functions and their derivatives at several fixed points, seeking the subregion in which the highest number of clusters can be separated. The number of clusters is estimated via the gap statistic. The functions are assigned to the new clusters by combining the k-means algorithm with the use of functional boxplots to identify functions that have been incorrectly classified because of their atypical local behavior. The DivClusFD method provides the number of clusters, the classification of the observed functions into the clusters and guidelines that may be for interpreting the clusters. A simulation study using synthetic data and tests of the performance of the DivClusFD method on real data sets indicate that this method is able to classify functions accurately. Fil: Justel, Ana. Universidad Autónoma de Madrid; España. Universidad Carlos III de Madrid; España Fil: Svarc, Marcela. Universidad de San Andrés; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina

Summary We suggest a new approach, which is applicable for general statistics computed from random samples of univariate or vector-valued or functional data, to assessing the influence that individual data have on the value of a statistic, and to ranking the data in terms of that influence. Our method is based on, first, perturbing the value of the statistic by ‘tilting’, or reweighting, each data value, where the total amount of tilt is constrained to be the least possible, subject to achieving a given small perturbation of the statistic, and, then, taking the ranking of the influence of data values to be that which corresponds to ranking the changes in data weights. It is shown, both theoretically and numerically, that this ranking does not depend on the size of the perturbation, provided that the perturbation is sufficiently small. That simple result leads directly to an elegant geometric interpretation of the ranks; they are the ranks of the lengths of projections of the weights onto a ‘line’ determined by the first empirical principal component function in a generalized measure of covariance. To illustrate the generality of the method we introduce and explore it in the case of functional data, where (for example) it leads to generalized boxplots. The method has the advantage of providing an interpretable ranking that depends on the statistic under consideration. For example, the ranking of data, in terms of their influence on the value of a statistic, is different for a measure of location and for a measure of scale. This is as it should be; a ranking of data in terms of their influence should depend on the manner in which the data are used. Additionally, the ranking recognizes, rather than ignores, sign, and in particular can identify left- and right-hand ‘tails’ of the distribution of a random function or vector.

We define a new boxplot that can deal with skewed and/or heavy-tailed distributions and possible outliers. The methodology relies on a rank-preserving transformation that allows to fit a so-called Tukey g -and-h distribution.

Atlas-building from population data is widely used in medical imaging. However, the emphasis of atlas-building approaches is typically to estimate a spatial alignment to compute a mean / median shape or image based on population data. In this work, we focus on the statistical characterization of the population data, once spatial alignment has been achieved. We introduce and propose the use of the weighted functional boxplot. This allows the generalization of concepts such as the median, percentiles, or outliers to spaces where the data objects are functions, shapes, or images, and allows spatio-temporal atlas-building based on kernel regression. In our experiments, we demonstrate the utility of the approach to construct statistical atlases for pediatric upper airways and corpora callosa revealing their growth patterns. We also define a score system based on the pediatric airway atlas to quantitatively measure the severity of subglottic stenosis (SGS) in the airway. This scoring allows the classification of pre- and post-surgery SGS subjects and radiographically normal controls. Experimental results show the utility of atlas information to assess the effect of airway surgery in children.

We propose notions of simplicial band depth for multivariate functional data that extend the univariate functional band depth. The proposed simplicial band depths provide simple and natural criteria to measure the centrality of a trajectory within a sample of curves. Based on these depths, a sample of multivariate curves can be ordered from the center outward and order statistics can be defined. Properties of the proposed depths, such as invariance and consistency, can be established. A simulation study shows the robustness of this new definition of depth and the advantages of using a multivariate depth versus the marginal depths for detecting outliers. Real data examples from growth curves and signature data are used to illustrate the performance and usefulness of the proposed depths.

Ensembles of numerical simulations are used in a variety of applications, such as meteorology or computational solid mechanics, in order to quantify the uncertainty or possible error in a model or simulation. Deriving robust statistics and visualizing the variability of an ensemble is a challenging task and is usually accomplished through direct visualization of ensemble members or by providing aggregate representations such as an average or pointwise probabilities. In many cases, the interesting quantities in a simulation are not dense fields, but are sets of features that are often represented as thresholds on physical or derived quantities. In this paper, we introduce a generalization of boxplots, called contour boxplots, for visualization and exploration of ensembles of contours or level sets of functions. Conventional boxplots have been widely used as an exploratory or communicative tool for data analysis, and they typically show the median, mean, confidence intervals, and outliers of a population. The proposed contour boxplots are a generalization of functional boxplots, which build on the notion of data depth. Data depth approximates the extent to which a particular sample is centrally located within its density function. This produces a center-outward ordering that gives rise to the statistical quantities that are essential to boxplots. Here we present a generalization of functional data depth to contours and demonstrate methods for displaying the resulting boxplots for two-dimensional simulation data in weather forecasting and computational fluid dynamics.

Spatio-temporal covariances are important for describing thespatio-temporal variability of underlying random fields in geostatistical data. For second-order stationary random fields, there exist subclasses of covariance functions that assume a simpler spatio-temporal dependence structure with separability and full symmetry. However, it is challenging to visualize and assess separability and full symmetry from spatio-temporal observations. In this work, we propose a functional data analysis approach that constructs test functions using the cross-covariances from time series observed at each pair of spatial locations. These test functions of temporal lags summarize the properties of separability or symmetry for the given spatial pairs. We use functional boxplots to visualize the functional median and the variability of the test functions, where the extent of departure from zero at all temporal lags indicates the degree of non-separability or asymmetry. We also develop a rank-based nonparametric testing procedure for assessing the significance of the non-separability or asymmetry. Essentially, the proposed methods only require the analysis of temporal covariance functions. Thus, a major advantage over existing approaches is that there is no need to estimate any covariance matrix for selected spatio-temporal lags. The performances of the proposed methods are examined by simulations with various commonly used spatio-temporal covariance models. To illustrate our methods in practical applications, we apply it to real datasets, including weather station data and climate model outputs.

We propose a new notion called `extremal depth' (ED) for functional data, discuss its properties, and compare its performance with existing concepts. The proposed notion is based on a measure of extreme `outlyingness'. ED has several desirable properties that are not shared by other notions and is especially well suited for obtaining central regions of functional data and function spaces. In particular: a) the central region achieves the nominal (desired) simultaneous coverage probability; b) there is a correspondence between ED-based (simultaneous) central regions and appropriate point-wise central regions; and c) the method is resistant to certain classes of functional outliers. The paper examines the performance of ED and compares it with other depth notions. Its usefulness is demonstrated through applications to constructing central regions, functional boxplots, outlier detection, and simultaneous confidence bands in regression problems., Comment: Journal of the American Statistical Association (Theory and Methods), To appear

A boxplot is an exploratory data analysis (EDA) tool for a compact distributional summary of a data set. It is designed to captures all typical observations and displays the location, spread, skewness and the tail of the data. The precision of some of this functionality is considered to be more reliable for symmetric data type and thus less appropriate for skewed data such as the extreme data. Many observations from extreme data were erroneously marked as outliers by the Tukeys standard boxplot. We proposed a modified boxplot fence adjustment using the Bowley coefficient, a robust skewness measure. The adjustment will enable us to detect inconsistent observations without any parametric assumption about the distribution of the data. The new boxplot is capable of displaying some additional features such as the location parameter region of the Gumbel fitted extreme data. A simulated and real life data were used to show the advantages of this development over those found in the literature.

Band depth is an important nonparametric measure that generalizes order statistics and makes univariate methods based on order statistics possible for functional data. However, the computational burden of band depth limits its applicability when large functional or image datasets are considered. This paper proposes an exact fast method to speed up the band depth computation when bands are defined by two curves. Remarkable computational gains are demonstrated through simulation studies comparing our proposal with the original computation and one existing approximate method. For example, we report an experiment where our method can rank one million curves, evaluated at fifty time points each, in 12.4 seconds with Matlab. Copyright © 2012 John Wiley & Sons, Ltd.

This article proposes a simulation-based method to adjust functional boxplots for correlations when visualizing functional and spatio-temporal data, as well as detecting outliers. We start by investigating the relationship between the spatiotemporal dependence and the 1.5 times the 50% central region empirical outlier detection rule. Then, we propose to simulate observations without outliers on the basis of a robust estimator of the covariance function of the data. We select the constant factor in the functional boxplot to control the probability of correctly detecting no outliers. Finally, we apply the selected factor to the functional boxplot of the original data. As applications, the factor selection procedure and the adjusted functional boxplots are demonstrated on sea surface temperatures, spatio-temporal precipitation and general circulation model (GCM) data. The outlier detection performance is also compared before and after the factor adjustment. Copyright © 2011 John Wiley & Sons, Ltd.

The statistical analysis of functional data is a growing need in many research areas. We propose a new depth notion for functional observations based on the graphic representation of the curves. Given a collection of functions, it allows to establish the centrality of a function and provides a natural center-outward ordering of the sample curves. Robust statistics such as the median function or a trimmed mean function can be defined from this depth definition. Its finite-dimensional version provides a new depth for multivariate data that is computationally very fast and turns out to be convenient to study high-dimensional observations. The natural properties are established for the new depth and the uniform consistency of the sample depth is proved. Simulation results show that the trimmed mean presents a better behavior than the mean for contaminated models. Several real data sets are considered to illustrate this new concept of depth. Finally, we use this new depth to generalize to functions the Wilcoxon rank sum test. It allows to decide whether two groups of curves come from the same population. This functional rank test is applied to girls and boys growth curves concluding that they present different growth patterns.

Many model-based methods have been developed over the last several decades for analysis of electroencephalograms (EEG) in order to understand electrical neural data. In this work, we propose to use the functional boxplot to analyze log periodograms of EEG time series data in the spectral domain. The functional bloxplot approach produces a median curve -- which is not equivalent to connecting medians obtained from frequency-specific boxplots. In addition, this approach identifies a functional median, summarizes variability and detects potential outliers. By extending functional boxplots analysis from one-dimensional curves to surfaces, surface boxplots are also used to explore the variation of the spectral power for the alpha (8-12 Hertz) and beta (16-32 Hertz) frequency bands across the brain cortical surface. By using rank-based nonparametric tests, we also investigate the stationarity of EEG traces across an exam acquired during resting-state by comparing the spectrum during the early vs. late phases of a single resting-state EEG exam.

This research was supported by the King Abdullah University of Science and Technology (KAUST).

In this paper we introduce a micro-clustering strategy for Functional Boxplots. The aim is to summarize a set of streaming time series splitted in non overlapping windows. It is a two step strategy which performs at first, an on-line summarization by means of functional data structures, named Functional Boxplot micro-clusters; then it reveals the final summariza- tion by processing, off-line, the functional data structures. Our main contribute consists in providing a new definition of micro-cluster based on Functional Boxplots and, in defining a proximity measure which allows to compare and update them. This allows to get a finer graph- ical summarization of the streaming time series by five functional basic statistics of data. The obtained synthesis will be able to keep track of the dynamic evolution of the multiple streams.

Functional data are occurring more and more often in practice, and various statistical techniques have been developed to analyze them. In this paper we consider multivariate functional data, where for each curve and each time point a p-dimensional vector of measurements is observed. For functional data the study of outlier detection has started only recently, and was mostly limited to univariate curves (p=1). In this paper we set up a taxonomy of functional outliers, and construct new numerical and graphical techniques for the detection of outliers in multivariate functional data, with univariate curves included as a special case. Our tools include statistical depth functions and distance measures derived from them. The methods we study are affine invariant in p-dimensional space, and do not assume elliptical or any other symmetry. ispartof: Statistical Methods and Applications vol:24 issue:2 pages:177-202 status: published

The exploratory approach, developed by John W. Tukey, focuses on steps that precede any formal inferences. At its core lie four main themes: display (usually graphical), reexpression (via a mathematical transformation), resistance (to unusual behavior in data), and residuals (from subtracting a summary or fitted model). The most familiar exploratory data analysis techniques include the stem-and-leaf display, the boxplot, and median polish. The article discusses themes and techniques and illustrates their application to data in four examples.

In practice, the use of functional data is often preferable to that of large finitedimensional vectors obtained by discrete approximations of functions. In this paper a new concept of data depth is introduced for functional data. The aim is to measure the centrality of a given curve within a group of curves. This concept is used to define ranks and trimmed means for functional data. Some theoretical and practical aspects are discussed and a simulation study is given. The results show a good performance of our method, in terms of efficiency and robustness, when compared with the mean. Finally, a real-data example based on the Nasdaq 100 index is discussed.