Search

AbstractIndexes are useful for summarizing multivariate information into single metrics for monitoring, communicating, and decision-making. While most work has focused on defining new indexes for specific purposes, more attention needs to be directed towards making it possible to understand index behavior in different data conditions, and to determine how their structure affects their values and the variability therein. Here we discuss a modular data pipeline recommendation to assemble indexes. It is universally applicable to index computation and allows investigation of index behavior as part of the development procedure. One can compute indexes with different parameter choices, adjust steps in the index definition by adding, removing, and swapping them to experiment with various index designs, calculate uncertainty measures, and assess indexes’ robustness. The paper presents three examples to illustrate the usage of the pipeline framework: comparison of two different indexes designed to monitor the spatio-temporal distribution of drought in Queensland, Australia; the effect of dimension reduction choices on the Global Gender Gap Index (GGGI) on countries’ ranking; and how to calculate bootstrap confidence intervals for the Standardized Precipitation Index (SPI). The methods are supported by a new R package, called tidyindex. Supplemental materials for the article are available online. [ABSTRACT FROM AUTHOR]

AbstractRegression experts consistently recommend plotting residuals for model diagnosis, despite the availability of many numerical hypothesis test procedures designed to use residuals to assess problems with a model fit. Here we provide evidence for why this is good advice using data from a visual inference experiment. We show how conventional tests are too sensitive, which means that too often the conclusion would be that the model fit is inadequate. The experiment uses the lineup protocol which puts a residual plot in the context of null plots. This helps generate reliable and consistent reading of residual plots for better model diagnosis. It can also help in an obverse situation where a conventional test would fail to detect a problem with a model due to contaminated data. The lineup protocol also detects a range of departures from good residuals simultaneously. Supplemental materials for the article are available online. [ABSTRACT FROM AUTHOR]

Children and young people (CYP) can be empowered to take on roles as agents of change in their own communities. CYP want to be heard and should be actively involved in the co-production, design and development of services aimed at them to ensure that the resulting services are acceptable and accessible. Little analysis of the framing and discourse of co-production in different contexts has been undertaken. Building on Children's Advocacy Center models from the United States of America (which are held in high esteem by local communities), there is perceived value of such a center in the UK. A service development initiative was designed to work with children from Greater Manchester (UK) to determine the potential for the establishment of a children's advocacy center in the North of England. This report presents the design and outcome of the initiative and contributes to the literature on the co-production of such service development projects with CYP, notably the means of achieving that outcome. Recommendations are made for the piloting of an Advocacy House model in the UK with collaborative efforts between CYP as well as health, education, law enforcement, social care providers, charities and voluntary groups. A community-inclusive partnership, underpinned by the principles of co-production and co-design, is integral to the further development of this pilot. [ABSTRACT FROM AUTHOR]

Textbook data is essential for teaching statistics and data science methods because it is clean, allowing the instructor to focus on methodology. Ideally textbook datasets are refreshed regularly, especially when they are subsets taken from an ongoing data collection. It is also important to use contemporary data for teaching, to imbue the sense that the methodology is relevant today. This article describes the trials and tribulations of refreshing a textbook dataset on wages, extracted from the National Longitudinal Survey of Youth (NLSY79) in the early 1990s. The data is useful for teaching modeling and exploratory analysis of longitudinal data. Subsets of NLSY79, including the wages data, can be found in supplementary materials from numerous textbooks and research articles. The NLSY79 database has been continually updated through to 2018, so new records are available. Here we describe our journey to refresh the wages data, and document the process so that the data can be regularly updated into the future. Our journey was difficult because the steps and decisions taken to get from the raw data to the wages textbook subset have not been clearly articulated. We have been diligent to provide a reproducible workflow for others to follow, which also hopefully inspires more attempts at refreshing data for teaching. Three new datasets and the code to produce them are provided in the open source R package called yowie. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Multivariate data is often visualized using linear projections, produced by techniques such as principal component analysis, linear discriminant analysis, and projection pursuit. A problem with projections is that they obscure low and high density regions near the center of the distribution. Sections, or slices, can help to reveal them. This article develops a section pursuit method, building on the extensive work in projection pursuit, to search for interesting slices of the data. Linear projections are used to define sections of the parameter space, and to calculate interestingness by comparing the distribution of observations, inside and outside a section. By optimizing this index, it is possible to reveal features such as holes (low density) or grains (high density). The optimization is incorporated into a guided tour so that the search for structure can be dynamic. The approach can be useful for problems when data distributions depart from uniform or normal, as in visually exploring nonlinear manifolds, and functions in multivariate space. Two applications of section pursuit are shown: exploring decision boundaries from classification models, and exploring subspaces induced by complex inequality conditions from a multiple parameter model. The new methods are available in R, in the tourr package. for this article are available online. [ABSTRACT FROM AUTHOR]

In high-dimensional data analysis, the curse of dimensionality reasons that points tend to be far away from the center of the distribution and on the edge of high-dimensional space. Contrary to this, is that projected data tends to clump at the center. This gives a sense that any structure near the center of the projection is obscured, whether this is true or not. A geometric transformation to reverse the curse, is defined in this article, which uses radial transformations on the projected data. It is integrated seamlessly into the grand tour algorithm, and we have called it a burning sage tour, to indicate that it reverses the curse. The work is implemented into the tourr package in R. Several case studies are included that show how the sage visualizations enhance exploratory clustering and classification problems. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]

This study was conducted to understand the reasons parents of children with minor conditions attend the Children's Emergency Department (ED), and their views about onsite pediatric same day care (SDC) service as an alternative treatment center. The study was a cross-sectional survey of parents attending an inner-city, district general hospital children's ED, with children aged under 16 years old who were allocated to low triage categories. A convenience sample of 58 parents of 58 children were recruited. All the 58 responses were analyzed. Incomplete questionnaires were not excluded. 47% of attendances were because of minor injury. Most presentations were within 24 hours of the injury or illness. 72% of parents were employed. 91% were registered with a General Practitioner (GP). 29% contacted a GP before the ED visit. The majority of participants who contacted a GP were referred to the ED; others were advised to wait to see if the child's condition improved and to attend the ED if there were any concerns or the child deteriorated in any way. About 50% of those that did not contact GP said the GP surgery was closed and 8% felt the GP could not help. 90% of parents perceived their child's condition as urgent requiring immediate treatment. About 33% of parents said they would be happy for their children to be treated at an onsite SDC center. The study showed limited access to GP services in the community and dissatisfaction with community services and perceived urgency of treatment prompted parents of children with minor conditions to attend the ED. This could mean significant ED attendance by children with minor conditions. The majority of the parents in the study would welcome an onsite pediatric SDC if appropriate to meet their children's care needs. Establishing an onsite SDC may help relieve the ED pressures to attend to more clinically urgent and emergency cases. [ABSTRACT FROM AUTHOR]

This article presents a new ensemble learning method for classification problems called projection pursuit random forest (PPF). PPF uses the PPtree algorithm where trees are constructed by splitting on linear combinations of randomly chosen variables. Projection pursuit is used to choose a projection of the variables that best separates the classes. Using linear combinations of variables to separate classes takes the correlation between variables into account which allows PPF to outperform a traditional random forest when separations between groups occurs in combinations of variables. The method presented here can be used in multi-class problems and is implemented into an R package, PPforest, which is available on CRAN. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]

Kaggle is a data modeling competition service, where participants compete to build a model with lower predictive error than other participants. Several years ago they released a simplified service that is ideal for instructors to run competitions in a classroom setting. This article describes the results of an experiment to determine if participating in a predictive modeling competition enhances learning. The evidence suggests it does. In addition, students were surveyed to examine if the competition improved engagement and interest in the class. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Taking projections of high-dimensional data is a common analytical and visualization technique in statistics for working with high-dimensional problems. Sectioning, or slicing, through high dimensions is less common, but can be useful for visualizing data with concavities, or nonlinear structure. It is associated with conditional distributions in statistics, and also linked brushing between plots in interactive data visualization. This short technical note describes a simple approach for slicing in the orthogonal space of projections obtained when running a tour, thus presenting the viewer with an interpolated sequence of sliced projections. The method has been implemented in R as an extension to the tourr package, and can be used to explore for concave and nonlinear structures in multivariate distributions. for this article are available online. [ABSTRACT FROM AUTHOR]

Calendars are broadly used in society to display temporal information and events. This article describes a new calendar display for plotting data that include a layout algorithm with many options, and faceting functionality. The functions use modular arithmetic on the date variable to restructure the data into a calendar format. The user can apply the grammar of graphics to create plots inside each calendar cell, and thus the displays synchronize neatly with ggplot2 graphics. The motivating application is studying pedestrian behavior in Melbourne, Australia, based on counts which are captured at hourly intervals by sensors scattered around the city. Faceting by the usual features such as day and month is insufficient to examine the behavior. Making displays on a monthly calendar format helps to understand pedestrian patterns relative to events such as work days, weekends, holidays, and special events. The functions for the calendar algorithm are available in the R package sugrrants. for this article are available online. [ABSTRACT FROM AUTHOR]

Mining temporal data for information is often inhibited by a multitude of formats: regular or irregular time intervals, point events that need aggregating, multiple observational units or repeated measurements on multiple individuals, and heterogeneous data types. This work presents a cohesive and conceptual framework for organizing and manipulating temporal data, which in turn flows into visualization, modeling, and forecasting routines. Tidy data principles are extended to temporal data by: (1) mapping the semantics of a dataset into its physical layout; (2) including an explicitly declared "index" variable representing time; (3) incorporating a "key" comprising single or multiple variables to uniquely identify units over time. This tidy data representation most naturally supports thinking of operations on the data as building blocks, forming part of a "data pipeline" in time-based contexts. A sound data pipeline facilitates a fluent workflow for analyzing temporal data. The infrastructure of tidy temporal data has been implemented in the R package, called tsibble. for this article are available online. [ABSTRACT FROM AUTHOR]

Graphics play a crucial role in statistical analysis and data mining. Being able to quantify structure in data that is visible in plots, and how people read the structure from plots is an ongoing challenge. The lineup protocol provides a formal framework for data plots, making inference possible. The data plot is treated like a test statistic, and lineup protocol acts like a comparison with the sampling distribution of the nulls. This article describes metrics for describing structure in data plots and evaluates them in relation to the choices that human readers made during several large Amazon Turk studies using lineups. The metrics that were more specific to the plot types tended to better match subject choices, than generic metrics. The process that we followed to evaluate metrics will be useful for general development of numerically measuring structure in plots, and also in future experiments on lineups for choosing blocks of pictures. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Temporal data are information measured in the context of time. This contextual structure provides components that need to be explored to understand the data and that can form the basis of interactions applied to the plots. In multivariate time series, we expect to see temporal dependence, long term and seasonal trends, and cross-correlations. In longitudinal data, we also expect within and between subject dependence. Time series and longitudinal data, although analyzed differently, are often plotted using similar displays. We provide a taxonomy of interactions on plots that can enable exploring temporal components of these data types, and describe how to build these interactions using data transformations. Because temporal data are often accompanied other types of data we also describe how to link the temporal plots with other displays of data. The ideas are conceptualized into a data pipeline for temporal data and implemented into the R packagecranvas. This package provides many different types of interactive graphics that can be used together to explore data or diagnose a model fit. [ABSTRACT FROM AUTHOR]

This article describes new user controls for examining high-dimensional data using low-dimensional linear projections and slices. A user can interactively change the contribution of a given variable to a low-dimensional projection, which is useful for exploring the sensitivity of structure to particular variables. The user can also interactively shift the center of a slice, for example, to explore how structure changes in local subspaces. The Mathematica package as well as example notebooks are provided, which contain functions enabling the user to experiment with these new manual controls, with one specifically for exploring regions and boundaries produced by classification models. The advantage of Mathematica is its linear algebra capabilities and interactive cursor location controls. Some limited implementation has also been made available in the R package tourr. for this article are available online. [ABSTRACT FROM AUTHOR]

In high-dimensional data, one often seeks a few interesting low-dimensional projections that reveal important features of the data. Projection pursuit is a procedure for searching high-dimensional data for interesting low-dimensional projections via the optimization of a criterion function called the projection pursuit index. Very few projection pursuit indices incorporate class or group information in the calculation. Hence, they cannot be adequately applied in supervised classification problems to provide low-dimensional projections revealing class differences in the data. This article introduces new indices derived from linear discriminant analysis that can be used for exploratory supervised classification. [ABSTRACT FROM AUTHOR]

This article presents aspects of the implementation of a bidirectional link between the Geographic Information System (GIS) ArcView[TM] and the interactive dynamic statistical graphics program XGobi. We describe the main functionality of the link, the underlying remote procedure call (RPC) mechanism, and internal data structures, and discuss topics such as security, concurrency, and linked brushing. We think that these topics are of particular interest to software authors intending to link similar software packages, and software users learning about strengths (and weaknesses) of the implementation of our link. [ABSTRACT FROM AUTHOR]

Discusses how to manually control visualization methods for high-dimensional projections. Functionality for controlling one and two-dimensional projections; Constrained manipulations; Applications of interactive manipulation tools to projection pursuit.

Proposes a rudimentary taxonomy of interactive data visualization based on finding Gestalt, posing queries and making comparisons. Interactive view manipulations; XGobi system for multivariate data visualization; Powers and limitations of particular focusing, linking and arranging tools; Illustration of asymptotic use of high-dimensional cubes for visualizing factorial experiments.

Discusses how two graphical methods for exploring high dimensional data--the `grand tour' dynamic model and the `projection pursuit' static model--complement each other for exploratory data analysis. Optimization adaptation of tour meovement; Monitoring window; Bitmap interface; Navigational tools; Index choices; Impact of sphering; Inclusion of user-defined index functions; Linked tours.

Deconstructing a time index into time granularities can assist in exploration and automated analysis of large temporal datasets. This article describes classes of time deconstructions using linear and cyclic time granularities. Linear granularities respect the linear progression of time such as hours, days, weeks and months. Cyclic granularities can be circular such as hour-of-the-day, quasi-circular such as day-of-the-month, and aperiodic such as public holidays. The hierarchical structure of granularities creates a nested ordering: hour-of-the-day and second-of-the-minute are single-order-up. Hour-of-the-week is multiple-order-up, because it passes over day-of-the-week. Methods are provided for creating all possible granularities for a time index. A recommendation algorithm provides an indication whether a pair of granularities can be meaningfully examined together (a "harmony"), or when they cannot (a "clash"). Time granularities can be used to create data visualizations to explore for periodicities, associations and anomalies. The granularities form categorical variables (ordered or unordered) which induce groupings of the observations. Assuming a numeric response variable, the resulting graphics are then displays of distributions compared across combinations of categorical variables. The methods implemented in the open source R package gravitas are consistent with a tidy workflow, with probability distributions examined using the range of graphics available in ggplot2. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractThe usual goal of supervised learning is to find the best model, the one that optimizes a particular performance measure. However, what if the explanation provided by this model is completely different from another model and different again from another model despite all having similarly good fit statistics? Is it possible that the equally effective models put the spotlight on different relationships in the data? Inspired by Anscombe’s quartet, this article introduces a Rashomon Quartet, that is a set of four models built on a synthetic dataset which have practically identical predictive performance. However, the visual exploration reveals distinct explanations of the relations in the data. This illustrative example aims to encourage the use of methods for model visualization to compare predictive models beyond their performance. [ABSTRACT FROM AUTHOR]

The author presents his view on Leland Wilkinson's article "The Future of Statistical Computing" and discusses the impact of computing on statistics and vice versa. He states that statistics is a foundation for many areas of computing, and driven by these needs, much better statistical software are needed. He relates that robust libraries such as those developed by the numerical community needed in the future and statistics will be used as a service, not as a tool for experts. He mentions that the disseminated adoption of statistical computing platforms are going to alter the role of the statistician.

The complexity of linear mixed-effects (LME) models means that traditional diagnostics are rendered less effective. This is due to a breakdown of asymptotic results, boundary issues, and visible patterns in residual plots that are introduced by the model fitting process. Some of these issues are well known and adjustments have been proposed. Working with LME models typically requires that the analyst keeps track of all the special circumstances that may arise. In this article, we illustrate a simpler but generally applicable approach to diagnosing LME models. We explain how to use new visual inference methods for these purposes. The approach provides a unified framework for diagnosing LME fits and for model selection. We illustrate the use of this approach on several commonly available datasets. A large-scale Amazon Turk study was used to validate the methods. R code is provided for the analyses. Supplementary materials for this article are available online. [ABSTRACT FROM PUBLISHER]

Statistical graphics play a crucial role in exploratory data analysis, model checking, and diagnosis. The lineup protocol enables statistical significance testing of visual findings, bridging the gulf between exploratory and inferential statistics. In this article, inferential methods for statistical graphics are developed further by refining the terminology of visual inference and framing the lineup protocol in a context that allows direct comparison with conventional tests in scenarios when a conventional test exists. This framework is used to compare the performance of the lineup protocol against conventional statistical testing in the scenario of fitting linear models. A human subjects experiment is conducted using simulated data to provide controlled conditions. Results suggest that the lineup protocol performs comparably with the conventional tests, and expectedly outperforms them when data are contaminated, a scenario where assumptions required for performing a conventional test are violated. Surprisingly, visual tests have higher power than the conventional tests when the effect size is large. And, interestingly, there may be some super-visual individuals who yield better performance and power than the conventional test even in the most difficult tasks. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

This article develops a generalization of thescatterplot matrixbased on the recognition that most datasets include both categorical and quantitative information. Traditional grids of scatterplots often obscure important features of the data when one or more variables are categorical but coded as numerical. Thegeneralized pairs plotoffers a range of displays of paired combinations of categorical and quantitative variables. A mosaic plot, fluctuation diagram, or faceted bar chart may be used to display two categorical variables. A side-by-side boxplot, stripplot, faceted histogram, or density plot helps visualize a categorical and a quantitative variable. A traditional scatterplot is suitable for displaying a pair of numerical variables, but options also support density contours or annotating summary statistics such as the correlation and number of missing values, for example. By combining these, the generalized pairs plot may help to reveal structure in multivariate data that otherwise might go unnoticed in the process of exploratory data analysis. Two different R packages provide implementations of the generalized pairs plot,gpairsandGGally. Supplementary materials for this article are available online on the journal web site. [ABSTRACT FROM PUBLISHER]

This article describes constructing interactive and dynamic linked data views using the Java programming language. The data views are designed for data that have a multivariate component. The approach to displaying data comes from earlier research on building statistical graphics based on data pipelines, in which different aspects of data processing and graphical rendering are organized conceptually into segments of a pipeline. The software design takes advantage of the object-oriented nature of the Java language to open up the data pipeline, allowing developers to have greater control over their visualization applications. Importantly, new types of data views coded to adhere to a few simple design requirements can easily be integrated with existing pipe sections. This allows access to sophisticated linking and dynamic interaction across all (new and existing) view types. Pipe segments can be accessed from data analysis packages such as Omegahat or R, providing a tight coupling of visual and numerical methods. [ABSTRACT FROM AUTHOR]

The authors discuss the supplementary materials submitted for the "Journal of Computational and Graphical Statistics" (JCGS). According to the authors, supplemental materials, which include animated images, datasets or software, help readers to easily analyse a certain study and data from printed articles. These materials can also encourage more research on dynamic graphics. Meanwhile, the authors encourage other writers to provide research with supplementary materials.