Search

We investigate uncertainty guidance mechanisms to support proton therapy (PT) planning visualization. Uncertainties in the PT workflow pose significant challenges for navigating treatment plan data and selecting the most optimal plan among alternatives. Although guidance techniques have not yet been applied to PT planning scenarios, they have successfully supported sense- and decision-making processes in other contexts. We hypothesize that augmenting PT uncertainty visualization with guidance may influence the intended users’ perceived confidence and provide new insights. To this end, we follow an iterative co-design process with domain experts to develop a visualization dashboard enhanced with distinct level-of-detail uncertainty guidance mechanisms. Our approach classifies uncertainty guidance into two dimensions: degree of intrusiveness and detail-orientation. Our dashboard supports the comparison of multiple treatment plans (i.e., nominal plans with their translational variations) while accounting for multiple uncertainty factors. We subsequently evaluate the designed and developed strategies by assessing perceived confidence and effectiveness during a sense- and decision-making process. Our findings indicate that uncertainty guidance in PT planning visualization does not necessarily impact the perceived confidence of the users in the process. Nonetheless, it provides new insights and raises uncertainty awareness during treatment plan selection. This observation was particularly evident for users with longer experience in PT planning.

This research studies the occurrence of data breaches in healthcare provider settings regarding patient data. Using visual analytics and data visualization tools, we study the distribution of healthcare breaches by state. We review the main causes and types of breaches, as well as their impact on both providers and patients. The research shows a range of data breach victims. Network servers are the most popular location for common breaches, such as hacking and information technology (IT) incidents, unauthorized access, theft, loss, and improper disposal. We offer proactive recommendations to prepare for a breach. These include, but are not limited to, regulatory compliance, implementing policies and procedures, and monitoring network servers. Unfortunately, the results indicate that the probability of data breaches will continue to rise.

We present RCMVis, a visual analytics system to support interactive Route Choice Modeling analysis. It aims to model which characteristics of routes, such as distance and the number of traffic lights, affect travelers' route choice behaviors and how much they affect the choice during their trips. Through close collaboration with domain experts, we designed a visual analytics framework for Route Choice Modeling. The framework supports three interactive analysis stages: exploration, modeling, and reasoning. In the exploration stage, we help analysts interactively explore trip data from multiple origin-destination (OD) pairs and choose a subset of data they want to focus on. To this end, we provide coordinated multiple OD views with different foci that allow analysts to inspect, rank, and compare OD pairs in terms of their multidimensional attributes. In the modeling stage, we integrate a k-medoids clustering method and a path-size logit model into our system to enable analysts to model route choice behaviors from trips with support for feature selection, hyperparameter tuning, and model comparison. Finally, in the reasoning stage, we help analysts rationalize and refine the model by selectively inspecting the trips that strongly support the modeling result. For evaluation, we conducted a case study and interviews with domain experts. The domain experts discovered unexpected insights from numerous modeling results, allowing them to explore the hyperparameter space more effectively to gain better results. In addition, they gained OD- and road-level insights into which data mainly supported the modeling result, enabling further discussion of the model.

The task of developing effective text information classification systems requires the thoughtful analysis and synthesis of variable components of technology. These components strongly affect the practical efficiency and the requirements to the data. For this purpose, a typical technology was discussed, comparing the regular “learning from features” approach versus the more advanced “deep learning” approach, that studies from data. In order to implement the technology, the first approach was tested, which included the means (methods, algorithms) for analysis of the features of the source text, by applying the dimensionality transformation, and building model solutions that allow the correct classification of data by a set of features. As a result, all the steps of the technology are described, which allowed to determine the way of presenting data in terms of hidden features in data, their presentation in a standard visual form and evaluate the solution, as well as its practical efficiency, based on this set of features. In a depth study, the informational core of the document was studied, using the regression and T-stochastic grouping of features for dimensionality reduction.The separate results contain estimation of practical efficiency of the algorithms in terms of time and relative performance for each step of the proposed technology. This estimation gives a possibility to obtain the best algorithm of intelligent data processing that is useful for a given dataset and application. In order to estimate the best suited algorithm for separation in reduced dimension an experiment was carried out which allowed the selection of the best range of data classification algorithms, in particular boosting methods. As a result of the analysis of the technology, the necessary steps of this technology were discussed and the classification on real text data was conducted, which allowed to identify the most important stages of the technology for text classification., Розглянуто задачу якісного аналізу процесу перетворення текстової інформації на набір ознак і відповідно перетворення цих ознак на набір, зручний для візуального аналізу. Розглянуто структуру типової технології з аналізу текстової інформації та визначено її основні елементи. Детально наведено опис кожного елементу технології аналізу та класифікації текстової інформації із залученням методів класифікації та групування ознак. Проведено експериментальні випробування окремих компонентів цієї технології.

Varied edge bundling methods have been used to reduce visual clutter in parallel coordinates plots (PCP). However, existing edge-bundled PCP do not scale well for visual analysis of multidimensional big data and often overplot the bundles in the area near the axes. In this study, we propose a scalable lightweight bundling method to support visual analysis of multidimensional big data in PCP. It helps the users discover trends and detect outliers in the data by bundling the edges between each two adjacent axes independently. We integrate human judgments into the two-dimensional data binning by novel interactions to accelerate the clustering process of the data. We use the frequency-based representation to render the clusters as histogram-like bundles to reveal the distribution of the data and eliminate the overplotting of the bundles. Based on our method, we build a lightweight web-based visual analytics system for exploring multidimensional big data in PCP. The scalability analysis of our method shows that its clustering time increases linearly with the size of the data. Its rendering time is independent of the size of the data. We conduct two case studies and a user study to compare our method with classic PCP and two state-of-the-art edge-bundled PCP.

To improve the transportation efficiency of advanced manufacturing, cameras have been extensively deployed to enhance the on-road perception of autonomous ground vehicles in smart industrial parks. Considering the informative yet substantial data volume of contents generated by those cameras, we employ vehicle-to-everything links to deliver the captured video frames to neighboring vehicles, road-side units, or base stations, in order to respond to vehicle-control-related video queries. To help vehicles obtain low-latency and high-accuracy on-road information for autonomous driving, an optimization problem is formulated, taking into account the impact of vehicle mobility and diverse resource demands of different video queries. Then, a two-stage algorithm is proposed to determine the frame rate of video cameras, as well as the destination of the associated video frames based on matching theory. Extensive simulation results show that this approach can improve the accuracy by up to 18% and reduce the response delay by 13.2%.

Projection techniques are often used to visualize high-dimensional data, allowing users to better understand the overall structure of multi-dimensional spaces on a 2D screen. Although many such methods exist, comparably little work has been done on generalizable methods of inverse-projection -- the process of mapping the projected points, or more generally, the projection space back to the original high-dimensional space. In this paper we present NNInv, a deep learning technique with the ability to approximate the inverse of any projection or mapping. NNInv learns to reconstruct high-dimensional data from any arbitrary point on a 2D projection space, giving users the ability to interact with the learned high-dimensional representation in a visual analytics system. We provide an analysis of the parameter space of NNInv, and offer guidance in selecting these parameters. We extend validation of the effectiveness of NNInv through a series of quantitative and qualitative analyses. We then demonstrate the method's utility by applying it to three visualization tasks: interactive instance interpolation, classifier agreement, and gradient visualization.

Bagging and boosting are two popular ensemble methods in machine learning (ML) that produce many individual decision trees. Due to the inherent ensemble characteristic of these methods, they typically outperform single decision trees or other ML models in predictive performance. However, numerous decision paths are generated for each decision tree, increasing the overall complexity of the model and hindering its use in domains that require trustworthy and explainable decisions, such as finance, social care, and health care. Thus, the interpretability of bagging and boosting algorithms, such as random forest and adaptive boosting, reduces as the number of decisions rises. In this paper, we propose a visual analytics tool that aims to assist users in extracting decisions from such ML models via a thorough visual inspection workflow that includes selecting a set of robust and diverse models (originating from different ensemble learning algorithms), choosing important features according to their global contribution, and deciding which decisions are essential for global explanation (or locally, for specific cases). The outcome is a final decision based on the class agreement of several models and the explored manual decisions exported by users. We evaluated the applicability and effectiveness of VisRuler via a use case, a usage scenario, and a user study. The evaluation revealed that most users managed to successfully use our system to explore decision rules visually, performing the proposed tasks and answering the given questions in a satisfying way., Comment: This manuscript is accepted for publication in the Information Visualization (IV) - SAGE Journals

This study aims to demonstrate how interactions between a Visual Analytics (VA) application and students shape an interactive and multimodal reading practice. VA is a technology offering support with analysing vast amounts of data through visualisations. Such information-rich interactive interfaces provide possibilities for students to gain insights, find correlations, and draw conclusions, but they also generate complexities concerning how to ‘read’ multimodal information on a screen. Inspired by Design-Based Research, interventions were designed and conducted in five social science secondary classrooms. The interactions between the VA application Statistics eXplorer and the students were video captured. A socio-material semiotic approach guides the analyses of how interactions between all actors (the interactive visualisations, the written text, the teachers, students, etc.) produce a reading network. The results show a reading characterised by being performative, collaborative, and dynamic. A combination of visuals and text supports the reading. However, visuals such as colour, highlighting and movement dominantly attract students’ attention, while written text often becomes subordinate and sometimes even ‘invisible’. Hence, this paper argues that it is vital for teachers to didactically support students’ visual reading skills.

The COVID-19 pandemic and its dramatic worldwide impact has required global multidisciplinary actions to mitigate its effects. Mobile phone activity-based digital contact tracing (DCT) via Bluetooth low energy technology has been considered a powerful pandemic monitoring tool, yet it sparked a controversial debate about privacy risks for people. In order to explore the potential benefits of a DCT system in the context of occupational risk prevention, this article presents the potential of visual analytics methods to summarize and extract relevant information from complex DCT data collected during a long-term experiment at our research center. Visual tools were combined with quantitative metrics to provide insights into contact patterns among volunteers. Results showed that crucial actors, such as participants acting as bridges between groups could be easily identified—ultimately allowing for making more informed management decisions aimed at containing the potential spread of a disease. This research work has been carried out within the context of the RAPIDm initiative, fostered by the Basque Government as part of the fast reaction program (PRAP Euskadi, led by SPRI—the entity of the Economic Development, Sustainability, and Environment Department of the Basque Government for promoting the Basque industry) with the aim to boost the Basque industrial sector by maintaining the productive activity in the context of the threat of the COVID-19 pandemic. Three research centers of BRTAn (Basque Research and Technology Alliance) have collaborated in this R&D initiative: Tecnalia, Ikerlan, and Vicomtech. Among the different research lines carried out in the RAPID initiative, Vicomtech has been responsible for the centralized BLE-based DCT system and visual analytics of the obtained data which has been selected as one of the representative cases by the OECDo of pandemic reaction report.

Dimensionality reduction is commonly used for identifying and analyzing patterns in the visual analysis of multi-dimensional datasets. The selection of subspaces is a core building block in projecting high-dimensional data to low-dimensional space, which is usually illustrated as a scatterplot for analysts to easily understand and explore. This process involves human prior knowledge and domain-specific requirements. Thus, quantifying and tracking the changes of projections across subspaces remain challenging. Existing methods can neither quantify the subsets-based changes of dimensionality reduction results when switching subspaces nor automatically and comprehensively display overall and subtle differences among dimensionality reduction results.To address this, we developed EvoSets, a novel visual analytics system designed to help users understand how subspaces affect dimensionality reduction results. The effects are quantified based on subsets' distribution to tracking the sensitivity of dimensionality reduction results across subspaces. System supports the exploration of the overall evolution of the dimensionality reduction results for helping users track the convergence and divergence behavior changes of subsets based on an extended Bubble Sets visualization. Similarities are intuitively illustrated, and dissimilarities are highlighted among projections based on layout constraints. The usefulness and effectiveness are evaluated with a user study and two case studies on multi-dimensional datasets.

The analysis of variance in complex text traditions is an arduous task when carried out manually. Text alignment algorithms provide domain experts with a robust alternative to such repetitive tasks. Existing white-box approaches allow the digital humanities to establish syntax-based metrics taking into account the spelling, morphology and order of words. However, they produce limited results, as semantic meanings are typically not taken into account. Our interdisciplinary collaboration between visualization and digital humanities combined a semi-supervised text alignment approach based on word embeddings that take not only syntactic but also semantic text features into account, thereby improving the overall quality of the alignment. In our collaboration, we developed different visual interfaces that communicate the word distribution in high-dimensional vector space generated by the underlying neural network for increased transparency, assessment of the tool's reliability and overall improved hypothesis generation. We further offer visual means to enable the expert reader to feed domain knowledge into the system at multiple levels with the aim of improving both the product and the process of text alignment. This ultimately illustrates how visualization can engage with and augment complex modes of reading in the humanities.

Anomaly detection is a common analytical task that aims to identify rare cases that differ from the typical cases that make up the majority of a dataset. When analyzing event sequence data, the task of anomaly detection can be complex because the sequential and temporal nature of such data results in diverse definitions and flexible forms of anomalies. This, in turn, increases the difficulty in interpreting detected anomalies. In this article, we propose a visual analytic approach for detecting anomalous sequences in an event sequence dataset via an unsupervised anomaly detection algorithm based on Variational AutoEncoders. We further compare the anomalous sequences with their reconstructions and with the normal sequences through a sequence matching algorithm to identify event anomalies. A visual analytics system is developed to support interactive exploration and interpretations of anomalies through novel visualization designs that facilitate the comparison between anomalous sequences and normal sequences. Finally, we quantitatively evaluate the performance of our anomaly detection algorithm, demonstrate the effectiveness of our system through case studies, and report feedback collected from study participants.

Genomic research emerges from collaborative work within and across different scientific disciplines. A diverse range of visualisation techniques has been employed to aid this research, yet relatively little is known as to how these techniques facilitate collaboration. We conducted a case study of collaborative research within a biomedical institute to learn more about the role visualisation plays in genomic mapping. Interviews were conducted with molecular biologists (N = 5) and bioinformaticians (N = 6). We found that genomic research comprises a variety of distinct disciplines engaged in complex analytic tasks that each resist simplification, and their complexity influences how visualisations were used. Visualisation use was impacted by group-specific interactions and temporal work patterns. Visualisations were also crucial to the scientific workflow, used for both question formation and confirmation of hypotheses, and acted as an anchor for the communication of ideas and discussion. In the latter case, two approaches were taken: providing collaborators with either interactive or static imagery representing a viewpoint. The use of generic software for simplified visualisations, and quick production and curation was also noted. We discuss these findings with reference to group-specific interactions and present recommendations for improving collaborative practices through visual analytics.

As a decentralized training approach, horizontal federated learning (HFL) enables distributed clients to collaboratively learn a machine learning model while keeping personal/private information on local devices. Despite the enhanced performance and efficiency of HFL over local training, clues for inspecting the behaviors of the participating clients and the federated model are usually lacking due to the privacy-preserving nature of HFL. Consequently, the users can only conduct a shallow-level analysis of potential abnormal behaviors and have limited means to assess the contributions of individual clients and implement the necessary intervention. Visualization techniques have been introduced to facilitate the HFL process inspection, usually by providing model metrics and evaluation results as a dashboard representation. Although the existing visualization methods allow a simple examination of the HFL model performance, they cannot support the intensive exploration of the HFL process. In this article, strictly following the HFL privacy-preserving protocol, we design an exploratory visual analytics system for the HFL process termed HFLens, which supports comparative visual interpretation at the overview, communication round, and client instance levels. Specifically, the proposed system facilitates the investigation of the overall process involving all clients, the correlation analysis of clients' information in one or different communication round(s), the identification of potential anomalies, and the contribution assessment of each HFL client. Two case studies confirm the efficacy of our system. Experts' feedback suggests that our approach indeed helps in understanding and diagnosing the HFL process better.

Extracting and analyzing crime patterns in big cities is a challenging spatiotemporal problem. The hardness of the problem is linked to two main factors, the sparse nature of the crime activity and its spread in large spatial areas. Sparseness hampers most time series (crime time series) comparison methods from working properly, while the handling of large urban areas tends to render the computational costs of such methods impractical. Visualizing different patterns hidden in crime time series data is another issue in this context, mainly due to the number of patterns that can show up in the time series analysis. In this article, we present a new methodology to deal with the issues above, enabling the analysis of spatiotemporal crime patterns in a street-level of detail. Our approach is made up of two main components designed to handle the spatial sparsity and spreading of crimes in large areas of the city. The first component relies on a stochastic mechanism from which one can visually analyze probable×intensive crime hotspots. Such analysis reveals important patterns that can not be observed in the typical intensity-based hotspot visualization. The second component builds upon a deep learning mechanism to embed crime time series in Cartesian space. From the embedding, one can identify spatial locations where the crime time series have similar behavior. The two components have been integrated into a web-based analytical tool called CriPAV (Crime Pattern Analysis and Visualization), which enables global as well as a street-level view of crime patterns. Developed in close collaboration with domain experts, CriPAV has been validated through a set of case studies with real crime data in São Paulo - Brazil. The provided experiments and case studies reveal the effectiveness of CriPAV in identifying patterns such as locations where crimes are not intense but highly probable to occur as well as locations that are far apart from each other but bear similar crime patterns.

Visually identifying effective bio-markers from human brain networks poses non-trivial challenges to the field of data visualization and analysis. Existing methods in the literature and neuroscience practice are generally limited to the study of individual connectivity features in the brain (e.g., the strength of neural connection among brain regions). Pairwise comparisons between contrasting subject groups (e.g., the diseased and the healthy controls) are normally performed. The underlying neuroimaging and brain network construction process is assumed to have 100% fidelity. Yet, real-world user requirements on brain network visual comparison lean against these assumptions. In this work, we present MV^2Net, a visual analytics system that tightly integrates multi-variate multi-view visualization for brain network comparison with an interactive wrangling mechanism to deal with data uncertainty. On the analysis side, the system integrates multiple extraction methods on diffusion and geometric connectivity features of brain networks, an anomaly detection algorithm for data quality assessment, single- and multi-connection feature selection methods for bio-marker detection. On the visualization side, novel designs are introduced which optimize network comparisons among contrasting subject groups and related connectivity features. Our design provides level-of-detail comparisons, from juxtaposed and explicit-coding views for subject group comparisons, to high-order composite view for correlation of network comparisons, and to fiber tract detail view for voxel-level comparisons. The proposed techniques are inspired and evaluated in expert studies, as well as through case analyses on diffusion and geometric bio-markers of certain neurology diseases. Results in these experiments demonstrate the effectiveness and superiority of MV^2Net over state-of-the-art approaches.

Deep reinforcement learning (DRL) targets to train an autonomous agent to interact with a pre-defined environment and strives to achieve specific goals through deep neural networks (DNN). Recurrent neural network (RNN) based DRL has demonstrated superior performance, as RNNs can effectively capture the temporal evolution of the environment and respond with proper agent actions. However, apart from the outstanding performance, little is known about how RNNs understand the environment internally and what has been memorized over time. Revealing these details is extremely important for deep learning experts to understand and improve DRLs, which in contrast, is also challenging due to the complicated data transformations inside these models. In this article, we propose Deep Reinforcement Learning Interactive Visual Explorer (DRLIVE), a visual analytics system to effectively explore, interpret, and diagnose RNN-based DRLs. Having focused on DRL agents trained for different Atari games, DRLIVE accomplishes three tasks: game episode exploration, RNN hidden/cell state examination, and interactive model perturbation. Using the system, one can flexibly explore a DRL agent through interactive visualizations, discover interpretable RNN cells by prioritizing RNN hidden/cell states with a set of metrics, and further diagnose the DRL model by interactively perturbing its inputs. Through concrete studies with multiple deep learning experts, we validated the efficacy of DRLIVE.

This paper presents a hybrid prediction and search approach (HPS) for building visualization systems of big data. The basic idea is training a regression model to predict a coarse range on the dataset and then searching target records that satisfy the query conditions within the range. The prediction reduces the storage cost without preprocessing a data structure storing aggregate values of queriable attribute range combinations. Meanwhile, the search eliminates the prediction bias inevitable for machine learning models. Experiments on multiple open datasets demonstrate HPS’s comparable query speed to existing techniques and the potential of continuous performance improvement by investing more hardware resources. In addition, the feature of returning original records instead of aggregate values brings better use flexibility, enabling to construct visualization systems with display/query functions that are unavailable for existing techniques.

The increased availability of quantitative historical datasets has provided new research opportunities for multiple disciplines in social science. In this article, we work closely with the constructors of a new dataset, CGED-Q (China Government Employee Database-Qing), that records the career trajectories of over 340,000 government officials in the Qing bureaucracy in China from 1760 to 1912. We use these data to study career mobility from a historical perspective and understand social mobility and inequality. However, existing statistical approaches are inadequate for analyzing career mobility in this historical dataset with its fine-grained attributes and long time span, since they are mostly hypothesis-driven and require substantial effort. We propose CareerLens, an interactive visual analytics system for assisting experts in exploring, understanding, and reasoning from historical career data. With CareerLens, experts examine mobility patterns in three levels-of-detail, namely, the macro-level providing a summary of overall mobility, the meso-level extracting latent group mobility patterns, and the micro-level revealing social relationships of individuals. We demonstrate the effectiveness and usability of CareerLens through two case studies and receive encouraging feedback from follow-up interviews with domain experts.

Comparing text documents is an essential task for a variety of applications within diverse research fields, and several different methods have been developed for this. However, calculating text similarity is an ambiguous and context-dependent task, so many open challenges still exist. In this paper, we present a novel method for text similarity calculations based on the combination of embedding technology and ensemble methods. By using several embeddings, instead of only one, we show that it is possible to achieve higher quality, which in turn is a key factor for developing high-performing applications for text similarity exploitation. We also provide a prototype visual analytics tool which helps the analyst to find optimal performing ensembles and gain insights to the inner workings of the similarity calculations. Furthermore, we discuss the generalizability of our key ideas to fields beyond the scope of text analysis. Funding Agencies|ELLIIT environment for strategic research in Sweden

In this article, we discuss how Visualization (VIS) with Machine Learning (ML) could mutually benefit from each other. We do so through the lens of our own experience working at this intersection for the last decade. Particularly we focus on describing how VIS supports explaining ML models and aids ML-based Dimensionality Reduction techniques in solving tasks such as parameter space analysis. In the other direction, we discuss approaches showing how ML helps improve VIS, such as applying ML-based automation to improve visualization design. Based on the examples and our own perspective, we describe a number of open research challenges that we frequently encountered in our endeavors to combine ML and VIS.

With the rise of large-scale environmental models comes new challenges for how we best utilize this information in research, management and decision making. Interactive data visualizations can make large and complex datasets easier to access and explore, which can lead to knowledge discovery, hypothesis formation and improved understanding. Here, we present a web-based interactive data visualization framework, the Interactive Catchment Explorer (ICE), for exploring environmental datasets and model outputs. Using a client-based architecture, the ICE framework provides a highly interactive user experience for discovering spatial patterns, evaluating relationships between variables and identifying specific locations using multivariate criteria.

International audience; We present a novel, web-based visual eye-tracking analytics tool called Gazealytics. Our open-source toolkit features a unified combination of gaze analytics features that support flexible exploratory analysis, along with annotation of areas of interest (AOI) and filter options based on multiple criteria to visually analyse eye tracking data across time and space. Gazealytics features coordinated views unifying spatiotemporal exploration of fixations and scanpaths for various analytical tasks. A novel matrix representation allows analysis of relationships between such spatial or temporal features. Data can be grouped across samples, user-defined AOIs or time windows of interest (TWIs) to support aggregate or filtered analysis of gaze activity. This approach exceeds the capabilities of existing systems by supporting flexible comparison between and within subjects, hypothesis generation, data analysis and communication of insights. We demonstrate in a walkthrough that Gazealytics supports multiple types of eye tracking datasets and analytical tasks.

This study examines how scientists respond to visual design in scientific communications. Specifically, we determine the impact of visual design on the Graphical Abstract (GA), an overview figure that attracts potential readers and visually summarizes what a paper is about. We show that GAs designed in accordance with classic visual design principles significantly enhance readers’ first impressions of a paper. Well-designed GAs make papers seem more interesting, more clearly written and more scientifically rigorous. These results confirm that visual design enhances rather than detracts from the perception of intellectual and scientific competence.

As cultural institutions are digitizing their artifacts and interlinking their collections, new opportunities emerge to engage with cultural heritage. However, it is the often comprehensive and complex nature of collections that can make it difficult to grasp their distribution and extent across a variety of dimensions. After a brief introduction to the research area of collection visualizations, this paper presents a design study visualizing an aggregated collection from diverse cultural institutions in Germany. We detail our iterative design process leading to prototypical implementations of four stylistically and functionally coordinated visualizations, each one focusing on different facets of the collection.

This deliverable explains in detail two open source alternatives of analytics dashboards that have been developed and implemented as part of PLATOON project. The document contains the complementary information for the actual developed open source visualisation dashboards publicly available in the PLATOON GitHub repository.

Random Forests (RFs) are a machine learning (ML) technique widely used across industries. The interpretation of a given RF usually relies on the analysis of statistical values and is often only possible for data analytics experts. To make RFs accessible to experts with no data analytics background, we present RfX, a Visual Analytics (VA) system for the analysis of a RF's decision-making process. RfX allows to interactively analyse the properties of a forest and to explore and compare multiple trees in a RF. Thus, its users can identify relationships within a RF's feature subspace and detect hidden patterns in the model's underlying data. We contribute a design study in collaboration with an automotive company. A formative evaluation of RFX was carried out with two domain experts and a summative evaluation in the form of a field study with five domain experts. In this context, new hidden patterns such as increased eccentricities in an engine's rotor by observing secondary excitations of its bearings were detected using analyses made with RfX. Rules derived from analyses with the system led to a change in the company's testing procedures for electrical engines, which resulted in 80% reduced testing time for over 30% of all components.

Bus bunching can deteriorate bus service quality and jeopardise the effectiveness of public transit in promoting sustainable urban transport. This study applies artificial intelligence (AI) techniques in visual analytics to examine the spatiotemporal characteristics of bus bunching from a stop-based perspective. A total of 25 405 real-time traffic images were extracted and analysed from 11 bus stops in Hong Kong in August 2019 (7am–7pm, Tuesdays–Thursdays). An image classification model was developed with an overall accuracy of 85%. Three distinct groups were identified based on the spatiotemporal characteristics of bus bunching at different bus stops. They are (a) systematic peak hour, (b) minor and intermittent and (c) random bus bunching. Specific traffic conditions in terms of traffic speed, traffic composition and bus occupancy rate are highly associated with the onset of bus bunching. Overall traditional bus management tactics, such as schedule synchronisation and headways control, need to be supplemented by stop-based strategies that consider the temporal profiles of bus bunching and associated traffic conditions on road.

Deep learning (DL) models have achieved impressive performance in various domains such as medicine, finance, and autonomous vehicle systems with advances in computing power and technologies. However, due to the black-box structure of DL models, the decisions of these learning models often need to be explained to end-users. Explainable Artificial Intelligence (XAI) provides explanations of black-box models to reveal the behavior and underlying decision-making mechanisms of the models through tools, techniques, and algorithms. Visualization techniques help to present model and prediction explanations in a more understandable, explainable, and interpretable way. This survey paper aims to review current trends and challenges of visual analytics in interpreting DL models by adopting XAI methods and present future research directions in this area. We reviewed literature based on two different aspects, model usage and visual approaches. We addressed several research questions based on our findings and then discussed missing points, research gaps, and potential future research directions. This survey provides guidelines to develop a better interpretation of neural networks through XAI methods in the field of visual analytics.

Analysts often make visual causal inferences about possible data-generating models. However, visual analytics (VA) software tends to leave these models implicit in the mind of the analyst, which casts doubt on the statistical validity of informal visual "insights". We formally evaluate the quality of causal inferences from visualizations by adopting causal support -- a Bayesian cognition model that learns the probability of alternative causal explanations given some data -- as a normative benchmark for causal inferences. We contribute two experiments assessing how well crowdworkers can detect (1) a treatment effect and (2) a confounding relationship. We find that chart users' causal inferences tend to be insensitive to sample size such that they deviate from our normative benchmark. While interactively cross-filtering data in visualizations can improve sensitivity, on average users do not perform reliably better with common visualizations than they do with textual contingency tables. These experiments demonstrate the utility of causal support as an evaluation framework for inferences in VA and point to opportunities to make analysts' mental models more explicit in VA software.

Visual query of spatiotemporal data is becoming an increasingly important function in visual analytics applications. Various works have been presented for querying large spatiotemporal data in real time. However, the real-time query of spatiotemporal data distribution is still an open challenge. As spatiotemporal data become larger, methods of aggregation, storage and querying become critical. We propose a new visual query system that creates a low-memory storage component and provides real-time visual interactions of spatiotemporal data. We first present a peak-based kernel density estimation method to produce the data distribution for the spatiotemporal data. Then a novel density dictionary learning approach is proposed to compress temporal density maps and accelerate the query calculation. Moreover, various intuitive query interactions are presented to interactively gain patterns. The experimental results obtained on three datasets demonstrate that the presented system offers an effective query for visual analytics of spatiotemporal data.

The efficiency of warehouses is vital to e-commerce. Fast order processing at the warehouses ensures timely deliveries and improves customer satisfaction. However, monitoring, analyzing, and manipulating order processing in the warehouses in real time are challenging for traditional methods due to the sheer volume of incoming orders, the fuzzy definition of delayed order patterns, and the complex decision-making of order handling priorities. In this paper, we adopt a data-driven approach and propose OrderMonitor, a visual analytics system that assists warehouse managers in analyzing and improving order processing efficiency in real time based on streaming warehouse event data. Specifically, the order processing pipeline is visualized with a novel pipeline design based on the sedimentation metaphor to facilitate real-time order monitoring and suggest potentially abnormal orders. We also design a novel visualization that depicts order timelines based on the Gantt charts and Marey's graphs. Such a visualization helps the managers gain insights into the performance of order processing and find major blockers for delayed orders. Furthermore, an evaluating view is provided to assist users in inspecting order details and assigning priorities to improve the processing performance. The effectiveness of OrderMonitor is evaluated with two case studies on a real-world warehouse dataset.

Tactic analysis is a major issue in badminton as the effective usage of tactics is the key to win. The tactic in badminton is defined as a sequence of consecutive strokes. Most existing methods use statistical models to find sequential patterns of strokes and apply 2D visualizations such as glyphs and statistical charts to explore and analyze the discovered patterns. However, in badminton, spatial information like the shuttle trajectory, which is inherently 3D, is the core of a tactic. The lack of sufficient spatial awareness in 2D visualizations largely limited the tactic analysis of badminton. In this work, we collaborate with domain experts to study the tactic analysis of badminton in a 3D environment and propose an immersive visual analytics system, TIVEE, to assist users in exploring and explaining badminton tactics from multi-levels. Users can first explore various tactics from the third-person perspective using an unfolded visual presentation of stroke sequences. By selecting a tactic of interest, users can turn to the first-person perspective to perceive the detailed kinematic characteristics and explain its effects on the game result. The effectiveness and usefulness of TIVEE are demonstrated by case studies and an expert interview.

Fastening is a recurrent assembly operation at the aerospace industry. Among the many different types of fasteners being used blind ones offer particular advantages but there is yet a lack of reliable installation monitoring methods for their massive adoption. The present paper proposes an installation evaluation solution for blind fasteners that integrates the effect of the previous drilling operation and allows the visualization of the relationships between hole quality parameters, installation variables and installation quality. The results show high precision values of 0.95 and accuracy of 0.9.

International audience; We present a visual analytics tool, MiningVis, to explore the long-term historical evolution and dynamics of the Bitcoin mining ecosystem. Bitcoin is a cryptocurrency that attracts much attention but remains difficult to understand. Particularly important to the success, stability, and security of Bitcoin is a component of the system called "mining.'' Miners are responsible for validating transactions and are incentivized to participate by the promise of a monetary reward. Mining pools have emerged as collectives of miners that ensure a more stable and predictable income. MiningVis aims to help analysts understand the evolution and dynamics of the Bitcoin mining ecosystem, including mining market statistics, multi-measure mining pool rankings, and pool hopping behavior. Each of these features can be compared to external data concerning pool characteristics and Bitcoin news. In order to assess the value of MiningVis, we conducted online interviews and insight-based user studies with Bitcoin miners. We describe research questions tackled and insights made by our participants and illustrate practical implications for visual analytics systems for Bitcoin mining.

We introduce a new research area in visual analytics (VA) aiming to bridge existing gaps between methods of interactive machine learning (ML) and eXplainable Artificial Intelligence (XAI), on one side, and human minds, on the other side. The gaps are, first, a conceptual mismatch between ML/XAI outputs and human mental models and ways of reasoning, and second, a mismatch between the information quantity and level of detail and human capabilities to perceive and understand. A grand challenge is to adapt ML and XAI to human goals, concepts, values, and ways of thinking. Complementing the current efforts in XAI towards solving this challenge, VA can contribute by exploiting the potential of visualization as an effective way of communicating information to humans and a strong trigger of human abstractive perception and thinking. We propose a cross-disciplinary research framework and formulate research directions for VA.

Earth scientists are increasingly employing time series data with multiple dimensions and high temporal resolution to study the impacts of climate and environmental changes on Earth's atmosphere, biosphere, hydrosphere, and lithosphere. However, the large number of variables and varying time scales of antecedent conditions contributing to natural phenomena hinder scientists from completing more than the most basic analyses. In this paper, we present EVis (Environmental Visualization), a new visual analytics prototype to help scientists analyze and explore recurring environmental events (e.g. rock fracture, landslides, heat waves, floods) and their relationships with high dimensional time series of continuous numeric environmental variables, such as ambient temperature and precipitation. EVis provides coordinated scatterplots, heatmaps, histograms, and RadViz for foundational analyses. These features allow users to interactively examine relationships between events and one, two, three, or more environmental variables. EVis also provides a novel visual analytics approach to allowing users to discover temporally lagging relationships related to antecedent conditions between events and multiple variables, a critical task in Earth sciences. In particular, this latter approach projects multivariate time series onto trajectories in a 2D space using RadViz, and clusters the trajectories for temporal pattern discovery. Our case studies with rock cracking data and interviews with domain experts from a range of sub-disciplines within Earth sciences illustrate the extensive applicability and usefulness of EVis.

A growing number of longitudinal cohort studies are generating data with extensive patient observations across multiple timepoints. Such data offers promising opportunities to better understand the progression of diseases. However, these observations are usually treated as general events in existing visual analysis tools. As a result, their capabilities in modeling disease progression are not fully utilized. To fill this gap, we designed and implemented ThreadStates, an interactive visual analytics tool for the exploration of longitudinal patient cohort data. The focus of ThreadStates is to identify the states of disease progression by learning from observation data in a human-in-the-loop manner. We propose a novel Glyph Matrix design and combine it with a scatter plot to enable seamless identification, observation, and refinement of states. The disease progression patterns are then revealed in terms of state transitions using Sankey-based visualizations. We employ sequence clustering techniques to find patient groups with distinctive progression patterns, and to reveal the association between disease progression and patient-level features. The design and development were driven by a requirement analysis and iteratively refined based on feedback from domain experts over the course of a 10-month design study. Case studies and expert interviews demonstrate that ThreadStates can successively summarize disease states, reveal disease progression, and compare patient groups.

Semantic segmentation is a critical component in autonomous driving and has to be thoroughly evaluated due to safety concerns. Deep neural network (DNN) based semantic segmentation models are widely used in autonomous driving. However, it is challenging to evaluate DNN-based models due to their black-box-like nature, and it is even more difficult to assess model performance for crucial objects, such as lost cargos and pedestrians, in autonomous driving applications. In this work, we propose VASS, a Visual Analytics approach to diagnosing and improving the accuracy and robustness of Semantic Segmentation models, especially for critical objects moving in various driving scenes. The key component of our approach is a context-aware spatial representation learning that extracts important spatial information of objects, such as position, size, and aspect ratio, with respect to given scene contexts. Based on this spatial representation, we first use it to create visual summarization to analyze models' performance. We then use it to guide the generation of adversarial examples to evaluate models' spatial robustness and obtain actionable insights. We demonstrate the effectiveness of VASS via two case studies of lost cargo detection and pedestrian detection in autonomous driving. For both cases, we show quantitative evaluation on the improvement of models' performance with actionable insights obtained from VASS.

International audience; Situated visualization is an emerging concept within visualization, in which data is visualized in situ, where it is relevant to people. The concept has gained interest from multiple research communities, including visualization, human-computer interaction (HCI) and augmented reality. This has led to a range of explorations and applications of the concept, however, this early work has focused on the operational aspect of situatedness leading to inconsistent adoption of the concept and terminology. First, we contribute a literature survey in which we analyze 44 papers that explicitly use the term "situated visualization" to provide an overview of the research area, how it defines situated visualization, common application areas and technology used, as well as type of data and type of visualizations. Our survey shows that research on situated visualization has focused on technology-centric approaches that foreground a spatial understanding of situatedness. Secondly, we contribute five perspectives on situatedness (space, time, place, activity, and community) that together expand on the prevalent notion of situatedness in the corpus. We draw from six case studies and prior theoretical developments in HCI. Each perspective develops a generative way of looking at and working with situatedness in design and research. We outline future directions, including considering technology, material and aesthetics, leveraging the perspectives for design, and methods for stronger engagement with target audiences. We conclude with opportunities to consolidate situated visualization research.

In this design study, we present IRVINE, a Visual Analytics (VA) system, which facilitates the analysis of acoustic data to detect and understand previously unknown errors in the manufacturing of electrical engines. In serial manufacturing processes, signatures from acoustic data provide valuable information on how the relationship between multiple produced engines serves to detect and understand previously unknown errors. To analyze such signatures, IRVINE leverages interactive clustering and data labeling techniques, allowing users to analyze clusters of engines with similar signatures, drill down to groups of engines, and select an engine of interest. Furthermore, IRVINE allows to assign labels to engines and clusters and annotate the cause of an error in the acoustic raw measurement of an engine. Since labels and annotations represent valuable knowledge, they are conserved in a knowledge database to be available for other stakeholders. We contribute a design study, where we developed IRVINE in four main iterations with engineers from a company in the automotive sector. To validate IRVINE, we conducted a field study with six domain experts. Our results suggest a high usability and usefulness of IRVINE as part of the improvement of a real-world manufacturing process. Specifically, with IRVINE domain experts were able to label and annotate produced electrical engines more than 30% faster.

The spatial time series generated by city sensors allow us to observe urban phenomena like environmental pollution and traffic congestion at an unprecedented scale. However, recovering causal relations from these observations to explain the sources of urban phenomena remains a challenging task because these causal relations tend to be time-varying and demand proper time series partitioning for effective analyses. The prior approaches extract one causal graph given long-time observations, which cannot be directly applied to capturing, interpreting, and validating dynamic urban causality. This paper presents Compass, a novel visual analytics approach for in-depth analyses of the dynamic causality in urban time series. To develop Compass, we identify and address three challenges: detecting urban causality, interpreting dynamic causal relations, and unveiling suspicious causal relations. First, multiple causal graphs over time among urban time series are obtained with a causal detection framework extended from the Granger causality test. Then, a dynamic causal graph visualization is designed to reveal the time-varying causal relations across these causal graphs and facilitate the exploration of the graphs along the time. Finally, a tailored multi-dimensional visualization is developed to support the identification of spurious causal relations, thereby improving the reliability of causal analyses. The effectiveness of Compass is evaluated with two case studies conducted on the real-world urban datasets, including the air pollution and traffic speed datasets, and positive feedback was received from domain experts.

Building a visual overview of temporal event sequences with an optimal level-of-detail (i.e. simplified but informative) is an ongoing challenge - expecting the user to zoom into every important aspect of the overview can lead to missing insights. We propose a technique to build a multilevel overview of event sequences, whose granularity can be transformed across sequence clusters (vertical level-of-detail) or longitudinally (horizontal level-of-detail), using hierarchical aggregation and a novel cluster data representation Align-Score-Simplify. By default, the overview shows an optimal number of sequence clusters obtained through the average silhouette width metric - then users are able to explore alternative optimal sequence clusterings. The vertical level-of-detail of the overview changes along with the number of clusters, whilst the horizontal level-of-detail refers to the level of summarization applied to each cluster representation. The proposed technique has been implemented into a visualization system called Sequence Cluster Explorer (Sequen-C) that allows multilevel and detail-on-demand exploration through three coordinated views, and the inspection of data attributes at cluster, unique sequence, and individual sequence level. We present two case studies using real-world datasets in the healthcare domain: CUREd and MIMIC-III; which demonstrate how the technique can aid users to obtain a summary of common and deviating pathways, and explore data attributes for selected patterns., Comment: This is the author's version of the article to be published in IEEE Transactions on Visualization and Computer Graphics

In this paper, we design novel interactive deep learning methods to improve semantic interactions in visual analytics applications. The ability of semantic interaction to infer analysts' precise intents during sensemaking is dependent on the quality of the underlying data representation. We propose the $\text{DeepSI}_{\text{finetune}}$ framework that integrates deep learning into the human-in-the-loop interactive sensemaking pipeline, with two important properties. First, deep learning extracts meaningful representations from raw data, which improves semantic interaction inference. Second, semantic interactions are exploited to fine-tune the deep learning representations, which then further improves semantic interaction inference. This feedback loop between human interaction and deep learning enables efficient learning of user- and task-specific representations. To evaluate the advantage of embedding the deep learning within the semantic interaction loop, we compare $\text{DeepSI}_{\text{finetune}}$ against a state-of-the-art but more basic use of deep learning as only a feature extractor pre-processed outside of the interactive loop. Results of two complementary studies, a human-centered qualitative case study and an algorithm-centered simulation-based quantitative experiment, show that $\text{DeepSI}_{\text{finetune}}$ more accurately captures users' complex mental models with fewer interactions.

Indiana University-Purdue University Indianapolis (IUPUI), Protein malfunction can cause human diseases, which makes the protein a target in the process of drug discovery. In-depth knowledge of how protein functions can widely contribute to the understanding of the mechanism of these diseases. Protein functions are determined by protein structures and their dynamic properties. Protein dynamics refers to the constant physical movement of atoms in a protein, which may result in the transition between different conformational states of the protein. These conformational transitions are critically important for the proteins to function. Understanding protein dynamics can help to understand and interfere with the conformational states and transitions, and thus with the function of the protein. If we can understand the mechanism of conformational transition of protein, we can design molecules to regulate this process and regulate the protein functions for new drug discovery. Protein Dynamics can be simulated by Molecular Dynamics (MD) Simulations. The MD simulation data generated are spatial-temporal and therefore very high dimensional. To analyze the data, distinguishing various atomic interactions within a protein by interpreting their 3D coordinate values plays a significant role. Since the data is humongous, the essential step is to find ways to interpret the data by generating more efficient algorithms to reduce the dimensionality and developing user-friendly visualization tools to find patterns and trends, which are not usually attainable by traditional methods of data process. The typical allosteric long-range nature of the interactions that lead to large conformational transition, pin-pointing the underlying forces and pathways responsible for the global conformational transition at atomic level is very challenging. To address the problems, Various analytical techniques are performed on the simulation data to better understand the mechanism of protein dynamics at atomic level by developing a new program called Probing Long-distance interactions by Tapping into Paired-Distances (PLITIP), which contains a set of new tools based on analysis of paired distances to remove the interference of the translation and rotation of the protein itself and therefore can capture the absolute changes within the protein. Firstly, we developed a tool called Decomposition of Paired Distances (DPD). This tool generates a distance matrix of all paired residues from our simulation data. This paired distance matrix therefore is not subjected to the interference of the translation or rotation of the protein and can capture the absolute changes within the protein. This matrix is then decomposed by DPD using Principal Component Analysis (PCA) to reduce dimensionality and to capture the largest structural variation. To showcase how DPD works, two protein systems, HIV-1 protease and 14-3-3 σ, that both have tremendous structural changes and conformational transitions as displayed by their MD simulation trajectories. The largest structural variation and conformational transition were captured by the first principal component in both cases. In addition, structural clustering and ranking of representative frames by their PC1 values revealed the long-distance nature of the conformational transition and locked the key candidate regions that might be responsible for the large conformational transitions. Secondly, to facilitate further analysis of identification of the long-distance path, a tool called Pearson Coefficient Spiral (PCP) that generates and visualizes Pearson Coefficient to measure the linear correlation between any two sets of residue pairs is developed. PCP allows users to fix one residue pair and examine the correlation of its change with other residue pairs. Thirdly, a set of visualization tools that generate paired atomic distances for the shortlisted candidate residue and captured significant interactions among them were developed. The first tool is the Residue Interaction Network Graph for Paired Atomic Distances (NG-PAD), which not only generates paired atomic distances for the shortlisted candidate residues, but also display significant interactions by a Network Graph for convenient visualization. Second, the Chord Diagram for Interaction Mapping (CD-IP) was developed to map the interactions to protein secondary structural elements and to further narrow down important interactions. Third, a Distance Plotting for Direct Comparison (DP-DC), which plots any two paired distances at user’s choice, either at residue or atomic level, to facilitate identification of similar or opposite pattern change of distances along the simulation time. All the above tools of PLITIP enabled us to identify critical residues contributing to the large conformational transitions in both HIV-1 protease and 14-3-3σ proteins. Beside the above major project, a side project of developing tools to study protein pseudo-symmetry is also reported. It has been proposed that symmetry provides protein stability, opportunities for allosteric regulation, and even functionality. This tool helps us to answer the questions of why there is a deviation from perfect symmetry in protein and how to quantify it.

It is difficult to explore large text collections if no or little information is available on the contained documents. Hence, starting analytic tasks on such corpora is challenging for many stakeholders from various domains. As a remedy, recent visualization research suggests to use visual spatializations of representative text documents or tags to explore text collections. With PyramidTags, we introduce a novel approach for summarizing large text collections visually. In contrast to previous work, PyramidTags in particular aims at creating an improved representation that incorporates both temporal evolution and semantic relationship of visualized tags within the summarized document collection. As a result, it equips analysts with a visual starting point for interactive exploration to not only get an overview of the main terms and phrases of the corpus, but also to grasp important ideas and stories. Analysts can hover and select multiple tags to explore relationships and retrieve the most relevant documents. In this work, we apply PyramidTags to hundreds of thousands of web-crawled news reports. Our benchmarks suggest that PyramidTags creates time- and context-aware layouts, while preserving the inherent word order of important pairs.