Search

Dimensionality Reduction (DR) scatterplot layouts have become a ubiquitous visualization tool for analyzing multidimensional datasets. Despite their popularity, such scatterplots suffer from occlusion, especially when informative glyphs are used to represent data instances, potentially obfuscating critical information for the analysis under execution. Different strategies have been devised to address this issue, either producing overlap-free layouts that lack the powerful capabilities of contemporary DR techniques in uncovering interesting data patterns or eliminating overlaps as a post-processing strategy. Despite the good results of post-processing techniques, most of the best methods typically expand or distort the scatterplot area, thus reducing glyphs’ size (sometimes) to unreadable dimensions, defeating the purpose of removing overlaps. This article presents Distance Grid (DGrid) , a novel post-processing strategy to remove overlaps from DR layouts that faithfully preserves the original layout's characteristics and bounds the minimum glyph sizes. We show that DGrid surpasses the state-of-the-art in overlap removal (through an extensive comparative evaluation considering multiple different metrics) while also being one of the fastest techniques, especially for large datasets. A user study with 51 participants also shows that DGrid is consistently ranked among the top techniques for preserving the original scatterplots’ visual characteristics and the aesthetics of the final results.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model’s predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance.This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the models predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance.This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the models predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance.This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the models predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model’s predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

To improve the understanding of chemo-refractory high-grade serous ovarian cancers (HGSOCs), we characterized the proteogenomic landscape of 242 (refractory and sensitive) HGSOCs, representing one discovery and two validation cohorts across two biospecimen types (formalin-fixed paraffin-embedded and frozen). We identified a 64-protein signature that predicts with high specificity a subset of HGSOCs refractory to initial platinum-based therapy and is validated in two independent patient cohorts. We detected significant association between lack of Ch17 loss of heterozygosity (LOH) and chemo-refractoriness. Based on pathway protein expression, we identified 5 clusters of HGSOC, which validated across two independent patient cohorts and patient-derived xenograft (PDX) models. These clusters may represent different mechanisms of refractoriness and implicate putative therapeutic vulnerabilities.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model’s predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model’s predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance.This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the models predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model's predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model’s predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance.This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the models predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

BackgroundApplying directed acyclic graph (DAG) models to proteogenomic data has been shown effective for detecting causal biomarkers of complex diseases. However, there remain unsolved challenges in DAG learning to jointly model binary clinical outcome variables and continuous biomarker measurements.ResultsIn this paper, we propose a new tool, DAGBagM, to learn DAGs with both continuous and binary nodes. By using appropriate models, DAGBagM allows for either continuous or binary nodes to be parent or child nodes. It employs a bootstrap aggregating strategy to reduce false positives in edge inference. At the same time, the aggregation procedure provides a flexible framework to robustly incorporate prior information on edges.ConclusionsThrough extensive simulation experiments, we demonstrate that DAGBagM has superior performance compared to alternative strategies for modeling mixed types of nodes. In addition, DAGBagM is computationally more efficient than two competing methods. When applying DAGBagM to proteogenomic datasets from ovarian cancer studies, we identify potential protein biomarkers for platinum refractory/resistant response in ovarian cancer. DAGBagM is made available as a github repository at https://github.com/jie108/dagbagM .

BackgroundApplying directed acyclic graph (DAG) models to proteogenomic data has been shown effective for detecting causal biomarkers of complex diseases. However, there remain unsolved challenges in DAG learning to jointly model binary clinical outcome variables and continuous biomarker measurements.ResultsIn this paper, we propose a new tool, DAGBagM, to learn DAGs with both continuous and binary nodes. By using appropriate models, DAGBagM allows for either continuous or binary nodes to be parent or child nodes. It employs a bootstrap aggregating strategy to reduce false positives in edge inference. At the same time, the aggregation procedure provides a flexible framework to robustly incorporate prior information on edges.ConclusionsThrough extensive simulation experiments, we demonstrate that DAGBagM has superior performance compared to alternative strategies for modeling mixed types of nodes. In addition, DAGBagM is computationally more efficient than two competing methods. When applying DAGBagM to proteogenomic datasets from ovarian cancer studies, we identify potential protein biomarkers for platinum refractory/resistant response in ovarian cancer. DAGBagM is made available as a github repository at https://github.com/jie108/dagbagM .

Dimensionality reduction has become an important research topic as demand for interpreting high-dimensional datasets has been increasing rapidly in recent years. There have been many dimensionality reduction methods with good performance in preserving the overall relationship among data points when mapping them to a lower-dimensional space. However, these existing methods fail to incorporate the difference in importance among features. To address this problem, we propose a novel meta-method, DimenFix, which can be operated upon any base dimensionality reduction method that involves a gradient-descent-like process. By allowing users to define the importance of different features, which is considered in dimensionality reduction, DimenFix creates new possibilities to visualize and understand a given dataset. Meanwhile, DimenFix does not increase the time cost or reduce the quality of dimensionality reduction with respect to the base dimensionality reduction used.

Visual Analytics (VA) tools provide ways for users to harness insights and knowledge from datasets. Recalling and retelling user experiences while utilizing VA tools has attracted significant interest. Nevertheless, each user sessions are unique. Even when different users have the same intention when using a VA tool, they may follow different paths and uncover different insights. Current methods of manually processing such data to recall and retell users' knowledge discovery paths may also be time-consuming, especially when there is the need to present users' findings to third parties. This paper presents a novel system that collects user intentions, behavior, and insights during knowledge discovery sessions, automatically structure the data, and extracts narrations of knowledge discovery as PowerPoint slide decks. The system is powered by a Knowledge Graph designed based on a formal and reproducible modeling process. To evaluate our system, we have attached it to two existing VA tools where users were asked to perform pre-defined tasks. Several slide decks and other analysis metrics were extracted from the generated Knowledge Graph. Experts scrutinized and confirmed the usefulness of our automated process for using the slide decks to disclose knowledge discovery paths to others and to verify whether the VA tools themselves were effective.

Dimensionality reduction has become an important research topic as demand for interpreting high-dimensional datasets has been increasing rapidly in recent years. There have been many dimensionality reduction methods with good performance in preserving the overall relationship among data points when mapping them to a lower-dimensional space. However, these existing methods fail to incorporate the difference in importance among features. To address this problem, we propose a novel meta-method, DimenFix, which can be operated upon any base dimensionality reduction method that involves a gradient-descent-like process. By allowing users to define the importance of different features, which is considered in dimensionality reduction, DimenFix creates new possibilities to visualize and understand a given dataset. Meanwhile, DimenFix does not increase the time cost or reduce the quality of dimensionality reduction with respect to the base dimensionality reduction used.

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model's predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts., Comment: This manuscript is accepted for publication in Computer Graphics Forum (CGF)

The primary goal of Visual Analytics (VA) is to enable user-guided knowledge generation. Theoretical VA works to explain how the different aspects of a VA tool bring forth new insights through user interactivity, which itself can be captured through tracking methods for reproduction or evaluation. However, the process of automatically capturing the user's thought process, such as intent and insights, and associating it with user's interaction events are largely ignored. Also, two forms of interactivity capture are typically ambiguous and intermixed: the temporal aspect, which indicates sequences of events, and the atemporal aspect, which explains the workflow as sequences of states within a state-space. In this work, we propose Visual Analytics Knowledge Graph (VAKG), a conceptual framework that brings VA modeling theory to practice through a novel Set-Theory formalization of knowledge modeling. By extracting such a model from a VA tool, VAKG structures a 4-way temporal knowledge graph that describes user behavior and its associated knowledge gain process. Such knowledge graphs can be populated manually or automatically during user analysis sessions, which can then be analyzed using graph analysis methods. VAKG is demonstrated by modeling and collecting Tableau and visual text-mining workflows, where comparative user satisfaction, tool efficacy, and overall workflow shortcomings can be extracted from the knowledge graph., Comment: 14 pgs, submitted to Computer Graphics Forum 2023

© 2020 The Authors The integration of mass spectrometry-based proteomics with next-generation DNA and RNA sequencing profiles tumors more comprehensively. Here this “proteogenomics” approach was applied to 122 treatment-naive primary breast cancers accrued to preserve post-translational modifications, including protein phosphorylation and acetylation. Proteogenomics challenged standard breast cancer diagnoses, provided detailed analysis of the ERBB2 amplicon, defined tumor subsets that could benefit from immune checkpoint therapy, and allowed more accurate assessment of Rb status for prediction of CDK4/6 inhibitor responsiveness. Phosphoproteomics profiles uncovered novel associations between tumor suppressor loss and targetable kinases. Acetylproteome analysis highlighted acetylation on key nuclear proteins involved in the DNA damage response and revealed cross-talk between cytoplasmic and mitochondrial acetylation and metabolism. Our results underscore the potential of proteogenomics for clinical investigation of breast cancer through more accurate annotation of targetable pathways and biological features of this remarkably heterogeneous malignancy.

Streaming data applications are becoming more common due to the ability ofdifferent information sources to continuously capture or produce data, such as sensors and social media. Although there are recent advances, most visualization approaches, particularly Dimensionality Reduction (DR) techniques, cannot be directly applied in such scenarios due to the transient nature of streaming data. A few DR methods currently address this limitation using online or incremental strategies, continuously updating the visualization as data is received. Despite their relative success, most impose the need to store and access the data multiple times to produce a complete projection, not being appropriate for streaming where data continuously grow. Others do not impose such requirements but cannot update the position of the data already projected, potentially resulting in visual artifacts. This paper presents Xtreaming, a novel incremental DR technique that continuously updates the visual representation to reflect new emerging structures or patterns without visiting the high-dimensional data more than once. Our tests show that in streaming scenarios where data is not fully stored in-memory, Xtreaming is competitive in terms of quality compared to other streaming and incremental techniques while being orders of magnitude faster.

Streaming data applications are becoming more common due to the ability ofdifferent information sources to continuously capture or produce data, such as sensors and social media. Although there are recent advances, most visualization approaches, particularly Dimensionality Reduction (DR) techniques, cannot be directly applied in such scenarios due to the transient nature of streaming data. A few DR methods currently address this limitation using online or incremental strategies, continuously updating the visualization as data is received. Despite their relative success, most impose the need to store and access the data multiple times to produce a complete projection, not being appropriate for streaming where data continuously grow. Others do not impose such requirements but cannot update the position of the data already projected, potentially resulting in visual artifacts. This paper presents Xtreaming, a novel incremental DR technique that continuously updates the visual representation to reflect new emerging structures or patterns without visiting the high-dimensional data more than once. Our tests show that in streaming scenarios where data is not fully stored in-memory, Xtreaming is competitive in terms of quality compared to other streaming and incremental techniques while being orders of magnitude faster., Funding agencies: Fundacao de Amparo a Pesquisa do Estado de SAo Paulo (FAPESP) [2013/07375-0]; Natural Sciences and Engineering Research Council of Canada (NSERC)

SummaryA primary goal of the US National Cancer Institute's Ras initiative at the Frederick National Laboratory for Cancer Research is to develop methods to quantify RAS signaling to facilitate development of novel cancer therapeutics. We use targeted proteomics technologies to develop a community resource consisting of 256 validated multiple reaction monitoring (MRM)-based, multiplexed assays for quantifying protein expression and phosphorylation through the receptor tyrosine kinase, MAPK, and AKT signaling networks. As proof of concept, we quantify the response of melanoma (A375 and SK-MEL-2) and colorectal cancer (HCT-116 and HT-29) cell lines to BRAF inhibition by PLX-4720. These assays replace over 60 Western blots with quantitative mass spectrometry-based assays of high molecular specificity and quantitative precision, showing the value of these methods for pharmacodynamic measurements and mechanism of action studies. Methods, fit-for-purpose validation, and results are publicly available as a resource for the community at assays.cancer.gov.MotivationA lack of quantitative, multiplexable assays for phosphosignaling limits comprehensive investigation of aberrant signaling in cancer and evaluation of novel treatments. To alleviate this limitation, we sought to develop assays using targeted mass spectrometry for quantifying protein expression and phosphorylation through the receptor tyrosine kinase, MAPK, and AKT signaling networks. The resulting assays provide a resource for replacing over 60 Western blots in examining cancer signaling and tumor biology with high molecular specificity and quantitative rigor.

SummaryA primary goal of the US National Cancer Institute's Ras initiative at the Frederick National Laboratory for Cancer Research is to develop methods to quantify RAS signaling to facilitate development of novel cancer therapeutics. We use targeted proteomics technologies to develop a community resource consisting of 256 validated multiple reaction monitoring (MRM)-based, multiplexed assays for quantifying protein expression and phosphorylation through the receptor tyrosine kinase, MAPK, and AKT signaling networks. As proof of concept, we quantify the response of melanoma (A375 and SK-MEL-2) and colorectal cancer (HCT-116 and HT-29) cell lines to BRAF inhibition by PLX-4720. These assays replace over 60 Western blots with quantitative mass spectrometry-based assays of high molecular specificity and quantitative precision, showing the value of these methods for pharmacodynamic measurements and mechanism of action studies. Methods, fit-for-purpose validation, and results are publicly available as a resource for the community at assays.cancer.gov.MotivationA lack of quantitative, multiplexable assays for phosphosignaling limits comprehensive investigation of aberrant signaling in cancer and evaluation of novel treatments. To alleviate this limitation, we sought to develop assays using targeted mass spectrometry for quantifying protein expression and phosphorylation through the receptor tyrosine kinase, MAPK, and AKT signaling networks. The resulting assays provide a resource for replacing over 60 Western blots in examining cancer signaling and tumor biology with high molecular specificity and quantitative rigor.

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, and be used as means to disclose cluster patterns in datasets. GNG, however, creates highly connected graphs when trained on high dimensional data, which in turn leads to highly clutter representations that fail to disclose any meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in large and high-dimensional datasets. The first one focuses on providing more meaningful and accurate cluster pattern representations of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology, which may, in turn, lead to visual cluttering in 2D representations. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower-dimensional graph structures, which in turn provide more accurate and meaningful cluster representations; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in distributed settings., CC BY 4.0

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, and be used as means to disclose cluster patterns in datasets. GNG, however, creates highly connected graphs when trained on high dimensional data, which in turn leads to highly clutter representations that fail to disclose any meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in large and high-dimensional datasets. The first one focuses on providing more meaningful and accurate cluster pattern representations of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology, which may, in turn, lead to visual cluttering in 2D representations. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower-dimensional graph structures, which in turn provide more accurate and meaningful cluster representations; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in distributed settings., CC BY 4.0

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the most populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, disclosing cluster patterns in datasets. It is common, however, for GNG to result in highly connected graphs when trained on high-dimensional data, which in turn leads to highly cluttered 2D representations that may fail to disclose meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in largescale and high-dimensional datasets. The first one focuses on providing more accurate and meaningful 2D visual representations for cluster patterns of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology which may, in turn, result in overplotting and clutter. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower dimensional graph structures that provide more meaningful (and sometimes more accurate) cluster representations with less overplotting and clutter; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in large-scale and distributed settings. (C) 2021 The Author(s). Published by Elsevier Inc.

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, and be used as means to disclose cluster patterns in datasets. GNG, however, creates highly connected graphs when trained on high dimensional data, which in turn leads to highly clutter representations that fail to disclose any meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in large and high-dimensional datasets. The first one focuses on providing more meaningful and accurate cluster pattern representations of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology, which may, in turn, lead to visual cluttering in 2D representations. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower-dimensional graph structures, which in turn provide more accurate and meaningful cluster representations; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in distributed settings., CC BY 4.0

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the most populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, disclosing cluster patterns in datasets. It is common, however, for GNG to result in highly connected graphs when trained on high-dimensional data, which in turn leads to highly cluttered 2D representations that may fail to disclose meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in large-scale and high-dimensional datasets. The first one focuses on providing more accurate and meaningful 2D visual representations for cluster patterns of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology which may, in turn, result in overplotting and clutter. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower-dimensional graph structures that provide more meaningful (and sometimes more accurate) cluster representations with less overplotting and clutter; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in large-scale and distributed settings.

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, and be used as means to disclose cluster patterns in datasets. GNG, however, creates highly connected graphs when trained on high dimensional data, which in turn leads to highly clutter representations that fail to disclose any meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in large and high-dimensional datasets. The first one focuses on providing more meaningful and accurate cluster pattern representations of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology, which may, in turn, lead to visual cluttering in 2D representations. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower-dimensional graph structures, which in turn provide more accurate and meaningful cluster representations; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in distributed settings., CC BY 4.0

Visual Analytics (VA) tools and techniques have been instrumental in supporting users to build better classification models, interpret models' overall logic, and audit results. In a different direction, VA has recently been applied to transform classification models into descriptive mechanisms instead of predictive. The idea is to use such models as surrogates for data patterns, visualizing the model to understand the phenomenon represented by the data. Although very useful and inspiring, the few proposed approaches have opted to use low complex classification models to promote straightforward interpretation, presenting limitations to capture intricate data patterns. In this paper, we present VAX (multiVariate dAta eXplanation), a new VA method to support the identification and visual interpretation of patterns in multivariate datasets. Unlike the existing similar approaches, VAX uses the concept of Jumping Emerging Patterns to identify and aggregate several diversified patterns, producing explanations through logic combinations of data variables. The potential of VAX to interpret complex multivariate datasets is demonstrated through use cases employing two real-world datasets covering different scenarios.

In exploratory tasks involving high-dimensional datasets, dimensionality reduction (DR) techniques help analysts to discover patterns and other useful information. Although scatter plot representations of DR results allow for cluster identification and similarity analysis, such a visual metaphor presents problems when the number of instances of the dataset increases, resulting in cluttered visualizations. In this work, we propose a scatter plot-based multilevel approach to display DR results and address clutter-related problems when visualizing large datasets, together with the definition of a methodology to use focus+context interaction on non-hierarchical embeddings. The proposed technique, called ExplorerTree, uses a sampling selection technique on scatter plots to reduce visual clutter and guide users through exploratory tasks. We demonstrate ExplorerTree's effectiveness through a use case, where we visually explore activation images of the convolutional layers of a neural network. Finally, we also conducted a user experiment to evaluate ExplorerTree's ability to convey embedding structures using different sampling strategies.

Dimensionality reduction (DR) techniques help analysts understand patterns in high-dimensional spaces. These techniques, often represented by scatter plots, are employed in diverse science domains and facilitate similarity analysis among clusters and data samples. For datasets containing many granularities or when analysis follows the information visualization mantra, hierarchical DR techniques are the most suitable approach since they present major structures beforehand and details on demand. However, current hierarchical DR techniques are not fully capable of addressing literature problems because they do not preserve the projection mental map across hierarchical levels or are not suitable for most data types. This work presents HUMAP, a novel hierarchical dimensionality reduction technique designed to be flexible in preserving local and global structures and the mental map throughout hierarchical exploration. We provide empirical evidence of our technique's superiority compared with current hierarchical approaches and show two case studies to demonstrate its strengths.

Machine Learning (ML) is widely used to automatically extract meaningful information from Electronic Health Records (EHR) to support operational, clinical, and financial decision-making. However, ML models require a large number of annotated examples to provide satisfactory results, which is not possible in most healthcare scenarios due to the high cost of clinician-labeled data. Active Learning (AL) is a process of selecting the most informative instances to be labeled by an expert to further train a supervised algorithm. We demonstrate the effectiveness of AL in multi-label text classification in the clinical domain. In this context, we apply a set of well-known AL methods to help automatically assign ICD-9 codes on the MIMIC-III dataset. Our results show that the selection of informative instances provides satisfactory classification with a significantly reduced training set (8.3\% of the total instances). We conclude that AL methods can significantly reduce the manual annotation cost while preserving model performance., Comment: It was accepted in the ACM CHIL 2021 workshop track

As neural networks become the tool of choice to solve an increasing variety of problems in our society, adversarial attacks become critical. The possibility of generating data instances deliberately designed to fool a network's analysis can have disastrous consequences. Recent work has shown that commonly used methods for model training often result in fragile abstract representations that are particularly vulnerable to such attacks. This paper presents a visual framework to investigate neural network models subjected to adversarial examples, revealing how models' perception of the adversarial data differs from regular data instances and their relationships with class perception. Through different use cases, we show how observing these elements can quickly pinpoint exploited areas in a model, allowing further study of vulnerable features in input data and serving as a guide to improving model training and architecture., Comment: Submitted to EUROVIS 2021

Knowing how to construct text-based Search Queries (SQs) for use in Search Engines (SEs) such as Google or Wikipedia has become a fundamental skill. Though much data are available through such SEs, most structured datasets live outside their scope. Visualization tools aid in this limitation, but no such tools come close to the sheer amount of information available through general-purpose SEs. To fill this gap, this paper presents Q4EDA, a novel framework that converts users' visual selection queries executed on top of time series visual representations, providing valid and stable SQs to be used in general-purpose SEs and suggestions of related information. The usefulness of Q4EDA is presented and validated by users through an application linking a Gapminder's line-chart replica with a SE populated with Wikipedia documents, showing how Q4EDA supports and enhances exploratory analysis of United Nations world indicators. Despite some limitations, Q4EDA is unique in its proposal and represents a real advance towards providing solutions for querying textual information based on user interactions with visual representations., Comment: 12 Figures, 21 pages, submitted to Information 2022 special issue: Trends and Opportunities in Visualization and Visual Analytics

The growing neural gas (GNG) is an unsupervised topology learning algorithm that models a data space through interconnected units that stand on the populated areas of that space. Its output is a graph that can be visually represented on a two-dimensional plane, and be used as means to disclose cluster patterns in datasets. GNG, however, creates highly connected graphs when trained on high dimensional data, which in turn leads to highly clutter representations that fail to disclose any meaningful patterns. Moreover, its sequential learning limits its potential for faster executions on local datasets, and, more importantly, its potential for training on distributed datasets while leveraging from the computational resources of the infrastructures in which they reside. This paper presents two methods that improve GNG for the visualization of cluster patterns in large and high-dimensional datasets. The first one focuses on providing more meaningful and accurate cluster pattern representations of high-dimensional datasets, by avoiding connections that lead to high-dimensional graphs in the modeled topology, which may, in turn, lead to visual cluttering in 2D representations. The second method presented in this paper enables the use of GNG on big and distributed datasets with faster execution times, by modeling and merging separate parts of a dataset using the MapReduce model. Quantitative and qualitative evaluations show that the first method leads to the creation of lower-dimensional graph structures, which in turn provide more accurate and meaningful cluster representations; and that the second method preserves the accuracy and meaning of the cluster representations while enabling its execution in distributed settings., CC BY 4.0

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and-error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early systemfeed back, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity ofthe view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and-error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early systemfeed back, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity ofthe view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early system feedback, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity of the view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early system feedback, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity of the view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

Over the past decades, classification models have proven to be essential machine learning tools given their potential and applicability in various domains. In these years, the north of the majority of the researchers had been to improve quantitative metrics, notwithstanding the lack of information about models' decisions such metrics convey. This paradigm has recently shifted, and strategies beyond tables and numbers to assist in interpreting models' decisions are increasing in importance. Part of this trend, visualization techniques have been extensively used to support classification models' interpretability, with a significant focus on rule-based models. Despite the advances, the existing approaches present limitations in terms of visual scalability, and the visualization of large and complex models, such as the ones produced by the Random Forest (RF) technique, remains a challenge. In this paper, we propose Explainable Matrix (ExMatrix), a novel visualization method for RF interpretability that can handle models with massive quantities of rules. It employs a simple yet powerful matrix-like visual metaphor, where rows are rules, columns are features, and cells are rules predicates, enabling the analysis of entire models and auditing classification results. ExMatrix applicability is confirmed via different examples, showing how it can be used in practice to promote RF models interpretability., Comment: IEEE VIS VAST 2020

Streaming data applications are becoming more common due to the ability of different information sources to continuously capture or produce data, such as sensors and social media. Despite recent advances, most visualization approaches, in particular, multidimensional projection or dimensionality reduction techniques, cannot be directly applied in such scenarios due to the transient nature of streaming data. Currently, only a few methods address this limitation using online or incremental strategies, continuously processing data, and updating the visualization. Despite their relative success, most of them impose the need for storing and accessing the data multiple times, not being appropriate for streaming where data continuously grow. Others do not impose such requirements but are not capable of updating the position of the data already projected, potentially resulting in visual artifacts. In this paper, we present Xtreaming, a novel incremental projection technique that continuously updates the visual representation to reflect new emerging structures or patterns without visiting the multidimensional data more than once. Our tests show that Xtreaming is competitive in terms of global distance preservation if compared to other streaming and incremental techniques, but it is orders of magnitude faster. To the best of our knowledge, it is the first methodology that is capable of evolving a projection to faithfully represent new emerging structures without the need to store all data, providing reliable results for efficiently and effectively projecting streaming data., Comment: 12 pages, 11 figures

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and-error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early systemfeed back, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity ofthe view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and-error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early systemfeed back, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity ofthe view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

As large datasets become more common, so becomes the necessity for exploratory approaches that allow iterative, trial-and-error analysis. Without such solutions, hypothesis testing and exploratory data analysis may become cumbersome due to long waiting times for feedback from computationally-intensive algorithms. This work presents a process model for progressive multidimensional projections (P-MDPs) that enables early feedback and user involvement in the process, complementing previous work by providing a lower level of abstraction and describing the specific elements that can be used to provide early systemfeed back, and those which can be enabled for user interaction. Additionally, we outline a set of design constraints that must be taken into account to ensure the usability of a solution regarding feedback time, visual cluttering, and the interactivity ofthe view. To address these constraints, we propose the use of incremental vector quantization (iVQ) as a core step within the process. To illustrate the feasibility of the model, and the usefulness of the proposed iVQ-based solution, we present a prototype that demonstrates how the different usability constraints can be accounted for, regardless of the size of a dataset.

Na era da informação, desenvolvemos a capacidade de coletar e armazenar dados, criar métodos sofisticados de mineração de dados e gerar visualizações ricas para compartilhar as informações resultantes do processo de análise de dados. No entanto, analisar e gerenciar dados brutos ainda é uma parte desafiadora desse processo, principalmente no que diz respeito ao pré-processamento de dados, que visa transformar esses dados brutos em um formato apropriado para análises subsequentes. Embora possamos encontrar estudos propondo implicações ou recomendações para futuras soluções de visualização no escopo da análise de dados, eles não se concentram nos desafios da fase de pré-processamento, nem em como a visualização pode suportá-la. Da mesma forma, os modelos atuais de análise visual não consideram o pré-processamento como uma fase igualmente importante em seus processos. Assim, com este estudo, pretendemos contribuir para a discussão de como podemos usar e combinar métodos de visualização e mineração de dados para auxiliar os analistas de dados durante as atividades de pré-processamento.Para isso, apresentamos um modelo de pré-processamento com análise visual, que contempla um conjunto de recursos para inspirar a implementação de novas soluções. Por sua vez, esses recursos foram projetados considerando uma lista de ideias(Insights) que obtivemos durante um estudo de entrevista com treze analistas de dados. As principais contribuições de nosso estudo são três: (a) O modelo de análise visual para auxiliar durante a fase de pré-processamento. (b) A lista de dez Insights, como um conjunto consolidado de requisitos para futuros estudos de pesquisa de visualização aplicados ao pré-processamento e à mineração de dados. (c) Os detalhes sobre o perfil dos analistas de dados, os principais desafios que eles enfrentam e as oportunidades que surgem enquanto eles estão envolvidos em projetos de mineração de dados em diversas áreas da organização., In the information age, we have evolved the ability to collect and store data, create sophisticated data mining methods, and generate rich visualizations to share the informa- tion resulting from the data analysis process. However, analyzing and managing raw data is still a challenging part of this process, mainly with regards to data preprocessing, which aims to transform this raw data into an appropriate format for subsequent analysis. Although we can find studies proposing design implications or recommendations for future visualization solutions in the data analysis scope, they do not focus on the challenges during the Preprocessing phase and on how visualization can support it. Likewise, the current Visual Analytics Models are not considering preprocessing an equally important phase in their pro- cess, such as Data, Models, Visualization, and Knowledge. Thus, with this study, we aim to contribute to the discussion of how we can use and combine methods of visualization and data mining to assist data analysts during the preprocessing activities. To achieve that, we are introducing the Preprocessing Profiling Model for Visual Analytics, which contemplates a set of features to inspire the implementation of new solutions. In turn, these features were designed considering a list of insights we obtained during an interview study with thirteen data analysts. The main contributions in our study are three: (a) the Preprocessing Profiling Model for Visual Analytics as a solution to assist during Preprocessing phase. (b) The list of ten insights, as a consolidated set of requirements for future visualization research studies applied to preprocessing and data mining. (c) The details on the profile of the data analysts, the main challenges they face, and the opportunities that arise while they are engaged in data mining projects in diverse organizational areas.

Na era da informação, desenvolvemos a capacidade de coletar e armazenar dados, criar métodos sofisticados de mineração de dados e gerar visualizações ricas para compartilhar as informações resultantes do processo de análise de dados. No entanto, analisar e gerenciar dados brutos ainda é uma parte desafiadora desse processo, principalmente no que diz respeito ao pré-processamento de dados, que visa transformar esses dados brutos em um formato apropriado para análises subsequentes. Embora possamos encontrar estudos propondo implicações ou recomendações para futuras soluções de visualização no escopo da análise de dados, eles não se concentram nos desafios da fase de pré-processamento, nem em como a visualização pode suportá-la. Da mesma forma, os modelos atuais de análise visual não consideram o pré-processamento como uma fase igualmente importante em seus processos. Assim, com este estudo, pretendemos contribuir para a discussão de como podemos usar e combinar métodos de visualização e mineração de dados para auxiliar os analistas de dados durante as atividades de pré-processamento.Para isso, apresentamos um modelo de pré-processamento com análise visual, que contempla um conjunto de recursos para inspirar a implementação de novas soluções. Por sua vez, esses recursos foram projetados considerando uma lista de ideias(Insights) que obtivemos durante um estudo de entrevista com treze analistas de dados. As principais contribuições de nosso estudo são três: (a) O modelo de análise visual para auxiliar durante a fase de pré-processamento. (b) A lista de dez Insights, como um conjunto consolidado de requisitos para futuros estudos de pesquisa de visualização aplicados ao pré-processamento e à mineração de dados. (c) Os detalhes sobre o perfil dos analistas de dados, os principais desafios que eles enfrentam e as oportunidades que surgem enquanto eles estão envolvidos em projetos de mineração de dados em diversas áreas da organização., In the information age, we have evolved the ability to collect and store data, create sophisticated data mining methods, and generate rich visualizations to share the informa- tion resulting from the data analysis process. However, analyzing and managing raw data is still a challenging part of this process, mainly with regards to data preprocessing, which aims to transform this raw data into an appropriate format for subsequent analysis. Although we can find studies proposing design implications or recommendations for future visualization solutions in the data analysis scope, they do not focus on the challenges during the Preprocessing phase and on how visualization can support it. Likewise, the current Visual Analytics Models are not considering preprocessing an equally important phase in their pro- cess, such as Data, Models, Visualization, and Knowledge. Thus, with this study, we aim to contribute to the discussion of how we can use and combine methods of visualization and data mining to assist data analysts during the preprocessing activities. To achieve that, we are introducing the Preprocessing Profiling Model for Visual Analytics, which contemplates a set of features to inspire the implementation of new solutions. In turn, these features were designed considering a list of insights we obtained during an interview study with thirteen data analysts. The main contributions in our study are three: (a) the Preprocessing Profiling Model for Visual Analytics as a solution to assist during Preprocessing phase. (b) The list of ten insights, as a consolidated set of requirements for future visualization research studies applied to preprocessing and data mining. (c) The details on the profile of the data analysts, the main challenges they face, and the opportunities that arise while they are engaged in data mining projects in diverse organizational areas.

Na era da informação, desenvolvemos a capacidade de coletar e armazenar dados, criar métodos sofisticados de mineração de dados e gerar visualizações ricas para compartilhar as informações resultantes do processo de análise de dados. No entanto, analisar e gerenciar dados brutos ainda é uma parte desafiadora desse processo, principalmente no que diz respeito ao pré-processamento de dados, que visa transformar esses dados brutos em um formato apropriado para análises subsequentes. Embora possamos encontrar estudos propondo implicações ou recomendações para futuras soluções de visualização no escopo da análise de dados, eles não se concentram nos desafios da fase de pré-processamento, nem em como a visualização pode suportá-la. Da mesma forma, os modelos atuais de análise visual não consideram o pré-processamento como uma fase igualmente importante em seus processos. Assim, com este estudo, pretendemos contribuir para a discussão de como podemos usar e combinar métodos de visualização e mineração de dados para auxiliar os analistas de dados durante as atividades de pré-processamento.Para isso, apresentamos um modelo de pré-processamento com análise visual, que contempla um conjunto de recursos para inspirar a implementação de novas soluções. Por sua vez, esses recursos foram projetados considerando uma lista de ideias(Insights) que obtivemos durante um estudo de entrevista com treze analistas de dados. As principais contribuições de nosso estudo são três: (a) O modelo de análise visual para auxiliar durante a fase de pré-processamento. (b) A lista de dez Insights, como um conjunto consolidado de requisitos para futuros estudos de pesquisa de visualização aplicados ao pré-processamento e à mineração de dados. (c) Os detalhes sobre o perfil dos analistas de dados, os principais desafios que eles enfrentam e as oportunidades que surgem enquanto eles estão envolvidos em projetos de mineração de dados em diversas áreas da organização., In the information age, we have evolved the ability to collect and store data, create sophisticated data mining methods, and generate rich visualizations to share the informa- tion resulting from the data analysis process. However, analyzing and managing raw data is still a challenging part of this process, mainly with regards to data preprocessing, which aims to transform this raw data into an appropriate format for subsequent analysis. Although we can find studies proposing design implications or recommendations for future visualization solutions in the data analysis scope, they do not focus on the challenges during the Preprocessing phase and on how visualization can support it. Likewise, the current Visual Analytics Models are not considering preprocessing an equally important phase in their pro- cess, such as Data, Models, Visualization, and Knowledge. Thus, with this study, we aim to contribute to the discussion of how we can use and combine methods of visualization and data mining to assist data analysts during the preprocessing activities. To achieve that, we are introducing the Preprocessing Profiling Model for Visual Analytics, which contemplates a set of features to inspire the implementation of new solutions. In turn, these features were designed considering a list of insights we obtained during an interview study with thirteen data analysts. The main contributions in our study are three: (a) the Preprocessing Profiling Model for Visual Analytics as a solution to assist during Preprocessing phase. (b) The list of ten insights, as a consolidated set of requirements for future visualization research studies applied to preprocessing and data mining. (c) The details on the profile of the data analysts, the main challenges they face, and the opportunities that arise while they are engaged in data mining projects in diverse organizational areas.