Search

Autoregressive and recurrent networks have achieved remarkable progress across various fields, from weather forecasting to molecular generation and Large Language Models. Despite their strong predictive capabilities, these models lack a rigorous framework for addressing uncertainty, which is key in scientific applications such as PDE solving, molecular generation and Machine Learning Force Fields. To address this shortcoming we present BARNN: a variational Bayesian Autoregressive and Recurrent Neural Network. BARNNs aim to provide a principled way to turn any autoregressive or recurrent model into its Bayesian version. BARNN is based on the variational dropout method, allowing to apply it to large recurrent neural networks as well. We also introduce a temporal version of the "Variational Mixtures of Posteriors" prior (tVAMP-prior) to make Bayesian inference efficient and well-calibrated. Extensive experiments on PDE modelling and molecular generation demonstrate that BARNN not only achieves comparable or superior accuracy compared to existing methods, but also excels in uncertainty quantification and modelling long-range dependencies.

The iterative bleaching extends multiplexity (IBEX) Knowledge-Base is a central portal for researchers adopting IBEX and related 2D and 3D immunofluorescence imaging methods. The design of the Knowledge-Base is modeled after efforts in the open-source software community and includes three facets: a development platform (GitHub), static website, and service for data archiving. The Knowledge-Base facilitates the practice of open science throughout the research life cycle by providing validation data for recommended and non-recommended reagents, e.g., primary and secondary antibodies. In addition to reporting negative data, the Knowledge-Base empowers method adoption and evolution by providing a venue for sharing protocols, videos, datasets, software, and publications. A dedicated discussion forum fosters a sense of community among researchers while addressing questions not covered in published manuscripts. Together, scientists from around the world are advancing scientific discovery at a faster pace, reducing wasted time and effort, and instilling greater confidence in the resulting data.

Recent advances in archaeogenomics have granted access to previously unavailable biological information with the potential to further our understanding of past social dynamics at a range of scales. However, to properly integrate these data within archaeological narratives, new methodological and theoretical tools are required. Effort must be put into finding new methods for weaving together different datasets where material culture and archaeogenomic data are both constitutive elements. This is true on a small scale, when we study relationships at the individual level, and at a larger scale when we deal with social and population dynamics. Specifically, in the study of kinship systems it is essential to contextualize and make sense of biological relatedness through social relations, which, in archaeology, is achieved by using material culture as a proxy. In this paper we propose a Network Science framework to integrate archaeogenomic data and material culture at an intrasite scale to study biological relatedness and social organization at the Neolithic site of \c{C}atalh\"oy\"uk. Methodologically, we propose the use of network variance to investigate the concentration of biological relatedness and material culture within networks of houses. This approach allowed us to observe how material culture similarity between buildings gives valuable information on potential biological relationships between individuals and how biogenetic ties concentrate at specific localities on site.

Vibrotactile stimulation has been explored to accelerate the acquisition of motor skills involving finger movements (Gemicioglu et al. 22, Markow et al. 2010, Seim et al. 17). This study evaluates the effectiveness of vibrotactile stimulation compared to visual feedback in learning a 14-note one-handed tune on the piano. In the experiment, 14 subjects with no prior piano experience were exposed to both vibrotactile and visual stimulation to determine which was more effective. Subjects were randomized 1:1 in a group that first receives vibrotactile stimulation, then visual stimulation or in a group that first receives visual stimulation, then vibrotactile stimulation. Effectiveness was measured by evaluating the timing error and accuracy. Results from our study indicated that the timing error for vibrotactile stimulation was 12.1% (SD 6.0%), while the equivalent for visual stimulation was 22.3% (SD 10.3%). The accuracy for vibrotactile stimulation was 69.2% (SD 27.2%), while the equivalent for visual stimulation was 91.3% (SD 13.5%). It was observed that vibrotactile stimulation was generally more effective at minimizing the timing error at which the notes were hit compared to visual stimulation, and no statistically significant differences were found in accuracy.

Intelligent tutoring systems leverage AI models of expert learning and student knowledge to deliver personalized tutoring to students. While these intelligent tutors have demonstrated improved student learning outcomes, it is still unclear how teachers might integrate them into curriculum and course planning to support responsive pedagogy. In this paper, we conducted a design study with five teachers who have deployed Apprentice Tutors, an intelligent tutoring platform, in their classes. We characterized their challenges around analyzing student interaction data from intelligent tutoring systems and built VisTA (Visualizations for Tutor Analytics), a visual analytics system that shows detailed provenance data across multiple coordinated views. We evaluated VisTA with the same five teachers, and found that the visualizations helped them better interpret intelligent tutor data, gain insights into student problem-solving provenance, and decide on necessary follow-up actions - such as providing students with further support or reviewing skills in the classroom. Finally, we discuss potential extensions of VisTA into sequence query and detection, as well as the potential for the visualizations to be useful for encouraging self-directed learning in students., Comment: 9 pages, 5 figures, ACM AVI 2024

Scientists studying deep ocean microbial ecosystems use limited numbers of sediment samples collected from the seafloor to characterize important life-sustaining biogeochemical cycles in the environment. Yet conducting fieldwork to sample these extreme remote environments is both expensive and time consuming, requiring tools that enable scientists to explore the sampling history of field sites and predict where taking new samples is likely to maximize scientific return. We conducted a collaborative, user-centered design study with a team of scientific researchers to develop DeepSee, an interactive data workspace that visualizes 2D and 3D interpolations of biogeochemical and microbial processes in context together with sediment sampling history overlaid on 2D seafloor maps. Based on a field deployment and qualitative interviews, we found that DeepSee increased the scientific return from limited sample sizes, catalyzed new research workflows, reduced long-term costs of sharing data, and supported teamwork and communication between team members with diverse research goals., Comment: Accepted to CHI 2024. 16 pages, 7 figures, 2 tables. For a demo video, see https://youtu.be/HJ4zbueJ9cs . For a live demo, visit https://www.its.caltech.edu/~datavis/deepsee/ . The source code is available at https://github.com/orphanlab/DeepSee

The recent explosion in popularity of large language models (LLMs) has inspired learning engineers to incorporate them into adaptive educational tools that automatically score summary writing. Understanding and evaluating LLMs is vital before deploying them in critical learning environments, yet their unprecedented size and expanding number of parameters inhibits transparency and impedes trust when they underperform. Through a collaborative user-centered design process with several learning engineers building and deploying summary scoring LLMs, we characterized fundamental design challenges and goals around interpreting their models, including aggregating large text inputs, tracking score provenance, and scaling LLM interpretability methods. To address their concerns, we developed iScore, an interactive visual analytics tool for learning engineers to upload, score, and compare multiple summaries simultaneously. Tightly integrated views allow users to iteratively revise the language in summaries, track changes in the resulting LLM scores, and visualize model weights at multiple levels of abstraction. To validate our approach, we deployed iScore with three learning engineers over the course of a month. We present a case study where interacting with iScore led a learning engineer to improve their LLM's score accuracy by three percentage points. Finally, we conducted qualitative interviews with the learning engineers that revealed how iScore enabled them to understand, evaluate, and build trust in their LLMs during deployment., Comment: Accepted to IUI 2024. 16 pages, 5 figures, 1 table. For a demo video, see https://youtu.be/EYJX-_fQPf0 . For a live demo, visit https://adamcoscia.com/papers/iscore/demo/ . The source code is available at https://github.com/AdamCoscia/iScore

Recent growth in the popularity of large language models has led to their increased usage for summarizing, predicting, and generating text, making it vital to help researchers and engineers understand how and why they work. We present KnowledgeVis, a human-in-the-loop visual analytics system for interpreting language models using fill-in-the-blank sentences as prompts. By comparing predictions between sentences, KnowledgeVis reveals learned associations that intuitively connect what language models learn during training to natural language tasks downstream, helping users create and test multiple prompt variations, analyze predicted words using a novel semantic clustering technique, and discover insights using interactive visualizations. Collectively, these visualizations help users identify the likelihood and uniqueness of individual predictions, compare sets of predictions between prompts, and summarize patterns and relationships between predictions across all prompts. We demonstrate the capabilities of KnowledgeVis with feedback from six NLP experts as well as three different use cases: (1) probing biomedical knowledge in two domain-adapted models; and (2) evaluating harmful identity stereotypes and (3) discovering facts and relationships between three general-purpose models., Comment: Accepted to IEEE TVCG. 20 pages, 10 figures, 1 table. For a demo video, see https://youtu.be/hBX4rSUMr_I . For a live demo, visit https://adamcoscia.com/papers/knowledgevis/demo/ . The source code is available at https://github.com/AdamCoscia/KnowledgeVIS

Data integration is often performed to consolidate information from multiple disparate data sources during visual data analysis. However, integration operations are usually separate from visual analytics operations such as encode and filter in both interface design and empirical research. We conducted a preliminary user study to investigate whether and how data integration should be incorporated directly into the visual analytics process. We used two interface alternatives featuring contrasting approaches to the data preparation and analysis workflow: manual file-based ex-situ integration as a separate step from visual analytics operations; and automatic UI-based in-situ integration merged with visual analytics operations. Participants were asked to complete specific and free-form tasks with each interface, browsing for patterns, generating insights, and summarizing relationships between attributes distributed across multiple files. Analyzing participants' interactions and feedback, we found both task completion time and total interactions to be similar across interfaces and tasks, as well as unique integration strategies between interfaces and emergent behaviors related to satisficing and cognitive bias. Participants' time spent and interactions revealed that in-situ integration enabled users to spend more time on analysis tasks compared with ex-situ integration. Participants' integration strategies and analytical behaviors revealed differences in interface usage for generating and tracking hypotheses and insights. With these results, we synthesized preliminary guidelines for designing future visual analytics interfaces that can support integrating attributes throughout an active analysis process., Comment: Accepted to IEEE TVCG. 13 pages, 5 figures. For a study breakdown video, see https://youtu.be/NzVxHn-OpqQ . The source code, data and analysis are available at https://github.com/AdamCoscia/Integration-Guidelines-VA

Recently, the first author proposed a measure to calculate Pearson correlations for node values expressed in a network, by taking into account distances or metrics defined on the network. In this technical note, we show that using an arbitrary choice of distances might result in imaginary or unbounded correlation values, which is undesired. We prove that this problem is solved by restricting to a special class of distances: negative type metrics. We also discuss two natural classes of negative type metrics on graphs, for which the network correlations are properly defined.

Complex networks are a useful tool to investigate various phenomena in social science, economics, and logistics. Node Vector Distance (NVD) is an emerging set of techniques allowing us to estimate the distance and correlation between variables defined on the nodes of a network. One drawback of NVD is its high computational complexity. Here we show that a subset of NVD techniques, the ones calculating the Generalized Euclidean measure on networks, can be efficiently tackled with Laplacian solvers. In experiments, we show that this provides a significant runtime speedup with negligible approximation errors, which opens the possibility to scale the techniques to large networks.

Data clustering, the task of grouping observations according to their similarity, is a key component of unsupervised learning -- with real world applications in diverse fields such as biology, medicine, and social science. Often in these fields the data comes with complex interdependencies between the dimensions of analysis, for instance the various characteristics and opinions people can have live on a complex social network. Current clustering methods are ill-suited to tackle this complexity: deep learning can approximate these dependencies, but not take their explicit map as the input of the analysis. In this paper, we aim at fixing this blind spot in the unsupervised learning literature. We can create network-aware embeddings by estimating the network distance between numeric node attributes via the generalized Euclidean distance. Differently from all methods in the literature that we know of, we do not cluster the nodes of the network, but rather its node attributes. In our experiments we show that having these network embeddings is always beneficial for the learning task; that our method scales to large networks; and that we can actually provide actionable insights in applications in a variety of fields such as marketing, economics, and political science. Our method is fully open source and data and code are available to reproduce all results in the paper.

In this work, we present GAROM, a new approach for reduced order modelling (ROM) based on generative adversarial networks (GANs). GANs have the potential to learn data distribution and generate more realistic data. While widely applied in many areas of deep learning, little research is done on their application for ROM, i.e. approximating a high-fidelity model with a simpler one. In this work, we combine the GAN and ROM framework, by introducing a data-driven generative adversarial model able to learn solutions to parametric differential equations. The latter is achieved by modelling the discriminator network as an autoencoder, extracting relevant features of the input, and applying a conditioning mechanism to the generator and discriminator networks specifying the differential equation parameters. We show how to apply our methodology for inference, provide experimental evidence of the model generalisation, and perform a convergence study of the method.

This paper estimates the impact of violence on domestic and international migration in Mexico during 2005-2020. Unlike earlier studies, we use network analysis to quantify changes in the degree of interconnectedness among all municipalities in Mexico and with the US, breaking it down by origin-destination pairs. To identify the impact of violence on the changes of the migration network, we use instrumental variables. We exploit the exogenous variation in heroin street prices in the US, local prices of corn in Mexico, and differences in which regions received more Chinese immigrant communities during the 1930s, members of which introduced opium cultivation and drug smuggling networks in Mexico. These instruments help us identify the areas with a current presence of drug trafficking organizations and facing increases in violence. We show that municipalities that experience rises in homicide rates continue to attract immigrants within the country, but they face a larger rise in domestic emigration. The violence also discourages the return migration of Mexicans from the US and non-transit international emigration. Consistent with our findings, areas affected by violence also experience a long-term reduction in vehicle trips made in the highway network connecting them with the rest of the country.

Malware authors often use cryptographic tools such as XOR encryption and block ciphers like AES to obfuscate part of the malware to evade detection. Use of cryptography may give the impression that these obfuscation techniques have some provable guarantees of success. In this paper, we take a closer look at the use of cryptographic tools to obfuscate malware. We first find that most techniques are easy to defeat (in principle), since the decryption algorithm and the key is shipped within the program. In order to clearly define an obfuscation technique's potential to evade detection we propose a principled definition of malware obfuscation, and then categorize instances of malware obfuscation that use cryptographic tools into those which evade detection and those which are detectable. We find that schemes that are hard to de-obfuscate necessarily rely on a construct based on environmental keying. We also show that cryptographic notions of obfuscation, e.g., indistinghuishability and virtual black box obfuscation, may not guarantee evasion detection under our model. However, they can be used in conjunction with environmental keying to produce hard to de-obfuscate version of programs., Comment: This is the full version of the paper with the same title to appear in the Journal of Computer Virology and Hacking Techniques

Multiplex networks allow us to study a variety of complex systems where nodes connect to each other in multiple ways, for example friend, family, and co-worker relations in social networks. Link prediction is the branch of network analysis allowing us to forecast the future status of a network: which new connections are the most likely to appear in the future? In multiplex link prediction we also ask: of which type? Because this last question is unanswerable with classical link prediction, here we investigate the use of graph association rules to inform multiplex link prediction. We derive such rules by identifying all frequent patterns in a network via multiplex graph mining, and then score each unobserved link's likelihood by finding the occurrences of each rule in the original network. Association rules add new abilities to multiplex link prediction: to predict new node arrivals, to consider higher order structures with four or more nodes, and to be memory efficient. We improve over previous work by creating a framework that is also efficient in terms of runtime, which enables an increase in prediction performance. This increase in efficiency allows us to improve a case study on a signed multiplex network, showing how graph association rules can provide valuable insights to extend social balance theory., Comment: arXiv admin note: substantial text overlap with arXiv:2008.08351

Convolutional Neural Network (CNN) is one of the most important architectures in deep learning. The fundamental building block of a CNN is a trainable filter, represented as a discrete grid, used to perform convolution on discrete input data. In this work, we propose a continuous version of a trainable convolutional filter able to work also with unstructured data. This new framework allows exploring CNNs beyond discrete domains, enlarging the usage of this important learning technique for many more complex problems. Our experiments show that the continuous filter can achieve a level of accuracy comparable to the state-of-the-art discrete filter, and that it can be used in current deep learning architectures as a building block to solve problems with unstructured domains as well.