Search

This paper introduces a novel measure of software innovation based on open source software (OSS) development activity on GitHub. We examine the dependency growth and release complexity among $\sim$200,000 unique releases from 28,000 unique packages across the JavaScript, Python, and Ruby ecosystems over two years post-release. We find that major versions show differential, strong prediction of one-year lagged log change in dependencies. In addition, semantic versioning of OSS releases is correlated with their complexity and predict downstream adoption. We conclude that major releases of OSS packages count as a unit of innovation complementary to scientific publications, patents, and standards, offering applications for policymakers, managers, and researchers.

In recent years, with the rapid advancement of large language models (LLMs), multi-agent systems have become increasingly more capable of practical application. At the same time, the software development industry has had a number of new AI-powered tools developed that improve the software development lifecycle (SDLC). Academically, much attention has been paid to the role of multi-agent systems to the SDLC. And, while single-agent systems have frequently been examined in real-world applications, we have seen comparatively few real-world examples of publicly available commercial tools working together in a multi-agent system with measurable improvements. In this experiment we test context sharing between Crowdbotics PRD AI, a tool for generating software requirements using AI, and GitHub Copilot, an AI pair-programming tool. By sharing business requirements from PRD AI, we improve the code suggestion capabilities of GitHub Copilot by 13.8% and developer task success rate by 24.5% -- demonstrating a real-world example of commercially-available AI systems working together with improved outcomes., Comment: 7 pages, 8 figures

Looped Transformers offer advantages in parameter efficiency and Turing completeness. However, their expressive power for function approximation and approximation rate remains underexplored. In this paper, we establish approximation rates of Looped Transformers by defining the concept of the modulus of continuity for sequence-to-sequence functions. This reveals a limitation specific to the looped architecture. That is, the analysis prompts us to incorporate scaling parameters for each loop, conditioned on timestep encoding. Experimental results demonstrate that increasing the number of loops enhances performance, with further gains achieved through the timestep encoding architecture.

Can advanced multi-modal models effectively tackle complex web-based tasks? Such tasks are often found on crowdsourcing platforms, where crowdworkers engage in challenging micro-tasks within web-based environments. Building on this idea, we present TurkingBench, a benchmark consisting of tasks presented as web pages with textual instructions and multi-modal contexts. Unlike previous approaches that rely on artificially synthesized web pages, our benchmark uses natural HTML pages originally designed for crowdsourcing workers to perform various annotation tasks. Each task's HTML instructions are instantiated with different values derived from crowdsourcing tasks, creating diverse instances. This benchmark includes 32.2K instances spread across 158 tasks. To support the evaluation of TurkingBench, we have developed a framework that links chatbot responses to actions on web pages (e.g., modifying a text box, selecting a radio button). We assess the performance of cutting-edge private and open-source models, including language-only and vision-language models (such as GPT4 and InternVL), on this benchmark. Our results show that while these models outperform random chance, there is still significant room for improvement. We hope that this benchmark will drive progress in the evaluation and development of web-based agents.

GitHub is the world's largest platform for collaborative software development, with over 100 million users. GitHub is also used extensively for open data collaboration, hosting more than 800 million open data files, totaling 142 terabytes of data. This study highlights the potential of open data on GitHub and demonstrates how it can accelerate AI research. We analyze the existing landscape of open data on GitHub and the patterns of how users share datasets. Our findings show that GitHub is one of the largest hosts of open data in the world and has experienced an accelerated growth of open data assets over the past four years. By examining the open data landscape on GitHub, we aim to empower users and organizations to leverage existing open datasets and improve their discoverability -- ultimately contributing to the ongoing AI revolution to help address complex societal issues. We release the three datasets that we have collected to support this analysis as open datasets at https://github.com/github/open-data-on-github., Comment: In submission to NeurIPS 2023 Track Datasets and Benchmarks

Deep neural networks (DNNs) are the de facto standard for essential use cases, such as image classification, computer vision, and natural language processing. As DNNs and datasets get larger, they require distributed training on increasingly larger clusters. A main bottleneck is the resulting communication overhead where workers exchange model updates (i.e., gradients) on a per-round basis. To address this bottleneck and accelerate training, a widely-deployed approach is compression. However, previous deployments often apply bi-directional compression schemes by simply using a uni-directional gradient compression scheme in each direction. This results in significant computational overheads at the parameter server and increased compression error, leading to longer training and lower accuracy. We introduce Tensor Homomorphic Compression (THC), a novel bi-directional compression framework that enables the direct aggregation of compressed values and thus eliminating the aforementioned computational overheads. Moreover, THC is compatible with in-network aggregation (INA), which allows for further acceleration. Our evaluation shows that training representative vision and language models with THC reaches target accuracy by 1.40x to 1.47x faster using INA and 1.28x to 1.33x faster using a software PS compared with state-of-the-art systems., Comment: 12 pages body, 21 pages total

Excitonic semiconductors have been a subject of research for photovoltaic applications for many decades. Among them, the organic polymers and small molecules based solar cells have now exceeded 19% power conversion efficiency (PCE). While organic photovoltaics (OPVs) are approaching maturity, the advent of strongly excitonic inorganic semiconductors such as two-dimensional transition metal dichalcogenides (TMDCs) has renewed interest in excitonic solar cells due to their high-optical constants, stable inorganic structure and sub-nm film thicknesses. While several reports have been published on TMDC based PVs, achieving power conversion efficiencies higher than 6% under one-sun AM1.5G illumination has remained challenging. Here, we perform a full optical and electronic analysis of design, structure and performance of monolayer TMDC based, single-junction excitonic PVs. Our computational model with optimized properties predicts a PCE of 9.22% in a superlattice device structure. Our analysis suggests that, while the PCE for 2D excitonic solar cells may be limited to < 10%, a specific power > 100 W g-1 may be achieved with our proposed designs, making them attractive in aerospace, distributed remote sensing, and wearable electronics., Comment: Manuscript plus supplement

Kidney transplantation is the preferred treatment for people suffering from end-stage renal disease. Successful kidney transplants still fail over time, known as graft failure; however, the time to graft failure, or graft survival time, can vary significantly between different recipients. A significant biological factor affecting graft survival times is the compatibility between the human leukocyte antigens (HLAs) of the donor and recipient. We propose to model HLA compatibility using a network, where the nodes denote different HLAs of the donor and recipient, and edge weights denote compatibilities of the HLAs, which can be positive or negative. The network is indirectly observed, as the edge weights are estimated from transplant outcomes rather than directly observed. We propose a latent space model for such indirectly-observed weighted and signed networks. We demonstrate that our latent space model can not only result in more accurate estimates of HLA compatibilities, but can also be incorporated into survival analysis models to improve accuracy for the downstream task of predicting graft survival times., Comment: This work has been accepted to BIBM 2022

The pricing of housing properties is determined by a variety of factors. However, post-pandemic markets have experienced volatility in the Chicago suburb area, which have affected house prices greatly. In this study, analysis was done on the Naperville/Bolingbrook real estate market to predict property prices based on these housing attributes through machine learning models, and to evaluate the effectiveness of such models in a volatile market space. Gathering data from Redfin, a real estate website, sales data from 2018 up until the summer season of 2022 were collected for research. By analyzing these sales in this range of time, we can also look at the state of the housing market and identify trends in price. For modeling the data, the models used were linear regression, support vector regression, decision tree regression, random forest regression, and XGBoost regression. To analyze results, comparison was made on the MAE, RMSE, and R-squared values for each model. It was found that the XGBoost model performs the best in predicting house prices despite the additional volatility sponsored by post-pandemic conditions. After modeling, Shapley Values (SHAP) were used to evaluate the weights of the variables in constructing models.

We present a general method to analyze the topological nature of the domain boundary connectivity that appeared in relaxed moir\'e superlattice patterns at the interface of 2-dimensional (2D) van der Waals (vdW) materials. At large enough moir\'e lengths, all moir\'e systems relax into commensurated 2D domains separated by networks of dislocation lines. The nodes of the 2D dislocation line network can be considered as vortex-like topological defects. We find that a simple analogy to common topological systems with an $S^1$ order parameter, such as a superconductor or planar ferromagnet, cannot correctly capture the topological nature of these defects. For example, in twisted bilayer graphene, the order parameter space for the relaxed moir\'e system is homotopy equivalent to a punctured torus. Here, the nodes of the 2D dislocation network can be characterized as elements of the fundamental group of the punctured torus, the free group on two generators, endowing these network nodes with non-Abelian properties. Extending this analysis to consider moir\'e patterns generated from any relative strain, we find that antivortices occur in the presence of anisotropic heterostrain, such as shear or anisotropic expansion, while arrays of vortices appear under twist or isotropic expansion between vdW materials. Experimentally, utilizing the dark field imaging capability of transmission electron microscopy (TEM), we demonstrate the existence of vortex and antivortex pair formation in a moir\'e system, caused by competition between different types of heterostrains in the vdW interfaces. We also present a methodology for mapping the underlying heterostrain of a moir\'e structure from experimental TEM data, which provides a quantitative relation between the various components of heterostrain and vortex-antivortex density in moir\'e systems., Comment: 15 pages with 11 figures

Dynamic or temporal networks enable representation of time-varying edges between nodes. Conventional adjacency-based data structures used for storing networks such as adjacency lists were designed without incorporating time and can thus quickly retrieve all edges between two sets of nodes (a node-based slice) but cannot quickly retrieve all edges that occur within a given time interval (a time-based slice). We propose a hybrid data structure for storing temporal networks that stores edges in both an adjacency dictionary, enabling rapid node-based slices, and an interval tree, enabling rapid time-based slices. Our hybrid structure also enables compound slices, where one needs to slice both over nodes and time, either by slicing first over nodes or slicing first over time. We further propose an approach for predictive compound slicing, which attempts to predict whether a node-based or time-based compound slice is more efficient. We evaluate our hybrid data structure on many real temporal network data sets and find that they achieve much faster slice times than existing data structures with only a modest increase in creation time and memory usage., Comment: Extended version of COMPLEX NETWORKS 2021 paper

Networks and temporal point processes serve as fundamental building blocks for modeling complex dynamic relational data in various domains. We propose the latent space Hawkes (LSH) model, a novel generative model for continuous-time networks of relational events, using a latent space representation for nodes. We model relational events between nodes using mutually exciting Hawkes processes with baseline intensities dependent upon the distances between the nodes in the latent space and sender and receiver specific effects. We demonstrate that our proposed LSH model can replicate many features observed in real temporal networks including reciprocity and transitivity, while also achieving superior prediction accuracy and providing more interpretable fits than existing models., Comment: To appear in UAI 2022. Code available at https://github.com/IdeasLabUT/Latent-Space-Hawkes

The stochastic block model (SBM) is one of the most widely used generative models for network data. Many continuous-time dynamic network models are built upon the same assumption as the SBM: edges or events between all pairs of nodes are conditionally independent given the block or community memberships, which prevents them from reproducing higher-order motifs such as triangles that are commonly observed in real networks. We propose the multivariate community Hawkes (MULCH) model, an extremely flexible community-based model for continuous-time networks that introduces dependence between node pairs using structured multivariate Hawkes processes. We fit the model using a spectral clustering and likelihood-based local refinement procedure. We find that our proposed MULCH model is far more accurate than existing models both for predictive and generative tasks., Comment: To appear at ICML 2022. Code available at https://github.com/IdeasLabUT/Multivariate-Community-Hawkes

User-generated data on social media contain rich information about who we are, what we like and how we make decisions. In this paper, we survey representative work on learning a concise latent user representation (a.k.a. user embedding) that can capture the main characteristics of a social media user. The learned user embeddings can later be used to support different downstream user analysis tasks such as personality modeling, suicidal risk assessment and purchase decision prediction. The temporal nature of user-generated data on social media has largely been overlooked in much of the existing user embedding literature. In this survey, we focus on research that bridges the gap by incorporating temporal/sequential information in user representation learning. We categorize relevant papers along several key dimensions, identify limitations in the current work and suggest future research directions.

Kidney transplantation can significantly enhance living standards for people suffering from end-stage renal disease. A significant factor that affects graft survival time (the time until the transplant fails and the patient requires another transplant) for kidney transplantation is the compatibility of the Human Leukocyte Antigens (HLAs) between the donor and recipient. In this paper, we propose 4 new biologically-relevant feature representations for incorporating HLA information into machine learning-based survival analysis algorithms. We evaluate our proposed HLA feature representations on a database of over 100,000 transplants and find that they improve prediction accuracy by about 1%, modest at the patient level but potentially significant at a societal level. Accurate prediction of survival times can improve transplant survival outcomes, enabling better allocation of donors to recipients and reducing the number of re-transplants due to graft failure with poorly matched donors., Comment: Extended version of AIME 2021 conference paper

In many application settings involving networks, such as messages between users of an on-line social network or transactions between traders in financial markets, the observed data consist of timestamped relational events, which form a continuous-time network. We propose the Community Hawkes Independent Pairs (CHIP) generative model for such networks. We show that applying spectral clustering to an aggregated adjacency matrix constructed from the CHIP model provides consistent community detection for a growing number of nodes and time duration. We also develop consistent and computationally efficient estimators for the model parameters. We demonstrate that our proposed CHIP model and estimation procedure scales to large networks with tens of thousands of nodes and provides superior fits than existing continuous-time network models on several real networks., Comment: 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada. Source code is available at https://github.com/IdeasLabUT/CHIP-Network-Model

Mid-infrared molecular hydrogen (H$_2$) emission is a powerful cooling agent in galaxy mergers and in radio galaxies; it is a potential key tracer of gas evolution and energy dissipation associated with mergers, star formation, and accretion onto supermassive black holes. We detect mid-IR H$_2$ line emission in at least one rotational transition in 91\% of the 214 Luminous Infrared Galaxies (LIRGs) observed with Spitzer as part of the Great Observatories All-sky LIRG Survey (GOALS). We use H$_2$ excitation diagrams to estimate the range of masses and temperatures of warm molecular gas in these galaxies. We find that LIRGs in which the IR emission originates mostly from the Active Galactic Nuclei (AGN) have about 100K higher H$_2$ mass-averaged excitation temperatures than LIRGs in which the IR emission originates mostly from star formation. Between 10 and 15\% of LIRGs have H$_2$ emission lines that are sufficiently broad to be resolved or partially resolved by the high resolution modules of Spitzer's Infrared Spectrograph (IRS). Those sources tend to be mergers and contain AGN. This suggests that a significant fraction of the H$_2$ line emission is powered by AGN activity through X-rays, cosmic rays, and turbulence. We find a statistically significant correlation between the kinetic energy in the H$_2$ gas and the H$_2$ to IR luminosity ratio. The sources with the largest warm gas kinetic energies are mergers. We speculate that mergers increase the production of bulk in-flows leading to observable broad H$_2$ profiles and possibly denser environments., Comment: 31 pages, 12 figures, 7 tables, accepted for publication in the Astrophysical Journal

On-line social networks (OSNs) often contain many different types of relationships between users. When studying the structure of OSNs such as Facebook, two of the most commonly studied networks are friendship and interaction networks. The link prediction problem in friendship networks has been heavily studied. There has also been prior work on link prediction in interaction networks, independent of friendship networks. In this paper, we study the predictive power of combining friendship and interaction networks. We hypothesize that, by leveraging friendship networks, we can improve the accuracy of link prediction in interaction networks. We augment several interaction link prediction algorithms to incorporate friendships and predicted friendships. From experiments on Facebook data, we find that incorporating friendships into interaction link prediction algorithms results in higher accuracy, but incorporating predicted friendships does not when compared to incorporating current friendships., Comment: To appear in ICWSM 2018. This version corrects some minor errors in Table 1. MATLAB code available at https://github.com/IdeasLabUT/Friendship-Interaction-Prediction

The temporal dynamics of a complex system such as a social network or a communication network can be studied by understanding the patterns of link appearance and disappearance over time. A critical task along this understanding is to predict the link state of the network at a future time given a collection of link states at earlier time points. In existing literature, this task is known as link prediction in dynamic networks. Solving this task is more difficult than its counterpart in static networks because an effective feature representation of node-pair instances for the case of dynamic network is hard to obtain. To overcome this problem, we propose a novel method for metric embedding of node-pair instances of a dynamic network. The proposed method models the metric embedding task as an optimal coding problem where the objective is to minimize the reconstruction error, and it solves this optimization task using a gradient descent method. We validate the effectiveness of the learned feature representation by utilizing it for link prediction in various real-life dynamic networks. Specifically, we show that our proposed link prediction model, which uses the extracted feature representation for the training instances, outperforms several existing methods that use well-known link prediction features.

Understanding mechanisms driving link formation in dynamic social networks is a long-standing problem that has implications to understanding social structure as well as link prediction and recommendation. Social networks exhibit a high degree of transitivity, which explains the successes of common neighbor-based methods for link prediction. In this paper, we examine mechanisms behind link formation from the perspective of an ego node. We introduce the notion of personalized degree for each neighbor node of the ego, which is the number of other neighbors a particular neighbor is connected to. From empirical analyses on four on-line social network datasets, we find that neighbors with higher personalized degree are more likely to lead to new link formations when they serve as common neighbors with other nodes, both in undirected and directed settings. This is complementary to the finding of Adamic and Adar that neighbor nodes with higher (global) degree are less likely to lead to new link formations. Furthermore, on directed networks, we find that personalized out-degree has a stronger effect on link formation than personalized in-degree, whereas global in-degree has a stronger effect than global out-degree. We validate our empirical findings through several link recommendation experiments and observe that incorporating both personalized and global degree into link recommendation greatly improves accuracy., Comment: To appear at the 10th International Workshop on Modeling Social Media co-located with the Web Conference 2019

We consider the problem of analyzing timestamped relational events between a set of entities, such as messages between users of an on-line social network. Such data are often analyzed using static or discrete-time network models, which discard a significant amount of information by aggregating events over time to form network snapshots. In this paper, we introduce a block point process model (BPPM) for continuous-time event-based dynamic networks. The BPPM is inspired by the well-known stochastic block model (SBM) for static networks. We show that networks generated by the BPPM follow an SBM in the limit of a growing number of nodes. We use this property to develop principled and efficient local search and variational inference procedures initialized by regularized spectral clustering. We fit BPPMs with exponential Hawkes processes to analyze several real network data sets, including a Facebook wall post network with over 3,500 nodes and 130,000 events., Comment: To appear at The Web Conference 2019

The importance of modeling the spread of epidemics through a population has led to the development of mathematical models for infectious disease propagation. A number of empirical studies have collected and analyzed data on contacts between individuals using a variety of sensors. Typically one uses such data to fit a probabilistic model of network contacts over which a disease may propagate. In this paper, we investigate the effects of different contact network models with varying levels of complexity on the outcomes of simulated epidemics using a stochastic Susceptible-Infectious-Recovered (SIR) model. We evaluate these network models on six datasets of contacts between people in a variety of settings. Our results demonstrate that the choice of network model can have a significant effect on how closely the outcomes of an epidemic simulation on a simulated network match the outcomes on the actual network constructed from the sensor data. In particular, preserving degrees of nodes appears to be much more important than preserving cluster structure for accurate epidemic simulations., Comment: To appear at International Conference on Social Informatics (SocInfo) 2017

One of the main benefits of a wrist-worn computer is its ability to collect a variety of physiological data in a minimally intrusive manner. Among these data, electrodermal activity (EDA) is readily collected and provides a window into a person's emotional and sympathetic responses. EDA data collected using a wearable wristband are easily influenced by motion artifacts (MAs) that may significantly distort the data and degrade the quality of analyses performed on the data if not identified and removed. Prior work has demonstrated that MAs can be successfully detected using supervised machine learning algorithms on a small data set collected in a lab setting. In this paper, we demonstrate that unsupervised learning algorithms perform competitively with supervised algorithms for detecting MAs on EDA data collected in both a lab-based setting and a real-world setting comprising about 23 hours of data. We also find, somewhat surprisingly, that incorporating accelerometer data as well as EDA improves detection accuracy only slightly for supervised algorithms and significantly degrades the accuracy of unsupervised algorithms., Comment: To appear at International Symposium on Wearable Computers (ISWC) 2017

The Spectral and Photometric Imaging Receiver (SPIRE) was launched as one of the scientific instruments on board of the space observatory Herschel. The SPIRE photometer opened up an entirely new window in the Submillimeter domain for large scale mapping, that up to then was very difficult to observe. There are already several catalogs that were produced by individual Herschel science projects. Yet, we estimate that the objects of only a fraction of these maps will ever be systematically extracted and published by the science teams that originally proposed the observations. The SPIRE instrument performed its standard photometric observations in an optically very stable configuration, only moving the telescope across the sky, with variations in its configuration parameters limited to scan speed and sampling rate. This and the scarcity of features in the data that require special processing steps made this dataset very attractive for producing an expert reduced catalog of point sources that is being described in this document. The Catalog was extracted from a total of 6878 unmodified SPIRE scan map observations. The photometry was obtained by a systematic and homogeneous source extraction procedure, followed by a rigorous quality check that emphasized reliability over completeness. Having to exclude regions affected by strong Galactic emission, that pushed the limits of the four source extraction methods that were used, this catalog is aimed primarily at the extragalactic community. The result can serve as a pathfinder for ALMA and other Submillimeter and Far-Infrared facilities. 1,693,718 sources are included in the final catalog, splitting into 950688, 524734, 218296 objects for the 250\mu m, 350\mu m, and 500\mu m bands, respectively. The catalog comes with well characterized environments, reliability, completeness, and accuracies, that single programs typically cannot provide.

A common problem in large-scale data analysis is to approximate a matrix using a combination of specifically sampled rows and columns, known as CUR decomposition. Unfortunately, in many real-world environments, the ability to sample specific individual rows or columns of the matrix is limited by either system constraints or cost. In this paper, we consider matrix approximation by sampling predefined \emph{blocks} of columns (or rows) from the matrix. We present an algorithm for sampling useful column blocks and provide novel guarantees for the quality of the approximation. This algorithm has application in problems as diverse as biometric data analysis to distributed computing. We demonstrate the effectiveness of the proposed algorithms for computing the Block CUR decomposition of large matrices in a distributed setting with multiple nodes in a compute cluster, where such blocks correspond to columns (or rows) of the matrix stored on the same node, which can be retrieved with much less overhead than retrieving individual columns stored across different nodes. In the biometric setting, the rows correspond to different users and columns correspond to users' biometric reaction to external stimuli, {\em e.g.,}~watching video content, at a particular time instant. There is significant cost in acquiring each user's reaction to lengthy content so we sample a few important scenes to approximate the biometric response. An individual time sample in this use case cannot be queried in isolation due to the lack of context that caused that biometric reaction. Instead, collections of time segments ({\em i.e.,} blocks) must be presented to the user. The practical application of these algorithms is shown via experimental results using real-world user biometric data from a content testing environment., Comment: shorter version to appear in ECML-PKDD 2018