Search

We study the problem of learning latent community structure from multiple correlated networks, focusing on edge-correlated stochastic block models with two balanced communities. Recent work of Gaudio, R\'acz, and Sridhar (COLT 2022) determined the precise information-theoretic threshold for exact community recovery using two correlated graphs; in particular, this showcased the subtle interplay between community recovery and graph matching. Here we study the natural setting of more than two graphs. The main challenge lies in understanding how to aggregate information across several graphs when none of the pairwise latent vertex correspondences can be exactly recovered. Our main result derives the precise information-theoretic threshold for exact community recovery using any constant number of correlated graphs, answering a question of Gaudio, R\'acz, and Sridhar (COLT 2022). In particular, for every $K \geq 3$ we uncover and characterize a region of the parameter space where exact community recovery is possible using $K$ correlated graphs, even though (1) this is information-theoretically impossible using any $K-1$ of them and (2) none of the latent matchings can be exactly recovered., Comment: 53 pages, 4 figures. To appear in Advances in Neural Information Processing Systems (NeurIPS) 2024

We study learning problems on correlated stochastic block models with two balanced communities. Our main result gives the first efficient algorithm for graph matching in this setting. In the most interesting regime where the average degree is logarithmic in the number of vertices, this algorithm correctly matches all but a vanishing fraction of vertices with high probability, whenever the edge correlation parameter $s$ satisfies $s^2 > \alpha \approx 0.338$, where $\alpha$ is Otter's tree-counting constant. Moreover, we extend this to an efficient algorithm for exact graph matching whenever this is information-theoretically possible, positively resolving an open problem of R\'acz and Sridhar (NeurIPS 2021). Our algorithm generalizes the recent breakthrough work of Mao, Wu, Xu, and Yu (STOC 2023), which is based on centered subgraph counts of a large family of trees termed chandeliers. A major technical challenge that we overcome is dealing with the additional estimation errors that are necessarily present due to the fact that, in relevant parameter regimes, the latent community partition cannot be exactly recovered from a single graph. As an application of our results, we give an efficient algorithm for exact community recovery using multiple correlated graphs in parameter regimes where it is information-theoretically impossible to do so using just a single graph., Comment: 73 pages, 8 figures; To appear in Advances in Neural Information Processing Systems (NeurIPS) 2024

We consider the task of estimating the latent vertex correspondence between two edge-correlated random graphs with generic, inhomogeneous structure. We study the so-called \emph{$k$-core estimator}, which outputs a vertex correspondence that induces a large, common subgraph of both graphs which has minimum degree at least $k$. We derive sufficient conditions under which the $k$-core estimator exactly or partially recovers the latent vertex correspondence. Finally, we specialize our general framework to derive new results on exact and partial recovery in correlated stochastic block models, correlated Chung-Lu graphs, and correlated random geometric graphs., Comment: 9 pages

We propose a simple and efficient local algorithm for graph isomorphism which succeeds for a large class of sparse graphs. This algorithm produces a low-depth canonical labeling, which is a labeling of the vertices of the graph that identifies its isomorphism class using vertices' local neighborhoods. Prior work by Czajka and Pandurangan showed that the degree profile of a vertex (i.e., the sorted list of the degrees of its neighbors) gives a canonical labeling with high probability when $n p_n = \omega( \log^{4}(n) / \log \log n )$ (and $p_{n} \leq 1/2$); subsequently, Mossel and Ross showed that the same holds when $n p_n = \omega( \log^{2}(n) )$. We first show that their analysis essentially cannot be improved: we prove that when $n p_n = o( \log^{2}(n) / (\log \log n)^{3} )$, with high probability there exist distinct vertices with isomorphic $2$-neighborhoods. Our first main result is a positive counterpart to this, showing that $3$-neighborhoods give a canonical labeling when $n p_n \geq (1+\delta) \log n$ (and $p_n \leq 1/2$); this improves a recent result of Ding, Ma, Wu, and Xu, completing the picture above the connectivity threshold. Our second main result is a smoothed analysis of graph isomorphism, showing that for a large class of deterministic graphs, a small random perturbation ensures that $3$-neighborhoods give a canonical labeling with high probability. While the worst-case complexity of graph isomorphism is still unknown, this shows that graph isomorphism has polynomial smoothed complexity., Comment: v2 contains major updates; in particular, the results have been extended to a smoothed analysis of graph isomorphism. The changes are also reflected in the new title. 30 pages, 3 figures

This paper studies how a centralized planner can modify the structure of a social or information network to reduce polarization. First, polarization is found to be highly dependent on degree and structural properties of the network -- including the well-known isoperimetric number (i.e., Cheeger constant). We then formulate the planner's problem under full information, and motivate disagreement-seeking and coordinate descent heuristics. A novel setting for the planner in which the population's innate opinions are adversarially chosen is introduced, and shown to be equivalent to maximization of the Laplacian's spectral gap. We prove bounds for the effectiveness of a strategy that adds edges between vertices on opposite sides of the cut induced by the spectral gap's eigenvector. Finally, these strategies are evaluated on six real-world and synthetic networks. In several networks, we find that polarization can be significantly reduced through the addition of a small number of edges., Comment: 29 pages, 12 figures

Tree-based models are used in many high-stakes application domains such as finance and medicine, where robustness and interpretability are of utmost importance. Yet, methods for improving and certifying their robustness are severely under-explored, in contrast to those focusing on neural networks. Targeting this important challenge, we propose deterministic smoothing for decision stump ensembles. Whereas most prior work on randomized smoothing focuses on evaluating arbitrary base models approximately under input randomization, the key insight of our work is that decision stump ensembles enable exact yet efficient evaluation via dynamic programming. Importantly, we obtain deterministic robustness certificates, even jointly over numerical and categorical features, a setting ubiquitous in the real world. Further, we derive an MLE-optimal training method for smoothed decision stumps under randomization and propose two boosting approaches to improve their provable robustness. An extensive experimental evaluation on computer vision and tabular data tasks shows that our approach yields significantly higher certified accuracies than the state-of-the-art for tree-based models. We release all code and trained models at https://github.com/eth-sri/drs., Comment: NeurIPS 2022 Paper

Randomized Smoothing (RS) is considered the state-of-the-art approach to obtain certifiably robust models for challenging tasks. However, current RS approaches drastically decrease standard accuracy on unperturbed data, severely limiting their real-world utility. To address this limitation, we propose a compositional architecture, ACES, which certifiably decides on a per-sample basis whether to use a smoothed model yielding predictions with guarantees or a more accurate standard model without guarantees. This, in contrast to prior approaches, enables both high standard accuracies and significant provable robustness. On challenging tasks such as ImageNet, we obtain, e.g., $80.0\%$ natural accuracy and $28.2\%$ certifiable accuracy against $\ell_2$ perturbations with $r=1.0$. We release our code and models at https://github.com/eth-sri/aces., Comment: Presented at the ICLR 2022 Workshop on Socially Responsible Machine Learning

We consider the problem of learning latent community structure from multiple correlated networks. We study edge-correlated stochastic block models with two balanced communities, focusing on the regime where the average degree is logarithmic in the number of vertices. Our main result derives the precise information-theoretic threshold for exact community recovery using multiple correlated graphs. This threshold captures the interplay between the community recovery and graph matching tasks. In particular, we uncover and characterize a region of the parameter space where exact community recovery is possible using multiple correlated graphs, even though (1) this is information-theoretically impossible using a single graph and (2) exact graph matching is also information-theoretically impossible. In this regime, we develop a novel algorithm that carefully synthesizes algorithms from the community recovery and graph matching literatures., Comment: 54 pages, 6 figures

We study random graphs with latent geometric structure, where the probability of each edge depends on the underlying random positions corresponding to the two endpoints. We focus on the setting where this conditional probability is a general monotone increasing function of the inner product of two vectors; such a function can naturally be viewed as the cumulative distribution function of some independent random variable. We consider a one-parameter family of random graphs, characterized by the variance of this random variable, that smoothly interpolates between a random dot product graph and an Erd\H{o}s--R\'enyi random graph. We prove phase transitions of detecting geometry in these graphs, in terms of the dimension of the underlying geometric space and the variance parameter of the conditional probability. When the dimension is high or the variance is large, the graph is similar to an Erd\H{o}s--R\'enyi graph with the same edge density that does not possess geometry; in other parameter regimes, there is a computationally efficient signed triangle statistic that distinguishes them. The proofs make use of information-theoretic inequalities and concentration of measure phenomena., Comment: 33 pages, 1 figure

We consider the task of learning latent community structure from multiple correlated networks. First, we study the problem of learning the latent vertex correspondence between two edge-correlated stochastic block models, focusing on the regime where the average degree is logarithmic in the number of vertices. We derive the precise information-theoretic threshold for exact recovery: above the threshold there exists an estimator that outputs the true correspondence with probability close to 1, while below it no estimator can recover the true correspondence with probability bounded away from 0. As an application of our results, we show how one can exactly recover the latent communities using multiple correlated graphs in parameter regimes where it is information-theoretically impossible to do so using just a single graph., Comment: 42 pages, 4 figures

Randomized Smoothing (RS) is a promising method for obtaining robustness certificates by evaluating a base model under noise. In this work, we: (i) theoretically motivate why ensembles are a particularly suitable choice as base models for RS, and (ii) empirically confirm this choice, obtaining state-of-the-art results in multiple settings. The key insight of our work is that the reduced variance of ensembles over the perturbations introduced in RS leads to significantly more consistent classifications for a given input. This, in turn, leads to substantially increased certifiable radii for samples close to the decision boundary. Additionally, we introduce key optimizations which enable an up to 55-fold decrease in sample complexity of RS for predetermined radii, thus drastically reducing its computational overhead. Experimentally, we show that ensembles of only 3 to 10 classifiers consistently improve on their strongest constituting model with respect to their average certified radius (ACR) by 5% to 21% on both CIFAR10 and ImageNet, achieving a new state-of-the-art ACR of 0.86 and 1.11, respectively. We release all code and models required to reproduce our results at https://github.com/eth-sri/smoothing-ensembles., Comment: ICLR 2022 Spotlight Paper

We study the problem of detecting latent geometric structure in random graphs. To this end, we consider the soft high-dimensional random geometric graph $\mathcal{G}(n,p,d,q)$, where each of the $n$ vertices corresponds to an independent random point distributed uniformly on the sphere $\mathbb{S}^{d-1}$, and the probability that two vertices are connected by an edge is a decreasing function of the Euclidean distance between the points. The probability of connection is parametrized by $q \in [0,1]$, with smaller $q$ corresponding to weaker dependence on the geometry; this can also be interpreted as the level of noise in the geometric graph. In particular, the model smoothly interpolates between the spherical hard random geometric graph $\mathcal{G}(n,p,d)$ (corresponding to $q = 1$) and the Erd\H{o}s-R\'enyi model $\mathcal{G}(n,p)$ (corresponding to $q = 0$). We focus on the dense regime (i.e., $p$ is a constant). We show that if $nq \to 0$ or $d \gg n^{3} q^{2}$, then geometry is lost: $\mathcal{G}(n,p,d,q)$ is asymptotically indistinguishable from $\mathcal{G}(n,p)$. On the other hand, if $d \ll n^{3} q^{6}$, then the signed triangle statistic provides an asymptotically powerful test for detecting geometry. These results generalize those of Bubeck, Ding, Eldan, and R\'acz (2016) for $\mathcal{G}(n,p,d)$, and give quantitative bounds on how the noise level affects the dimension threshold for losing geometry. We also prove analogous results under a related but different distributional assumption, and we further explore generalizations of signed triangles in order to understand the intermediate regime left open by our results., Comment: 50 pages, 2 figures

Tree trace reconstruction aims to learn the binary node labels of a tree, given independent samples of the tree passed through an appropriately defined deletion channel. In recent work, Davies, R\'acz, and Rashtchian used combinatorial methods to show that $\exp(\mathcal{O}(k \log_{k} n))$ samples suffice to reconstruct a complete $k$-ary tree with $n$ nodes with high probability. We provide an alternative proof of this result, which allows us to generalize it to a broader class of tree topologies and deletion models. In our proofs, we introduce the notion of a subtrace, which enables us to connect with and generalize recent mean-based complex analytic algorithms for string trace reconstruction., Comment: 13 pages, 2 figures

In the usual trace reconstruction problem, the goal is to exactly reconstruct an unknown string of length $n$ after it passes through a deletion channel many times independently, producing a set of traces (i.e., random subsequences of the string). We consider the relaxed problem of approximate reconstruction. Here, the goal is to output a string that is close to the original one in edit distance while using much fewer traces than is needed for exact reconstruction. We present several algorithms that can approximately reconstruct strings that belong to certain classes, where the estimate is within $n/\mathrm{polylog}(n)$ edit distance, and where we only use $\mathrm{polylog}(n)$ traces (or sometimes just a single trace). These classes contain strings that require a linear number of traces for exact reconstruction and which are quite different from a typical random string. From a technical point of view, our algorithms approximately reconstruct consecutive substrings of the unknown string by aligning dense regions of traces and using a run of a suitable length to approximate each region. To complement our algorithms, we present a general black-box lower bound for approximate reconstruction, building on a lower bound for distinguishing between two candidate input strings in the worst case. In particular, this shows that approximating to within $n^{1/3 - \delta}$ edit distance requires $n^{1 + 3\delta/2}/\mathrm{polylog}(n)$ traces for $0< \delta < 1/3$ in the worst case.

Large pools of synthetic DNA molecules have been recently used to reliably store significant volumes of digital data. While DNA as a storage medium has enormous potential because of its high storage density, its practical use is currently severely limited because of the high cost and low throughput of available DNA synthesis technologies. We study the role of batch optimization in reducing the cost of large scale DNA synthesis, which translates to the following algorithmic task. Given a large pool $\mathcal{S}$ of random quaternary strings of fixed length, partition $\mathcal{S}$ into batches in a way that minimizes the sum of the lengths of the shortest common supersequences across batches. We introduce two ideas for batch optimization that both improve (in different ways) upon a naive baseline: (1) using both $(ACGT)^{*}$ and its reverse $(TGCA)^{*}$ as reference strands, and batching appropriately, and (2) batching via the quantiles of an appropriate ordering of the strands. We also prove asymptotically matching lower bounds on the cost of DNA synthesis, showing that one cannot improve upon these two ideas. Our results uncover a surprising separation between two cases that naturally arise in the context of DNA data storage: the asymptotic cost savings of batch optimization are significantly greater in the case where strings in $\mathcal{S}$ do not contain repeats of the same character (homopolymers), as compared to the case where strings in $\mathcal{S}$ are unconstrained., Comment: Improved Theorem 1.2 and its proof

Recent work, motivated by anonymous messaging platforms, has introduced adaptive diffusion protocols which can obfuscate the source of a rumor: a "snapshot adversary" with access to the subgraph of "infected" nodes can do no better than randomly guessing the entity of the source node. What happens if the adversary has access to multiple independent snapshots? We study this question when the underlying graph is the infinite $d$-regular tree. We show that (1) a weak form of source obfuscation is still possible in the case of two independent snapshots, but (2) already with three observations there is a simple algorithm that finds the rumor source with constant probability, regardless of the adaptive diffusion protocol. We also characterize the tradeoff between local spreading and source obfuscation for adaptive diffusion protocols (under a single snapshot). These results raise questions about the robustness of anonymity guarantees when spreading information in social networks., Comment: 30 pages, 3 figures

We introduce a new model of correlated randomly growing graphs and study the fundamental questions of detecting correlation and estimating aspects of the correlated structure. The model is simple and starts with any model of randomly growing graphs, such as uniform attachment (UA) or preferential attachment (PA). Given such a model, a pair of graphs $(G_1, G_2)$ is grown in two stages: until time $t_{\star}$ they are grown together (i.e., $G_1 = G_2$), after which they grow independently according to the underlying growth model. We show that whenever the seed graph has an influence in the underlying graph growth model---this has been shown for PA and UA trees and is conjectured to hold broadly---then correlation can be detected in this model, even if the graphs are grown together for just a single time step. We also give a general sufficient condition (which holds for PA and UA trees) under which detection is possible with probability going to $1$ as $t_{\star} \to \infty$. Finally, we show for PA and UA trees that the amount of correlation, measured by $t_{\star}$, can be estimated with vanishing relative error as $t_{\star} \to \infty$., Comment: 62 pages, 5 figures

Recent years have seen a marked increase in the spread of misinformation, a phenomenon which has been accelerated and amplified by social media such as Facebook and Twitter. While some actors spread misinformation to push a specific agenda, it has also been widely documented that others aim to simply disrupt the network by increasing disagreement and polarization across the network and thereby destabilizing society. Popular social networks are also vulnerable to large-scale attacks. Motivated by this reality, we introduce a simple model of network disruption where an adversary can take over a limited number of user profiles in a social network with the aim of maximizing disagreement and/or polarization in the network. We investigate this model both theoretically and empirically. We show that the adversary will always change the opinion of a taken-over profile to an extreme in order to maximize disruption. We also prove that an adversary can increase disagreement / polarization at most linearly in the number of user profiles it takes over. Furthermore, we present a detailed empirical study of several natural algorithms for the adversary on both synthetic networks and real world (Reddit and Twitter) data sets. These show that even simple, unsophisticated heuristics, such as targeting centrists, can disrupt a network effectively, causing a large increase in disagreement / polarization. Studying the problem of network disruption through the lens of an adversary thus highlights the seriousness of the problem., Comment: 20 pages, 6 figures

We consider a variant of the planted clique problem where we are allowed unbounded computational time but can only investigate a small part of the graph by adaptive edge queries. We determine (up to logarithmic factors) the number of queries necessary both for detecting the presence of a planted clique and for finding the planted clique. Specifically, let $G \sim G(n,1/2,k)$ be a random graph on $n$ vertices with a planted clique of size $k$. We show that no algorithm that makes at most $q = o(n^2 / k^2 + n)$ adaptive queries to the adjacency matrix of $G$ is likely to find the planted clique. On the other hand, when $k \geq (2+\epsilon) \log_2 n$ there exists a simple algorithm (with unbounded computational power) that finds the planted clique with high probability by making $q = O( (n^2 / k^2) \log^2 n + n \log n)$ adaptive queries. For detection, the additive $n$ term is not necessary: the number of queries needed to detect the presence of a planted clique is $n^2 / k^2$ (up to logarithmic factors)., Comment: 14 pages, 1 figure

We study the problem of learning a node-labeled tree given independent traces from an appropriately defined deletion channel. This problem, tree trace reconstruction, generalizes string trace reconstruction, which corresponds to the tree being a path. For many classes of trees, including complete trees and spiders, we provide algorithms that reconstruct the labels using only a polynomial number of traces. This exhibits a stark contrast to known results on string trace reconstruction, which require exponentially many traces, and where a central open problem is to determine whether a polynomial number of traces suffice. Our techniques combine novel combinatorial and complex analytic methods., Comment: Major revisions in the new version including algorithm descriptions, more details in section 3.1, and several new figures

Consider algorithms with unbounded computation time that probe the entries of the adjacency matrix of an $n$ vertex graph, and need to output a clique. We show that if the input graph is drawn at random from $G_{n,\frac{1}{2}}$ (and hence is likely to have a clique of size roughly $2\log n$), then for every $\delta < 2$ and constant $\ell$, there is an $\alpha < 2$ (that may depend on $\delta$ and $\ell$) such that no algorithm that makes $n^{\delta}$ probes in $\ell$ rounds is likely (over the choice of the random graph) to output a clique of size larger than $\alpha \log n$., Comment: 15 pages

It is well known that sequential decision making may lead to information cascades. That is, when agents make decisions based on their private information, as well as observing the actions of those before them, then it might be rational to ignore their private signal and imitate the action of previous individuals. If the individuals are choosing between a right and a wrong state, and the initial actions are wrong, then the whole cascade will be wrong. This issue is due to the fact that cascades can be based on very little information. We show that if agents occasionally disregard the actions of others and base their action only on their private information, then wrong cascades can be avoided. Moreover, we study the optimal asymptotic rate at which the error probability at time $t$ can go to zero. The optimal policy is for the player at time $t$ to follow their private information with probability $p_{t} = c/t$, leading to a learning rate of $c'/t$, where the constants $c$ and $c'$ are explicit., Comment: 19 pages; v2: minor changes

Starting from a unit mass on a vertex of a graph, we investigate the minimum number of "\emph{controlled diffusion}" steps needed to transport a constant mass $p$ outside of the ball of radius $n$. In a step of a controlled diffusion process we may select any vertex with positive mass and topple its mass equally to its neighbors. Our initial motivation comes from the maximum overhang question in one dimension, but the more general case arises from optimal mass transport problems. On $\mathbb{Z}^{d}$ we show that $\Theta( n^{d+2} )$ steps are necessary and sufficient to transport the mass. We also give sharp bounds on the comb graph and $d$-ary trees. Furthermore, we consider graphs where simple random walk has positive speed and entropy and which satisfy Shannon's theorem, and show that the minimum number of controlled diffusion steps is $\exp{( n \cdot h / \ell ( 1 + o(1) ))}$, where $h$ is the Avez asymptotic entropy and $\ell$ is the speed of random walk. As examples, we give precise results on Galton-Watson trees and the product of trees $\mathbb{T}_d \times \mathbb{T}_k$., Comment: 32 pages, 2 figures

It is well known that an $n \times n$ Wishart matrix with $d$ degrees of freedom is close to the appropriately centered and scaled Gaussian Orthogonal Ensemble (GOE) if $d$ is large enough. Recent work of Bubeck, Ding, Eldan, and Racz, and independently Jiang and Li, shows that the transition happens when $d = \Theta ( n^{3} )$. Here we consider this critical window and explicitly compute the total variation distance between the Wishart and GOE matrices when $d / n^{3} \to c \in (0, \infty)$. This shows, in particular, that the phase transition from Wishart to GOE is smooth., Comment: 7 pages, 1 figure

Extracting information from large graphs has become an important statistical problem since network data is now common in various fields. In this minicourse we will investigate the most natural statistical questions for three canonical probabilistic models of networks: (i) community detection in the stochastic block model, (ii) finding the embedding of a random geometric graph, and (iii) finding the original vertex in a preferential attachment tree. Along the way we will cover many interesting topics in probability theory such as P\'olya urns, large deviation theory, concentration of measure in high dimension, entropic central limit theorems, and more., Comment: 38 pages, 10 figures. Lecture notes for a graduate minicourse presented at University of Washington and the XX Brazilian School of Probability in June/July 2016

The prevalent technique for DNA sequencing consists of two main steps: shotgun sequencing, where many randomly located fragments, called reads, are extracted from the overall sequence, followed by an assembly algorithm that aims to reconstruct the original sequence. There are many different technologies that generate the reads: widely-used second-generation methods create short reads with low error rates, while emerging third-generation methods create long reads with high error rates. Both error rates and error profiles differ among methods, so reconstruction algorithms are often tailored to specific shotgun sequencing technologies. As these methods change over time, a fundamental question is whether there exist reconstruction algorithms which are robust, i.e., which perform well under a wide range of error distributions. Here we study this question of sequence assembly from corrupted reads. We make no assumption on the types of errors in the reads, but only assume a bound on their magnitude. More precisely, for each read we assume that instead of receiving the true read with no errors, we receive a corrupted read which has edit distance at most $\epsilon$ times the length of the read from the true read. We show that if the reads are long enough and there are sufficiently many of them, then approximate reconstruction is possible: we construct a simple algorithm such that for almost all original sequences the output of the algorithm is a sequence whose edit distance from the original one is at most $O(\epsilon)$ times the length of the original sequence., Comment: 13 pages, 2 figures