Search

Model ecosystems with quenched, symmetric interspecies interactions have been extensively studied using the replica method of the statistical mechanics of disordered systems. Here, we consider a more general scenario in which both the species abundances and the interspecies interactions change with time, albeit in widely separated timescales. The equilibrium of the coupled dynamics is studied analytically within the partial annealing framework, in which the number of replicas n takes on positive as well as negative values. In the case n > 0, which describes ecosystems characterized by the cooperative interspecies interactions, we find a discontinuous transition to a regime of zero diversity, whereas in the case where competition prevails, n < 0, we find that the species diversity is maximum.

The statistical properties of the energy landscape of the low autocorrelated binary string problem (LABSP) are studied numerically and compared with those of several classic disordered models. Using two global measures of landscape structure which have been introduced in the Simulated Annealing literature, namely, depth and difficulty, we find that the landscape of LABSP, except perhaps for a very large degeneracy of the local minima energies, is qualitatively similar to some well-known landscapes such as that of the mean-field 2-spin glass model. Furthermore, we consider a mean-field approximation to the pure model proposed by Bouchaud and Mezard (1994, J. Physique I France 4 1109) and show both analytically and numerically that it describes extremely well the statistical properties of LABSP.

Within the replica framework we study analytically the instance space of the number partitioning problem. This classic integer programming problem consists of partitioning a sequence of N positive real numbers $\{a_1, a_2,..., a_N}$ (the instance) into two sets such that the absolute value of the difference of the sums of $a_j$ over the two sets is minimized. We show that there is an upper bound $\alpha_c N$ to the number of perfect partitions (i.e. partitions for which that difference is zero) and characterize the statistical properties of the instances for which those partitions exist. In particular, in the case that the two sets have the same cardinality (balanced partitions) we find $\alpha_c=1/2$. Moreover, we show that the disordered model resulting from hte instance space approach can be viewed as a model of replicators where the random interactions are given by the Hebb rule., Comment: 7 pages

The mean number of metastable states in spin glass models with short-ranged p-spin interactions is estimated analytically using a variational method introduced by Tanaka and Edwards (1980 J. Phys. F: Met. Phys. 10 2769) for very large coordination numbers. For lattices with small connectivities, numerical simulations do not show any significant dependence on the relative positions of the interacting spins on the lattice, indicating thus that these systems can be described by a few macroscopic parameters. As an extremely anisotropic model we consider the low-autocorrelated binary spin model and we show through numerical simulations that its landscape has an exceptionally large number of local optima.

The thermodynamics of the infinite-range Ising spin glass with p-spin interactions in the presence of an external magnetic field h is investigated analytically using the replica method. We give emphasis to the analysis of the transition between the replica symmetric and the one-step replica symmetry breaking regimes. In particular, we derive analytical conditions for the onset of the continuous transition, as well as for the location of the tricritical point at which the transition between those two regimes becomes discontinuous., 21 pages, latex. Submitted to J. Phys. A

We investigate analytically the emergence of the categorization ability in the pseudo-inverse attractor neural network. More pointedly, we consider the problem of learning an extensive number of concepts by storing a finite number of examples s of each concept. We find that there is a critical value beyond which the categorization error, as measured by the average fraction of unstable sites in the concepts, decreases monotonically with s.

The statistical properties of the local optima (metastable states) of the infinite range Ising spin glass with p-spin interactions in the presence of an external magnetic field h are investigated analytically. The average number of optima as well as the typical overlap between pairs of identical optima are calculated for general p. Similarly to the thermodynamic order parameter, for p>2 and small h the typical overlap q_t is a discontinuous function of the energy. The size of the jump in q_t increases with p and decreases with h, vanishing at finite values of the magnetic field., 12 pages,tex

We numerically investigate the structure of the basins of attraction and the nature of the spurious attractors of the pseudo-inverse and the optimal weights attractor neural networks. We show that the number of attractors in the optimal weights model increases as the margin parameter increases, and that the basins of attraction of the stored patterns are not significantly enlarged by . Moreover, the number of attractors is smaller than in the pseudo-inverse model.

We extend the standard calculation of the number of metastable states (fixed points) so as to take into account cyclic attractors as well. We calculate analytically the average number of cycles in the Little model as a function of the fraction of flipping spins (1-q)/2. We find that the cycles of length two are by far the most common type of attractors. In particular, for small values of the storage parameter, the inverse cycles (q=-1) are the dominant attractors.

We investigate the effect of dilution after learning on the generalization ability of single-layer Boolean perceptrons that learn from noisy examples. We present a thorough comparison between the relative performances of several well known learning rules. In particular, we show that the effect of dilution is always deleterious, and that the Bayes algorithm always gives the best generalization performance.

We investigate the dependence of the retrieval properties of the pseudo-inverse and optimal attractor neural networks on the fraction of stored patterns , the temperature T and the margin parameter . Phase diagrams in the full space of parameters are presented in the regime of extreme dilution, i.e. when the connectivity C satisfies the condition , where N is the number of neurons. Furthermore, we study analytically the neighbourhood of a stored pattern for both models by calculating the average fraction of unstable sites in a pattern that differs by d sites from a given stored pattern. This analysis may shed light on the properties of the basins of attraction of the stored patterns.

The dependence of the optimal solution average cost of the set covering problem on the density of 1's of the incidence matrix () and on the number of constraints (P) is investigated in the limit where the number of items (N) goes to infinity. The annealed approximation is employed to study two stochastic models: the constant density model, where the elements of the incidence matrix are statistically independent random variables, and the Karp model, where the rows of the incidence matrix possess the same number of 1's. Lower bounds for are presented in the case that P scales with ln N and is of order 1, as well as in the case that P scales linearly with N and is of order 1/N. It is shown that in the case that P scales with exp N and is of order 1 the annealed approximation yields exact results for both models.

We investigate, through statistical mechanics techniques, the problem of finding N-tuples J=(Ji,J2,...,JN) with a fixed fraction k of non-vanishing entries that solve a set of P= alpha N linear equations. In particular, we determine the region in the plane ( alpha ,k) where these solutions exist with a probability of 1 in the limit of large N. We also obtain at insight into the microscopic structure of these solutions by calculating the distribution of the probability of an arbitrary non-vanishing entry Ji. Moreover we evaluate analytically the performances of several easy-to-implement heuristic algorithms and compare them with the optimal solution, a task that is suited very well to the statistical mechanics approach.

We calculate the distribution of learning times of the optimal stability perceptron algorithm of Krauth and Mezard (1987) for the learning from noisy examples problem. In particular, we find that in the case of noiseless examples the average total number of learning steps scales with ( alpha 2, where alpha is the training set size, although the number of examples that must effectively be learned tends to zero as alpha -1.

The minimal fraction of errors a perceptron can make on P random dichotomies is investigated. An exact upper bound is presented as well as comparisons between the replica theory estimates and a rigorous lower bound.

We study the equilibrium properties of the real weights linear perceptron within the replica formalism framework, focusing on the effects of the normalization of the weights on the learning and generalization capabilities of the network. We also investigate the effects of static noise corrupting the training data and of dynamical noise acting on the weights during the training stage.

The authors investigate the effects of dilution (lesions) on the memorization and generalization abilities of a single-layer perceptron whose non-zero weights are constrained to take on binary (+or-1) values only. The diluted perceptron is trained to realize a Boolean linearly separable mapping generated by a fully connected perceptron. In the case where the training process is disturbed by noise and the vanishing weights are chosen so as to minimize the training error, one finds that the dilution can improve the storage capacity and the generalization ability of the network. If the weights are cut randomly, however, the dilution will always degrade the network's performance. In this case they show that the main effect of dilution is to introduce an effective noise in the training examples.

The authors investigate the problem of loading a single-layered perceptron of Ising (+or-1) weights, focusing on the cases of linear and binary output neurons. Previous studies of this problem have been made using the canonical ensemble, which in many cases require the breaking of replica symmetry. They consider here an alternative approach, based on the microcanonical ensemble, and show that many results that were obtained previously using replica symmetry breaking within the canonical ensemble can be easily obtained using the replica-symmetric assumption within the microcanonical ensemble. Since the nature of the replica symmetry breaking in many of the models with discrete weights is similar, they believe that the results can immediately be extended to other cases such as learning a rule from examples, and utilizing discrete weights of larger synaptic depth.

The authors study learning and generalization in single-layer feedforward networks, whose weights are constrained to take on a discrete set of values. Their analytic results are obtained within the replica approach, which is verified through Monte Carlo simulations. It is shown that, depending on the architecture of the network and on the source of the training examples, three qualitatively different behaviours emerge. This distinction, which is manifested through the dependence of the training and generalization errors on the size of the training set suggests a possible way to determine the suitability of the architecture to the learning task. They conjecture that this distinction is relevant to the more interesting case of multi-layered networks.

The adaptive map is a simple model for evolutionary searches on a fitness landscape. The map is defined in the set M=(1,2,. . .M), with each integer representing both an organism and its fitness. An integer can be mapped on itself or on any larger integer with the same probability. The author calculates the probability distribution of the number of attractors in a map, as well as the distribution of the lengths of the searches. He studies the statistical structure of the basins of attraction and compares it with the structures obtained in the random map model and in the mean-field spin glass.

Axelrod's model in the square lattice with nearest-neighbors interactions exhibits culturally homogeneous as well as culturally fragmented absorbing configurations. In the case the agents are characterized by $F=2$ cultural features and each feature assumes $k$ states drawn from a Poisson distribution of parameter $q$ these regimes are separated by a continuous transition at $q_c = 3.10 \pm 0.02$. Using Monte Carlo simulations and finite size scaling we show that the mean density of cultural domains $\mu$ is an order parameter of the model that vanishes as $\mu \sim \left ( q - q_c \right)^\beta$ with $\beta = 0.67 \pm 0.01$ at the critical point. In addition, for the correlation length critical exponent we find $\nu = 1.63 \pm 0.04$ and for Fisher's exponent, $\tau = 1.76 \pm 0.01$. This set of critical exponents places the continuous phase transition of Axelrod's model apart from the known universality classes of nonequilibrium lattice models.

The authors describe a simple variant of Hopfield's (1982) or Little's (1974) model for neural networks, which is able to act as an associative memory even in the heavily overloaded spin-glass phase, using non-equilibrium states as attractors.

Neural networks are only useful as associative memories if an outside observer is able to ascertain whether a particular input corresponds to one of the memory states or not, i.e. the memory must be transparent. The authors design a model system which enters a 2-cycle whenever an input is not within the basin of attraction of some memory state.

The authors propose a mechanism to enhance the learning of an arbitrary but finite number of patterns (marked patterns) without damaging the ability of the network to retrieve the rest of the patterns. They find that the process of forgetting the marked patterns may become continuous, presenting then an unusual behaviour for symmetric neural networks.

The authors study the retrieval properties of Hopfield's model (1982) when there is a finite correlation between two of the memorised patterns.