Search

Spot patterns, in which vegetation patches form a hexagonal lattice, are frequent in nature and could serve as an early-warning indicator of abrupt vegetation collapses. Consequently, they have been intensively studied using both individual-based models and density-based field equations. Yet, the relationship between these two approaches remains unclear, particularly in scenarios where vegetation dynamics exhibit strong long-range spatial correlations and traditional mean-field approximations fail. To solve this issue, we develop a new method that refines mean-field approximations by describing both the dynamics of the biomass density field and its spatial correlations. This new approach harnesses the strengths of both individual and density-based mdoels, treating spatial correlations explicitly and allowing for the identification of spatial instabilities resulting in periodic patterns. Our results indicate that this new approximation predicts the parameter regimes where regular periodic patterns emerge more accurately than mean-field models, suggesting that it could provide a more robust framework to perform further nonlinear analysis.

Generalizing language-conditioned robotic policies to new tasks remains a significant challenge, hampered by the lack of suitable simulation benchmarks. In this paper, we address this gap by introducing GemBench, a novel benchmark to assess generalization capabilities of vision-language robotic manipulation policies. GemBench incorporates seven general action primitives and four levels of generalization, spanning novel placements, rigid and articulated objects, and complex long-horizon tasks. We evaluate state-of-the-art approaches on GemBench and also introduce a new method. Our approach 3D-LOTUS leverages rich 3D information for action prediction conditioned on language. While 3D-LOTUS excels in both efficiency and performance on seen tasks, it struggles with novel tasks. To address this, we present 3D-LOTUS++, a framework that integrates 3D-LOTUS's motion planning capabilities with the task planning capabilities of LLMs and the object grounding accuracy of VLMs. 3D-LOTUS++ achieves state-of-the-art performance on novel tasks of GemBench, setting a new standard for generalization in robotic manipulation. The benchmark, codes and trained models are available at \url{https://www.di.ens.fr/willow/research/gembench/}.

Many natural phenomena are quantified by counts of observable events, from the annihilation of quasiparticles in a lattice to predator-prey encounters on a landscape to spikes in a neural network. These events are triggered at random intervals when an underlying dynamical system occupies a set of reactive states in its phase space. We derive a general expression for the distribution of times between events in such counting processes assuming the underlying triggering dynamics is a stochastic process that converges to a stationary distribution. Our results contribute to resolving a long-standing dichotomy in the study of reaction-diffusion processes, showing the inter-reaction point process interpolates between a reaction- and a diffusion-limited regime. At low reaction rates, the inter-reaction process is Poisson with a rate depending on stationary properties of the event-triggering stochastic process. At high reaction rates, inter-reaction times are dominated by the hitting times to the reactive states. To further illustrate the power of this approach we apply our framework to obtain the counting statistics of two counting processes appearing in several biophysical scenarios. First, we study the common situation of estimating an animal's activity level by how often it crosses a detector, showing that the mean number of crossing events can decrease monotonically with the hitting rate, a seemingly 'paradoxical' result that could possibly lead to misinterpretation of experimental count data. Second, we derive the ensemble statistics for the detection of many particles, recovering and generalizing known results in the biophysics of chemosensation. Overall, we develop a unifying theoretical framework to quantify inter-event time distributions in reaction-diffusion systems that clarifies existing debates in the literature and provide examples of application to real-world scenarios., Comment: 14 pages, 3 figures

Populations very often self-organize into regular spatial patterns with important ecological and evolutionary consequences. Yet, most existing models neglect the effect that external biophysical drivers might have both on pattern formation and the spatiotemporal population dynamics once patterns form. Here, we investigate the effect of environmental flows on pattern formation and population dynamics using a spatially nonlocal logistic model (or Fisher-Kolmogorov equation) coupled to a simple shear and a Rankine vortex flow. We find that, whereas population abundance generally decreases with increasing flow intensity, the effect of the flow on the pattern instability depends on the spatial structure of the flow velocity field. This result shows that the velocity field interacts with the spatial feedbacks responsible for pattern formation in non-trivial ways, leading to a variety of spatiotemporal population dynamics regimes in which the total population abundance can exhibit either regular oscillations with a characteristic frequency or more erratic dynamics without a well-defined period. More generally, the diversity of spatiotemporal population dynamics caused by the interplay between self-organizing feedbacks and environmental flows highlights the importance of incorporating environmental and biophysical processes when studying both ecological pattern formation and its consequences., Comment: 27 pages, 10 figures

Learning generalizable visual representations from Internet data has yielded promising results for robotics. Yet, prevailing approaches focus on pre-training 2D representations, being sub-optimal to deal with occlusions and accurately localize objects in complex 3D scenes. Meanwhile, 3D representation learning has been limited to single-object understanding. To address these limitations, we introduce a novel 3D pre-training framework for robotics named SUGAR that captures semantic, geometric and affordance properties of objects through 3D point clouds. We underscore the importance of cluttered scenes in 3D representation learning, and automatically construct a multi-object dataset benefiting from cost-free supervision in simulation. SUGAR employs a versatile transformer-based model to jointly address five pre-training tasks, namely cross-modal knowledge distillation for semantic learning, masked point modeling to understand geometry structures, grasping pose synthesis for object affordance, 3D instance segmentation and referring expression grounding to analyze cluttered scenes. We evaluate our learned representation on three robotic-related tasks, namely, zero-shot 3D object recognition, referring expression grounding, and language-driven robotic manipulation. Experimental results show that SUGAR's 3D representation outperforms state-of-the-art 2D and 3D representations., Comment: Accepted to CVPR 2024. Project webpage: https://cshizhe.github.io/projects/robot_sugar.html

The emergence of a spatially-organized population distribution depends on the dynamics of the population and mediators of interaction (activators and inhibitors). Two broad classes of models have been used to investigate when and how self-organization is triggered, namely, reaction-diffusion and spatially nonlocal models. Nevertheless, these models implicitly assume smooth propagation scenarios, neglecting that individuals many times interact by exchanging short and abrupt pulses of the mediating substance. A recently proposed framework advances in the direction of properly accounting for these short-scale fluctuations by applying a coarse-graining procedure on the pulse dynamics. In this paper, we generalize the coarse-graining procedure and apply the extended formalism to new scenarios in which mediators influence individuals' reproductive success or their motility. We show that, in the slow- and fast-mediator limits, pulsed interactions recover, respectively, the reaction-diffusion and nonlocal models, providing a mechanistic connection between them. Furthermore, at each limit, the spatial stability condition is qualitatively different, leading to a timescale-induced transition where spatial patterns emerge as mediator dynamics becomes sufficiently fast.

The ability for robots to comprehend and execute manipulation tasks based on natural language instructions is a long-term goal in robotics. The dominant approaches for language-guided manipulation use 2D image representations, which face difficulties in combining multi-view cameras and inferring precise 3D positions and relationships. To address these limitations, we propose a 3D point cloud based policy called PolarNet for language-guided manipulation. It leverages carefully designed point cloud inputs, efficient point cloud encoders, and multimodal transformers to learn 3D point cloud representations and integrate them with language instructions for action prediction. PolarNet is shown to be effective and data efficient in a variety of experiments conducted on the RLBench benchmark. It outperforms state-of-the-art 2D and 3D approaches in both single-task and multi-task learning. It also achieves promising results on a real robot., Comment: Accepted to CoRL 2023. Project website: https://www.di.ens.fr/willow/research/polarnet/

Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR). While previous work mainly evaluates DR for disembodied tasks, such as pose estimation and object detection, here we systematically explore visual domain randomization methods and benchmark them on a rich set of challenging robotic manipulation tasks. In particular, we propose an off-line proxy task of cube localization to select DR parameters for texture randomization, lighting randomization, variations of object colors and camera parameters. Notably, we demonstrate that DR parameters have similar impact on our off-line proxy task and on-line policies. We, hence, use off-line optimized DR parameters to train visuomotor policies in simulation and directly apply such policies to a real robot. Our approach achieves 93% success rate on average when tested on a diverse set of challenging manipulation tasks. Moreover, we evaluate the robustness of policies to visual variations in real scenes and show that our simulator-trained policies outperform policies learned using real but limited data. Code, simulation environment, real robot datasets and trained models are available at https://www.di.ens.fr/willow/research/robust_s2r/.

Ecologists have long investigated how demographic and movement parameters determine the spatial distribution and critical habitat size of a population. However, most models oversimplify movement behavior, neglecting how landscape heterogeneity influences individual movement. We relax this assumption and introduce a reaction-advection-diffusion equation that describes population dynamics when individuals exhibit space-dependent movement bias toward preferred regions. Our model incorporates two types of these preferred regions: a high-quality habitat patch, termed `habitat', which is included to model avoidance of degraded habitats like deforested regions; and a preferred location, such as a chemoattractant source or a watering hole, that we allow to be asymmetrically located with respect to habitat edges. In this scenario, the critical habitat size depends on both the relative position of the preferred location and the movement bias intensities. When preferred locations are near habitat edges, the critical habitat size can decrease when diffusion increases, a phenomenon called the drift paradox. Also, ecological traps arise when the habitat overcrowds due to excessive attractiveness or the preferred location is near a low-quality region. Our results highlight the importance of species-specific movement behavior and habitat preference as drivers of population dynamics in fragmented landscapes and, therefore, in the design of protected areas., Comment: 21 pages, 8 figures

Intraspecific interactions are key drivers of population dynamics because they establish relations between individual fitness and population density. The component Allee effect is defined as a positive correlation between any fitness component of a focal organism and population density, and it can lead to positive density dependence in the population per capita growth rate. The spatial structure is key to determining whether and to which extent a component Allee effect will manifest at the demographic level because it determines how individuals interact with one another. However, existing spatial models to study the Allee effect impose a fixed spatial structure, which limits our understanding of how a component Allee effect and the spatial dynamics jointly determine the existence of demographic Allee effects. To fill this gap, we introduce a spatially-explicit theoretical framework where spatial structure and population dynamics are emergent properties of the individual-level demographic rates. Depending on the intensity of the individual processes the population exhibits a variety of spatial patterns that determine the demographic-level by-products of an existing individual-level component Allee effect. We find that aggregation increases population abundance and allows populations to survive in harsher environments and at lower global population densities when compared with uniformly distributed organisms. Moreover, aggregation can prevent the component Allee effect from manifesting at the population level or restrict it to the level of each independent group. These results provide a mechanistic understanding of how component Allee effects operate for different spatial population structures and show at the population level. Our results contribute to better understanding population dynamics in the presence of Allee effects and can potentially inform population management strategies.

Marine plankton play a crucial role in carbon storage, oxygen production, global climate, and ecosystem function. Planktonic ecosystems are embedded in a Lagrangian patches of water that are continuously moving, stretching, and diluting. These processes drive inhomegeneities on a range of scales, with implications for the integrated ecosystem properties, but are hard to characterize. We present a theoretical framework which accounts for all these aspects; tracking the water patch hosting a drifting ecosystem along with its physical, environmental, and biochemical features. The model resolves patch dilution and internal physical mixing as a function of oceanic strain and diffusion. Ecological dynamics are parameterized by an idealized nutrient and phytoplankton population and we specifically capture the propagation of the biochemical spatial variances to represent within-patch heterogeneity. We find that, depending only on the physical processes to which the water patch is subjected, the plankton biomass response to a resource perturbation can vary several fold. This work indicates that we must account for these processes when interpreting and modelling marine ecosystems and provides a framework with which to do so.

In human environments, robots are expected to accomplish a variety of manipulation tasks given simple natural language instructions. Yet, robotic manipulation is extremely challenging as it requires fine-grained motor control, long-term memory as well as generalization to previously unseen tasks and environments. To address these challenges, we propose a unified transformer-based approach that takes into account multiple inputs. In particular, our transformer architecture integrates (i) natural language instructions and (ii) multi-view scene observations while (iii) keeping track of the full history of observations and actions. Such an approach enables learning dependencies between history and instructions and improves manipulation precision using multiple views. We evaluate our method on the challenging RLBench benchmark and on a real-world robot. Notably, our approach scales to 74 diverse RLBench tasks and outperforms the state of the art. We also address instruction-conditioned tasks and demonstrate excellent generalization to previously unseen variations., Comment: Accepted in CoRL 2022 (oral); project page at https://guhur.github.io/hiveformer/

We study those operators on a Hilbert space that can be lifted / extended to any twisted Hilbert space. We prove that these form an ideal of operators which contains all the Schatten classes. We characterize those multiplication operators on $\ell_p$ that are liftable / extensible through centralizers.

Image segmentation is often ambiguous at the level of individual image patches and requires contextual information to reach label consensus. In this paper we introduce Segmenter, a transformer model for semantic segmentation. In contrast to convolution-based methods, our approach allows to model global context already at the first layer and throughout the network. We build on the recent Vision Transformer (ViT) and extend it to semantic segmentation. To do so, we rely on the output embeddings corresponding to image patches and obtain class labels from these embeddings with a point-wise linear decoder or a mask transformer decoder. We leverage models pre-trained for image classification and show that we can fine-tune them on moderate sized datasets available for semantic segmentation. The linear decoder allows to obtain excellent results already, but the performance can be further improved by a mask transformer generating class masks. We conduct an extensive ablation study to show the impact of the different parameters, in particular the performance is better for large models and small patch sizes. Segmenter attains excellent results for semantic segmentation. It outperforms the state of the art on both ADE20K and Pascal Context datasets and is competitive on Cityscapes., Comment: ICCV 2021. Code available at https://github.com/rstrudel/segmenter

Self-organized spatial patterns of vegetation are frequent in drylands and, because pattern shape correlates with water availability, they have been suggested as important indicators of ecosystem health. However, the mechanisms underlying pattern emergence remain unclear. Some theories hypothesize that patterns could result from a water-mediated scale-dependent feedback (SDF) whereby interactions favoring plant growth dominate at short distances and growth-inhibitory interactions dominate in the long range. However, we know little about how the presence of a focal plant affects the fitness of its neighbors as a function of the inter-individual distance, which is expected to be highly ecosystem-dependent. This lack of empirical knowledge and system dependency challenge the relevance of SDF as a unifying theory for vegetation pattern formation. Assuming that plant interactions are always inhibitory and only their intensity is scale-dependent, alternative theories also recover the typical vegetation patterns observed in nature. Importantly, although these alternative hypotheses lead to visually indistinguishable patterns, they predict contrasting desertification dynamics, which questions the potential use of vegetation patterns as ecosystem-state indicators. To help resolve this issue, we first review existing theories for vegetation self-organization and their conflicting predictions about desertification dynamics. Second, we discuss potential empirical tests via manipulative experiments to identify pattern-forming mechanisms and link them to specific desertification dynamics. A comprehensive view of models, the mechanisms they intend to capture, and experiments to test them in the field will help to better understand both how patterns emerge and improve predictions on the fate of the ecosystems where they form., Comment: 20 pages, 6 figures

We introduce and analyze a spatial Lotka-Volterra competition model with local and nonlocal interactions. We study two alternative classes of nonlocal competition that differ in how each species' characteristics determine the range of the nonlocal interactions. In both cases, nonlocal interactions can create spatial patterns of population densities in which highly populated clumps alternate with unpopulated regions. This non-populated regions provide spatial niches for a weaker competitor to establish in the community and persist in conditions in which local models predict competitive exclusion. Moreover, depending on the balance between local and nonlocal competition intensity, the clumps of the weaker competitor vary from M-like structures with higher densities of individuals accumulating at the edges of each clump to triangular structures with most individuals occupying their centers. These results suggest that long-range competition, through the creation of spatial patterns in population densities, might be an important driving force behind the rich diversity of species observed in real ecological communities.

We introduce an asymmetric noisy voter model to study the joint effect of immigration and a competition-dispersal tradeoff in the dynamics of two species competing for space in regular lattices. Individuals of one species can invade a nearest-neighbor site in the lattice, while individuals of the other species are able to invade sites at any distance but are less competitive locally, i.e., they establish with a probability $g \le 1$. The model also accounts for immigration, modeled as an external noise that may spontaneously replace an individual at a lattice site by another individual of the other species. This combination of mechanisms gives rise to a rich variety of outcomes for species competition, including exclusion of either species, mono-stable coexistence of both species at different population proportions, and bi-stable coexistence with proportions of populations that depend on the initial condition. Remarkably, in the bi-stable phase, the system undergoes a discontinuous transition as the intensity of immigration overcomes a threshold, leading to a half loop dynamics associated to a cusp catastrophe, which causes the irreversible loss of the species with the shortest dispersal range., Comment: 13 pages, 9 figures, 3 appendices

Motion planning and obstacle avoidance is a key challenge in robotics applications. While previous work succeeds to provide excellent solutions for known environments, sensor-based motion planning in new and dynamic environments remains difficult. In this work we address sensor-based motion planning from a learning perspective. Motivated by recent advances in visual recognition, we argue the importance of learning appropriate representations for motion planning. We propose a new obstacle representation based on the PointNet architecture and train it jointly with policies for obstacle avoidance. We experimentally evaluate our approach for rigid body motion planning in challenging environments and demonstrate significant improvements of the state of the art in terms of accuracy and efficiency., Comment: CoRL 2020. See the project webpage at https://www.di.ens.fr/willow/research/nmp_repr/