Search

As interactions with autonomous agents-ranging from robots in physical settings to avatars in virtual and augmented realities-become more prevalent, developing advanced cognitive architectures is critical for enhancing the dynamics of human-avatar groups. This paper presents a reinforcement-learning-based cognitive architecture, trained via a sim-to-real approach, designed to improve synchronization in periodic motor tasks, crucial for applications in group rehabilitation and sports training. Extensive numerical validation consistently demonstrates improvements in synchronization. Theoretical derivations and numerical investigations are complemented by preliminary experiments with real participants, showing that our avatars can integrate seamlessly into human groups, often being indistinguishable from humans.

We present a data-driven control architecture for modifying the kinematics of robots and artificial avatars to encode specific information such as the presence or not of an emotion in the movements of an avatar or robot driven by a human operator. We validate our approach on an experimental dataset obtained during the reach-to-grasp phase of a pick-and-place task.

In addressing control problems such as regulation and tracking through reinforcement learning, it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error prior to deployment. Motivated by this necessity, we present a set of results and a systematic reward shaping procedure that (i) ensures the optimal policy generates trajectories that align with specified control requirements and (ii) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Inverted Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep reinforcement learning methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.

We consider the problem of optimizing the interconnection graphs of complex networks to promote synchronization. When traditional optimization methods are inapplicable, due to uncertain or unknown node dynamics, we propose a data-driven approach leveraging datasets of relevant examples. We analyze two case studies, with linear and nonlinear node dynamics. First, we show how including node dynamics in the objective function makes the optimal graphs heterogeneous. Then, we compare various design strategies, finding that the best either utilize data samples close to a specific Pareto front or a combination of a neural network and a genetic algorithm, with statistically better performance than the best examples in the datasets.

Geometric pattern formation is an important emergent behavior in many applications involving large-scale multi-agent systems, such as sensor networks deployment and collective transportation. Attraction/repulsion virtual forces are the most common control approach to achieve such behavior in a distributed and scalable manner. Nevertheless, for most existing solutions only numerical and/or experimental evidence of their convergence is available. Here, we revisit the problem of achieving pattern formation giving sufficient conditions to prove analytically that under the influence of appropriate virtual forces, a large-scale multi-agent swarming system locally converges towards a stable and robust triangular lattice configuration. Specifically, the proof is carried out using LaSalle's invariance principle and geometry-based arguments. Our theoretical results are complemented by exhaustive numerical simulations confirming their effectiveness and estimating the region of asymptotic stability of the triangular configuration.

The problem of partitioning a power grid into a set of islands can be a solution to restore power dispatchment in sections of a grid affected by an extreme failure. Current solutions to this problem usually involve finding the partition of the grid into islands that minimizes the sum of their absolute power imbalances. This combinatorial problem is often solved through heuristic offline methods. In this paper, we propose instead a distributed online algorithm through which nodes can migrate among islands, self-organizing the network into a suitable partition. We prove that, under a set of appropriate assumptions, the proposed solution yields a partition whose absolute power imbalance falls within a given bound of the optimal solution. We validate our analytical results by testing our partitioning strategy on the IEEE 118 and 300 benchmark problems.

One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approximate model of the system, without any assumption about the knowledge of the system's dynamics. There is no expectation that it will be able to achieve the control objective if used stand-alone. During learning, the tutor occasionally suggests an action, thus partially guiding exploration. We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing. We demonstrate that CT-DQN is able to achieve better or equivalent data efficiency with respect to the classic function approximation solutions.

Geometric pattern formation is crucial in many tasks involving large-scale multi-agent systems. Examples include mobile agents performing surveillance, swarm of drones or robots, or smart transportation systems. Currently, most control strategies proposed to achieve pattern formation in network systems either show good performance but require expensive sensors and communication devices, or have lesser sensor requirements but behave more poorly. Also, they often require certain prescribed structural interconnections between the agents (e.g., regular lattices, all-to-all networks etc). In this paper, we provide a distributed displacement-based control law that allows large group of agents to achieve triangular and square lattices, with low sensor requirements and without needing communication between the agents. Also, a simple, yet powerful, adaptation law is proposed to automatically tune the control gains in order to reduce the design effort, while improving robustness and flexibility. We show the validity and robustness of our approach via numerical simulations and experiments, comparing it with other approaches from the existing literature.

Given a flow network with variable suppliers and fixed consumers, the minimax flow problem consists in minimizing the maximum flow between nodes, subject to flow conservation and capacity constraints. We solve this problem over acyclic graphs in a distributed manner by showing that it can be recast as a consensus problem between the maximum downstream flows, which we define here for the first time. Additionally, we present a distributed algorithm to estimate these quantities. Finally, exploiting our theoretical results, we design an online distributed controller to prevent overcurrent in microgrids consisting of loads and droop-controlled inverters. Our results are validated numerically on the CIGRE benchmark microgrid.

Shimmy is a dangerous phenomenon that occurs when aircraft's nose landing gears oscillate in a rapid and uncontrollable fashion. In this paper, we propose the use of two nonlinear control approaches (zero average control and model reference adaptive control based on minimal control synthesis) as simple yet effective strategies to suppress undesired oscillations, even in the presence of uncertainties and partial state measurements. Numerical results are presented to validate the proposed control approaches.

We study convergence in networks of piecewise-smooth (PWS) systems that commonly arise in applications to model dynamical systems whose evolution is affected by macroscopic events such as switches and impacts. Existing approaches were typically oriented toward guaranteeing global bounded synchronizability, local stability of the synchronization manifold, or achieving synchronization by exerting a control action on each node. Here we start by generalizing existing results on QUAD systems to the case of PWS systems, accounting for a large variety of nonlinear coupling laws. Then, we propose that a discontinuous coupling can be used to guarantee global synchronizability of a network of N PWS agents under mild assumptions on the individual dynamics. We provide extensive numerical simulations to gain insights on larger networks.

The problem of partitioning a power grid into a set of microgrids, or islands, is of interest for both the design of future smart grids, and as a last resort to restore power dispatchment in sections of a grid affected by an extreme failure. In the literature this problem is usually solved by turning it into a combinatorial optimization problem, often solved through generic heruristic methods such as Genetic Algorithms or Tabu Search. In this paper, we take a different route and obtain the grid partition by exploiting the synchronization dynamics of a cyberlayer of Kuramoto oscillators, each parameterized as a rough approximation of the dynamics of the grid's node it corresponds to. We present first a centralised algorithm and then a decentralised strategy. In the former, nodes are aggregated based on their internode synchronization times while in the latter they exploit synchronization of the oscillators in the cyber layer to selforganise into islands. Our preliminary results show that the heuristic synchronization based algorithms do converge towards partitions that are comparable to those obtained via other more cumbersome and computationally expensive optimization-based methods., Comment: 6 pages, 4 figures

This paper is concerned with the design of intermittent non-pharmaceutical strategies to mitigate the spread of the COVID-19 epidemic exploiting network epidemiological models. Specifically, by studying a variational equation for the dynamics of the infected in a network model of the epidemic spread, we derive, using contractivity arguments, a condition that can be used to guarantee that, in epidemiological terms, the effective reproduction number is less than unity. This condition has three advantages: (i) it is easily computable; (ii) it is directly related to the model parameters; (iii) it can be used to enforce a scalability condition that prohibits the amplification of disturbances within the network system. We then include satisfaction of such a condition as a constraint in a Model Predictive Control problem so as to mitigate (or suppress) the spread of the epidemic while minimizing the economic impact of the interventions. A data-driven model of Italy as a network of three macro-regions (North, Center, and South), whose parameters are identified from real data, is used to illustrate and evaluate the effectiveness of the proposed control strategy., Comment: Accepted by Control and Decision Conference 2021

The COVID-19 epidemic that emerged in Wuhan China at the end of 2019 hit Italy particularly hard, yielding the implementation of strict national lockdown rules (Phase 1). There is now a hot ongoing debate in Italy and abroad on what the best strategy is to restart a country to exit a national lockdown (Phase 2). Previous studies have focused on modelling possible restarting scenarios at the national level, overlooking the fact that Italy, as other nations around the world, is divided in administrative regions who can independently oversee their own share of the Italian National Health Service. In this study, we show that regionalism, and heterogeneity between regions, is essential to understand the spread of the epidemic and, more importantly, to design effective post Lock-Down strategies to control the disease. To achieve this, we model Italy as a network of regions and parameterize the model of each region on real data spanning almost two months from the initial outbreak. Using the model, we confirm the effectiveness at the regional level of the national lockdown strategy implemented so far by the Italian government to mitigate the spread of the disease and show its efficacy at the regional level. We also propose that differentiated, albeit coordinated, regional interventions can be effective in Phase 2 to restart the country and avoid future recurrence of the epidemic, while avoiding saturation of the regional health systems and mitigating impact on costs. Our study and methodology can be easily extended to other levels of granularity (provinces or counties in the same region or states in other federal countries, etc.) to support policy- and decision-makers., Comment: 35 pages

Synchronization is a crucial phenomenon in many natural and artificial complex network systems. Applications include neuronal networks, formation control and coordination in robotics, and frequency synchronization in electrical power grids. In this paper, we propose the use of a distributed discontinuous coupling protocol to achieve convergence and synchronization in networks of non-identical nonlinear dynamical systems. We show that the synchronous dynamics is a solution to the average of the nodes' vector fields, and derive analytical estimates of the critical coupling gains required to achieve convergence. Numerical simulations are used to illustrate and validate the theoretical results.

The Painlev\'e paradox is a phenomenon that causes instability in mechanical systems subjects to unilateral constraints. While earlier studies were mostly focused on abstract theoretical settings, recent work confirmed the occurrence of the paradox in realistic set-ups. In this paper, we investigate the dynamics and presence of the Painlev\'e phenomenon in a twolinks robot in contact with a moving belt, through a bifurcation study. Then, we use the results of this analysis to inform the design of control strategies able to keep the robot sliding on the belt and avoid the onset of undesired lift-off. To this aim, through numerical simulations, we synthesise and compare a PID strategy and a hybrid force/motion control scheme, finding that the latter is able to guarantee better performance and avoid the onset of bouncing motion due to the Painlev\'e phenomenon.

Complex networks are a successful framework to describe collective behaviour in many applications, but a notable gap remains in the current literature, that of proving asymptotic convergence in networks of piecewise-smooth systems. Indeed, a wide variety of physical systems display discontinuous dynamics that change abruptly, including dry friction mechanical oscillators, electrical power converters, and biological neurons. In this paper, we study how to enforce global asymptotic state-synchronization in these networks. Specifically, we propose the addition of a distributed discontinuous coupling action to the commonly used diffusive coupling protocol. Moreover, we provide analytical estimates of the thresholds on the coupling gains required for convergence, and highlight the importance of a new connectivity measure, which we named minimum density. The theoretical results are illustrated by a set of representative examples.

The aim of this paper is to present the application of an approach to study contraction theory recently developed for piecewise smooth and switched systems. The approach that can be used to analyze incremental stability properties of so-called Filippov systems (or variable structure systems) is based on the use of regularization, a procedure to make the vector field of interest differentiable before analyzing its properties. We show that by using this extension of contraction theory to nondifferentiable vector fields, it is possible to design observers for a large class of piecewise smooth systems using not only Euclidean norms, as also done in previous literature, but also non-Euclidean norms. This allows greater flexibility in the design and encompasses the case of both piecewise-linear and piecewise-smooth (nonlinear) systems. The theoretical methodology is illustrated via a set of representative examples., Comment: Preprint accepted to IFAC World Congress 2017

Atomic force microscopes have proved to be fundamental research tools in many situations where a gentle imaging process is required, and in a variety of environmental conditions, such as the study of biological samples. Among the possible modes of operation, intermittent contact mode is one that causes less wear to both the sample and the instrument; therefore, it is ideal when imaging soft samples. However, intermittent contact mode is not particularly fast when compared to other imaging strategies. In this paper, we introduce three enhanced control approaches, applied at both the dither and z-axis piezos, to address the limitations of existing control schemes. Our proposed strategies are able to eliminate different image artefacts, automatically adapt scan speed to the sample being scanned and predict its features in real time. The result is that both the image quality and the scan time are improved., Comment: 11 pages