Search

This paper presents a novel reinforcement learning framework for trajectory tracking of unmanned aerial vehicles in cluttered environments using a dual-agent architecture. Traditional optimization methods for trajectory tracking face significant computational challenges and lack robustness in dynamic environments. Our approach employs deep reinforcement learning (RL) to overcome these limitations, leveraging 3D pointcloud data to perceive the environment without relying on memory-intensive obstacle representations like occupancy grids. The proposed system features two RL agents: one for predicting UAV velocities to follow a reference trajectory and another for managing collision avoidance in the presence of obstacles. This architecture ensures real-time performance and adaptability to uncertainties. We demonstrate the efficacy of our approach through simulated and real-world experiments, highlighting improvements over state-of-the-art RL and optimization-based methods. Additionally, a curriculum learning paradigm is employed to scale the algorithms to more complex environments, ensuring robust trajectory tracking and obstacle avoidance in both static and dynamic scenarios.

This paper proposes an observer-based formation tracking control approach for multi-vehicle systems with second-order motion dynamics, assuming that vehicles' relative or global position and velocity measurements are unavailable. It is assumed that all vehicles are equipped with sensors capable of sensing the bearings relative to neighboring vehicles and only one leader vehicle has access to its global position. Each vehicle estimates its absolute position and velocity using relative bearing measurements and the estimates of neighboring vehicles received over a communication network. A distributed observer-based controller is designed, relying only on bearing and acceleration measurements. This work further explores the concept of the \textit{Bearing Persistently Exciting} (BPE) formation by proposing new algorithms for bearing-based localization and state estimation of second-order systems in centralized and decentralized manners. It also examines conditions on the desired formation to guarantee the exponential stability of distributed observer-based formation tracking controllers. In support of our theoretical results, some simulation results are presented to illustrate the performance of the proposed observers as well as the observer-based tracking controllers.

In this paper, we investigate the problem of decomposing 2D environments for robot coverage planning. Coverage path planning (CPP) involves computing a cost-minimizing path for a robot equipped with a coverage or sensing tool so that the tool visits all points in the environment. CPP is an NP-Hard problem, so existing approaches simplify the problem by decomposing the environment into the minimum number of sectors. Sectors are sub-regions of the environment that can each be covered using a lawnmower path (i.e., along parallel straight-line paths) oriented at an angle. However, traditional methods either limit the coverage orientations to be axis-parallel (horizontal/vertical) or provide no guarantees on the number of sectors in the decomposition. We introduce an approach to decompose the environment into possibly overlapping rectangular sectors. We provide an approximation guarantee on the number of sectors computed using our approach for a given environment. We do this by leveraging the submodular property of the sector coverage function, which enables us to formulate the decomposition problem as a submodular set cover (SSC) problem with well-known approximation guarantees for the greedy algorithm. Our approach improves upon existing coverage planning methods, as demonstrated through an evaluation using maps of complex real-world environments., Comment: Extended version of the 2024 IEEE CDC paper, 8 pages, 3 figures

Sensor network localization (SNL) is a challenging problem due to its inherent non-convexity and the effects of noise in inter-node ranging measurements and anchor node position. We formulate a non-convex SNL problem as a multi-player non-convex potential game and investigate the existence and uniqueness of a Nash equilibrium (NE) in both the ideal setting without measurement noise and the practical setting with measurement noise. We first show that the NE exists and is unique in the noiseless case, and corresponds to the precise network localization. Then, we study the SNL for the case with errors affecting the anchor node position and the inter-node distance measurements. Specifically, we establish that in case these errors are sufficiently small, the NE exists and is unique. It is shown that the NE is an approximate solution to the SNL problem, and that the position errors can be quantified accordingly. Based on these findings, we apply the results to case studies involving only inter-node distance measurement errors and only anchor position information inaccuracies., Comment: arXiv admin note: text overlap with arXiv:2311.03326, arXiv:2401.02471

In this paper we develop a multiple model reference adaptive controller (MMRAC) with blending. The systems under consideration are non-square, i.e., the number of inputs is not equal to the number of states; multi-input, linear, time-invariant with uncertain parameters that lie inside of a known, compact, and convex set. Moreover, the full state of the plant is available for feedback. A multiple model online identification scheme for the plant's state and input matrices is developed that guarantees the estimated parameters converge to the underlying plant model under the assumption of persistence of excitation. Using an exact matching condition, the parameter estimates are used in a control law such that the plant's states asymptotically track the reference signal generated by a state-space model reference. The control architecture is proven to provide boundedness of all closed-loop signals and to asymptotically drive the state tracking error to zero. Numerical simulations illustrate the stability and efficacy of the proposed MMRAC scheme., Comment: 10 pages, 7 figures, IEEE Journal Submission

Consider a sensor network consisting of both anchor and non-anchor nodes. We address the following sensor network localization (SNL) problem: given the physical locations of anchor nodes and relative measurements among all nodes, determine the locations of all non-anchor nodes. The solution to the SNL problem is challenging due to its inherent non-convexity. In this paper, the problem takes on the form of a multi-player non-convex potential game in which canonical duality theory is used to define a complementary dual potential function. After showing the Nash equilibrium (NE) correspondent to the SNL solution, we provide a necessary and sufficient condition for a stationary point to coincide with the NE. An algorithm is proposed to reach the NE and shown to have convergence rate $\mathcal{O}(1/\sqrt{k})$. With the aim of reducing the information exchange within a network, a distributed algorithm for NE seeking is implemented and its global convergence analysis is provided. Extensive simulations show the validity and effectiveness of the proposed approach to solve the SNL problem., Comment: arXiv admin note: text overlap with arXiv:2311.03326

In this paper, we propose a method to replan coverage paths for a robot operating in an environment with initially unknown static obstacles. Existing coverage approaches reduce coverage time by covering along the minimum number of coverage lines (straight-line paths). However, recomputing such paths online can be computationally expensive resulting in robot stoppages that increase coverage time. A naive alternative is greedy detour replanning, i.e., replanning with minimum deviation from the initial path, which is efficient to compute but may result in unnecessary detours. In this work, we propose an anytime coverage replanning approach named OARP-Replan that performs near-optimal replans to an interrupted coverage path within a given time budget. We do this by solving linear relaxations of integer linear programs (ILPs) to identify sections of the interrupted path that can be optimally replanned within the time budget. We validate OARP-Replan in simulation and perform comparisons against a greedy detour replanner and other state-of-the-art coverage planners. We also demonstrate OARP-Replan in experiments using an industrial-level autonomous robot., Comment: 16 pages, 18 figures, Paper submitted to IEEE T-RO

Sensor network localization (SNL) problems require determining the physical coordinates of all sensors in a network. This process relies on the global coordinates of anchors and the available measurements between non-anchor and anchor nodes. Attributed to the intrinsic non-convexity, obtaining a globally optimal solution to SNL is challenging, as well as implementing corresponding algorithms. In this paper, we formulate a non-convex multi-player potential game for a generic SNL problem to investigate the identification condition of the global Nash equilibrium (NE) therein, where the global NE represents the global solution of SNL. We employ canonical duality theory to transform the non-convex game into a complementary dual problem. Then we develop a conjugation-based algorithm to compute the stationary points of the complementary dual problem. On this basis, we show an identification condition of the global NE: the stationary point of the proposed algorithm satisfies a duality relation. Finally, simulation results are provided to validate the effectiveness of the theoretical results.

In this paper, we tackle the problem of planning an optimal coverage path for a robot operating indoors. Many existing approaches attempt to discourage turns in the path by covering the environment along the least number of coverage lines, i.e., straight-line paths. This is because turning not only slows down the robot but also negatively affects the quality of coverage, e.g., tools like cameras and cleaning attachments commonly have poor performance around turns. The problem of minimizing coverage lines however is typically solved using heuristics that do not guarantee optimality. In this work, we propose a turn-minimizing coverage planning method that computes the optimal number of axis-parallel (horizontal/vertical) coverage lines for the environment in polynomial time. We do this by formulating a linear program (LP) that optimally partitions the environment into axis-parallel ranks (non-intersecting rectangles of width equal to the tool width). We then generate coverage paths for a set of real-world indoor environments and compare the results with state-of-the-art coverage approaches., Comment: 8 pages, 9 figures, submitted to RA-L with IROS 2022 option

Adaptive control strategies usually are designed based on gradient methods for the sake of simplicity in Lyapunov analysis. However, least squares (LS)-based parameter identifiers, with proper selection of design parameters, exhibit better transient performance than the gradient-based ones, from the aspects of convergence speed and robustness to measurement noise. On the other hand, most of the LS-based adaptive control procedures are designed via the indirect adaptive control approaches, due to the difficulty in integrating an LS-based adaptive law within the direct approaches starting with a certain Lyapunov-like cost function to be driven to (a neighborhood of) zero. In this paper, a formal constructive analysis framework is proposed to integrate the recursive LS-based parameter identification with direct adaptive control. To this end, a Lyapunov-like function is proposed for the analysis to achieve adaptive laws, which guarantee the exponential convergence of the parameters. Application of the proposed procedure in adaptive cruise control design is studied through Matlab/Simulink and CarSim simulations, validating the analytical results., Comment: This is the second version of the arXiv:2007.08578 in Arxiv and it is submitted to IEEE CDC 2022. 8 pages, 3 figures

Extremum seeking (ES) optimization approach has been very popular due to its non-model based analysis and implementation. This approach has been mostly used with gradient based search algorithms. Since least squares (LS) algorithms are typically observed to be superior, in terms of convergence speed and robustness to measurement noises, over gradient algorithms, it is expected that LS based ES schemes will also provide faster convergence and robustness to sensor noises. In this paper, with this motivation, a recursive least squares (RLS) estimation based ES scheme is designed and analysed for application to scalar parameter and vector parameter static map and dynamic systems. Asymptotic convergence to the extremum is established for all the cases. Simulation studies are provided to validate the performance of proposed scheme., Comment: 6 pages, 8 figures, will be submitted to L-CSS with CDC Option 2020

This paper focuses on the problem of estimating bearing vectors between the agents in a two dimensional multi-agent network based on subtended angle measurements, called edge localization problem. We propose an edge localization graph to investigate the solvability of this problem and a distributed estimation method via orientation estimation of virtual agents to solve the problem. Under the proposed method, the estimated bearing vector exponentially converges to the real one with a common bias if and only if the edge localization graph has an oriented spanning tree. Furthermore, the estimated variables exponentially converge to the true values if the edge localization graph has an oriented spanning tree with a root knowing the bearing vector from it to one of its neighbors., Comment: 12 pages, 12 figures, Brief version of a paper named "Distributed Bearing Vector Estimation in Multi-agent Networks" published in Automatica

We consider trajectories where the sign of the derivative of each entry is opposite to that of the corresponding entry in the gradient of an energy function. We show that this condition guarantees convergence when the energy function is quadratic and positive definite and partly extend that result to some classes of positive semi-definite quadratic functions including those defined using a graph Laplacian. We show how this condition allows establishing the convergence of a platoon application in which it naturally appears, due to deadzones in the control laws designed to avoid instabilities caused by inconsistent measurements of the same distance by different agents., Comment: 6 pages, 2 figures, double columns

In this paper we study continuous time adaptive extremum localization of an arbitrary quadratic function $F(\cdot)$ based on Hessian estimation, using measured the signal intensity by a sensory agent. The function $F(\cdot)$ represents a signal field as a result of a source located at the maximum point of $F(\cdot)$ and is decreasing as moving away from the source location. Stability of the proposed adaptive estimation and localization scheme is analyzed and the Hessian parameter and location estimates are shown to asymptotically converge to the true values. Moreover, the stability and convergence properties of algorithm are shown to be robust to drift in the extremum location. Simulation test results are displayed to verify the established properties of the proposed scheme as well as robustness to signal measurement noise.

In this paper, a combined formation acquisition and cooperative extremum seeking control scheme is proposed for a team of three robots moving on a plane. The extremum seeking task is to find the maximizer of an unknown two-dimensional function on the plane. The function represents the signal strength field due to a source located at maximizer, and is assumed to be locally concave around maximizer and monotonically decreasing in distance to the source location. Taylor expansions of the field function at the location of a particular lead robot and the maximizer are used together with a gradient estimator based on signal strength measurements of the robots to design and analyze the proposed control scheme. The proposed scheme is proven to exponentially and simultaneously (i) acquire the specified geometric formation and (ii) drive the lead robot to a specified neighborhood disk around maximizer, whose radius depends on the specified desired formation size as well as the norm bounds of the Hessian of the field function. The performance of the proposed control scheme is evaluated using a set of simulation experiments., Comment: Presented at the 2018 IEEE Conference on Decision and Control (CDC), Miami Beach, FL, USA

Road congestion in urban environments, especially near signalized intersections, has been a major cause of significant fuel and time waste. Various solutions have been proposed to solve the problem of increasing idling times and number of stops of vehicles at signalized intersections, ranging from infrastructure to vehicle-based techniques. However, all the solutions introduced to solve the problem have approached the problem from a single vehicle point of view. This research introduces a game-theoretic cooperative speed optimization framework to minimize vehicles' idling times and number of stops at signalized intersections. This framework consists of three modules to cover individual autonomous vehicle speed optimization; conflict recognition; and cooperative speed decision making. A time token allocation algorithm is introduced through the proposed framework to allow the vehicles to cooperate and agree on certain speed actions such that the average idling times and number of stops are minimized. Simulation to test and validate the proposed framework is conducted and results are reported., Comment: 25 pages, journal paper

This paper is concerned with the network-theoretic properties of so-called $k$-nearest neighbor intelligent vehicular platoons, where each vehicle communicates with $k$ vehicles, both in front and behind. The network-theoretic properties analyzed in this paper play major roles in quantifying the resilience and robustness of three generic distributed estimation and control algorithms against communication failures and disturbances, namely resilient distributed estimation, resilient distributed consensus, and robust network formation. Based on the results for the connectivity measures of the $k$-nearest neighbor platoon, we show that extending the traditional platooning topologies (which were based on interacting with nearest neighbors) to $k$-nearest neighbor platoons increases the resilience of distributed estimation and control algorithms to both communication failures and disturbances. Finally, we discuss how the performance of each algorithm scales with the size of the vehicle platoon.

Reliable estimation (or measurement) of vehicle states has always been an active topic of research in the automotive industry and academia. Among the vehicle states, vehicle speed has a priority due to its critical importance in traction and stability control. Moreover, the emergence of new generation of communication technologies has brought a new avenue to traditional studies on vehicle estimation and control. To this end, this paper introduces a set of distributed function calculation algorithms for vehicle networks, robust to communication failures. The introduced algorithms enable each vehicle to gather information from other vehicles in the network in a distributed manner. A procedure to use such a bank of information for a single vehicle to diagnose and correct a possible fault in its own speed estimation/measurement is discussed. The functionality and performance of the proposed algorithms are verified via illustrative examples and simulation results.

As bulk synchronous generators in the power grid are replaced by distributed generation interfaced through power electronics, inertia is removed from the system, prompting concerns over grid stability. Different metrics are available for quantifying grid stability and performance; however, no theoretical results are available comparing and contrasting these metrics. This paper presents a rigorous system-theoretic study of performance metrics for low-inertia stability. For networks with uniform parameters, we derive explicit expressions for the eigenvalue damping ratios, and for the $\mathcal{H}_{2}$ and $\mathcal{H}_{\infty}$ norms of the linearized swing dynamics, from external power disturbances to different phase/frequency performance outputs.These expressions show the dependence of system performance on inertia constants, damping constants, and on the grid topology. Surprisingly, we find that the $\mathcal{H}_2$ and $\mathcal{H}_{\infty}$ norms can display contradictory behavior as functions of the system inertia, indicating that low-inertia performance depends strongly on the chosen performance metric.

We consider a graph theoretic approach to the performance and robustness of a platoon of vehicles, where each vehicle communicates with its $k$-nearest neighbors. In particular, we quantify the platoon's stability margin, robustness to disturbances (in terms of system $\mathcal{H}_{\infty}$ norm), and maximum delay tolerance via graph-theoretic notions such as nodal degrees and (grounded) Laplacian matrix eigenvalues. Our results show that there is a trade-off between robustness to time delay and robustness to disturbances. Both first-order dynamics (reference velocity tracking) and second-order dynamics (controlling inter-vehicular distance) are analyzed in this direction. Theoretical contributions are confirmed via simulation results., Comment: 16 pages, 5 figures

We present a graph-theoretic approach to analyzing the robustness of leader-follower consensus dynamics to disturbances and time delays. Robustness to disturbances is captured via the system $\mathcal{H}_2$ and $\mathcal{H}_{\infty}$ norms and robustness to time delay is defined as the maximum allowable delay for the system to remain asymptotically stable. Our analysis is built on understanding certain spectral properties of the grounded Laplacian matrix that play a key role in such dynamics. Specifically, we give graph-theoretic bounds on the extreme eigenvalues of the grounded Laplacian matrix which quantify the impact of disturbances and time-delays on the leader-follower dynamics. We then provide tight characterizations of these robustness metrics in Erdos-Renyi random graphs and random regular graphs. Finally, we view robustness to disturbances and time delay as network centrality metrics, and provide conditions under which a leader in a network optimizes each robustness objective. Furthermore, we propose a sufficient condition under which a single leader optimizes both robustness objectives simultaneously.