Search

Indoor environments have abundant presence of high-level semantic information which can provide a better understanding of the environment for robots to improve the uncertainty in their pose estimate. Although semantic information has proved to be useful, there are several challenges faced by the research community to accurately perceive, extract and utilize such semantic information from the environment. In order to address these challenges, in this paper we present a lightweight and real-time visual semantic SLAM framework running on board aerial robotic platforms. This novel method combines low-level visual/visual-inertial odometry (VO/VIO) along with geometrical information corresponding to planar surfaces extracted from detected semantic objects. Extracting the planar surfaces from selected semantic objects provides enhanced robustness and makes it possible to precisely improve the metric estimates rapidly, simultaneously generalizing to several object instances irrespective of their shape and size. Our graph-based approach can integrate several state of the art VO/VIO algorithms along with the state of the art object detectors in order to estimate the complete 6DoF pose of the robot while simultaneously creating a sparse semantic map of the environment. No prior knowledge of the objects is required, which is a significant advantage over other works. We test our approach on a standard RGB-D dataset comparing its performance with the state of the art SLAM algorithms. We also perform several challenging indoor experiments validating our approach in presence of distinct environmental conditions and furthermore test it on board an aerial robot., This work was supported by the MISTI-Spain for the financial support in the project entitled Drone Autonomy and the Spanish Ministry of Economy and Competitivity for its funding Project (Complex Coordinated Inspection and Security missions by UAVs in cooperation with UGV) under Grant RTI2018-100847-B-C21. The work of Paloma de la Puente was supported in part by the Spanish Ministry of Economics and Competitivity under Grant DPI2017-86915-C3-3-R COGDRIVE.

Planning, control, perception, and learning are current research challenges in multirobot systems. The transition dynamics of the robots may be unknown or stochastic, making it difficult to select the best action each robot must take at a given time. The observation model, a function of the robots' sensor systems, may be noisy or partial, meaning that deterministic knowledge of the team's state is often impossible to attain. Moreover, the actions each robot can take may have an associated success rate and/or a probabilistic completion time. Robots designed for real-world applications require careful consideration of such sources of uncertainty, regardless of the control scheme or planning or learning algorithms used for a specific problem. Understanding the underlying mechanisms of planning algorithms can be challenging due to the latent variables they often operate on. When performance testing such algorithms on hardware, the simultaneous use of the debugging and visualization tools available on a workstation can be difficult. This transition from experimentation to implementation becomes especially challenging when the experiments need to replicate some feature of the software tool set in hardware, such as simulation of visually complex environments. This article details a robotics prototyping platform, called measurable augmented reality for prototyping cyberphysical systems (MAR-CPS), that directly addresses this problem, allowing for the real-time visualization of latent state information to aid hardware prototyping and performance testing of algorithms.

A key challenge facing modern airborne delivery systems, such as parafoils, is the ability to accurately and consistently deliver supplies into difficult, complex terrain. Robustness is a primary concern, given that environmental wind disturbances are often highly uncertain and time-varying. This paper presents a new online trajectory-planning algorithm that enables a large, autonomous parafoil with underactuated dynamics to robustly execute collision avoidance and precision landing on mapped terrain, even with significant wind uncertainties. This algorithm is designed to handle arbitrary initial altitudes, approach geometries, and terrain surfaces, and is robust to wind disturbances that may be highly dynamic throughout terminal approach. Real-time wind uncertainty modeling and classification are used to anticipate future disturbances, while a novel uncertainty-sampling technique ensures that robustness to future wind variation is efficiently maintained. The designed cost-to-go function enables selection of partial paths that intelligently trade off between current and reachable future states, while encouraging upwind landings. Simulation results demonstrate that this algorithm reduces the worst-case impact of wind disturbances relative to state-of-the-art approaches., Charles Stark Draper Laboratory. Independent Research and Development

Robust environment perception is essential for decision-making on robots operating in complex domains. Intelligent task execution requires principled treatment of uncertainty sources in a robot's observation model. This is important not only for low-level observations (e.g., accelerom-eter data), but also for high-level observations such as semantic object labels. This paper formalizes the concept of macro-observations in Decentralized Partially Observable Semi-Markov Decision Processes (Dec-POSMDPs), allowing scalable semantic-level multi-robot decision making. A hierarchical Bayesian approach is used to model noise statistics of low-level classifier outputs, while simultaneously allowing sharing of domain noise characteristics between classes. Classification accuracy of the proposed macro-observation scheme, called Hierarchical Bayesian Noise Inference (HBNI), is shown to exceed existing methods. The macro-observation scheme is then integrated into a Dec-POSMDP planner, with hardware experiments running onboard a team of dynamic quadrotors in a challenging domain where noise-agnostic filtering fails. To the best of our knowledge, this is the first demonstration of a real-time, convolutional neural net-based classification framework running fully onboard a team of quadrotors in a multi-robot decision-making domain., Boeing Company

Autonomous Mobility On Demand (MOD) systems can utilize fleet management strategies in order to provide a high customer quality of service (QoS). Previous works on autonomous MOD systems have developed methods for rebalancing single capacity vehicles, where QoS is maintained through large fleet sizing. This work focuses on MOD systems utilizing a small number of vehicles, such as those found on a campus, where additional vehicles cannot be introduced as demand for rides increases. A predictive positioning method is presented for improving customer QoS by identifying key locations to position the fleet in order to minimize expected customer wait time. Ridesharing is introduced as a means for improving customer QoS as arrival rates increase. However, with ridesharing perceived QoS is dependent on an often unknown customer preference. To address this challenge, a customer ratings model, which learns customer preference from a 5-star rating, is developed and incorporated directly into a ridesharing algorithm. The predictive positioning and ridesharing methods are applied to simulation of a real-world campus MOD system. A combined predictive positioning and ridesharing approach is shown to reduce customer service times by up to 29%. and the customer ratings model is shown to provide the best overall MOD fleet management performance over a range of customer preferences., Ford Motor Company, Ford-MIT Alliance

This paper presents the first ever approach for solving continuous-observation Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and their semi-Markovian counterparts, Dec-POSMDPs. This contribution is especially important in robotics, where a vast number of sensors provide continuous observation data. A continuous-observation policy representation is introduced using Stochastic Kernel-based Finite State Automata (SK-FSAs). An SK-FSA search algorithm titled Entropy-based Policy Search using Continuous Kernel Observations (EPSCKO) is introduced and applied to the first ever continuous-observation Dec-POMDP/Dec-POSMDP domain, where it significantly outperforms state-of-the-art discrete approaches. This methodology is equally applicable to Dec-POMDPs and Dec-POSMDPs, though the empirical analysis presented focuses on Dec-POSMDPs due to their higher scalability. To improve convergence, an entropy injection policy search acceleration approach for both continuous and discrete observation cases is also developed and shown to improve convergence rates without degrading policy quality., Boeing Company

In autonomous Mobility on Demand (MOD) systems, customers request rides from a fleet of shared vehicles that can be automatically positioned in response to customer demand. Recent approaches to MOD systems have focused on environments where customers can only request rides through an app or by waiting at a station. This paper develops MOD fleet management approaches for ride hailing, where customers may instead request rides simply by hailing a passing vehicle, an approach of particular importance for campus MOD systems. The challenge for ride hailing is that customer demand is not explicitly provided as it would be with an app, but rather customers are only served if a vehicle happens to be located at the arrival location. This work focuses on maximizing the number of served hailing customers in an MOD system by learning and utilizing customer demand. A Bayesian framework is used to define a novel customer demand model which incorporates observed pedestrian traffic to estimate customer arrival locations with a quantification of uncertainty. An exploration planner is proposed which routes MOD vehicles in order to reduce arrival rate uncertainty. A robust ride hailing fleet management planner is proposed which routes vehicles under the presence of uncertainty using a chance-constrained formulation. Simulation of a real-world MOD system on MIT's campus demonstrates the effectiveness of the planners. The customer demand model and exploration planner are demonstrated to reduce estimation error over time and the ride hailing planner is shown to improve the fraction of served customers in the system by 73% over a baseline exploration approach., Ford-MIT Alliance, Ford Motor Company

An off-policy Bayesian nonparameteric approximate reinforcement learning framework, termed as GPQ, that employs a Gaussian processes (GP) model of the value (Q) function is presented in both the batch and online settings. Sufficient conditions on GP hyperparameter selection are established to guarantee convergence of off-policy GPQ in the batch setting, and theoretical and practical extensions are provided for the online case. Empirical results demonstrate GPQ has competitive learning speed in addition to its convergence guarantees and its ability to automatically choose its own bases locations., United States. Office of Naval Research (Autonomy Program N000140910625)

We interpret several dynamical system verification questions, e.g., region of attraction and reachability analyses, as data classification problems. We discuss some of the tradeoffs between conventional optimization-based certificate constructions with certainty in the outcomes and this new date-driven approach with quantified confidence in the outcomes. The new methodology is aligned with emerging computing paradigms and has the potential to extend systematic verification to systems that do not necessarily admit closed-form models from certain specialized families. We demonstrate its effectiveness on a collection of both conventional and unconventional case studies including model reference adaptive control systems, nonlinear aircraft models, and reinforcement learning problems., United States. Air Force Research Laboratory (FA8650-15-C-2546), United States. Office of Naval Research (N000141310778), United States. Army Research Office (W911NF- 15-1-0592), National Science Foundation (U.S.) (1550212), United States. Defense Advanced Research Projects Agency (W911NF-16-1-0001)

For safe navigation in dynamic environments, an autonomous vehicle must be able to identify and predict the future behaviors of other mobile agents. A promising data-driven approach is to learn motion patterns from previous observations using Gaussian process (GP) regression, which are then used for online prediction. GP mixture models have been subsequently proposed for finding the number of motion patterns using GP likelihood as a similarity metric. However, this paper shows that using GP likelihood as a similarity metric can lead to non-intuitive clustering configurations - such as grouping trajectories with a small planar shift with respect to each other into different clusters - and thus produce poor prediction results. In this paper we develop a novel modeling framework, Dirichlet process active region (DPAR), that addresses the deficiencies of the previous GP-based approaches. In particular, with a discretized representation of the environment, we can explicitly account for planar shifts via a max pooling step, and reduce the computational complexity of the statistical inference procedure compared with the GP-based approaches. The proposed algorithm was applied on two real pedestrian trajectory datasets collected using a 3D Velodyne Lidar, and showed 15% improvement in prediction accuracy and 4.2 times reduction in computational time compared with a GP-based algorithm., Ford Motor Company

Incorrect landmark and loop closure measurements can cause standard SLAM algorithms to fail catastrophically. Recently, several SLAM algorithms have been proposed that are robust to loop closure errors, but it is shown in this paper that they cannot provide robust solutions when landmark measurement errors occur. The root cause of this problem is that the robust SLAM algorithms only focus on generating solutions that are locally consistent (i.e. each measurement agrees with its corresponding estimates) rather than globally consistent (i.e. all of the measurements in the solution agree with each other). Moreover, these algorithms do not attempt to maximize the number of correct measurements included in the solution, meaning that often correct measurements are ignored and the solution quality suffers as a result. This paper proposes a new formulation of the robust SLAM problem that seeks a globally consistent map that also maximizes the number of measurements included in the solution. In addition, a novel incremental SLAM algorithm, called incremental SLAM with consistency-checking, is developed to solve the new robust SLAM problem. Finally, simulated and experimental results show that the new algorithm significantly outperforms state-of-the-art robust SLAM methods for datasets with incorrect landmark measurements and can match their performance for datasets with incorrect loop closures., Charles Stark Draper Laboratory. Internal Research and Development Program

Structural regularities in man-made environments reflect in the distribution of their surface normals. Describing these surface normal distributions is important in many computer vision applications, such as scene understanding, plane segmentation, and regularization of 3D reconstructions. Based on the small-variance limit of Bayesian nonparametric von-Mises-Fisher (vMF) mixture distributions, we propose two new flexible and efficient k-means-like clustering algorithms for directional data such as surface normals. The first, DP-vMF-means, is a batch clustering algorithm derived from the Dirichlet process (DP) vMF mixture. Recognizing the sequential nature of data collection in many applications, we extend this algorithm to DDP-vMF-means, which infers temporally evolving cluster structure from streaming data. Both algorithms naturally respect the geometry of directional data, which lies on the unit sphere. We demonstrate their performance on synthetic directional data and real 3D surface normals from RGB-D sensors. While our experiments focus on 3D data, both algorithms generalize to high dimensional directional data such as protein backbone configurations and semantic word vectors., United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N00014- 11-1-0688), United States. Army Research Office. Multidisciplinary University Research Initiative (Grant W911NF-11-1-0391)

We describe a probabilistic framework for synthesizing control policies for general multi-robot systems, given environment and sensor models and a cost function. Decentralized, partially observable Markov decision processes (Dec-POMDPs) are a general model of decision processes where a team of agents must cooperate to optimize some objective (specified by a shared reward or cost function) in the presence of uncertainty, but where communication limitations mean that the agents cannot share their state, so execution must proceed in a decentralized fashion. While Dec-POMDPs are typically intractable to solve for real-world problems, recent research on the use of macro-actions in Dec-POMDPs has significantly increased the size of problem that can be practically solved as a Dec-POMDP. We describe this general model, and show how, in contrast to most existing methods that are specialized to a particular problem class, it can synthesize control policies that use whatever opportunities for coordination are present in the problem, while balancing off uncertainty in outcomes, sensor information, and information about other agents. We use three variations on a warehouse task to show that a single planner of this type can generate cooperative behavior using task allocation, direct communication, and signaling, as appropriate.

Autonomous learning through interaction with the physical world is a promising approach to designing controllers and decision-making policies for robots. Unfortunately, learning on robots is often difficult due to the large number of samples needed for many learning algorithms. Simulators are one way to decrease the samples needed from the robot by incorporating prior knowledge of the dynamics into the learning algorithm. In this paper we present a novel method for transferring data from a simulator to a robot, using simulated data as a prior for real-world learning. A Bayesian nonparametric prior is learned from a potentially black-box simulator. The mean of this function is used as a prior for the Probabilistic Inference for Learning Control (PILCO) algorithm. The simulated prior improves the convergence rate and performance of PILCO by directing the policy search in areas of the state-space that have not yet been observed by the robot. Simulated and hardware results show the benefits of using the prior knowledge in the learning framework.

Most current model reference adaptive control (MRAC) methods rely on parametric adaptive elements, in which the number of parameters of the adaptive element are fixed a priori, often through expert judgment. An example of such an adaptive element is radial basis function networks (RBFNs), with RBF centers preallocated based on the expected operating domain. If the system operates outside of the expected operating domain, this adaptive element can become noneffective in capturing and canceling the uncertainty, thus rendering the adaptive controller only semiglobal in nature. This paper investigates a Gaussian process-based Bayesian MRAC architecture (GP-MRAC), which leverages the power and flexibility of GP Bayesian nonparametric models of uncertainty. The GP-MRAC does not require the centers to be preallocated, can inherently handle measurement noise, and enables MRAC to handle a broader set of uncertainties, including those that are defined as distributions over functions. We use stochastic stability arguments to show that GP-MRAC guarantees good closed-loop performance with no prior domain knowledge of the uncertainty. Online implementable GP inference methods are compared in numerical simulations against RBFN-MRAC with preallocated centers and are shown to provide better tracking and improved long-term learning., United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N000141110688), National Science Foundation (U.S.) (Grant ECS 0846750)

In persistent missions, taking system’s health and capability degradation into account is an essential factor to predict and avoid failures. The state space in health-aware planning problems is often a mixture of continuous vehicle-level and discrete mission-level states. This in particular poses a challenge when the mission domain is partially observable and restricts the use of computationally expensive forward search methods. This paper presents a method that exploits a structure that exists in many health-aware planning problems and performs a two-layer planning scheme. The lower layer exploits the local linearization and Gaussian distribution assumption over vehicle-level states while the higher layer maintains a non-Gaussian distribution over discrete mission-level variables. This two-layer planning scheme allows us to limit the expensive online forward search to the mission-level states, and thus predict system’s behavior over longer horizons in the future. We demonstrate the performance of the method on a long duration package delivery mission using a quadrotor in a partially-observable domain in the presence of constraints and health/capability degradation.

We present a framework for reinforcement learning (RL) in a scenario where multiple simulators are available with decreasing amounts of fidelity to the real-world learning scenario. Our framework is designed to limit the number of samples used in each successively higher-fidelity/cost simulator by allowing the agent to choose to run trajectories at the lowest level that will still provide it with information. The approach transfers state-action Q-values from lower-fidelity models as heuristics for the “Knows What It Knows” family of RL algorithms, which is applicable over a wide range of possible dynamics and reward representations. Theoretical proofs of the framework's sample complexity are given and empirical results are demonstrated on a remote controlled car with multiple simulators. The approach allows RL algorithms to find near-optimal policies for the real world with fewer expensive real-world samples than previous transfer approaches or learning without simulators.

Real-world robots commonly have to act in complex, poorly understood environments where the true world dynamics are unknown. To compensate for the unknown world dynamics, we often provide a class of models to a learner so it may select a model, typically using a minimum prediction error metric over a set of training data. Often in real-world domains the model class is unable to capture the true dynamics, due to either limited domain knowledge or a desire to use a small model. In these cases we call the model class misspecified, and an unfortunate consequence of misspecification is that even with unlimited data and computation there is no guarantee the model with minimum prediction error leads to the best performing policy. In this work, our approach improves upon the standard maximum likelihood model selection metric by explicitly selecting the model which achieves the highest expected reward, rather than the most likely model. We present an algorithm for which the highest performing model from the model class is guaranteed to be found given unlimited data and computation. Empirically, we demonstrate that our algorithm is often superior to the maximum likelihood learner in a batch learning setting for two common RL benchmark problems and a third real-world system, the hydrodynamic cart-pole, a domain whose complex dynamics cannot be known exactly., United States. Office of Naval Research. Multidisciplinary University Research Initiative (N00014-11-1-0688)

A lightweight solution for estimating position and velocity relative to a known marker is presented. The marker consists of three infrared (IR) LEDs in a fixed pattern. Using an IR camera with a 100 Hz update rate, the range and bearing to the marker are calculated. This information is then fused with inertial sensor information to produce state estimates at 1 kHz using a sigma point Kalman filter. The computation takes place on a 14 gram custom autopilot, yielding a lightweight system for generating high-rate relative state information. The estimation scheme is compared to data recorded with a motion capture system., National Science Foundation (U.S.) (Graduate Research Fellowship under Grant No. 0645960)

Commonly used Proportional-Integral-Derivative based UAV flight controllers are often seen to provide adequate trajectory-tracking performance, but only after extensive tuning. The gains of these controllers are tuned to particular platforms, which makes transferring controllers from one UAV to other time-intensive. This paper formulates the problem of control-transfer from a source system to a transfer system and proposes a solution that leverages well-studied techniques in adaptive control. It is shown that concurrent learning adaptive controllers improve the trajectory tracking performance of a quadrotor with the baseline linear controller directly imported from another quadrotor whose inertial characteristics and throttle mapping are very different. Extensive flight-testing, using indoor quadrotor platforms operated in MIT's RAVEN environment, is used to validate the method., United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N000141110688)

A multi-UAV system relies on communications to operate. Failure to communicate remotely sensed mission data to the base may render the system ineffective, and the inability to exchange command and control messages can lead to system failures. This paper describes a unique method to control communications through distributed task allocation to engage under-utilized UAVs to serve as communication relays and to ensure that the network supports mission tasks. The distributed algorithm uses task assignment information, including task location and proposed execution time, to predict the network topology and plan support using relays. By explicitly coupling task assignment and relay creation processes the team is able to optimize the use of agents to address the needs of dynamic complex missions. The framework is designed to consider realistic network communication dynamics including path loss, stochastic fading, and information routing. The planning strategy is shown to ensure that agents support both datarate and interconnectivity bit-error-rate requirements during task execution. System performance is characterized through experiments both in simulation and in outdoor flight testing with a team of three UAVs., Aurora Flight Sciences Corp. (Fellowship Program)

Inverse reinforcement learning (IRL) is the task of learning the reward function of a Markov Decision Process (MDP) given knowledge of the transition function and a set of expert demonstrations. While many IRL algorithms exist, Bayesian IRL [1] provides a general and principled method of reward learning by casting the problem in the Bayesian inference framework. However, the algorithm as originally presented suffers from several inefficiencies that prohibit its use for even moderate problem sizes. This paper proposes modifications to the original Bayesian IRL algorithm to improve its efficiency and tractability in situations where the state space is large and the expert demonstrations span only a small portion of it. The key insight is that the inference task should be focused on states that are similar to those encountered by the expert, as opposed to making the naive assumption that the expert demonstrations contain enough information to accurately infer the reward function over the entire state space. A modified algorithm is presented and experimental results show substantially faster convergence while maintaining the solution quality of the original method., United States. Office of Naval Research (Science of Autonomy Program Contract N000140910625))

This paper presents the design, analysis and experimental testing of a variable- pitch quadrotor. A custom in-lab built quadrotor with on-board attitude stabi- lization is developed and tested. An analysis of the dynamic di erences in thrust output between a xed-pitch and variable-pitch propeller is given and validated with simulation and experimental results. It is shown that variable-pitch actuation has signi cant advantages over the conventional xed-pitch con guration, includ- ing increased thrust rate of change, decreased control saturation, and the ability to quickly and e ciently reverse thrust. These advantages result in improved quadro-tor tracking of linear and angular acceleration command inputs in both simulation and hardware testing. The bene ts should enable more aggressive and aerobatic ying with the variable-pitch quadrotor than with standard xed-pitch actuation, while retaining much of the mechanical simplicity and robustness of the xed-pitch quadrotor., Aurora Flight Sciences Corp., National Science Foundation (U.S.) (Graduate Research Fellowship Grant 0645960)

Risk and reward are fundamental concepts in the cooperative control of unmanned systems. This paper focuses on a constructive relationship between a cooperative planner and a learner in order to mitigate the learning risk while boosting the asymptotic performance and safety of agent behavior. Our framework is an instance of the intelligent cooperative control architecture (iCCA) where a learner (Natural actor-critic, Sarsa) initially follows a “safe” policy generated by a cooperative planner (consensus-based bundle algorithm). The learner incrementally improves this baseline policy through interaction, while avoiding behaviors believed to be “risky”. This paper extends previous work toward the coupling of learning and cooperative control strategies in real-time stochastic domains in two ways: (1) the risk analysis module supports stochastic risk models, and (2) learning schemes that do not store the policy as a separate entity are integrated with the cooperative planner extending the applicability of iCCA framework. The performance of the resulting approaches are demonstrated through simulation of limited fuel UAVs in a stochastic task assignment problem. Results show an 8% reduction in risk, while improving the performance up to 30%., United States. Air Force Office of Scientific Research (Grant FA9550-09-1-0522), Boeing Scientific Research Laboratories

This paper presents a decentralized algorithm that creates feasible assignments for a network of autonomous agents in the presence of coupled constraints. The coupled constraints address complex mission characteristics that include assignment relationships, where the value of a task is conditioned on whether or not another task has been assigned, and temporal relationships, where the value of a task is conditioned on when it is performed relative to other tasks. The new algorithm is developed as an extension to the Consensus-Based Bundle Algorithm (CBBA), introducing the notion of pessimistic or optimistic bidding strategies and the relative timing constraints between tasks. This extension, called Coupled-Constraint CBBA (CCBBA), is compared to the baseline in a complex mission simulation and is found to outperform the baseline, particularly for task-rich scenarios., United States. Air Force Office of Scientific Research (Grant FA9550-08-1-0086), United States. Air Force Office of Scientific Research. Multidisciplinary University Research Initiative (Grant FA9550-08-1-0356)

This paper addresses the problem of task allocation over a heterogeneous team of hu- man operators and robotic agents with the object of improving mission efficiency and reducing costs. A distributed systems-level predictive approach is presented which simul- taneously plans schedules for the human operators and robotic agents while accounting for agent availability, workload and coordination requirements. The approach is inspired by the Consensus-Based Bundle Algorithm (CBBA, a distributed task allocation frame- work previously developed by the authors, which is used to perform the task coordination for the team in a dynamic environment. Results show that predictive systems-level plan- ning improves mission performance, distributes workload efficiently among agents, reduces operator over-utilization and leads to coordinated agent behavior., United States. Air Force Office of Scientific Research (FA9550-08-1-0086), United States. Office of Naval Research (MURI grant FA9550-08-1-0356)

The performance of many complex UAV decision-making problems can be extremely sensitive to small errors in the model parameters. One way of mitigating this sensitivity is by designing algorithms that more effectively learn the model throughout the course of a mission. This paper addresses this important problem by considering model uncertainty in a multi-agent Markov Decision Process (MDP) and using an active learning approach to quickly learn transition model parameters. We build on previous research that allowed UAVs to passively update model parameter estimates by incorporating new state transition observations. In this work, however, the UAVs choose to actively reduce the uncertainty in their model parameters by taking exploratory and informative actions. These actions result in a faster adaptation and, by explicitly accounting for UAV fuel dynamics, also mitigates the risk of the exploration. This paper compares the nominal, passive learning approach against two methods for incorporating active learning into the MDP framework: (1) All state transitions are rewarded equally, and (2) State transition rewards are weighted according to the expected resulting reduction in the variance of the model parameter. In both cases, agent behaviors emerge that enable faster convergence of the uncertain model parameters to their true values.

This paper introduces a hardware platform for automated battery changing and charging for multiple UAV agents. The automated station holds a bu er of 8 batteries in a novel dual-drum structure that enables a "hot" battery swap, thus allowing the vehicle to remain powered on throughout the battery changing process. Each drum consists of four battery bays, each of which is connected to a smart-charger for proper battery maintenance and charging. The hot-swap capability in combination with local recharging and a large 8-battery capacity allow this platform to refuel multiple UAVs for long-duration and persistent missions with minimal delays and no vehicle shutdowns. Experimental results from the RAVEN indoor flight test facility are presented that demonstrate the capability and robustness of the battery change/charge station in the context of a multi-agent, persistent mission where surveillance is continuously required over a speci ed region., Boeing Scientific Research Laboratories, United States. Air Force Office of Scientific Research (FA9550-09-1-0522)

Unmanned Aircraft Systems (UAS) have the potential to perform many of the dangerous missions currently own by manned aircraft. Yet, the complexity of some tasks, such as air combat, have precluded UAS from successfully carrying out these missions autonomously. This paper presents a formulation of a level flight, fixed velocity, one-on-one air combat maneuvering problem and an approximate dynamic programming (ADP) approach for computing an efficient approximation of the optimal policy. In the version of the problem formulation considered, the aircraft learning the optimal policy is given a slight performance advantage. This ADP approach provides a fast response to a rapidly changing tactical situation, long planning horizons, and good performance without explicit coding of air combat tactics. The method's success is due to extensive feature development, reward shaping and trajectory sampling. An accompanying fast and e ffective rollout-based policy extraction method is used to accomplish on-line implementation. Simulation results are provided that demonstrate the robustness of the method against an opponent beginning from both off ensive and defensive situations. Flight results are also presented using micro-UAS own at MIT's Real-time indoor Autonomous Vehicle test ENvironment (RAVEN)., Defense University Research Instrumentation Program (U.S.) (grant number FA9550-07-1-0321), United States. Air Force Office of Scientific Research (AFOSR # FA9550-08-1-0086), American Society for Engineering Education (National Defense Science and Engineering Graduate Fellowship)