Search

Robust and generalizable robots that can autonomously manipulate objects in semi-structured environments can bring material benefits to society. Datadriven learning approaches are crucial for enabling such systems by identifying and exploiting patterns in semi-structured environments, allowing robots to adapt to novel scenarios with minimal human supervision. However, despite significant prior work in learning for robot manipulation, large gaps remain before robots can be widely deployed in the real world. This thesis addresses three particular challenges to advance toward this goal: sensing in semi-structured environments, adapting manipulation to novel scenarios, and flexible planning for diverse skills and tasks. A common theme among the discussed approaches is enabling efficient and generalizable learning by incorporating “structures," or priors specific to robot manipulation, into the design and implementation of learning algorithms. The thesis works follow the three challenges above. We first leverage contact-based sensing in scenarios that are difficult for visionbased perception. In one work, we use contact feedback to track in-hand object poses during dexterous manipulation. In another, we learn to localize contacts on the surface of robot arms to enable whole-arm sensing. Next, we explore adapting manipulation to novel objects and environments for both model-based and model-free skills. We show how learning task-oriented interactive perception can improve the performance of downstream modelbased skills by identifying relevant dynamics parameters. We also show how using object-centric action spaces can make deep reinforcement learning of model-free skills more efficient and generalizable. Lastly, we explore flexible planning methods to leverage low-level skills for more complex manipulation tasks. We develop a search-based task planner that relaxes assumptions on skill and task representations from prior works by learning skill-level dynamics models. This planner is then applied in a followup work that uses learned preconditions of Hybrid Force-Velocity Controllers to perform multi-step contact-rich manipulation tasks. We also explore planning for more flexible tasks described by natural language by using code as the structured action space. This is done by prompting Large Language Models to directly map natural language task instructions to robot policy code, which orchestrates existing robot perception and skill libraries to complete the task.

Designing is an open-ended process that involves multiple iterations of creating artifacts and evaluating them with multifaceted criteria. The challenges come from the fact that not only the artifact, but the criteria for design are gradually articulated and that they are unique to the domain, context, and designer’s intentions. Recent advancements in generative design have sought to aid the 3D design process by delegating the task of auto generating and visualizing multiple artifacts. However, these frameworks still come short by enforcing the designers to prematurely define constraints and prompting them to translate their intentions into a limited set of parametric or operational inputs which is non-trivial and confines the design to certain forms. This thesis introduces an alternative framework of 3D designing where the designer interactively trains a reinforcement learning (RL) based generative agent. It first aims to formulate a RL agent that can be trained to procedurally generate 3D forms given partial and evolving constraints. The proposed method leverages the capabilities of RL agents to procedurally generate feasible 3D forms while adapting to user-provided constraints. It also presents and experiments with an interactive framework between the designer and generative RL agent, that would allow simultaneous articulation of design criteria and artifacts, which is critical in early design development. The interactive framework allows designers to iteratively evaluate auto-generated artifacts and provide intermittent feedback that serves to articulate the implicit design criteria. As a result, this work sheds light on ways to integrate learning-based computational frameworks for idiosyncratic design tasks as well as presenting a novel mode of designing through training generative learning-based agents.

Due to the complex and safety-critical nature of autonomous driving, recent works typically test their ideas on simulators designed for the very purpose of advancing self-driving research. Despite the convenience of modeling autonomous driving as a trajectory optimization problem, few of these methods resort to online reinforcement learning (RL) to address challenging driving scenarios. This is mainly because classic online RL algorithms are originally designed for toy problems such as Atari games, which are solvable within hours. In contrast, it may take weeks or months to get satisfactory results on self-driving tasks using these online RL methods as a consequence of the time-consuming simulation and the difficulty of the problem itself. Thus, a promising online RL pipeline for autonomous driving should be efficiency driven. In this thesis, we investigate the inefficiency of directly applying generic single-agent or distributed RL algorithms to CARLA self-driving pipelines due to the expensive simulation cost. We propose two asynchronous distributed RL methods, Multi-Parallel SAC (off-policy) and Multi-Parallel PPO (on-policy), dedicated to accelerating the online RL training on the CARLA simulator via a specialized distributed framework that establishes both inter-process and intra-process parallelization. We demonstrate that our distributed multi-agent RL algorithms achieve stateof-the-art performances on various CARLA self-driving tasks in much shorter and reasonable time.

The ways that we embody interactions in virtual spaces are rapidly evolving with developments in the fields of deep learning and virtual reality. This is significant because these spaces are increasingly becoming extensions of different human social and creative practices. That being said, there is a lack of perspectives offered towards shaping technological usage that considers creative practices alongside immersive platforms and cutting-edge AI technologies. How then might we as designers, developers, and users, interrogate the intersection of deep learning and VR to better understand how the future landscape of design and virtual interaction might change? This thesis offers insight into how these technologies might overlap in future design environments. This thesis provides a framework for a design interaction between human and agent co-creator in VR supported by reinforcement learning, an unsupervised training method focused on crafting AI behavior. This thesis contributes in two ways, by firstly providing insight into how human-agent interactions can be facilitated given open-ended design problems, and secondly, by exploring RL as a design medium for crafting agent behaviors through creating human demonstrations in VR. This thesis helps create and guide future questioning on the relevancy of emerging AI technologies in open-ended design contexts.

Subterranean exploration has been thrust into the spotlight by the recent DARPA Subterranean Challenge. Teams are tasked with developing a multi-agent system that can rapidly navigate through unknown underground environments while sending back maps and other semantic information. A system with such capabilities has a wide variety of applications from surveying and mapping mines to search and rescue operations. Aerial platforms in particular have the ability to fly fast and discover large parts of an environment despite their lower operational endurance. They can also change altitude at will to explore narrow nooks in spaces that are otherwise difficult to access. The work done in this thesis aims to leverage these abilities to build a resilient autonomy pipeline that enables multiple drones to coordinate an underground exploration mission with the click of a button. Towards that end, we propose an integrated exploration and coordination pipeline that is designed to generalize across a variety of subterranean environments. An ensemble of planners has been built in to tackle a large set of operating conditions ensuring the robot is never still for too long. Coordination map sharing enables the aerial platforms to minimize the overlap in explored volumes between them and other robots on the team. An extensive set of field experiments show that the approach works well in subterranean spaces such as caves, mines and urban infrastructure. We conclude with a discussion on further work planned for improving the system in the future.

We focus on relatively low dimensional robot motion planning problems, such as planning for navigation of a self-driving vehicle, unmanned aerial vehicles (UAVs), and footstep planning for humanoids. In these problems, there is a need for fast planning, potentially compromising the solution quality. Often, we want to plan fast but are also interested in controlling the solution quality because we desire efficient planning. Bounded-suboptimal heuristic search algorithms are a popular alternative to optimal heuristic search algorithms that compromise solution quality for computation speed. Specifically, these searches aim to find a solution that can be computed faster than the optimal solution while guaranteeing that its cost is bounded within a specified factor of the optimal cost (bounded-suboptimality). Currently, most bounded-suboptimal heuristic search algorithms speed up planning by guiding the search using cost-to-goal estimates. Cost-to-goal estimates are not necessarily correlated with the search effort required to reach the goal. The key idea of this thesis is to explicitly reason about anticipated search efforts required to reach the goal instead of cost-to-goal estimates, and achieve higher speedup by guiding the search along solutions that minimize these anticipated search efforts while ensuring bounded-suboptimality of the computed solutions. Also, bounded-suboptimal heuristic search algorithms have been largely investigated in the context of deterministic planning. In this thesis, we use the key idea of this thesis to speed up planning while ensuring bounded suboptimality in the context of deterministic as well as probabilistic planning. To this end, our first contribution is to speed up deterministic robot planning problems formulated as bounded-suboptimal heuristic search. Weighted A* (wA*) search is a popular bounded-suboptimal heuristic search algorithm. We investigate the problem of computing heuristics that explicitly aim to reduce the search efforts of wA*. For heuristic computation, it is common to solve a simpler planning problem in a relaxed space formed by relaxing some constraints in the original search space. We introduce conservative heuristics—novel heuristics that anticipate search efforts required to reach the goal. We first introduce the notion of a conservative path in the relaxed space, whose existence guarantees the existence of a feasible path in the original space. Conservative heuristics are computed such that if a conservative path exists in the relaxed space, then the search can follow the heuristic gradient to find a feasible path in the original space while expanding only those states that appear on this path. We propose an algorithm to compute conservative heuristics. We evaluate conservative heuristics theoretically as well as empirically using simulated experiments in humanoid footstep planning, planning for a UAV, and a real-world experiment in footstep planning for a NAO robot. Our second contribution is to speed up a particular class of planning under uncertainty problems. Many real-world planning problems involve robots having to plan in partially known environments. This frequently requires planning under uncertainty over missing information about the environment. Unfortunately, the computational expense of such planning often precludes its scalability to real-world problems. The Probabilistic Planning with Clear Preferences (PPCP) framework computes optimal policies in a scalable way for a subset of such planning problems wherein there exist clear preferences over the actual values of missing information. It runs a series of fast, deterministic, A*-like searches to construct and refine a policy, guaranteeing optimality under certain conditions. Aligning with the key idea of this thesis, we anticipate that while running a series of A*-like searches to compute a policy, search efforts are correlated with the amount of missing information each path relies upon to reach the goal. We observe that by exploiting this correlation and minimizing the amount of missing information each path relies upon, marginally suboptimal policies can be computed significantly faster. To that end, we introduce Fast-PPCP, a novel planning algorithm that computes a provably bounded-suboptimal policy using significantly lesser number of searches than that required to find an optimal policy, for the same subset of problems that can be solved by PPCP. We evaluate Fast-PPCP theoretically and experimentally, showing its benefit over popular baselines. Moreover, to evaluate Fast-PPCP in discounted-reward problems such as RockSample, we also formulate a transformation that converts discounted-reward problems into discounted-cost problems to which Fast-PPCP can be applied. In the final part of the thesis, we are motivated by the application of Fast-PPCP to real-world robot navigation problems in partially-known environments. In such problems, Fast-PPCP can potentially waste search efforts in exploring multiple partial policies that use an unknown region to reach the goal before discovering that those partial policies are invalid. To reduce this wastage, we introduce an optimized version of Fast-PPCP for planning in environments with large unknown regions. We demonstrate its utility in off-road robot navigation in simulation and on a physical robot.

Swarms consist of numerous entities that work together to achieve an objective. There are twobroad schools of swarms, centralized and decentralized, which refers to the method of organizationemployed by the swarm. With the abundance and readiness of modern processing equipment, mostmodern swarms use a centralized model where control and instructions converge at one point, usually ata controller station. While practical, the centralized model opens the door for problems that can beavoided with some compromise by a decentralized system. However, one of the primary benefits ofusing a decentralized swarm model, security, is undermined by the inclusion of onboard centralprocessing units (CPUs). Our project, Autonomous Swarms for Extreme Conditions (ASEC) project aimedto demonstrate the feasibility of using a decentralized swarm with no central processing units.Specifically, we were tasked with:1. the design and fabrication of three rover-like swarm agents.2. creating a swarm agent that could withstand ���extreme��� conditions (defined later).3. producing these agents in a cost-efficient manner with a low-price final product.Decentralized swarms with no CPUs are often overlooked and overshadowed by their more commonCPU-based, centralized counterparts (CCCs); CCCs hold a clear advantage in the complexity of tasks theycan complete. Additionally, with modern encryption methods, they are secure to most common attacks.CCCs become ineffective when their cyber protection, communication, or required resources becomeunavailable. ASEC acts as a proof of concept for a rover-based, CPUless, decentralized swarm. To achievethese goals, the ASEC design group devised a solution that would require no CPU: taking inspiration frominsect behavior and utilizing analog electronics. The project yielded successful results but certain areas,namely swarm communication, underperformed.Our client, Dr. Borowczak, is a faculty member in Computer Science that specializes incybersecurity. Decentralized swarms are particularly valuable to him because they are much moreresilient to common attacks. Whereas an attack on the center of a centralized system can be devastating,a decentralized swarm can withstand individual attacks without compromising the rest of the system.The work performed by the ASEC team will aid Dr. Borowczak in understanding the challenges that facethe physical implementation of decentralized swarms (DS). The applications of such a swarm are quitenumerous. What is lost in processing capability by the non-inclusion of a CPU is gained in resilience. Withless sensitive electronics and simpler communication channels, DS can be deployed to areas withpunishing conditions such as space, irradiated environments, disaster zones, and conflict zones. Whileour specific design will not be implemented in any of these environments, the core principles of DS agentdesign can be carried over. Every component of the agents acts as a potential vector of attack andmanipulation; thus, the ASEC team was careful to weigh design choices with potential future impacts.

Teleoperated systems are used widely in deployed robots today, for such tasks as space exploration, disaster recovery, or assisted manipulation. However, teleoperated systems are difficult to control, especially when performing high-dimensional, contact-rich tasks like manipulation. One approach to ease teleoperated manipulation is shared control; this strategy combines the user’s direct control input with an autonomous plan to achieve the user’s goal, thereby speeding up tasks and reducing user effort. To do so, the system needs a prediction of the user’s goal. One common approach derives this goal prediction from the user’s control input itself, as it is already available to the system. Prior work using this prediction source in baseline tasks validates the usefulness of shared control. In this thesis, we prove that the effectiveness of control input for goal prediction is a consequence of how optimal users provide control input. When the user’s control input is restricted, however, the assistance may be suboptimal. To improve on this performance, we turn to another source for goal information: natural gaze. People’s natural, unconstrained eye gaze behavior reveals information about their immediate goals and their future tasks.The accuracy and timing of these predictions are different than those provided by the control input pipeline, making it a promising additional source. To effectively use natural gaze for goal predictions and to combine it with control input, we analyze the behavior of each signal and evaluate them in the context of the full assistive system. In this thesis, we show that control input and eye gaze complement each other for goal prediction during shared control. Control input gives local information about the user’s goal, making it particularly effective in simple tasks when people can act optimally but limiting its performance in more complex tasks. On the other hand, eye gaze provides global information about task intentions early, but it does not do so as reliably. We first formalize evaluation criteria for goal prediction sources and examine how goal prediction using control input, the current state of the art, affects the assistance. We show that the autonomous system does not always need to know the user’s specific goal to make progress in the task. One key advantage of control input as a prediction source is that when the user’s control is noisily optimal, the parts of the task where the autonomous system requires goal information coincide with those where the user’s control input is likely to provide that information. However, when user input is restricted so people cannot act optimally, the user’s control input is no longer as informative about the goal; this restriction occurs, for example, when using low-degree of freedom input devices. While the goal information from control input is still reliable when it is available, it may not come early enough in the task, so alternative goal prediction sources may help. Next, we analyze natural eye gaze as a source of global information to supplement the goal prediction given by control input. We collect a data set of natural gaze during a teleoperated manipulation task and show that while people do look at their goals, more often they look at the robot end-effector, and sometimes they complete tasks without ever looking at the goals. From this analysis, we develop a contextual representation for gaze behavior and use it to predict the user’s goal; this signal can give predictions earlier than are available from the control input, but the variability of people’s gaze behavior limits the reliability of the signal when used on its own. Finally, we integrate both signals into a system for online assisted manipulation, and we evaluate the model for the usefulness of each signal in a task that restricts the user’s input and requires multidimensional assistance. When using control input for goal prediction, the system reliably provides some assistance, but cannot do so in all dimensions. When we incorporate gaze-based goal prediction, an earlier goal prediction from gaze enables the assistance to act in all dimensions and increases user task performance. However, the assistance using only gaze performs worse than either other condition, so it benefits from the reliability of another goal prediction source like control input. Developing a model for how different goal prediction sources contribute to assistance quality during shared control enables this assistance strategy to work in more complex situations, such as ones with restricted user input or multipart assistance. The work in this thesis can help ground future explorations of input modalities for goal prediction. With this greater understanding of shared control for effective assistance, this work helps to bring it closer to real-world applications.

Computational context understanding refers to an agent’s ability to fuse disparate sources of information for decision-making and is, therefore, generally regarded as a prerequisite for sophisticated machine reasoning capabilities, as in artificial intelligence (AI). Data-driven and knowledge-driven methods are two classical techniques in the pursuit of such machine sense-making capability. However, while data-driven methods seek to model the statistical regularities of events by making observations in the real-world, they remain difficult to interpret and they lack mechanisms for naturally incorporating external knowledge. Conversely, knowledge-driven methods combine structured knowledge bases, enable symbolic reasoning based on axiomatic principles, and yield more interpretable predictions; however, they often lack the ability to estimate the statistical salience of an inference or robustly accommodate perturbations in the input. To combat these issues, we use hybrid AI methodology as a general framework for combining the strengths of both approaches. Specifically, we inherit the concept of neuro-symbolism as a way of using domain knowledge to guide the learning progress of deep neural networks. Domain knowledge appears in many forms, including: (i) graphical models, which characterise such relationships between entities as dependence, independence, causality, correlation, and partial correlation; (ii) commonsense knowledge, which covers spatial knowledge, affordances from objects’ physical properties, semantic relations, and functional knowledge; (iii) privileged information, in the form of demonstrations or soft labels from an expert agent; (iv) learned behaviour primitives and priors, which agents may compose for generalisable and transferable task-execution; and (v) auxiliary tasks, objectives, and constraints — carefully-chosen, for constrained optimisation. Regardless of the type of domain knowledge available, the same practical objective remains: to learn meaningful neural representations, for downstream tasks of interest. The underlying goal of neural representation learning is to statistically identify the best explanatory factors of variation in the agent’s input data or observations, often requiring intuition about the complementarity between multiple modalities or views in the input. While there has been much focus on learning effective neural representations for specific tasks, then transferring or adapting the learned representations to other tasks, comparatively less focus has been placed on representation learning in the presence of various types of domain knowledge. This knowledge could be used to recover information about the underlying generating process, to design effective modelling strategies in learning problems, to ensure model transferability or generalisability, or to understand the complementarity between views. This thesis studies the avenues by which the aforementioned types of domain knowledge can be combined with neural representations, in order to achieve improved model performance and generalisability for the following problem domains: neural commonsense reasoning, multimodal robot navigation, and autonomous driving. This thesis contributes a collection of tools, methodologies, tasks, international AI challenges and leaderboards, datasets, and knowledge graphs; additionally, this work led to the successful organisation of two international workshops in safe learning for autonomous driving.

Multi-agent reinforcement learning involves and facilitates a team of agents to solve complex decision-making problems in shared environments. This learning process is largely successful in many areas, but its inherently stochastic nature is problematic when applied to safety-critical domains.To solve this limitation, we propose our Multi-Agent Reinforcement Learning (AMARL), which uses a model checking technique called quantitative verification. Quantitative verification provides formal guarantees of agent compliance to safety, performance, and other non-functional requirements, while reinforcement learning occurs and after a policy has been learned.Our AMARL approach is demonstrated using three separate navigation domains, which contain patrolling problems. The multi-agent systems must learn to visit patrol points to satisfy mission objectives while limiting exposure to risky areas in these domains. Different reinforcement learning algorithms have been utilised within these domains: temporal difference learning, game theory, and direct policy search. The performance of these algorithms, while combined with our approach, are presented. Lastly, we demonstrate AMARL with differing system sizes in both homogeneous and heterogeneous multi-agent systems through our extensive experimentation. This experimentation shows that the use of AMARL leads to faster and more efficient performance than standard reinforcement learning and consistently meets safety requirements.

As more service robots are deployed in the world, human-robot interaction will not be limited to one-to-one interactions between users and robots. Instead, users will likely have to interact with multiple robots, simultaneously or sequentially, throughout their day to receive services and complete different tasks. In this dissertation, I describe work, in collaboration with my colleagues, broadened the knowledge on a crucial aspect of multi-robot human interaction: person transfer, or the act of transferring users between multiple service robots. We first investigated rationales for transfer and important aspects of transferring users. We then explored how person transfers should be designed and implemented in laboratory and field settings. We used a combination of design, behavioral, and technical methods to understand the challenges and nuances in realizing person transfers. Our research consisted of (1) A collection of Research through Design workshops to chart out the space of person transfers; (2) A lab study to understand how people perceive social interaction between robots and the flow of information in a person transfer scenario; (3) A description of an interactive system that implemented realistic person transfers in both laboratory and field settings; (4) A lab study that evaluated different robot joining strategies and people’s spatial behavioral responses during person transfers; and (5) An in-the-field demonstration of person transfers. Our work seeks to increase our understanding of this crucial phase and inform developers and designers about appropriate robot behaviors when a human is being transferred from one robot to another.

Socially interactive robots present numerous unique programming challenges for interaction developers. While modern authoring tools succeed at making the authoring experience approachable and convenient for developers from a wide variety of backgrounds, they are less successful at targeting assistance to developers based on the specific task or interaction being authored. We propose interaction templates, a data-driven solution for (1) matching in-progress robot programs to candidate task or interaction models and then (2) providing assistance to developers by using the matched models to generate modifications to in-progress programs. In this paper, we present the various dimensions that define first how interaction templates might be used, then how interaction templates may be represented, and finally how they might be collected.

Reinforcement learning requires large amounts of training data. For many systems, especially mobile robots, collecting this training data can be expensive and time consuming. We propose a novel domain adaptation method to reduce the amount of training data needed for model-based reinforcement learning methods to train policies for a target system. Using our method, the required amount of target system training data can be reduced by collecting data on a proxy system with similar, but not identical, dynamics on which training data is cheaper to collect. Our method models the underlying dynamics shared between the two systems using a System Invariant Dynamics Model (SIDM), and models each system’s relationship to the SIDM using encoders and decoders. When only limited amounts of target system training data is available, using target and proxy data to train the SIDM, encoders, and decoders can lead to more accurate dynamics models for the target system than using target system data alone. We demonstrate this approach using simulated wheeled robots driving over rough terrain, varying dynamics parameters between the target and proxy system, and find a reduction of 5-20x in the amount of data needed for these systems.

Planning problems in which an agent must perform tasks for reward by navigating its environment while constrained by time and location have a wide variety of applications inrobotics. Many real-world environments in which such planning problems apply, such as office buildings or city streets, are very structured. They consist of passages with notable locations along them such as streets or hallways, and notable locations might be grouped into clusters such as floors in a building divided by bottlenecks such as elevators.In this thesis, we introduce algorithms designed for an agent performing tasks for reward while constrained by time and location in a structured environment. Our goal is specificallyto exploit the structure of the environment when planning in order to improve the reward received. We present three algorithms that do so in different ways. First, we present the Task Graph algorithm, which groups actions the agent can perform into larger \tasks" and creates a plan from a sequence of tasks, rather than individual actions. We test the algorithm in a problem we call \Exploration Scheduling," in which an agent seeks to gather information in the free time between mandatory tasks in order to improve the reward received from the mandatory tasks, and find it yields strong results. Second, we present the Region Plan algorithm, which divides the environment into regionsbased on its structure and creates a separate plan for each region to choose from. We test the Region Plan algorithm's ability to solve a variant of the orienteering problem in simulated structured environments, and demonstrate environment structures and reward distributions in which the Region Plan approach allows the agent to avoid getting stuck at certain locally optimal solutions. Third, we present Route-Based Multi-Level Variable Neighborhood Search (Route- MLVNS), a variable neighborhood search algorithm that represents solutions to the problem as routes the agent takes through the environment, rather than sequences of actions. In this way, Route-MLVNS takes into account the locations that the agent passes by as it travels when evaluating the reward of a location, rather than only considering the reward of traveling to a single location at a time. We test Route-MLVNS in a simulation of the orienteering problem in structured environments, comparing it to an MLVNS algorithm from the literature that treats solutions as ordinary plans of tasks, and find Route-MLVNS to yield superior results. We also demonstrate a unique capability that Route-MLVNS has, the ability to solvethe orienteering problem for a continuous range of time constraints simultaneously rather than only for a single time constraint or discrete set of time constraints. Finally, we consider scenarios in which an agent solves repeated instances of the orienteering problem in a structured environment with stochastic rewards that are initially unknown. We present algorithms which treat locations in the environment as levers in a multi-armed bandit problem and assign them values based on both the expected reward and the potential value of the knowledge gained by visiting them, and use planning algorithms based on the environment structure to find a plan that maximizes the assigned value received. We perform tests demonstrating the algorithms' effectiveness at both receiving reward over time and learning an effective model. Additionally, we analyze the relationship between the structure of the environment and the distribution of the data received by the agent.Overall, our work demonstrates the benefit of an agent considering and exploiting the structure of its environment when planning tasks, and how the structure affects the data it can gather in environments with unknown stochastic rewards.

Human-in-the-loop control for mobile robots is an important aspect of robot operation, especially for navigation in unstructured environments or in the case of unexpected events. However, traditional paradigms of human-in-the-loop control have relied heavily on the human to provide precise and accurate control inputs to the robot, or reduced the role of the human to providing supervisory task specifications. In this thesis, we explore a new paradigm of human-robot collaboration called human-in-the-loop planning, where the robot can act semi-autonomously according to the human’s intention while having the human directly inform motion planning and trajectory generation. The proposed paradigm maximizes the strengths of the human and robot such that the human-robot system can perform with optimal efficiency. To this end, we first abstract away complex vehicle dynamics by way of motion primitive teleoperation, which allows an operator to control a vehicle via input reparameterization into trajectories. We then build upon motion primitive teleoperation and present a method for reactive collision avoidance. We then propose a novel method of local trajectory generation without end goal specifications for human-in-the-loop control. The method, called Biased Incremental Action Sampling, is a sample based approach to build motion primitive trees that optimize for non-goal based cost functions. We then introduce hierarchical human-in-the-loop planning, which incorporates intended motions as global paths such that generated local trajectories can follow the paths autonomously. Lastly, we introduce continuous dynamic autonomy by generating path predictions on semantic topological navigation maps. By incorporating environment contexts into human-in-the-loop control, this allows us to reason about the human’s intentions over the path space and generate predictions to assist navigation in unstructured, constrained environments.

This dissertation examines the impact of active shooters upon patrons attending large outdoor events. There has been a spike in shooters targeting densely populated spaces in recent years, to include open-air venues. The 2019 Gilroy Garlic Festival was selected for modeling replication using AnyLogic software to test various experiments designed to reduce casualties in the event of an active shooter situation. Through achievement of validation to produce identical outcomes of the real-world Gilroy Garlic Festival shooting, the researcher established a reliable foundational model for experimental purposes. This active shooter research project identifies the need for rapid response efforts to neutralize the shooter(s) as quickly as possible to minimize casualties. Key findings include the importance of armed officers patrolling event grounds to reduce response time, the need for adequate exits during emergency evacuations, incorporation of modern technology to identify the shooter’s location, and applicability of a 1:548 police to patron ratio.

We describe a new task called The Amazing Race: Robot Edition. In this task, the robot is placed in a real, unknown environment, without a map, and asked to find a designated location. It will need to explore its surroundings, find and approach people, engage them in a dialogue to obtain directions to the goal, and follow those directions to the hallway with the goal. We describe and implement a variety of robotic behaviors that performs each of these functions. We test these in the real world in test environments that were distinct from the training environments where we developed our methods and trained our models. Additionally, these test environments were completely unmodified and reflect the state of the real world.First, we describe how our robotic system solves this problem where the environment is constrained to a single floor or a single building. We demonstrate that we are able to find a goal location in never-before-seen environments. Next, we describe a machine-learned approach to the dialogue and components of our system to make it more robust to the diversity and noisiness of navigational instructions someone may provide.

Public use of multi-robot systems has significantly increased and such multi-robot systems are increasingly using open networks for operation, which poses new challenges for robotic systems. For example, a typical multi-robot application receives and transmits a great deal of information between sensors, actuators, and controllers via cyberspace, allowing potential security breaches. The hierarchical nature of multi-robot systems that commonly operated through supervisory control in unprotected communication channels makes itself more vulnerable to cyberthreats. These risks could lead to worst-case scenarios, such as security breaches and safety issues. In this dissertation, we deal with problems concerning the security of multi-robot systems, covering key aspects such as threat identification methodologies and countermeasure developments. Specifically, this dissertation proposes attack diagnosis methods and attack resilient control architectures to ensure the security of multi-robot systems with the following objectives: i) to introduce types of attacks that reveal specific vulnerabilities of robotic systems and corresponding design requirements to describe how a cyberthreat may affect the system functionality; ii) to develop an attack diagnosis scheme that enables each robot in multi-robot systems to continuously monitor undesirable actions caused by cyberthreat; iii) to design an attack-resilient control framework that adapts to undesirable situations so that the overall performance is guaranteed, thus mitigating the impact of attacks; and iv) to evaluate the performance of the proposed systems with both numerical simulations and experiments using real robots. With these objectives, the methodologies proposed in this dissertation are consist of two phases -- attack detection and countermeasure. The attack detection phase is to determine if any agent in multi-robot systems is being attacked. Any abrupt change or unexpected dynamic behavior is identified by a stochastic process-based local diagnosis system. The countermeasure phase is to protect the whole team from adversarial attacks. Attack resilient control algorithms are exploited to recover desirable performance and ensure continuous safe operation of the system. Thus, the proposed methods ensure the desired control performance of multi-robot systems in the presence of different types of attacks. This approach is also capable of providing attack resilient control algorithms of multi-vehicle systems in which each vehicle is modeled as a nonlinear system.

Enhancing the capability to control a tremendous range of physical actuators and sensors, combined with wireless technology and the Internet of Things (IoT), apparel technologies play a significant role in supporting safe, comfortable and healthy living, observing each customer’s conditions. Since apparel technologies have advanced to enable humans to work as a team with the clothing they wear, the interaction between a human and apparel is further enhanced with the introduction of sensors, wireless network, and artificially intelligent techniques. A variety of wearable technologies have been developed and spread to meet the needs of customers, however, some wearable devices are considered as non-practical tech-oriented, not consumer-oriented.The purpose of this research is to develop an apparel system which integrates intelligent autonomous agents, human-based sensors, wireless network protocol, mobile application management system and a zipper robot. This research is an augmentation to the existing research and literature, which are limited to the zipping and unzipping process without much built in intelligence. This research is to face the challenges of the elderly and people with self-care difficulties. The intent is to provide a scientific path for intelligent zipper robot systems with potential, not only to help people, but also to be commercialized.The research develops an intelligent system to control of zippers fixed on garments, based on the profile and desire of the human. The theoretical and practical elements of developing small, integrated, intelligent zipper robots that interact with an application by using a lightweight MQTT protocol for application in the daily lives of diverse populations of people with physical challenges. The system functions as intelligent automatized garment to ensure users could positively utilize a zipper robot device to assist in putting on garments which also makes them feel comfortable wearing and interacting with the system. This research is an approach towards the “future of fashion”, and the goal is to incentivize and inspire others to develop new instances of wearable robots and sensors that help people with specific needs to live a better life.

Multi-robot applications frequently seek to employ human operators to direct robot actions online because fully automated planners struggle to encode human expertise orhandle the extenuating circumstances that occur during real world operations. However, it is extremely challenging for a human to direct multi-robot teams, especially online, i.e., in real-time. From entertainment to defense, applications can require detailed inter-robot coordination direction which can be difficult for an operator to specify. Finding motion plans that then meet user intent is complicated by the need tooperate in cluttered environments while absolving the human operator from having to consider physical constraints of the system such as dynamic feasibility or safety.Existing multi-robot policy specification and planning methods have difficulty providing all the required capabilities to enable high-speed, online, human-directed multiagentmotion generation in cluttered environments. While reaction based or local control methods have been widely used for multi-robot applications due to their fast computation times, these methods can fail to produce coordinated responses or safety guarantees. In contrast, optimization methods and search approaches are frequently used to coordinate actions across robots and generate high quality motion plans. Unfortunately, the large number of inherent decision variables, especially as frequently incurred when considering numerous obstacles, often means these methods require computation times in excess of online operation limits. In this thesis, we enable a search based methodology to find safe and feasible multirobottrajectories at fast time scales even in highly cluttered environments by leveraging a group-based representation of multi-robot actions and offline acquired data. Ourapproach generalizes across environments and enables an operator to coordinate multirobot motions online for applications requiring high-frequency plan generation such as required by joystick control. We show results for two human-commanded multi-robot applications: theatrical performance and urban reconnaissance. Both applications require a multi-robot planner to resolve differences between operator input and feasibility requirements given the current state of the system, and generate dynamically feasible and safe trajectories for all agents within time and computation constraints. This thesis establishes an online multi-robot planning and control system for the specification of multi-robot behaviors, and we demonstrate the feasibility of the approach using a multi-quadrotor system, controlling up to 30 physical robots online in the context of the theatric application. We then address assumptions made under thetheatric context to meet the high-frequency planning and cluttered environment context of the urban reconnaissance application, and present a generalized formulation ofmulti-robot behaviors and leverage examples of multi-agent actions from real world data sets to inform an online search policy. We demonstrate that the generalized representationcoupled with the proposed data-based search heuristic enables high-speed multi-robot coordination in cluttered environments, providing needed capabilities to enable multi-robot solutions for complex real world applications.

Planetary rovers allow for science investigations in remote environments. Theyhave traversed many kilometers and made major scientific discoveries. However,rovers spend a considerable amount of time awaiting instructions from mission control.The reason is that they are designed for highly supervised data collection, notfor autonomous exploration. The exploration of farther worlds will face increasingchallenges and constraints. Such missions will demand a new approach.This work advocates Bayesian models as powerful tools for a new paradigmfor robotic explorers. In this approach, the explorer���s description of where to gois not prescribed by a fixed set of instructions, but instead by a model of what theexplorer believes. This formulation has several benefits. Bayesian models providea mathematically grounded framework to reason about uncertainty. They can allowrobots to gain a deeper understanding of the evolving scientific goals guiding themission. Furthermore, they can empower scientists by providing explainable results.To this end, this research develops models that allow for data interpretation bylearning and exploiting structure in the data and the environment. It shows how thesemodels enable robotic explorers to make intelligent decisions based on instantaneousinformation. Ultimately, it demonstrates how science productivity is improved bymeasuring science value with information-theoretic variables and by formulatingthe exploration problem in terms of Bayesian experimental design.This work makes several contributions to the field of science-driven robotic exploration.First, it introduces three different deep generative models for the analysisof data that enable scientists to quantify and interpret learned statistical dependencies.Then, it presents an adaptive exploration model that leverages contextual informationfrom remote data to efficiently extrapolate features from in situ observations,as well as a corresponding strategy for improving science productivity. Afterward, itestablishes a hierarchical probabilistic structure in which scientists initially describetheir abstract beliefs and hypotheses, and then this belief evolves as the robot makesraw measurements; additionally, science information gain is efficiently computedand maximized. Finally, it proposes a comprehensive model for planetary rover explorationthat considers both science productivity and risk.The presented Bayesian models are validated and evaluated in various scienceinvestigation scenarios that can provably benefit from autonomous robotic exploration.Such scenarios include terrestrial, airless, and Martian surface surveys, aswell as marine biology studies. Emphasis is placed on spectroscopic data, which iswidely used in the natural sciences for composition analysis. Promising results areshown in simulations and field experiments using the autonomous rover Zo��e.

Robots have inherently limited onboard processing, storage, and power capabilities. Cloud computing resources have the potential to provide significant advantages for robots in many applications. However, to make use of these resources, frameworks must be developed that facilitate robot interactions with cloud services. We first propose a cloud-based architecture called Smart Cloud that intends to overcome the physical limitations of single- or multi-robot systems through massively parallel computation that is provided on demand by cloud services. Smart Cloud is implemented on Amazon Web Services (AWS) and available for robots running on the Robot Operating System (ROS) and on non-ROS systems. Smart Cloud also features a first-of-its-kind architecture that incorporates JavaScript-based libraries to run various robotic applications related to machine learning and other methods. We later develop and validate three types of application offloading algorithms for cloud robotics. For the first algorithm, we evaluate the architecture performance with respect to a full offloading schema that offloads the whole computation onto the cloud service provider. For the full offloading schema, we deploy the application completely on the cloud without considering the local capabilities of the robot. We validated this schema with machine learning and navigation applications. We evaluated the performance of the architecture both with complete offloading and with no offloading in terms of CPU usage, latency, and security.The key to effectively offloading tasks is an application solution that does not underutilize the robot’s own computational capabilities and makes decisions based on crucial cost parameters such as latency and CPU availability. Hence for the second algorithm, we formulate the application offloading problem as a Markovian decision process and propose a deep reinforcement learning-based deep Q-network (DQN) approach. The state-space is formulated with the assumption that input data size directly impacts application execution time. The proposed algorithm is designed as a continuous task problem with discrete action space; i.e., we apply a choice of action at each time step and use the corresponding outcome to train the DQN to acquire the maximum rewards possible. To validate the proposed algorithm, we designed and implemented a robot navigation testbed. The results demonstrated that for the given state-space values, the proposed algorithm learned to take appropriate actions to reduce application latency and also learned a policy that takes actions based on input data size. Finally, we compared the proposed DQN algorithm with a long short-term memory (LSTM) algorithm in terms of accuracy. When trained and validated on the same dataset, the proposed DQN algorithm obtained at least 9 percentage points greater accuracy than the LSTM algorithm.Another key factor to enabling robot use of cloud computing is designing an efficient offloading algorithm that forms a quick consensus on when to offload without any prior knowledge or information about the application. The third algorithm is designed to be trained quickly after the application has been initiated. We propose a predictive algorithm to anticipate the time needed to execute an application for a given application data input size with the help of a small number of previous observations. To validate the algorithm, we train it on the previous N observations, which include independent (input data size) and dependent (execution time) variables. To understand how algorithm performance varies in terms of prediction accuracy and error, we tested various N values using linear regression and a mobile robot path planning application. From our experiments and analysis, we determined the algorithm to have acceptable error and prediction accuracy when N>40.

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple highly trained human operators to safely controla single vehicle. Our research focus is to build autonomous systems that can empower any individual with the full artistic capabilities of aerial cameras. We develop a system for active vision: in other words, one that not only passively processes the incoming sensor feed, but on the contrary, actively reasons about the cinematographic quality of viewpoints and safely generates sequences of shots. The theory and systems developed in this work can impactvideo generation for both real-world and simulated environments, such as professional and amateur movie-making, videogames, and virtual reality. First, we formalize the theory behind the aerial filming problem by incorporating cinematography guidelines into robot motion planning. We describe the problem in terms of its principal cost functions, and develop an efficient trajectory optimization framework for executing arbitrary types of shots while avoiding collisions and occlusions with obstacles. Second, we propose and develop a system for aerial cinematography in the wild. We combine several components into a real-time framework: vision-based target estimation, 3D signed-distance mapping for collision and occlusion avoidance, and trajectory optimization for camera motion. We extensively evaluate our system both in simulation and in fieldexperiments by filming dynamic targets moving through unstructured environments. Third, we take a step towards learning the intangible art of cinematography. We all know agood clip when we see it - but we cannot yet objectively specify a formula. We propose the use of deep reinforcement learning with a human evaluator in the loop to guide the selection of artistic shots, and show that the learned policies can incorporate intuitive concepts ofhuman aesthetics. Next, we develop novel data-driven framework to enable direct user control of camera positioning parameters in an intuitive learned semantic space (e.g. calm, enjoyable, establishing), and show its effectiveness in a series of user studies. Lastly, we take the first steps towards the concept of multi-camera collaboration for filming. The use of multiple simultaneous viewpoints is necessary when capturing real-world scenes such as sports or social events. In these situations it is difficult to capture the optimal viewpoint at all times employing a single aerial camera, specially because the events cannot be reenacted for additional takes. Here, we design motion planning algorithms for multicamera cinematography that are able to maximize the quality of multiple video streams simultaneously using limited onboard resources.

Robotic grasping has seen tremendous advancements in recent years. Yet, the current paradigm of manipulation research is typically some form of table-top manipulationin constrained setups or in simulation. Building general purpose personal robots that can autonomously grasp unknown objects in unstructured environments like homes isan open problem. In this thesis, we explore important directions in scaling data-driven grasping to the diversity and constraints imposed by the real world. We first discuss how we can go beyond picking individual objects in isolation to 6-DOF grasping in clutter. Most existing methods train policies on datasets collected in curated settings (in lab or simulation) and hence may not cope with the mismatchin data distribution when deployed in the wild. We build and open-source a low-cost mobile manipulator platform to parallelize data collection in challenging settings likehomes and show that policies trained on this data generalize to novel objects in unseen homes. As a result, we also discuss ideas for scaling robot learning with several robotsand transferring policies between different hardware. Yet, we hypothesize that visual perception alone is insufficient for robustness and present a self-supervised tactile-basedre-grasping framework to close the loop on grasp execution. Lastly, we strive to go beyond robotic pick-and-place and generalize to diverse semantic manipulation tasks.We do so by scaling task-oriented grasping datasets with crowdsourcing and learning from semantic information like knowledge graphs.

Report related to Objective 2

Report related to Objective 1.

Robotic-assisted minimally invasive surgery (RMIS) has been steadily increasing since its introduction in the early 2000s and now has become a medical standard of care in multiple surgical specialties. In RMIS, the leading surgeon teleoperates a surgical robot from a console distant from the patient. While on the patient side, there is at least one surgical assistant supporting the procedure by effective handling surgical instruments. One of the most important tasks done by the surgical assistant is blood suction and irrigation. This task is critical to maintaining a clear view of the surgical field and avoid contaminations and infections. When several tasks are competing for the surgical assistant’s attention, taking care of blood suction implies leaving unattended other assistive tasks, such as the exchange of robotic instruments and handling of sutures. An alternative approach to handle bleeding events is having the leading surgeon teleoperate the suction tool. Likewise, this leads to less attention allocated to the patient and an increase in their cognitive load. This thesis describes a semi-autonomous suction assistant to release the main surgeon of blood suction during a procedure and avoiding the associated cognitive demands of such task. At the heart of this system, there is a deep learning algorithm that segments and identifies the location of blood poolings from the endoscopic camera’s images. With the segmented images, the system extracts navigational information to provide automatic suction, allowing the leading surgeon to focus exclusively on the main task. The system was integrated into a Da Vinci Research Kit robot (DVRK). Additionally, an augmented reality (AR) and a real-time cognitive workload assessment module were developed to improve human-robot work dynamics. The AR module displayed semi-transparent annotations indicating the robot's next target location. These annotations allowed the user to better coordinate his actions with the surgical robot. The cognitive workload assessment module allowed to classify the users' mental state into low and high cognitive workload. Using this information, the robotic assistant provided suction during periods of high mental demands. To evaluate the proposed framework, a computational experiment and two user studies were conducted. The goal of the computational experiment was to assess the prediction performance of the proposed cognitive workload detection system under two different modalities: single-user models and multi-user models. In the single-user modality, an average classification accuracy of $76\%$ was achieved. This result demonstrates how to effectively use EEG and eye-tracker features to predict cognitive states in RMIS procedures. In the first user study, the goal was to evaluate the capability of the autonomous framework to improve the user's surgical performance. This was achieved by comparing the autonomous system against a condition of manual teleoperation of the suction tool. The study's main finding was a reduction in the completion time and the reported workload demands when using the autonomous system. The goal in the second study was to evaluate the integration of the autonomous system with the cognitive workload framework. In this setting, the robotic assistant would only act when the user's mental state was classified as high cognitive workload. To achieve the study's goal, the autonomous system was assessed against manual teleoperation of the suction tool. The main results for this study show a reduction in the completion time and improved human-robot collaboration fluency. Overall, the experiments' results shows how objective and real-time assessment of cognitive load can be used together with surgical autonomy to enhance RMIS surgical outcomes.

Report related to Objective 3.

Disaster relief scenarios require rapid and persistent situational awareness to inform first-responders of safe and viable routes through a constantly shifting environment.Knowing what roads have become flooded or are suddenly obstructed by debris can significantly improve response time and ease the distribution of resources. In a sufficiently large environment, deploying and maintaining fixed camera stands would be ineffective and prohibitively expensive, so we look to deploying teams of robots equipped with sensors to persistently cover the region of interest. In this case,the main challenge is determining how to distribute the robots to cover the entire region with limited travel speed and duration. Basic multi-robot coverage and exploration methods take a passive approach that direct robots to evenly cover the space and populate a map of the environment with the observations the robots acquire as they move. More reactive frontier-based approaches will continuously guide robots towards unobserved regions until the environment is fully known. These approaches, however, are less effective when the environment changes over time. When the number of robots are limited and can only operate for finite durations, the planner must prioritize which regions to visit in order to provide the most accurate map possible.In this thesis, we propose to plan deployments of teams of quadrotors equipped with range sensors to cooperatively cover an environment such that a map can be persistently updated as the environment topography evolves. Here, we present a systems-based approach that breaks up planning into stages and computes feasible plans over a sliding-window horizon. These plans are extended over subsequenthorizons up to the limit that each robot’s battery capacity will allow, ending with robots returning to home base and recharging. We initially show that the proposed planner is able to outperform greedy frontier assignment in terms of map accuracy and confidence. We then show how the objective function we initially used to distribute robots can be modified to incorporate the ‘goodness-of-fit’ of the environment dynamics model by biasing robots towards regions whose dynamics are less understood. The updated objective results in a system that quickly converges to robotsrevisiting regions only as often as they are expected to change. While there is a physical limitation to how much area a team of energy-constrained robots can cover persistently, the system we present leverages an understanding of environment dynamics to maximize model improvement over time in a computationally tractable fashion. This is shown with a comprehensive study of the computational complexity of each component of the proposed system and an evaluation of how overall performance evolves over time relative to the choice of system parameters and environment conditions. We additionally show that the system can be tuned to better address environments experiencing changes of greatly varying magnitudes and scale. The result is a complete system that enables perpetual deployment and efficient distribution of robots throughout the environment to ensure no changes go unobserved for too long.

Based on a nationally-representative public opinion survey of over 2000 Australian adults, the report examines key areas of public understanding, optimism and concern regarding the societal application of AI technologies. As industry and policymakers continue to develop, implement and manage AI across most areas of Australian society, this report explores the often overlooked views of the general public – in many ways, the ultimate ‘end users’ of these powerful technologies.

The onset of Industry 4.0 has provided considerable benefits to Intelligent Cyber-Physical Systems (ICPS), with technologies such as internet of things, wireless sensing, cognitive computing and artificial intelligence to improve automation and control. However, with increasing automation, the “human” element in industrial systems is often overlooked for the sake of standardization. While automation aims to redirect the workload of human to standardized and programmable entities, humans possess qualities such as cognitive awareness, perception and intuition which cannot be automated (or programmatically replicated) but can provide automated systems with much needed robustness and sustainability, especially in unstructured and dynamic environments. Incorporating tangible human skills and knowledge within industrial environments is an essential function of “Human-in-the-loop” (HITL) Systems, a term for systems powerfully augmented by different qualities of human agents. The primary challenge, however, lies in the realistic modelling and application of these qualities; an accurate human model must be developed, integrated and tested within different cyber-physical workflows to 1) validate the assumed advantages, investments and 2) ensure optimized collaboration between entities. Agricultural Robotic Systems (ARS) are an example of such cyber-physical systems (CPS) which, in order to reduce reliance on traditional human-intensive approaches, leverage sensor networks, autonomous robotics and vision systems and for the early detection of diseases in greenhouse plants. Complete elimination of humans from such environments can prove sub-optimal given that greenhouses present a host of dynamic conditions and interactions which cannot be explicitly defined or managed automatically. Supported by efficient algorithms for sampling, routing and search, HITL augmentation into ARS can provide improved detection capabilities, system performance and stability, while also reducing the workload of humans as compared to traditional methods. This research thus studies the modelling and integration of humans into the loop of ARS, using simulation techniques and employing intelligent protocols for optimized interactions. Human qualities are modelled in human “classes” within an event-based, discrete time simulation developed in Python. A logic controller based on collaborative intelligence (HUB-CI) efficiently dictates workflow logic, owing to the multi-agent and multi-algorithm nature of the system. Two integration hierarchies are simulated to study different types of integrations of HITL: Sequential, and Shared Integration. System performance metrics such as costs, number of tasks and classification accuracy are measured and compared for different collaboration protocols within each hierarchy, to verify the impact of chosen sampling and search algorithms. The experiments performed show the statistically significant advantages of HUB-CI based protocol over traditional protocols in terms of collaborative task performance and disease detectability, thus justifying added investment due to the inclusion of HITL. The results also discuss the competitive factors between both integrations, laying out the relative advantages and disadvantages and the scope for further research. Improving human modelling and expanding the range of human activities within the loop can help to improve the practicality and accuracy of the simulation in replicating an HITL-ARS. Finally, the research also discusses the development of a user-interface software based on ARS methodologies to test the system in the real-world.

Judge's choice joint 2nd prize-winning poster in the Postgraduate Research Poster Competition held by the Graduate School at the Open University on 10th June 2020.

Game balance has been a challenge for game developers since the time games have become more complex. There have been a handful of proposals for game balancing processes outside the manual labor-intensive play testing methods, which most game developers often are forced to use simply due to the lack of better methods. Simple solutions, like restrictive game play, are limited because of their inability to provide insight on interdependencies among the mechanisms in the game. Complex techniques framed around the potential of AI algorithms are limited by computational budgets or cognition inability to assess human actions. In order to find a middle ground we investigate Game Theory and Mechanism Design concepts. Both have proven to be effective tools to analyse strategic situations among interacting participants, or in this case `players'. We test the feasibility of using these techniques in an Real Time Strategy (RTS) game domain to understand game balance. MicroRTS, a small and simple execution of an RTS game is employed as our model. The results provide promising insight on the effectiveness of the method in detecting imbalances and further inspection to find the cause. An additional benefit out of this technique, besides detecting for game imbalances, the approach can be leveraged to create imbalances. This is useful when the designer or player desires to do so.

Water ice at the lunar poles could be the most valuable resource beyond planet Earth. However, that value is not foregone, and can only be determined by rovers that evaluate the distributions of abundance, concentration, and characteristics of this ice. The near-term explorations will be solar and unlikely to endure night and hence are limited to a single cycle of lunar daylight. Sun-synchronous polar routes promise order-of-magnitude greater duration andrange. Such routes are, however, brittle in the sense that a rover must be at precisely scheduled time and place. Methods for terrain-based localization that workat low latitudes are challenged at the lunar poles, where the grazing sunlight casts long shadows that obscure and change views over time. Moreover, the cadence ofexploration driven by the clocking sun requires terrain localization to be highly informative. Technologies in these areas are lacking, which results in a level of risk that currently makes these missions not viable. This thesis develops methods for terrain relative navigation that enable the robust execution of sun-synchronous routes at the lunar poles. This research advances terrain registration under sensor and map uncertainty, exploits shadows as features for localization, and develops planning algorithms that increase information gain from terrain registration.There are four major contributions of this research. The first is a scan matching algorithm that probabilistically geometrically registers rover perspective stereoimages to a digital elevation map which is evaluated against priors in both simulated and real small-scale experiments. The second is an extension of the geometric method to exploit shadows as features. The third is a method of predicting the uncertainty of terrain-based localization given terrain geometry and lighting. The fourth is a belief space planning algorithm that maximizes the probability of surviving sunsynchronous exploration by considering when and where to perform terrain relative corrections. Together, these methods are evaluated for route robustness and mission risk within the developed simulation framework and with scale field data. The work developed in this thesis enables robust sun-synchronous routes for multi-month exploration and increased science return at the lunar poles.

Swarm robotics investigates mechanisms for simple robots to work together on complex tasks; similar to biological insects. This thesis develops novel approaches to increase the adaptability of robotic swarms, allowing their operation in challenging environments. The three novel works explore: -behaviour creation, using evolutionary computation to evolve new behaviours; -behaviour adjustment, learning to utilise existing behaviours ; and -behaviour selection, using machine learning and other classification techniques to appropriately select behaviours for a given environment. These works are applied to a wireless communication task, having the swarm transport data between disconnected networking devices when traditional methods are unavailable.

Robotic navigation algorithms for real-world robots require dense and accurate probabilistic volumetric representations of the environment in order to traverse efficiently. Sensor data in a Simultaneous Localisation And Mapping (SLAM)context, however, always has associated acquisition noise and pose uncertainty, and encoding this within the map representation while still maintaining computationaltractability is a key challenge in deploying these systems outside of controlled laboratory settings. The occupancy inference problem is essentially a high dimensional searchin the space of all maps. By incorporating the physics of sensor formation using forward models, it is possible to reason in terms of the likelihood of the measurements for a given map hypothesis to obtain a solution that explainsthe noisy observations as well as possible. However, this approach to mapping has historically been prohibitively expensive to compute in real-time. Thus, conventional robotic mapping algorithms have primarily chosen to work with limiting assumptions to maintain tractability.In this thesis we present a framework that explicitly reasons about the conditional dependence imposed on the occupancy of voxels traversed by each ray of a depth camera as a Markov Random Field (MRF). The tight intra-and inter-ray coupling explicitly incorporates conditional dependence of the occupancy of individual voxels as opposed to making independent log-odds Bayes filterupdates as conventional occupancy maps do. Visibility constraints imposed by using a forward sensor model enables simplification of the otherwise high dimensional inference. The forward model allows incorporating learnt sensor noise characteristics for accurate inference. Instead of marginalising sensor data immediately, data from camera poses is retained, and can be added, moved, or removed in an ad-hoc fashion whilst performing inference. In order to avoid prohibitive sensor data storage costs, an extension to using the framework in a submapping setting with pose graphs is presented with sensor data marginalisationdeferred until as late as possible. Marginalisation is performed using succinct parametric Gaussian distribution representations. Finally, Gaussian mixture model map representations are then demonstrated to be capable ofproviding robust localisation in multi-hypothesis settings. All of this is made real-time feasible by the inherent parallelisability of the proposed framework and is implemented on GPUs.

This thesis develops a framework for automatic construction of complex swarm robot behaviours. It demonstrates the benefit of machine learning in assisting swarm robots in unknown and dynamic environments.

Robots have the tremendous potential of assisting people in their lives, allowing them to achieve goals that they would not be able to achieve by themselves. In particular, sociallyassistive robots provide assistance primarily through social interaction, in healthcare, therapy, and education contexts. Despite their potential, current socially assistive robotsstill lack robust interactive capabilities to allow them to carry out assistive tasks flexibly and autonomously. Some challenges for these robots include responding to and engaging in multi-modal behavior, operating with minimal expert intervention, and accommodating different user needs. Motivated by these challenges, this thesis aims at augmenting the algorithmic capabilities of such robots by leveraging the structure of existing standardized human-human interactions in assistive domains. Using therapy for Autism Spectrum Disorder (ASD) as a domain of focus, we explore two roles for a socially assistive robot: ‘provider’ and‘receiver’. In the provider role, the robot proactively engages in assistive tasks with a human receiver (namely a child with ASD), following standardized interactive tasks. We contribute a family of algorithms for automated action selection, whose goal is to build cost-optimal robot action sequences that account for a range of receiver profiles. We further estimate the action parameters needed to run these algorithms through empirical studies with children with ASD and psychology experts, and show that the algorithms are able to generate personalized action sequences according to different child profiles. In the receiver role, the robot simulates common behavioral responses of children withASD to the standardized actions, acting as an aid for providers in training. By reversing the standardized diagnosis pipeline, we first develop a simulation method that generates behaviors consistent with user-controllable receiver profiles. In a second step, we develop an interactive robot capable of responding to a therapist’s actions in an embodied fashion. Our evaluation studies conducted with therapists validate the designed robot behaviors and show promising results for the integration of such robots in clinical training. These contributions allow for a richer set of interactions with robots in assistive contexts, and are expected to increase their autonomy, flexibility, and effectiveness when dealing with diverse user populations.

Multi-robot systems have demonstrated strong capability in handling environmental operations. In this study, We examine how a team of robots can be utilized in covering and removing spill patches in a dynamic environment by executing three consecutive stages: deployment, coverage, and rendezvous. For the deployment problem, we aim for robot allocation based on the discreteness of the patches that need to be covered. With the deep neural network (DNN) based spill detector and remote sensing facilities such as drones with vision sensors and satellites, we are able to obtain the spill distribution in the workspace. Then, we formulate the allocation problem in a general optimization form and provide solutions using an integer linear programming (ILP) solver under several realistic constraints. After the allocation process is completed and the robot team is divided according to the number of spills, we deploy robots to their computed optimal goal positions. In the robot deployment part, control laws based on artificial potential field (APF) method are proposed and practiced on robots with a common unicycle model. For the coverage control problem, we show two strategies that are tailored for a wirelessly networked robot team. We propose strategies for coverage with and without path planning, depending on the availability of global information. Specifically, in terms of coverage with path planning, we partition the workspace from the aerial image into pieces and let each robot take care of one of the pieces. However, path-planning-based coverage relies on GPS signals or other external positioning systems, which are not applicable for indoor or GPS-denied circumstances. Therefore, we propose an asymptotic boundary shrink control that enables a collective coverage operation with the robot team. Such a strategy does not require a planned path, and because of its distributedness, it shows many advantages, including system scalability, dynamic spill adaptability, and collision avoidance. In case of a large-scale patch that poses challenges to robot connectivity maintenance during the operation, we propose a pivot-robot coverage strategy by mean of an a priori geometric tessellation (GT). In the pivot-robot-based coverage strategy, a team of robots is sent to perform complete coverage to every packing area of GT in sequence. Ultimately, the entire spill in the workspace can be covered and removed. For the rendezvous problem, we investigate the use of graph theory and propose control strategies based on network topology to motivate robots to meet at a designated or the optimal location. The rendezvous control strategies show a strong robustness to some common failures, such as mobility failure and communication failure. To expedite the rendezvous process and enable herding control in a distributed way, we propose a multi-robot multi-point rendezvous control strategy. To verify the validity of the proposed strategies, we carry out simulations in the Robotarium MATLAB platform, which is an open source swarm robotics experiment testbed, and conduct real experiments involving multiple mobile robots.

Self-organization in a multi-robot system is a spontaneous process where some form of overall order arises from local interactions between robots in an initially disordered system. Cooperative coordination strategies for self-organization promote teamwork to complete a task while increasing the total utility of the system. In this dissertation, we apply prosocial behavioral concepts such as altruism and cooperation in multi-robot systems and investigate their effects on overall system performance on given tasks. We stress the significance of this research in long-term applications involving minimal to no human supervision, where self-sustainability of the multi-robot group is of utmost importance for the success of the mission at hand and system re-usability in the future.For part of the research, we take bio-inspiration of cooperation from the huddling behavior of Emperor Penguins in the Antarctic which allows them to share body heat and survive one of the harshest environments on Earth as a group. A cyclic energy sharing concept is proposed for a convoying structured multi-robot group inspired from penguin movement dynamics in a huddle with carefully placed induction coils to facilitate directional energy sharing with neighbors and a position shuffling algorithm, allowing long-term survival of the convoy as a group in the field. Simulation results validate that the cyclic process allows individuals an equal opportunity to be at the center of the group identified as the most energy conserving position, and as a result robot groups were able to travel over 4 times the distance during convoying with the proposed method without any robot failing as opposed to without the shuffling and energy sharing process. An artificial potential based Adaptive Inter-agent Spacing (AIS) control law is also proposed for efficient energy distribution in an unstructured multi-robot group aimed at long-term survivability goals in the field. By design, as an altruistic behavior higher energy bearing robots are dispersed throughout the group based on their individual energy levels to counter skewed initial distributions for faster group energy equilibrium attainment. Inspired by multi-huddle merging and splitting behavior of Emperor Penguins, a clustering and sequential merging based systematic energy equilibrium attainment method is also proposed as a supplement to the AIS controller. The proposed system ensures that high energy bearing agents are not over crowded by low energy bearing agents. The AIS controller proposed for the unstructured energy sharing and distribution process yielded 55%, 42%, 23% and 33% performance improvements in equilibrium attainment convergence time for skewed, bi-modal, normal and random initial agent resource level distributions respectively on a 2D plane using the proposed energy distribution method over the control method of no adaptive spacing. Scalability analysis for both energy sharing concepts confirmed their application with consistently improved performances different sized groups of robots. Applicability of the AIS controller as a generalized resource distribution method under certain constraints is also discussed to establish its significance in various multi-robot applications.A concept of group based survival from damaging directional external stimuli is also adapted from the Emperor Penguin huddling phenomenon where individuals on the damaging stimuli side continuously relocate to the leeward side of the group following the group boundary using Gaussian Processes Machine Learning based global health-loss rate minima estimations in a distributed manner. The method relies on cooperation from all robots where individuals take turns being sheltered by the group from the damaging external stimuli. The distributed global health loss rate minima estimation allowed the development of two settling conditions. The global health loss rate minima settling method yielded 12.6%, 5.3%, 16.7% and 14.2% improvement in average robot health over the control case of no relocation, while an optimized health loss rate minima settling method further improved on the global health loss rate settling method by 3.9%, 1.9%, 1.7% and 0.6% for robot group sizes 26, 35, 70 and 107 respectively.As a direct application case study of collaboration in multi-robot systems, a distributed shape formation strategy is proposed where robots act as beacons to help neighbors settle in a prescribed formation by local signaling. The process is completely distributed in nature and does not require any external control due to the cooperation between robots. Beacon robots looking for a robot to settle as a neighbor and continue the shape formation process, generates a surface gradient throughout the formed shape that allow robots to determine the direction of the structure forming frontier along the dynamically changing structure surface and eventually reach the closest beacon. Simulation experiments validate complex shape formation in 2D and 3D using the proposed method. The importance of group collaboration is emphasized in this case study without which the shape formation process would not be possible, without a centralized control scheme directing individual agents to specific positions in the structure. As the final application case study, a collaborative multi-agent transportation strategy is proposed for unknown objects with irregular shape and uneven weight distribution. Although, the proposed system is robust to single robot object transportation, the proposed methodology of transport is focused on robots regulating their effort while pushing objects from an identified pushing location hoping other robots support the object moment on the other end of the center of mass to prevent unintended rotation and create an efficient path of the object to the goal. The design of the object transportation strategy takes cooperation cues from human behaviors when coordinating pushing of heavy objects from two ends. Collaboration is achieved when pushing agents can regulate their effort with one another to maintain an efficient path for the object towards the set goal. Numerous experiments of pushing simple shapes such as disks and rectangular boxes and complex arbitrary shapes with increasing number of robots validate the significance and effectiveness of the proposed method. Detailed robustness studies of changing weight of objects during transportation portrayed the importance of cooperation in multi-agent systems in countering unintended drift effects of the object and maintain a steady efficient path to the goal. Each case study is presented independent of one another with the Penguin huddling based self-organizations in response to internal and external stimuli focused on fundamental self-organization methods, and the structure formation and object transportation strategies focused on cooperation in specific applications. All case studies are validated by relevant simulation and experiments to establish the effectiveness of altruistic and cooperative behaviors in multi-robot systems.

By using flexible and stretchable materials in place of fixed components, soft robots can materially adapt or change to their environment, providing built-in safeties for robotic operation around humans or fragile, delicate objects. And yet, building a robot out of only soft and flexible materials can be a significant challenge depending on the tasks that the robot needs to perform, for example if the robot were to need to exert higher forces (even temporarily) or self-report its current state (as it deforms unexpectedly around external objects). Thus, the appeal of multifunctional materials for soft robots, wherein the materials used to build the body of the robot also provide actuation, sensing, or even simply electrical connections, all while maintaining the original vision of environmental adaptability or safe interactions. Multifunctional material systems are explored throughout the body of this dissertation in three ways: (1) Sensor integration into high strain actuators for state estimation and closed-loop control. (2) Simplified control of multifunctional material systems by enabling multiple functions through a single input stimulus (i.e., only requiring one source of input power). (3) Presenting a solution for the open challenge of controlling both well established and newly developed thermally-responsive soft robotic materials through an on-body, high strain, uniform, Joule-heating energy source. Notably, these explorations are not isolated from each other as, for example, work towards creating a new material for thermal control also facilitated embedded sensory feedback. The work presented in this dissertation paves a way forward for multifunctional material integration, towards the end-goal of full-functioning soft robots, as well as (more broadly) design methodologies for other safety-forward or adaptability-forward technologies.

There are millions of individuals in the world who currently experience limited mobility as a result of aging, stroke, injuries to the brain or spinal cord, and certain neurological diseases. Robotic Assistive Devices (RADs) have shown superiority in helping people with limited mobility by providing physical movement assistance. However, RADs currently existing on the market for people with limited mobility are still far from intelligent.Learning control strategies are developed in this study to make a Cable-Driven Assistive Device (CDAD) intelligent in assisting a human joint (e.g., a knee joint, an ankle joint, or a wrist joint). CDADs are a type of RADs designed based on Cable-Driven Parallel Robots (CDPRs). A PID–FNN control strategy and DDPG-based strategies are proposed to allow a CDAD to learn physical human-robot interactions when controlling the pose of the human joint. Both pose-tracking and trajectory-tracking tasks are designed to evaluate the PID–FNN control strategy and the DDPG-based strategies through simulations. Simulations are conducted in the Gazebo simulator using an example CDAD with three degrees of freedom and four cables. Simulation results show that the proposed PID–FNN control strategy and DDPG-based strategies work in controlling a CDAD with proper learning.

Multi-agent systems are a collection of agents working in shared environments, often with shared goals while being required to adhere to limited resources. These systems have universal applications and are often deemed as the future of automation in industry; however, an open issue within these systems is ensuring a degree of trustworthiness, allowing human counterparts to be confident that these systems and their individual agents will adhere to expected behaviours even when issues occur. The need for “Safety”, which is often defined in the literature in a post hoc fashion, in these systems can be seen at its most crucial within sensitive operations such as within military application and search and rescue operations. The current state of safety in agents, learning or otherwise, shows much promise with the use of quantitative analysis methods, to deliver a statistical foundation of how likely safety standards will be adhered to. In Multi-agent systems, a large area of literature is dedicated to Petri-net modelling, and using these models to constrict agent behaviour, however, Petri-nets require expertise to design, and analysis of these tools for safety remains an open question. This project aims to look further into the use of the Petri-net tool in modelling multi-agent systems to constrict “unsafe” behaviour while they learn to relatively optimise their behaviours and after this learning has concluded. The project aims to do this by increasing the accessibility of Petri-nets when modelling robot teams, and also further investigate ways to analysis these Petri-net models to deliver a high quality of trustworthiness.

While robotics has made tremendous progress over the last few decades, most success stories are still limited to carefully engineered and precisely modeled environments. Getting these robots to work in the complex and diverse world that we live in has proven to be a difficult challenge. Interestingly, one of the most significant successes in the last decade of AI has been the use of Machine Learning (ML) to generalize about and robustly handle diverse situations. From detecting objects in cluttered visual scenes to translating sentences between languages, ML has become an integral part of modern technology. So why don’twe just apply current learning algorithms to robots? Arguably, one of the biggest reason is the lack of large-scale data. A key ingredient that sparked progress in Computer Vision and Natural Language Processing was internet-scale data: 1-10 million labelled images for computer vision; 100 million word-pairs for language translation. Unfortunately, this scale of data is not available for robots. Hence, effectively integrating learning to robots requiresus to rethink our use of robotic data. In this thesis we draw inspiration from other data driven fields of AI (Computer Vision, Natural Language Processing, etc.) to develop data-centric robot learning techniques. However, unlike the aforementioned fields, robotics involves interactions with real hardware systems, which presents three unique challenges for large-scale learning. The first challengeis the physical nature of robotics where every piece of data needs to be executed on a real system. To scalably collect large amounts of data with minimal human supervision, wepresent ‘self-supervised’ robot learning techniques where the robot both collects and labels real-world data. From manipulating unseen objects in new homes to avoiding previously unseen obstacles while flying, we demonstrate that large-scale, self-supervised data, when combined with off-the-shelf ML tools, can produce generalizable robotic skills. The second challenge is that real robots are slow, which limits the amount of data we can collect even with ‘self-supervised’ techniques. This limitation necessitates the development of algorithms that can efficiently utilize available robot data. We do this by instilling notionsof robustness via adversaries and sharing representations across multiple tasks. Finally, in several applications, robotic data may not be easy or practical to collect. Insuch scenarios, we can turn to data from simulated models of the real world. Towards this, we present algorithms to learn generalizable skills in a simulator that can transfer to the real world.

Unmanned Aerial Vehicles (UAVs) are quickly becoming ubiquitous, largely driven by the commercialisation of consumer based UAV models. Many of these UAVs will adhere to local aviation laws, for example, observing predefined flight corridors, restricted airspace, operating away from populated areas and so on. However, it is anticipated a reasonable number of users will inevitably not obey the law. It is not known what regulations will be imposed on UAV usage in the future, or how the general population will seek to use them, but it is anticipated the need for systems that will be able to search for these UAVs, or specifically those not adhering to the local aviation laws. This research proposes a framework to address this perceived future need.

Path planning is the generation of paths for the robots to navigate based on some constraints. Coverage path planning is where the robots needs to cover an entire work space for various applications like sensing, inspection and so on. Though there are numerous works on 2D coverage and also coverage using a single robot, the works on 3D coverage and multi-agents are very limited. This thesis makes several contributions to multi-agent path planning for 3D structures.Motivated by the inspection of 3D structures, especially airplanes, we present a 3D coverage path planning algorithm for a multi-UAV system. We propose a unified method, where the viewpoints selection and path generation are done simultaneously for multiple UAVs. The approach is scalable in terms of number of UAVs and is also robust to models with variations in geometry. The proposed method also distributes the task uniformly amongst the multiple UAVs involved and hence making the best use of the robotics team. The uniform task distribution is an integral part of the path planner. Various performance measures of the paths generated in terms of coverage, path length and time also has been presented.

The thesis introduces an innovative UAV detection system. The commercial UAV market is booming. Meanwhile, the risks and threats from improper UAV usages are also booming. CUAS is to protect the public and facilities. The problem is a lack of an intelligent platform which can adapt many sensors for UAV detection. The hypothesis is that, the system can track the UAV’s movement by applying the multi-agent system (MAS) to UAV route track. The experiment proves that the multi-agent system benefits for the UAV track.

There is a famous tale in computer vision: Once, a graduate student asked the famous computer vision scientist Takeo Kanade: “What are the three most important problemsin computer vision?” Takeo replied: “Correspondence, correspondence, correspondence!” Indeed, even for the most commonly applied Convolutional Neural Networks (ConvNets), they are internally learning representations that lead to correspondence across objects or object parts. The way these networks learn is via human annotations on millions of static images. For example, humans will label images as dog, car, etc. However, this is not howwe humans learn. The visual system of an infant develops in a dynamic and continuous environment without using semantics until much later in life. In this thesis, I will argue that we need to go beyond images and exploit the massiveamount of correspondence in videos. In videos, we have millions of pixels linked to each other by time. I will discuss how to learn correspondence from continuous observations in videos without any human supervision. Once the correspondence is given, it can be utilized as supervision in training the ConvNets, eliminating the need for manual labels. Besides supervision, capturing long-range correspondence is also the key to video understanding aswell as interaction reasoning. The effectiveness of these ideas will be demonstrated on tasks including object recognition, tracking, action recognition, affordance and physical property estimation.

Methods to assess individual facial actions have potential to shed light on important behavioral phenomena ranging from emotion and social interaction to psychological disorders and health. However, manual coding of such actions is labor intensive and requires extensive training. To date, establishing reliable automated coding of unscripted facial actions has been a daunting challenge impeding development of psychological theories and applications requiring facial expression assessment. It is therefore essential that automated coding systems be developed with enough precision and robustness to ease the burden of manual coding in challenging data involving variation in participant gender, ethnicity, head pose, speech, and occlusion. We report a major advance in automated coding of spontaneous facial actions during an unscripted social interaction involving three strangers. For each participant (n = 80, 47 % women, 15 % Nonwhite), 25 facial action units (AUs) were manually coded from video using the Facial Action Coding System. Twelve AUs occurred more than 3 % of the time and were processed using automated FACS coding. Automated coding showed very strong reliability for the proportion of time that each AU occurred (mean intraclass correlation = 0.89), and the more stringent criterion of frame-by-frame reliability was moderate to strong (mean Matthew’s correlation = 0.61). With few exceptions, differences in AU detection related to gender, ethnicity, pose, and average pixel intensity were small. Fewer than 6 % of frames could be coded manually but not automatically. These findings suggest automated FACS coding has progressed sufficiently to be applied to observational research in emotion and related areas of study.

We describe a real-time pedestrian detection system intended for use in automotive applications. Our system demonstrates superior detection performance when compared to many state-of-the-art detectors and is able to run at a speed of 14 fps on an Intel Core i7 computer when applied to 640×480 images. Our approach uses an analysis of geometric constraints to efficiently search feature pyramids and increases detection accuracy by using a multiresolution representation of a pedestrian model to detect small pixel-sized pedestrians normally missed by a single representation approach. We have evaluated our system on the Caltech Pedestrian benchmark which is currently the largest publicly available pedestrian dataset at the time of this publication. Our system shows a detection rate of 61% with 1 false positive per image (FPPI) whereas recent other state-of-the-art detectors show a detection rate of 50% ~ 61% under the `reasonable' test scenario (explained later). Furthermore, we also demonstrate the practicality of our system by conducting a series of use case experiments on selected videos of Caltech dataset.