Search

Partially Observable Markov Decision Processes (POMDPs) are powerful models for sequential decision making under transition and observation uncertainties. This paper studies the challenging yet important problem in POMDPs known as the (indefinite-horizon) Maximal Reachability Probability Problem (MRPP), where the goal is to maximize the probability of reaching some target states. This is also a core problem in model checking with logical specifications and is naturally undiscounted (discount factor is one). Inspired by the success of point-based methods developed for discounted problems, we study their extensions to MRPP. Specifically, we focus on trial-based heuristic search value iteration techniques and present a novel algorithm that leverages the strengths of these techniques for efficient exploration of the belief space (informed search via value bounds) while addressing their drawbacks in handling loops for indefinite-horizon problems. The algorithm produces policies with two-sided bounds on optimal reachability probabilities. We prove convergence to an optimal policy from below under certain conditions. Experimental evaluations on a suite of benchmarks show that our algorithm outperforms existing methods in almost all cases in both probability guarantees and computation time., Comment: Accepted to the Conference on Uncertainty in Artificial Intelligence (UAI) 2024

Many sequential decision problems involve optimizing one objective function while imposing constraints on other objectives. Constrained Partially Observable Markov Decision Processes (C-POMDP) model this case with transition uncertainty and partial observability. In this work, we first show that C-POMDPs violate the optimal substructure property over successive decision steps and thus may exhibit behaviors that are undesirable for some (e.g., safety critical) applications. Additionally, online re-planning in C-POMDPs is often ineffective due to the inconsistency resulting from this violation. To address these drawbacks, we introduce the Recursively-Constrained POMDP (RC-POMDP), which imposes additional history-dependent cost constraints on the C-POMDP. We show that, unlike C-POMDPs, RC-POMDPs always have deterministic optimal policies and that optimal policies obey Bellman's principle of optimality. We also present a point-based dynamic programming algorithm for RC-POMDPs. Evaluations on benchmark problems demonstrate the efficacy of our algorithm and show that policies for RC-POMDPs produce more desirable behaviors than policies for C-POMDPs., Comment: Accepted to the Conference on Uncertainty in Artificial Intelligence (UAI) 2024

Robotic space missions have long depended on automation, defined in the 2015 NASA Technology Roadmaps as "the automatically-controlled operation of an apparatus, process, or system using a pre-planned set of instructions (e.g., a command sequence)," to react to events when a rapid response is required. Autonomy, defined there as "the capacity of a system to achieve goals while operating independently from external control," is required when a wide variation in circumstances precludes responses being pre-planned, instead autonomy follows an on-board deliberative process to determine the situation, decide the response, and manage its execution. Autonomy is increasingly called for to support adventurous space mission concepts, as an enabling capability or as a significant enhancer of the science value that those missions can return. But if autonomy is to be allowed to control these missions' expensive assets, all parties in the lifetime of a mission, from proposers through ground control, must have high confidence that autonomy will perform as intended to keep the asset safe to (if possible) accomplish the mission objectives. The role of mission assurance is a key contributor to providing this confidence, yet assurance practices honed over decades of spaceflight have relatively little experience with autonomy. To remedy this situation, researchers in JPL's software assurance group have been involved in the development of techniques specific to the assurance of autonomy. This paper summarizes over two decades of this research, and offers a vision of where further work is needed to address open issues., Comment: 9 pages, 0 figures. To be published in The 2nd International Conference on Assured Autonomy

Technology Readiness Levels are a mainstay for organizations that fund, develop, test, acquire, or use technologies. Technology Readiness Levels provide a standardized assessment of a technology's maturity and enable consistent comparison among technologies. They inform decisions throughout a technology's development life cycle, from concept, through development, to use. A variety of alternative Readiness Levels have been developed, including Algorithm Readiness Levels, Manufacturing Readiness Levels, Human Readiness Levels, Commercialization Readiness Levels, Machine Learning Readiness Levels, and Technology Commitment Levels. However, while Technology Readiness Levels have been increasingly applied to emerging disciplines, there are unique challenges to assessing the rapidly developing capabilities of autonomy. This paper adopts the moniker of Space Trusted Autonomy Readiness Levels to identify a two-dimensional scale of readiness and trust appropriate for the special challenges of assessing autonomy technologies that seek space use. It draws inspiration from other readiness levels' definitions, and from the rich field of trust and trustworthiness. The Space Trusted Autonomy Readiness Levels were developed by a collaborative Space Trusted Autonomy subgroup, which was created from The Space Science and Technology Partnership Forum between the United States Space Force, the National Aeronautics and Space Administration, and the National Reconnaissance Office.

A close partnership between people and partially autonomous machines has enabled decades of space exploration. But to further expand our horizons, our systems must become more capable. Increasing the nature and degree of autonomy - allowing our systems to make and act on their own decisions as directed by mission teams - enables new science capabilities and enhances science return. The 2011 Planetary Science Decadal Survey (PSDS) and on-going pre-Decadal mission studies have identified increased autonomy as a core technology required for future missions. However, even as scientific discovery has necessitated the development of autonomous systems and past flight demonstrations have been successful, institutional barriers have limited its maturation and infusion on existing planetary missions. Consequently, the authors and endorsers of this paper recommend that new programmatic pathways be developed to infuse autonomy, infrastructure for support autonomous systems be invested in, new practices be adopted, and the cost-saving value of autonomy for operations be studied., Comment: 10 pages (compared to 8 submitted to PSADS), 2 figures, submitted to National Academy of Sciences Planetary Science and Astrobiology Decadal Survey 2023-2032

Artificial intelligence (AI), which encompasses machine learning (ML), has become a critical technology due to its well-established success in a wide array of applications. However, the proper application of AI remains a central topic of discussion in many safety-critical fields. This has limited its success in autonomous systems due to the difficulty of ensuring AI algorithms will perform as desired and that users will understand and trust how they operate. In response, there is growing demand for trustability in AI to address both the expectations and concerns regarding its use. The Aerospace Corporation (Aerospace) developed a Framework for Trusted AI (henceforth referred to as the framework) to encourage best practices for the implementation, assessment, and control of AI-based applications. It is generally applicable, being based on terms and definitions that cut across AI domains, and thus is a starting point for practitioners to tailor to their particular application. To help demonstrate how the framework can be tailored into mission assurance guidance for the space domain, Aerospace sought the involvement of the Jet Propulsion Laboratory (JPL) to engage with actual examples of AI-based space autonomy.

As artificial intelligence (AI) is increasingly pro- posed for new and future capabilities in space missions, the question of how to trust AI-enabled space autonomy has been explored. Recently, a collaboration between The Aerospace Corporation (Aerospace) and NASA’s Jet Propulsion Labora- tory (JPL) investigated how Aerospace’s Trusted AI Frame- work could be applied to two JPL projects that planned on lev- eraging AI for critical autonomous tasks. This combined effort led to many insights in the practical implementation of trusted AI along with considerable updates to the Trusted AI Frame- work that tailored its topic threads to space exploration. This document cohesively summarizes the enhanced framework as tailored to space missions as well as estimation of the level of trust required as a function of mission criticality and key stakeholders. The goal of this work is to provide a set of best practices to inform autonomy researchers, flight engineers, mission and proposal reviewers, and instrument and mission principal investigators (PI’s) to drive AI-based autonomy that maximizes trust and lowers the barriers to mission adoption for both science and engineering applications.

The cost of quality vs cost of failure correction has been a long-running topic of discussion within the Aerospace community. It leads directly to concepts of “risk tolerance”, and risk-based decision-making. It would be valuable if there was a way to compute the optimal investment in customer-executed quality assurance activities using defect significance with respect to performance objectives, the activities’ defect detection effectiveness, and the cost-penalty for late discovery of impactful defects. This optimization is particularly of interest to projects whose budget constraints significantly limit their risk management options.The cost to fix defects (i.e., failure correction) escalates as the project matures. There have been studies attempting to determine the relative cost of fixing defects discovered during various phases of a project life cycle with important implications, all of which suggest growth factors are large. The commonly referred to 1:10:100 rule represents a cost multiplier for repair/rework across the Design to Fab to Test hardware development phases. Cost premiums for QA activities also accumulate when they are treated as mandatory (due to schedule drag) or are performed later than their assigned phase.This paper describes the modeling of development phase -dependencies in the conduct of typical customer-executed quality assurance activities. Our initial modeling encompasses:• Distinct phases of the production lifecycle• Multiple kinds of Defects, each with some a-priori likelihood of being present• Each defect’s impact on performance Objectives for a type of hardware• The cost and efficacy of assurance techniques at detecting such Defects• The costs of fixing those Defects detected in a given phase of the production lifecycleThe model captures assurance activities’ abilities to Detect defects. Upon detection it is assumed that the Defect is immediately fixed. Defects that “escape” detection by some activity may thereafter be detected by a later activity, but by then the cost of fixing the Defect may have escalated. Defects are related to the performance Objectives they would detract from, were those Defects to remain present in the operating system.We have constructed and are exploring, a model that relates the importance of hardware system elements to mission objectives, the impact of types of Defects on those hardware types, the cost of customer-executed assurance activities (i.e., supplier controls) and their effectiveness towards reducing an impactful quality escape, and the cost of Defect correction across production phase. We describe the approach taken to select the key model aspects, why they are relevant to our NASA mission, and our efforts to populate it with relevant and contemporary data. We use a notional example to illustrate model design and function.

The purpose of this workshop was to address the following questions: What are the impediments to making more use of autonomy on robotic space missions? Which of those impediments apply to which kinds of space missions? What is/can/could be done to overcome those impediments? Highlights of the workshop’s treatment of these topics are given herein.

The purpose of this workshop was to address the following questions: What are the impediments to making more use of autonomy on robotic space missions? Which of those impediments apply to which kinds of space missions? What is/can/could be done to overcome those impediments? Highlights of the workshop’s treatment of these topics are given herein.

Spacecraft, operating with a time-to-effect faster than human command response, or with absent or delayed communication, have depended on autonomy. It is likely that this reliance will continue, and increase further in future missions. Spacecraft autonomy is used when operator intervention is not available or feasible, or when dynamic aspects of a spacecraft situation cannot be predicted in advance. This paper provides case studies of several successfully verified autonomous software systems across multiple spacecraft, flight phases (launch, transit, entry and landing, science utilization), and software classifications, and can serve as examples to future missions in methods of successfully assuring spacecraft autonomy.

Over the past two years, JPL has used the ASTERIA (Arcsecond Space Telescope Enabling Research In Astrophysics) CubeSat as an in-flight test platform during extended missions. ASTERIA successfully completed its prime mission in early 2018, and continued to operate in low Earth orbit (LEO) for an additional twenty months. This paper describes demonstrations that were performed on the spacecraft and on the ground-based testbed during the extended mission. These demonstrations fall into three categories: Autonomy technology maturation, hardware characterization, and science discovery. Autonomy technology maturation supported three development efforts. The first shifted the spacecraft commanding paradigm from time-based sequences to Task Networks (tasknets), which allow simpler commanding and more robust onboard execution. The second demonstrated onboard orbit determination in Low Earth Orbit (LEO) without GPS. This activity used a fully-independent means of spacecraft orbit determination for Earth orbiters using only passive imaging. The third technology provided in situ hardware health state estimation using a model-based reasoning technique. These three technologies were demonstrated either in flight or on the testbed individually, and then were combined to demonstrate the capability to perform autonomous navigation on board without ground intervention, even in the presence of anomalies. Hardware characterization involved both onboard and ground-based activities. On board, nonstandard attitude control modes were commanded to characterize the spacecraft pointing jitter as a function of target brightness, reaction wheel speed, controller gain, and the number of guide stars. The results provide insights into the contribution of jitter to the ASTERIA photometry and inform the feasibility of future astrophysics small satellite missions for which jitter control is an enabling technology. On the ground, the ASTERIA Operations Team coordinated with Amazon Web Services (AWS) to configure their new ground stations to communicate with ASTERIA to prove out their viability. ASTERIA used AWS ground stations for nominal operations for the last four months of the mission. Finally, ASTERIA continued to perform exoplanet science as the spacecraft was well-suited to execute long-term monitoring of stars such as alpha Centauri to search for small transiting planets. The science team also imaged a number of interesting objects including a comet, an asteroid, cities at night, and the moon, and coordinated with other projects on Targets of Opportunity for follow-up confirmations and co-observations. Throughout the prime and the extended missions, the ASTERIA spacecraft proved to be a mighty platform that “will go into history as an innovative milestone.”[1 - Zurbuchen]

System-level autonomy refers to autonomously meeting the crosscutting needs of a system through awareness and coordinated control spanning the system's breadth of capabilities. In contrast to function-level autonomy, which focuses on capabilities required to achieve a specific function such as surface navigation or image recognition, system-level autonomy addresses the needs to coordinate and manage activities and resources, and estimate the state, across subsystems. This paper describes demonstrations that were conducted on a spacecraft workstation testbed. The autonomy was provided by system-level planning and execution integrated with system-level estimators of orbit knowledge and spacecraft hardware health. These components are embedded in a system-level framework defining how goals are formed and executed, which elements exist, and how control authority is distributed among components. The planning and execution system at the heart of the framework has the capability to schedule, execute and monitor completion of tasks, as well as plan around unexpected events including new science opportunities and anomalies. The planning and scheduling system is the Multi-mission EXECutive (MEXEC), supported by the system-level health state estimator Model-Based Off-Nominal State Identification and Detection (MONSID), and Autonomous Navigation (AutoNav) algorithms, which determine the orbital system state based on optical observation of other targets. These components are applicable to many kinds of missions on different platforms. These demonstrations were elaborations of earlier experiments conducted on the ASTERIA (Arcsecond Space Telescope Enabling Research In Astrophysics) CubeSat, described in a companion submission [1]. The spacecraft’s extended mission served as an in-flight test platform, during which some individual autonomous capabilities were flown successfully. The autonomy experiments described here were performed on the ASTERIA workstation testbed.

We describe our pilot study development of an Assurance Case arguing the robustness of a spacecraft demonstration of optical communication. Our Assurance Case addresses the concern that optical communication may be interrupted by the presence of cloud cover, threatening the success of the demonstration. Central to our Assurance Case is its use of a model of atmospheric attenuation to support a key portion of the robustness argument. The conclusion for the demonstration is that its schedule slack plus ability to store data for transmission later accommodates occasional weather-caused atmospheric attenuation.We indicate how the overall structure of the Assurance Case derives from the Objectives Hierarchy set forth in the NASA Reliability and Maintainability standard. We then present the portions of the Assurance Case that argue (1) the atmospheric attenuation model is sufficiently accurate, (2) application of the model shows the desired robustness of the demonstration, and (3) all the model assumptions are satisfied in its application. Lastly, we suggest how an engineering model of the demonstration system and its use could inform the development of the Assurance Case encompassing additional plausible hazards.

The traditional form of spacecraft commanding is with sequences that specify when commands should execute based on a schedule generated on the ground. Some sequences have control logic and event driven responses to increase flexibility, but it is limited. An approach to increase autonomy is to use goal-based planning and commanding. Using this paradigm, intention and behavior is modeled on board the spacecraft. In this paper we describe MEXEC (Multi-mission EXECutive), a multi-mission, task-based, onboard planning and execution software designed specifically to be used as flight software. As a path to infusion for future flight projects, we describe two experiments performed on the ASTERIA CubeSat and testbed that demonstrate that MEXEC can be integrated and used for spacecraft operations and increase robustness and science return compared to the standard sequences that were being used.

Over the past decade or so, the emergence of Model Based Systems Engineering (MBSE) has demonstrated its desirability and value in terms of 1) being a single source of truth, 2) unambiguous definitions and relationships, and 3) after representation, the ability to explore/extract any sets of data on demand. While much work has been done in showing the value to the system engineering discipline in these areas, how does that value translate to the Safety and Mission Assurance (S&MA) world? This paper provides a vision of a very desirable future of NASA S&MA after it is fully integrated into the MBSE framework. We explore the impact and consequences of the MBSE Value items discussed above and how they impact the disciplines of quality assurance, reliability and maintainability, system safety, and software assurance. We provide insight into how the MBSE modeling tools can be used to define S&MA processes (ideally as a result of Use Case [1] elaboration of processes represented in MagicDraw®), produce S&MA products (ViewEditor output of various items), and represent S&MA disciplines (S&MA inside of MagicDraw). We also provide insight into the degree to which some elements can be directly integrated into a SysML® model and when, as often happens, an interface to some external source must be provided. The desirability of this future is part of the reason for the NASA Office of Safety and Mission Assurance’s (OSMA) recent creation of a Model Based Mission Assurance (MBMA) Program [2] and the MBMA annual workshops. We briefly summarize the efforts to date to generate S&MA Use Cases for eventual deployment into pilot and project efforts. Even simple use of the SysML modeling tools can be used to capture quality assurance tasks and integrate them with the systems engineering and produce products that are easy to use by quality practitioners that are unfamiliar with these methods. We anticipate finding opportunities to pilot and implement various Quality Assurance (QA) Use Cases in FY20. The MBMA Program is focused on implementation; the NASA Office of the Chief Engineer's Community of Practice, as well as the SmallSat communities, are very interested in the integration of S&MA. Finally, as projects move forward utilizing whatever efficiency increases they can find in a cost-constrained environment, the S&MA community cannot be caught unawares and needs to continue preparing for the ever-growing implementation of MBSE across NASA and our government and commercial partners.

The work described addresses assurance of a planning and execution software system being added to an in-orbit CubeSat to demonstrate autonomous control of that spacecraft. Our focus was on how to develop assurance of the correct operation of the added software in its operational context, our approach to which was to use an assurance case to guide and organize the information involved.

A number of specific benefits that fit within the hallmarks of effective development are realized with implementation of model-based approaches to systems and assurance. Model Based Systems Engineering (MBSE) enabled by standardized modeling languages (e.g., SysML®) is at the core. These benefits in the context of spaceflight system challenges can include: Improved management of complex development, Reduced risk in the development process, Improved cost management, Improved design decisions. With appropriate modeling techniques the assurance community can improve early oversight and insight into project development. NASA has shown the basic constructs of SysML in an MBSE environment offer several key advantages, within a Model Based Mission Assurance (MBMA) initiative.

While there have been meetings on assurance of autonomy in other application areas, this meeting seeks to develop a roadmap for assurance of autonomy specifically for robotic space missions. Key characteristics that distinguish this application area include: the lack of detailed prior knowledge of the environments in which those missions are to operate; the challenges of mimicking deep space operating conditions for purposes of testing; the need to be highly assured of failsafe operation; limited and delayed (due to speed of light over solar system distances) communication; and the one/few-of-a-kind, must-work-the-first-time nature of most space missions.

UNKNOWN