Autonomous Systems, Trust, and Guarantees

Autonomy of systems has been gaining increasing attention from scientists and engineers in various disciplines such as robotics, artificial intelligence, embedded systems, and control. However, there are serious challenges on the way of designing and adopting such systems, which haven’t, in our perspective, received sufficient attention yet. In this position paper, we review some of those challenges and present potential approaches of coping with them. Moreover, we highlight topics that require immediate attention and research efforts of the community. In particular, we propose three approaches: (i) More dependable autonomy using deterministic methods that can provide similar controllability, dependability, and traceability as that of traditional systems. Among such solutions is providing guarantee bands as opposed to guaranteed narrow operation points. (ii) Orthogonal autonomy adds autonomy in the extra-functional plane of the system. (iii) Expressive system design allows autonomous systems to trace and explain their decisions and reactions even though they may be using less deterministic tools.