Reinforcement Learning as a Path to Autonomous Intelligent Cyber-Defense Agents in Vehicle Platforms

Technological advancement of vehicle platforms exposes opportunities for new attack paths and vulnerabilities. Static cyber defenses can help mitigate certain attacks, but those attacks must generally be known ahead of time, and the cyber defenses must be hand-crafted by experts. This research explores reinforcement learning (RL) as a path to achieve autonomous, intelligent cyber defense of vehicle control networks—namely, the controller area network (CAN) bus. We train an RL agent for the CAN bus using Toyota’s Portable Automotive Security Testbed with Adaptability (PASTA). We then apply the U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory’s methodology for quantitative measurement of cyber resilience to assess the agent’s effect on the vehicle testbed in a contested cyberspace environment. Despite all defenses having similar traditional performance measures, our RL agent averaged a 90% cyber resilience measurement during drive cycles executed on hardware versus 41% for a naïve static timing defense and 98% for the bespoke timing-based defense. Our results also show that an RL-based agent can detect and block injection attacks on a vehicle CAN bus in a laboratory environment with greater cyber resilience than prior learning approaches (1% for convolutional networks and 0% for recurrent networks). With further research, we believe there is potential for using RL in the autonomous intelligent cyber defense agent concept.