Automatic inverse treatment planning of Gamma Knife radiosurgery via deep reinforcement learning

Several inverse planning algorithms have been developed for Gamma Knife (GK) radiosurgery to determine a large number of plan parameters by solving an optimization problem, which typically consists of multiple objectives. The priorities among these objectives need to be repetitively adjusted to achieve a clinically good plan for each patient. This study aimed to achieve automatic and intelligent priority tuning by developing a deep reinforcement learning (DRL)-based method to model the tuning behaviors of human planners.We built a priority-tuning policy network using deep convolutional neural networks. Its input was a vector composed of multiple plan metrics that were used in our institution for GK plan evaluation. The network can determine which tuning action to take based on the observed quality of the intermediate plan. We trained the network using an end-to-end DRL framework to approximate the optimal action-value function. A scoring function was designed to measure the plan quality to calculate the received reward of a tuning action.Vestibular schwannoma was chosen as the test bed in this study. The number of training, validation and testing cases were 5, 5, and 16, respectively. For these three datasets, the average scores of the initial plans obtained with the same initial priority set were 3.63 ± 1.34, 3.83 ± 0.86 and 4.20 ± 0.78, respectively, while they were improved to 5.28 ± 0.23, 4.97 ± 0.44 and 5.22 ± 0.26 through manual priority tuning by human expert planners. Our network achieved competitive results with 5.42 ± 0.11, 5.10 ± 0. 42, 5.28 ± 0.20, respectively.Our network can generate GK plans of comparable or slightly higher quality than the plans generated by human planners via manual priority tuning for vestibular schwannoma cases. The network can potentially be incorporated into the clinical workflow as planning assistance to improve GK planning efficiency and help to reduce plan quality variation caused by interplanner variability. We also hope that our method can reduce the workload of GK planners and allow them to spend more time on more challenging cases.