Search

In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation.

Type 1 Diabetes is characterized by the lack of insulin-producing beta cells in the pancreas. The artificial pancreas promises to alleviate the burdens of self-management. While the physical components of the system – the continuous glucose monitor and insulin pump – have experienced rapid advances, a technological bottleneck remains in the control algorithm, which is responsible for translating data from the former into instructions for the latter. In this work, we propose to bring machine learning techniques to bear upon the challenges of blood glucose control. Specifically, we employ reinforcement learning to learn an optimal insulin policy. Learning is generalized using nonparametric regression with functional features, exploiting information contained in the shape of the glucose curve. Our algorithm is model-free, data-driven and personalized. In-silico simulations with T1D models demonstrate the potential of the proposed algorithm.