Search

The focus of this dissertation work is the formulation and improvement of anemia management process involving trial-and-error. A two-stage method is adopted toward this objective. Given a medical treatment process, a discrete Markov representation is first derived as a formal translation of the treatment process to a control problem under uncertainty. A simulative numerical solution of the control problem is then obtained on-the-fly in the form of a control law maximizing the long-term benefit at each decision stage. Approximate dynamic programming methods are employed in the proposed solution. The motivation underlying this choice is that, in reality, some patient characteristics, which are critical for the sake of treatment, cannot be determined through diagnosis and remain unknown until early stages of treatment, when the patient demonstrates them upon actions by the decision maker. A review of these simulative control tools, which are studied extensively in reinforcement learning theory, is presented. Two approximate dynamic programming tools, namely SARSA and Q -learning, are introduced. Their performance in discovering the optimal individualized drug dosing policy is illustrated on hypothetical patients made up as fuzzy models for simulations. As an addition to these generic reinforcement learning methods, a state abstraction scheme for the considered application domain is also proposed. The control methods of this study, capturing the essentials of a drug delivery problem, constitutes a novel computational framework for model-free medical treatment. Experimental evaluation of the dosing strategies produced by the proposed methods against the standard policy, which is being followed actually by human experts in Kidney Diseases Program, University of Louisville, shows the advantages for use of reinforcement learning in the drug dosing problem in particular and in medical decision making in general.