Search

In this paper, a model-free deep reinforcement learning (DRL) strategy is presented with an artificial neural network (ANN) as reaction simulation environment, to obtain a fed-batch control strategy for an experimental bioreactor. The proposed method is a fundamental attempt to control reactions by employing state-of-the-art machine learning tools without the aid of well-established mechanistic understanding of the reaction system. This application utilizes the Asynchronous Advantage Actor-Critic (A3C) algorithm, a member of the DRL family, that takes advantage of actor-critic algorithm and asynchronous learning by parallel learning agents to achieve stability and efficiency of the learning process. The resulting controller demonstrates robust performance in the fed-batch bioreactor since it can be adjusted to meet varying constraining factors including nutrient limitations and culture lengths. Results are presented for a bioreactor that produces cyanobacterial-phycocyanin (C-PC) in Plectonema sp. UTEX 1541. Experimental validations show a 52.1% increase in the product yield, and a 20.1% increase in C-PC concentration compared to a control group with the same total nutrient input replenished in a non-optimized manner. [ABSTRACT FROM AUTHOR]

• Control framework based on deep-Q-network is developed. • Monte-Carlo sampling method is integrated to generate simulation datasets. • The proposed model is applied to the extraction column in downstream process. • Interface based on component object model is suggested to integrate algorithms. This paper proposes a systematic framework to develop deep reinforcement learning (RL)-based algorithms for control system of downstream separation in biopharmaceutical process as follows. First, a simulation model as a digital twin is built and Monte-Carlo sampling generates substantial amounts of samples considering disturbances. Second, the deep RL-based control system is designed and the optimization subject to sample datasets is conducted. The methodology is implemented in a prototype software and relevant codes are shared by Mendeley Data. The proposed model is successfully applied to control the liquid-liquid extraction column for the recovery of fusidic acid as part of downstream processing. The resulting deep RL algorithm provides an operation performance with a better API recovery yield (32 % higher than open loop operation) and lower deviations (23 % lower than open loop operation) against disturbances. Image, graphical abstract [ABSTRACT FROM AUTHOR]