Search

Conventional reinforcement learning (RL) typically determines an appropriate primitive action at each timestep. However, by using a proper macro action, defined as a sequence of primitive actions, an RL agent is able to bypass intermediate states to a farther state and facilitate its learning procedure. The problem we would like to investigate is what associated beneficial properties that macro actions may possess. In this article, we unveil the properties of reusability and transferability of macro actions. The first property, reusability , means that a macro action derived along with one RL method can be reused by another RL method for training, while the second one, transferability , indicates that a macro action can be utilized for training agents in similar environments with different reward settings. In our experiments, we first derive macro actions along with RL methods. We then provide a set of analyses to reveal the properties of reusability and transferability of the derived macro actions.

Collecting training data from the physical world is usually time-consuming and even dangerous for fragile robots, and thus, recent advances in robot learning advocate the use of simulators as the training platform. Unfortunately, the reality gap between synthetic and real visual data prohibits direct migration of the models trained in virtual worlds to the real world. This paper proposes a modular architecture for tackling the virtual-to-real problem. The proposed architecture separates the learning model into a perception module and a control policy module, and uses semantic image segmentation as the meta representation for relating these two modules. The perception module translates the perceived RGB image to semantic image segmentation. The control policy module is implemented as a deep reinforcement learning agent, which performs actions based on the translated image segmentation. Our architecture is evaluated in an obstacle avoidance task and a target following task. Experimental results show that our architecture significantly outperforms all of the baseline methods in both virtual and real environments, and demonstrates a faster learning curve than them. We also present a detailed analysis for a variety of variant configurations, and validate the transferability of our modular architecture., 7 pages, accepted by IJCAI-18

Techniques of a dual-path error amplifier and two capacitor multipliers for providing on-chip frequency compensation and soft-start function are proposed in this paper. The concept of the dual-path error amplifier is to use two currents to charge and discharge a compensation capacitor simultaneously. As a result, the equivalent capacitance is enlarged significantly with little additional power and silicon area. The dc-dc converter with the dual-path architecture also has great performance in transient response because the compensation capacitor is reduced significantly. For the soft-start function, the subtractive-type and time-average capacitor multipliers are used to relax the restriction of the capacitance and the charging current. Consequently, it is easy to integrate the soft-start capacitance into a chip and the output overshoot voltage can be suppressed. A prototype converter fabricated with TSMC 0.35-μm 2P4M CMOS process verifies the effectiveness of the techniques of a dual-path error amplifier and two capacitor multipliers. Experimental results demonstrate the converter stability, transient response, and soft-start function. The transient recovery time and transient ripple are less than 20 μs and 25 mV, respectively, for the load current swing from 50 to 500 mA. Moreover, the soft-start time is up to 8 ms. With the proposed techniques, the external pins of the dc-dc converters are minimized and their performance is improved significantly.