How predictors affect the RL-based search strategy in Neural Architecture Search?

Predictor-based Neural Architecture Search is an important topic since it can efficiently reduce the computational cost of evaluating candidate architectures. Most existing predictor-based NAS algorithms aim to design different predictors to improve prediction performance. Unfortunately, even a promising performance predictor may suffer from the accuracy decline due to long-term and continuous usage, thus leading to the degraded performance of the search strategy. That naturally gives rise to the following problems: how do predictors affect search strategies and how to efficiently use the predictor? In this paper, we take Reinforcement Learning (RL) based search strategy to study theoretically and empirically the impact of predictors on search strategies. We first formulate an RL-Predictor-based NAS algorithm as model-based RL and analyze it with a guarantee of monotonic improvement. Then, based on this analysis, we propose a simple procedure of predictor usage, named m i x e d b a t c h , which contains ground-truth data and prediction data in a batch. The proposed procedure can efficiently reduce the impact of predictor errors on the RL-based search strategy with maintaining performance growth. Our algorithm, RL-Predictor-based Neural Architecture Search with Mixed batch (RPNASM), outperforms traditional NAS algorithms and prior state-of-the-art predictor-based NAS algorithms on three NAS-Bench-201 tasks and one NAS-Bench-ASR task. Our code is available at https://github.com/tjdeng/RPNASM. • Theoretically analyze predictor's impact on RL search strategy for the first time. • Perform comprehensive experiments to investigate RL-Predictor based NAS algorithms. • Propose RL-Predictor-based NAS framework to enhance search performance. [ABSTRACT FROM AUTHOR]