Coloring anime line art videos with transformation region enhancement network.

• We propose a multi-scale Transformation Region Enhancement Network (TRE-Net) to enhance the learning on geometric transformation regions. • We propose to locate geometric transformation regions with Multi-scale Euclidean Distance Difference (Multi-scale EDD) Map. To the best of our knowledge, the Multi-scale EDD Map is used for the first time in anime line art colorization. As a result, the network can adaptively leverage different strategies in different regions. Different from mask propagation algorithms for gray video colorization, our Multi-scale EDD Maps work well when dealing with line art sequences and act as robust guidance for our TRE-Net to improve the colorization quality and efficiency. • Feature Enhancement Module (FEM) and Attention loss are devised to generate color-aligned reference features by enhancing the feature learning on geometric transformation regions. • Extensive qualitative and quantitative experimental results demonstrate that our model can generate visually appealing colorized frames. [Display omitted] Automatic colorization of anime line art videos aims to produce color frames given line art frames and reference color images, which is challenging due to various motions and geometric transformations across frame sequences. Existing methods usually utilize the feature maps of reference images directly and treat all the regions in an image equally. However, this may overlook the details of the regions undergoing geometric transformations. To emphasize the regions with significant transformations between the reference and target frames, we propose a Transformation Region Enhancement Network (TRE-Net) to exploit useful reference information and enhance the colorization of key transformation regions with Region Localization Module (RLM) and Feature Enhancement Module (FEM). Specifically, we propose Multi-scale Euclidean Distance Difference (Multi-scale EDD) Maps in RLM which effectively locate geometric transformation regions by contrasting the Euclidean Distance Maps of two line arts and aggregating representations at multiple scales of the network. In addition, FEM is devised to enhance feature learning in the regions with geometric transformation and to ensure proper color alignment. FEM learns locally enhanced features through an attention-gating operation at a low computational cost. With the well-represented key geometric transformation regions, our method exploits the multi-scale reference information well for color alignment, thus produces perceptually pleasing frames. Comprehensive experimental results show that our proposed method is superior to existing methods in terms of the overall quality of colorized anime line art videos. [ABSTRACT FROM AUTHOR]