Search

Thanka murals are important cultural heritages of Tibet, but many precious murals were damaged during history. Thanka mural restoration is very important for the protection of Tibetan cultural heritage. Partial convolution has great potential for Thanka mural restoration due to its outstanding performance for inpainting irregular holes. However, three challenges prevent the existing partial convolution-based methods from solving Thanka restoration problems: 1) the features of multi-scale objects in Thanka murals cannot be extracted correctly because of single-scale partial convolution; 2) the stroke-like Thanka inpainting mode cannot be effectively simulated and learned by existing rectangular or arbitrary masks; and 3) the original content of damaged Thanka murals cannot be restored. To resolve these problems, we propose a Thanka mural inpainting method based on multi-scale adaptive partial convolution and stroke-like masks. The proposed method consists of three parts: 1) a kernel-level multi-scale adaptive partial convolution (MAPConv) to accurately discriminate valid pixels from invalid pixels, and to extract the features of multi-scale objects; 2) a parameter-configurable stroke-like mask generation method to simulate and learn the stroke-like Thanka inpainting mode; and 3) a 2-phase learning framework based on MAPConv Unet and different loss functions to restore the original content of Thanka murals. Experiments on both simulated and real damages of Thanka murals demonstrated that our approach works well on a small dataset (N=2780), generates realistic mural content, and restores the damaged Thanka murals with high speed (600 ms for multiple holes in $512\times 512$ images). The proposed end-to-end method can be applied to other small datasets-based inpainting tasks. [ABSTRACT FROM AUTHOR]

It is very challenging to recognize the information of occluded Tibetan document pages due to the lack of digitization and their long-term storage. Multiple pages are stuck, and textual characters are occluded with each other, which causes difficulties in restoration. Due to the large size of Tibetan documents, it is impossible to separate and repair these occluded pages by professionals. Therefore, the separation of overlapping pages and restoration of occluded pages play important roles in the digitization of Tibetan documents. We extract underlying pages by show-through scanning and eliminating the text area of top pages. In order to restore the occluded underlying pages, we present a side-enhanced U-Net (SEU-Net) that attaches side feature extraction module and side classification module to the U-Net to improve the classification of textual edges. Experiments performed on the dataset of Tibetan documents restoration patches show that SEU-Net is able to classify the textual pixels in the occluded pages accurately, and both side feature extraction module and side classification module improve performance independently. [ABSTRACT FROM AUTHOR]