Search

Abstract Ancient murals, invaluable cultural artifacts, frequently suffer damage from environmental and human factors, necessitating effective restoration techniques. Traditional methods, which rely on manual skills, are time-consuming and often inconsistent. This study introduces an innovative mural restoration approach using a generative adversarial network (GAN) within a UNet architecture. The generator integrates Transformer and convolutional neural network (CNN) components, effectively capturing and reconstructing complex mural features. This work's novelty lies in integrating the Group-wise Multi-scale Self-Attention (GMSA), an Encoder-Decoder Feature Interaction (EDFI) module, and a Local Feature Enhancement Block (LFEB). These components allow the model to better capture, reconstruct, and enhance mural features, leading to a significant improvement over traditional restoration methods. Tested on a dataset of Tang Dynasty murals, the method demonstrated superior performance in PSNR, SSIM, and LPIPS metrics compared to seven other techniques. Ablation studies confirmed the effectiveness of the heterogeneous network design and the critical contributions of the GMSA, EDFI, and LFEB modules. Practical restoration experiments showed the method's ability to handle various types of mural damage, providing seamless and visually authentic restorations. This novel approach offers a promising solution for the digital preservation and restoration of cultural heritage murals, with potential applications in practical restoration projects.

Abstract Ancient murals are precious cultural heritages. They suffer from various damages due to man-made destruction and long-time exposure to the environment. It is urgent to protect and restore the damaged ancient murals. Virtual restoration of ancient murals aims to fill damaged mural regions by using modern computer techniques. Most existing restoration approaches fail to fill the loss mural regions with rich details and complex structures. In this paper, we propose a virtual restoration network of ancient murals based on global–local feature extraction and structural information guidance (GLSI). The proposed network consists of two major sub-networks: the structural information generator (SIG) and the image content generator (ICG). In the first sub-network, SIG can predict the structural information and the coarse contents for the missing mural regions. In the second sub-network, ICG utilizes the predicted structural information and the coarse contents to generate the refined image contents for the missing mural regions. Moreover, we design an innovative BranchBlock module that can effectively extract and integrate the local and global features. We introduce a Fast Fourier Convolution (FFC) to improve the color restoration for the missing mural regions. We conduct experiments over simulated and real damaged murals. Experimental results show that our proposed method outperforms other three comparative state-of-the-art approaches in terms of structural continuity, color harmony and visual rationality of the restored mural images. In addition, the mural restoration results of our method can achieve comparatively high quantitative evaluation metrics.

Abstract Mural is an important component of culture and art of Dunhuang in China. Unfortunately, these murals had been ruined or are being ruined by some diseases such as cracking, hollowing, falling off, mildew, dirt, and so on. Existing image restoration algorithms have problems such as incomplete repair and disharmonious texture during large-area repair, so the effect of mural image disease area repair is poor. Due to lack of a standard mural datasets, Dunhuang mural datasets are created in the paper. Meanwhile, our network architecture SeparaFill is proposed which connects two generators based on U-Net. Based on the characteristics of the painting, the contour line pixel area of the mural image is innovatively separated from the content pixel area. Firstly, the contour restoration generator network with skip connect and hierarchical residual blocks is employed to repair contour lines. Then, the color mural image is repaired by the content completion network with guide of the repaired contour. Full resolution branches and generator branches of the U type are exploited in content completion generators. Convolution layers of different kernel sizes are fused to improve the reusability of the underlying features. Finally, global and local discriminant networks are applied to determine whether the repaired mural image is authentic in terms of both the modified and unmodified areas. The proposed SeparaFill shows good performance in restoring the line structure of the damaged mural images and retaining the contour information of the mural images. Compared with existing restoration algorithms in mural real damage repair experiment, our algorithm increases the peak signal-to-noise ratio (PSNR) by an average of 1.1–4.3 dB and the structural similarity (SSIM) values were slightly improved. Experimental results reveal the good performance of the proposed model, which can contribute to digital restorations of ancient murals.

Abstract Shenzhen is a modern metropolis, but it hides a variety of valuable cultural heritage, such as ancient murals. How to effectively preserve and repair the murals is a worthy of discussion question. Here, we propose a generation-discriminator network model based on artificial intelligence algorithms to perform digital image restoration of ancient damaged murals. In adversarial learning, this study optimizes the discriminative network model. First, the real mural images and damaged images are spliced together as input to the discriminator network. The network uses a 5-layer encoder unit to down-sample the 1024 × 1024 × 3 image to 32 × 32 × 256. Then, we connect a layer of ZeroPadding2D to expand the image to 34 × 34 × 256, and pass the Conv2D layer, down-sample to 31 × 31 × 256, perform batch normalization, and repeat the above steps to get a 30 × 30 × 1 matrix. Finally, this part of the loss is emphasized in the loss function as needed to improve the texture detail information of the image generated by the Generator. The experimental results show that compared with the traditional algorithm, the PSNR value of the algorithm proposed in this paper can be increased by 5.86 db at most. The SSIM value increased by 0.13. Judging from subjective vision. The proposed algorithm can effectively repair damaged murals with dot-like damage and complex texture structures. The algorithm we proposed may be helpful for the digital restoration of ancient murals, and may also provide reference for mural restoration workers.

Abstract How to effectively protect ancient murals has become an urgent and important problem. Digital image processing developments have made it possible to repair damaged murals to a certain extent. This study proposes a consistency-enhanced generative adversarial network (GAN) model to repair missing mural areas. First, the convolutional layer from a fully convolutional network (FCN) is used to extract deep image features; then, through deconvolution, the features are mapped to the size of the original image and the repaired image is output, thereby completing the regenerative network. Next, global and local discriminant networks are applied to determine whether the repaired mural image is “authentic” in terms of both the modified and unmodified areas. In adversarial learning, the generative and discriminant network models are optimized to better complete the mural repair. The network introduces a dilated convolution that increases the convolution kernel’s receptive field. Each network convolutional layer joins in the batch standardization (BN) process to accelerate network convergence and increase the number of network layers and adopts a residual module to avoid the vanishing gradient problem and further optimizing the network. Compared with existing mural restoration algorithms, the proposed algorithm increases the peak signal-to-noise ratio (PSNR) by an average of 6–8 dB and increases the structural similarity (SSIM) index by 0.08–0.12. From a visual perspective, this algorithm successfully complements mural images with complex textures and large missing areas; thus, it may contribute to digital restorations of ancient murals.