Search

The elastic bending energy model is commonly used to describe the shape transformation of biological lipid vesicles, making it a classical phase field model. In this paper, by coupling the elastic bending energy with the total variation (TV) regularization, we develop an elastic bending-TV model for image inpainting. By solving the energy minimization problem of this model, we obtain the results for image processing. We adopt an operator splitting method for the model and the numerical scheme involves the introduction of two vector- and scalar-valued functions to reconstruct this functional. The energy minimization problem is transformed into finding the steady state solution of artificial time-dependent PDE systems. At each fractional step, we can find either a closed-form solution or being solved by an efficient algorithm, which is a very robust and stable algorithm. Experimental results validate the superiority of our model and the effectiveness of the scheme for image inpainting. [ABSTRACT FROM AUTHOR]

Recent studies have shown that image structure can significantly facilitate image inpainting. However, current approaches mostly explore structure prior without considering its guidance to texture reconstruction, leading to performance degradation. To solve this issue, we propose a two-stream Tracking Structure Transformer (TSFormer), including structure target stream and image completion stream, to capture the synchronous and dynamic interplay between structure and texture. Specifically, we first design a structure enhancement module to restore the Histograms of Oriented Gradient (HOG) and the edge of an input image in a sketch space, which forms the input of the structure target stream. Meanwhile, in the image completion stream, we design a channel-space parallel-attention component to facilitate the efficient co-learning of channel and spatial visual cues. To build a bridge between the two streams, we further develop a structure-texture cross-attention module, wherein both structure and texture are synchronously extracted through self-attention, and texture extraction is implemented by dynamically tracking the structure in a cross-attention fashion, enabling the capture of the intricate interaction between structure and texture. Extensive experiments evaluated on three benchmark datasets, including CelebA, Places2, and Paris StreetView, demonstrate that the proposed TSFormer achieves state-of-the-art performance compared to its competitors. The code is available at https://github.com/GZHU-DVL/TSFormer. [ABSTRACT FROM AUTHOR]

Traditional dental prosthetics require a significant amount of work, labor, and time. To simplify the process, a method to convert teeth scan images, scanned using an intraoral scanner, into 3D images for design was developed. Furthermore, several studies have used deep learning to automate dental prosthetic processes. Tooth images are required to train deep learning models, but they are difficult to use in research because they contain personal patient information. Therefore, we propose a method for generating virtual tooth images using image-to-image translation (pix2pix) and contextual reconstruction fill (CR-Fill). Various virtual images can be generated using pix2pix, and the images are used as training images for CR-Fill to compare the real image with the virtual image to ensure that the teeth are well-shaped and meaningful. The experimental results demonstrate that the images generated by the proposed method are similar to actual images. In addition, only using virtual images as training data did not perform well; however, using both real and virtual images as training data yielded nearly identical results to using only real images as training data. [ABSTRACT FROM AUTHOR]

This study presents a new image inpainting model based on U-Net and incorporating the Wasserstein Generative Adversarial Network (WGAN). The model uses skip connections to connect every encoder block to the corresponding decoder block, resulting in a strictly symmetrical architecture referred to as Symmetric Connected U-Net (SC-Unet). By combining SC-Unet with a GAN, the study aims to reconstruct images more effectively and seamlessly. The traditional discriminators only differentiate the entire image as true or false. In this study, the discriminator calculated the probability of each pixel belonging to the hole and non-hole regions, which provided the generator with more gradient loss information for image inpainting. Additionally, every block of SC-Unet incorporated a Dilated Convolutional Neural Network (DCNN) to increase the receptive field of the convolutional layers. Our model also integrated Multi-Head Self-Attention (MHSA) into selected blocks to enable it to efficiently search the entire image for suitable content to fill the missing areas. This study adopts the publicly available datasets CelebA-HQ and ImageNet for evaluation. Our proposed algorithm demonstrates a 10% improvement in PSNR and a 2.94% improvement in SSIM compared to existing representative image inpainting methods in the experiment. [ABSTRACT FROM AUTHOR]

To enhance the appeal of residential real estate listings and captivate online customers, clean and visually convincing indoor scenes are highly desirable. In this research, we introduce an innovative image inpainting model designed to seamlessly replace undesirable elements within images of indoor residential spaces with realistic and coherent alternatives. While Generative Adversarial Networks (GANs) have demonstrated remarkable potential for removing unwanted objects, they can be resource-intensive and face difficulties in consistently producing high-quality outcomes, particularly when unwanted objects are scattered throughout the images. To empower small- and medium-sized businesses with a competitive edge, we present a novel GAN model that is resource-efficient and requires minimal training time using arbitrary mask generation and a novel half-perceptual loss function. Our GAN model achieves compelling results in removing unwanted elements from indoor scenes, demonstrating the capability to train within a single day using a single GPU, all while minimizing the need for extensive post-processing. [ABSTRACT FROM AUTHOR]

In recent years, deep learning models have dramatically influenced image inpainting. However, many existing studies still suffer from over‐smoothed or blurred textures when missing regions are large or contain rich visual details. To restore textures at a fine‐grained level, a multi‐stage inpainting approach is proposed, which applies a series of partial inpainting modules as well as a progressive inpainting module to inpaint missing areas from their boundaries to the centre successively. Some improvements are made on the partial convolutions to reduce artifacts like blurriness, which require a convolution kernel to contain known pixels more than a certain proportion. Towards photorealistic inpainting results, the intermediate outputs from each stage are used to compute the loss. Finally, to facilitate the training process, a multi‐step training is designed that progressively adds inpainting modules to optimize the model. Experiments show that this method outperforms the current excellent techniques on the publicly available datasets: CelebA, Places2 and Paris StreetView. [ABSTRACT FROM AUTHOR]

• Language-based Subject Generation faces challenge in accurate portrayal of subject. • Nowadays Reference Guided Generation lacks ability to preserve subject identity. • Exemplar-based instructions with visual tokens preserve visual details of subject. • Model based guidance samples better quality images with different pose. • Our model achieved highest CLIP, DINO score and user study compared to others. This paper presents a novel approach to subject-driven image generation that addresses the limitations of traditional text-to-image diffusion models. Our method generates images using reference images without relying on language-based prompts. We introduce a visual detail preserving module that captures intricate details and textures, addressing overfitting issues associated with limited training samples. The model's performance is further enhanced through a modified classifier-free guidance technique and feature concatenation, enabling the natural positioning and harmonization of subjects within diverse scenes. Quantitative assessments using CLIP, DINO and Quality scores (QS), along with a user study, demonstrate the superior quality of our generated images. Our work highlights the potential of pre-trained models and visual patch embeddings in subject-driven editing, balancing diversity and fidelity in image generation tasks. Our implementation is available at https://github.com/8eomio/Subject-Inpainting. [Display omitted] To create your abstract, type over the instructions in the template box below. Fonts or abstract dimensions should not be changed or altered. [ABSTRACT FROM AUTHOR]

Thangka, as a precious heritage of painting art, holds irreplaceable research value due to its richness in Tibetan history, religious beliefs, and folk culture. However, it is susceptible to partial damage and form distortion due to natural erosion or inadequate conservation measures. Given the complexity of textures and rich semantics in thangka images, existing image inpainting methods struggle to recover their original artistic style and intricate details. In this paper, we propose a novel approach combining discrete codebook learning with a transformer for image inpainting, tailored specifically for thangka images. In the codebook learning stage, we design an improved network framework based on vector quantization (VQ) codebooks to discretely encode intermediate features of input images, yielding a context-rich discrete codebook. The second phase introduces a parallel transformer module based on a cross-shaped window, which efficiently predicts the index combinations for missing regions under limited computational cost. Furthermore, we devise a multi-scale feature guidance module that progressively fuses features from intact areas with textural features from the codebook, thereby enhancing the preservation of local details in non-damaged regions. We validate the efficacy of our method through qualitative and quantitative experiments on datasets including Celeba-HQ, Places2, and a custom thangka dataset. Experimental results demonstrate that compared to previous methods, our approach successfully reconstructs images with more complete structural information and clearer textural details. [ABSTRACT FROM AUTHOR]

Deep learning has brought essential improvement to image inpainting technology. Conventional deep-learning methods primarily focus on creating visually appealing content in the missing parts of images. However, these methods usually generate edge variations and blurry structures in the filled images, which lead to imbalances in quantitative metrics PSNR/SSIM and LPIPS/FID. In this work, we introduce a pioneering model called PTG-Fill, which utilizes a coarse-to-fine architecture to achieve smooth-variation image inpainting. Our approach adopts the novel Stable-Partial Convolution to construct the coarse network, which integrates a smooth mask-update process to ensure its long-term operation. Meanwhile, we propose the novel Distinctive-Gated Convolution to construct the refined network, which diminishes pixel-level variations by the distinctive attention. Additionally, we build up a novel Transformer bridger to preserve the in-depth features for image refinement and facilitate the operation of the two-stage network. Our extensive experiments demonstrate that PTG-Fill outperforms previous state-of-the-art methods both quantitatively and qualitatively under various mask ratios on four benchmark datasets: CelebA-HQ, FFHQ, Paris StreetView, and Places2. Code and pre-trained weights are available at https://github.com/zeyuwang-zju/PTG-Fill. [ABSTRACT FROM AUTHOR]

Current single-view image inpainting methods often suffer from low image information utilization and suboptimal repair outcomes. To address these challenges, this paper introduces a novel image inpainting framework that leverages a structure-guided multi-scale attention pyramid network. This network consists of a structural repair network and a multi-scale attention pyramid semantic repair network. The structural repair component utilizes a dual-branch U-Net network for robust structure prediction under strong constraints. The predicted structural view then serves as auxiliary information for the semantic repair network. This latter network exploits the pyramid structure to extract multi-scale features of the image, which are further refined through an attention feature fusion module. Additionally, a separable gated convolution strategy is employed during feature extraction to minimize the impact of invalid information from missing areas, thereby enhancing the restoration quality. Experiments conducted on standard datasets such as Paris Street View and CelebA demonstrate the superiority of our approach over existing methods through quantitative and qualitative comparisons. Further ablation studies, by incrementally integrating proposed mechanisms into a baseline model, substantiate the effectiveness of our multi-view restoration strategy, separable gated convolution, and multi-scale attention feature fusion. [ABSTRACT FROM AUTHOR]

Removing photobombing elements from images is a challenging task that requires sophisticated image inpainting techniques. Despite the availability of various methods, their effectiveness depends on the complexity of the image and the nature of the distracting element. To address this issue, we conducted a benchmark study to evaluate 10 state-of-the-art photobombing removal methods on a dataset of over 300 images. Our study focused on identifying the most effective image inpainting techniques for removing unwanted regions from images. We annotated the photobombed regions that require removal and evaluated the performance of each method using peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Fréchet inception distance (FID). The results show that image inpainting techniques can effectively remove photobombing elements, but more robust and accurate methods are needed to handle various image complexities. Our benchmarking study provides a valuable resource for researchers and practitioners to select the most suitable method for their specific photobombing removal task. [ABSTRACT FROM AUTHOR]

A deep‐learning algorithm is proposed for the inpainting of Bragg coherent diffraction imaging (BCDI) patterns affected by detector gaps. These regions of missing intensity can compromise the accuracy of reconstruction algorithms, inducing artefacts in the final result. It is thus desirable to restore the intensity in these regions in order to ensure more reliable reconstructions. The key aspect of the method lies in the choice of training the neural network with cropped sections of diffraction data and subsequently patching the predictions generated by the model along the gap, thus completing the full diffraction peak. This approach enables access to a greater amount of experimental data for training and offers the ability to average overlapping sections during patching. As a result, it produces robust and dependable predictions for experimental data arrays of any size. It is shown that the method is able to remove gap‐induced artefacts on the reconstructed objects for both simulated and experimental data, which becomes essential in the case of high‐resolution BCDI experiments. [ABSTRACT FROM AUTHOR]

The integration of convolutional neural network (CNN) and transformer enhances the network's capacity for concurrent modeling of texture details and global structures. However, training challenges with transformer limit their effectiveness to low-resolution images, leading to increased artifacts in slightly larger images. In this paper, we propose a single-stage network utilizing large kernel attention (LKA) to address high-resolution damaged images. LKA enables the capture of both global and local details, akin to transformer and CNN networks, resulting in high-quality inpainting. Our method excels in: (1) reducing parameters, improving inference speed, and enabling direct training on 1024 × 1024 resolution images; (2) utilizing LKA for enhanced extraction of global high-frequency and local details; (3) demonstrating excellent generalization on irregular mask models and common datasets such as Places2, Celeba-HQ, FFHQ, and the random irregular mask dataset Pconv from NVIDIA. [ABSTRACT FROM AUTHOR]

The rapid advancement of deep learning technologies presents novel opportunities for restoring damaged patterns in ancient silk, which is pivotal for the preservation and propagation of ancient silk culture. This study systematically scrutinizes the evolutionary trajectory of image inpainting algorithms, with a particular emphasis on those firmly rooted in the Context-Encoder structure. To achieve this study's objectives, a meticulously curated dataset comprising 6996 samples of ancient Chinese silk (256 × 256 pixels) was employed. Context-Encoder-based image inpainting models—LISK, MADF, and MEDFE—were employed to inpaint damaged patterns. The ensuing restoration effects underwent rigorous evaluation, providing a comprehensive analysis of the inherent strengths and limitations of each model. This study not only provides a theoretical foundation for adopting image restoration algorithms grounded in the Context-Encoder structure but also offers ample scope for exploration in achieving more effective restorations of ancient damaged silk. [ABSTRACT FROM AUTHOR]

Automatic micro-defect detection is crucial for promoting efficiency in the production lines of patterned OLED panels. Recently, deep learning algorithms have emerged as promising solutions for micro-defect detection. However, in real-world industrial scenarios, the scarcity of training data or annotations results in a drop in performance. A multi-stage few-shot micro-defect detection approach is proposed for patterned OLED panels to deal with this problem. Firstly, we introduce a converter from defective to defect-free images based on our redesigned Vector Quantized-Variational AutoEncoder (VQ-VAE), aiming to inpaint defects with normal textures. Next, we exploit a region-growing method with automatic seed points to obtain the defect's segmentation and geometric parameters in each image block. Reliable seed points are provided by structural similarity index maps between defective sub-blocks and reconstructed reference. Finally, a multi-scale Siamese neural network is proposed to identify the category of extracted defects. With our proposed approach, detection and classification results of defects can be obtained successively. Our experimental results on samples at different array processes demonstrate the superb adaptability of VQ-VAE, with a defect detection rate ranging from 90.0% to 96.0%. Additionally, compared with existing classification models, our multi-scale Siamese neural network exhibits an impressive 98.6% classification accuracy for a long-tailed defect dataset without overfitting. In summary, the proposed approach shows great potential for practical micro-defect detection in industrial scenarios with limited training data. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

As a robust digital image inpainting technology, the Criminisi algorithm (CA) has been widely used. However, its high running time that it needs to search in the entire undamaged area of the image to determine an optimal matching block presents a challenge. To address this issue, this study proposes an improved version of CA, named PSO-CA, which incorporates the particle swarm optimization algorithm (PSO) with CA. The running time of the CA is significantly reduced benefiting from the parallel optimization capability of the PSO. In addition, the search space is restricted to the neighbouring region of the block that needs to be filled. The availability of the proposed PSO-CA algorithm is assessed in the laboratory colour model by the running time and three matching indices, such as the peak signal-to-noise ratio (PSNR). The experimental results indicate that PSO-CA significantly enhances the inpainting speed and produces the same or better results compared with the initial CA and the Criminisi with search space algorithm (CWSS). [ABSTRACT FROM AUTHOR]

An interpolation method, which estimates unknown values with constrained information, is based on mathematical calculations. In this study, we addressed interpolation from an image-based perspective and expanded the use of image inpainting to estimate values at unknown points. When chemical gas is dispersed through a chemical attack or terrorism, it is possible to determine the concentration of the gas at each location by utilizing the deployed sensors. By interpolating the concentrations, we can obtain the contours of gas concentration. Accurately distinguishing the contours of a contaminated region from a map enables the optimal response to minimize damage. However, areas with an insufficient number of sensors have less accurate contours than other areas. In order to achieve more accurate contour data, an image inpainting-based method is proposed to enhance reliability by erasing and reconstructing low-accuracy areas in the contour. Partial convolution is used as the machine learning approach for image-inpainting, with the modified loss function for optimization. In order to train the model, we developed a gas diffusion simulation model and generated a gas concentration contour dataset comprising 100,000 contour images. The results of the model were compared to those of Kriging interpolation, one of the conventional spatial interpolation methods, finally demonstrating 13.21% higher accuracy. This suggests that interpolation from an image-based perspective can achieve higher accuracy than numerical interpolation on well-trained data. The proposed method was validated using gas concentration contour data from the verified gas dispersion modeling software Nuclear Biological Chemical Reporting And Modeling System (NBC_RAMS), which was developed by the Agency for Defense Development, South Korea. [ABSTRACT FROM AUTHOR]

Image inpainting has made great achievements recently, but it is often tough to generate a semantically consistent image when faced with large missing areas in complex scenes. To address semantic and structural alignment in existing methods for image inpainting, this paper proposes an end-to-end attention-based gated convolution GAN with prior guidance named AGG, which designs the spatial and channel attention mechanisms for full extraction of semantic and structural features. Moreover, AGG constructs the attention-based upsampling module based on the channel attention to refine the feature map and capture more details from features of up-level low sizes. AGG uses the image contour as prior, allowing the gated convolution and attention mechanism may fill the image efficiently by focusing on the contour information. The attention-based gated convolution can effectively capture the global features and compensate for the limitations of the restricted receptive field of the naive convolution. Compared to other models, AGG generates images with finer outline features and no common problems such as the watermark and blur, which shows the best overall performance on the Paris StreetView, CelebA-HQ, and Places2 datasets. The best FID, LPIPS, PSNR, and SSIM values achieved by AGG are 2.18, 0.046, 30.82, and 0.951 on the CelebA-HQ dataset, with at least 3.21% and 6.52% performance improvement on FID and LPIPS compared to state-of-the-art methods, respectively. The source code will be available at https://github.com/Shawn-Yu-1/AGGNet. [ABSTRACT FROM AUTHOR]

Electron tomography based on scanning transmission electron microscopy (STEM) is used to analyze 3D structures of metal nanoparticles on the atomic scale. However, in the case of supported metal nanoparticle catalysts, the supporting material may interfere with the 3D reconstruction of metal nanoparticles. In this study, a deep learning‐based image inpainting method is applied to high‐angle annular dark field (HAADF)–STEM images of a supported metal nanoparticle to predict and remove the background image of the support. The inpainting method can separate an 11 nm Pd nanoparticle from the θ‐Al2O3 support in HAADF–STEM images of the θ‐Al2O3‐supported Pd catalyst. 3D reconstruction of the extracted images of the Pd nanoparticle reveals that the Pd nanoparticle adopts a deformed structure of the cuboctahedron model particle, resulting in high index surfaces, which account for the high catalytic activity for methane combustion. Using the xyz coordinate of each Pd atom, the local Pd–Pd bond distance and its variance in a real supported Pd nanoparticle are visualized, showing large strain and disorder at the Pd–Al2O3 interface. The results demonstrate that 3D atomic‐scale analysis enables atomic structure‐based understanding and design of supported metal catalysts. [ABSTRACT FROM AUTHOR]

The aim of image inpainting is to fill in damaged areas according to certain rules based on information about the adjacent positions of missing areas and the overall structure of the image, a technique that plays a key role in various tasks in computer vision. With the rapid development of deep learning, researchers have combined it with image inpainting and achieved excellent performance. To gain insight into the techniques involved, this paper summarizes the latest research advances in the field of image inpainting. Firstly, existing classical image inpainting methods are reviewed, and traditional image inpainting methods and their advantages and disadvantages are introduced. Secondly, three classical network models are outlined, and the image inpainting methods are classified into single-stage, multi-stage and a priori condition-guided approaches according to different network types and model structures. Representative algorithms among them are selected and their important technical improvement ideas are analyzed and summarized. Then, the common datasets commonly used in image inpainting tasks and the evaluation metrics used to evaluate inpainting results are introduced. The paper presents a comprehensive summary of the various algorithms in terms of network models and inpainting methods, and selects representative algorithms for quantitative and qualitative comparative analysis. Finally, the future development trends and research directions have prospected, and the current problems of image inpainting are summarized. [ABSTRACT FROM AUTHOR]

Image and video inpainting is a classic problem in computer vision and computer graphics, aiming to fill in the plausible and realistic content in the missing areas of images and videos. With the advance of deep learning, this problem has achieved significant progress recently. The goal of this paper is to comprehensively review the deep learning-based methods for image and video inpainting. Specifically, we sort existing methods into different categories from the perspective of their high-level inpainting pipeline, present different deep learning architectures, including CNN, VAE, GAN, diffusion models, etc., and summarize techniques for module design. We review the training objectives and the common benchmark datasets. We present evaluation metrics for low-level pixel and high-level perceptional similarity, conduct a performance evaluation, and discuss the strengths and weaknesses of representative inpainting methods. We also discuss related real-world applications. Finally, we discuss open challenges and suggest potential future research directions. [ABSTRACT FROM AUTHOR]

Image inpainting techniques have made rapid progresses in recent years. Recent advancements focus mainly on generating realistic and semantically plausible structure and texture features in missing regions. However, current popular inpainting methods rely typically on a single encoder–decoder or two separate encoder–decoders, which lead to inconsistent contextual semantics and blurry textures. To address the above issue, a dual-feature encoder implemented by structure and texture features is proposed. It utilizes skip connection to guide its corresponding decoder to fill image structure information (deep layer) and texture information (shallow layer) in the attention-based latent space. Additionally, we design multi-scale receptive fields to further improve the consistency of contextual semantics and image details. The experimental findings demonstrate that our method can effectively repair the structure and texture information of missing images with superior performance on three commonly used datasets. Furthermore, we build a mural dataset from the Mogao Grottoes and successfully restore them using our network. [ABSTRACT FROM AUTHOR]

In practical scenarios, such as in electronics, where barcodes on electronic component carriers often wear out, and in logistics, where package labels frequently get damaged, this type of damage makes the recognition of two-dimensional (2D) barcodes challenging. In this study, a new repair method was introduced for quick response (QR) and PDF417 codes. In addition, a structural fusion attention (SFA) mechanism with a recurrent feature reasoning network was integrated to enhance structural integrity and recognition rates. The proposed method significantly outperforms existing inpainting models in terms of accuracy and robustness, which is demonstrated by the custom dataset provided by the authors. Notably, the approach ensures near-perfect recognition rates despite extensive structural impairments. It achieves an accuracy of 98% for large-area PDF417 occlusions and maintains a recognition rate of 100% for QR codes with 75–90% structural damage. These findings highlight the exceptional ability of the proposed method to restore 2D barcodes impaired by diverse levels of structural occlusion. [ABSTRACT FROM AUTHOR]

Image inpainting infers the missing areas of a corrupted image according to the information of the undamaged part. Many existing image inpainting methods can generate plausible inpainted results from damaged images with the fast-developed deep-learning technology. However, they still suffer from over-smoothed textures or textural distortion in the cases of complex textural details or large damaged areas. To restore textures at a fine-grained level, we propose an image inpainting method based on a hierarchical VQ-VAE with a vector credibility mechanism. It first trains the hierarchical VQ-VAE with ground truth images to update two codebooks and to obtain two corresponding vector collections containing information on ground truth images. The two vector collections are fed to a decoder to generate the corresponding high-fidelity outputs. An encoder then is trained with the corresponding damaged image. It generates vector collections approximating the ground truth by the help of the prior knowledge provided by the codebooks. After that, the two vector collections pass through the decoder from the hierarchical VQ-VAE to produce the inpainted results. In addition, we apply a vector credibility mechanism to promote vector collections from damaged images and approximate vector collections from ground truth images. To further improve the inpainting result, we apply a refinement network, which uses residual blocks with different dilation rates to acquire both global information and local textural details. Extensive experiments conducted on several datasets demonstrate that our method outperforms the state-of-the-art ones. [ABSTRACT FROM AUTHOR]

Thanks to the rapid development of deep learning in recent years, image inpainting has made significant progress. As a fundamental task in the field of computer vision, many researchers are committed to exploring more efficient methods, and state-of-the-art research results prove that generative adversarial networks (GAN) have superior performance. However, due to the inherent ill-posedness of image inpainting tasks, these approaches suffer from lack of detailed information, local structural fractures or boundary artifacts. In this paper, we leverage the properties of GAN architecture to process images in more detail and more comprehensively. A novel dual U-Net GAN is designed to inpaint images, which is composed of a U-Net based generator and a U-Net-based discriminator. The former captures semantic information of different scales layer by layer and decodes it back to the original size to repair damaged images, while the latter optimizes the network by combining reconstruction loss, adversarial loss, perceptual loss and style loss. In particular, the U-Net-based discriminator allows per-pixel detail and global feedback to be provided to the generator, guaranteeing the global consistency of the inpainted image and the realism of local shapes and textures. Extensive experiments demonstrate that for different proportions of damage, the images inpainted by our proposed model have reasonable texture structure and contextual semantic information. Furthermore, the proposed model outperforms state-of-the-art models in both qualitative and quantitative comparisons. The code will be available at https://github.com/yjjswu. [ABSTRACT FROM AUTHOR]

Limited by the dynamic range of the detector, saturation artifacts usually occur in optical coherence tomography (OCT) imaging for high scattering media. The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images. We proposed a deep learning-based inpainting method of saturation artifacts in this paper. The generation mechanism of saturation artifacts was analyzed, and experimental and simulated datasets were built based on the mechanism. Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs. The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility, strong generalization, and robustness. [ABSTRACT FROM AUTHOR]

The fold that appeared in the micro-slice images needs to be inpainted exactly so that it can meet the requirements of the scientific experiments. The biological slice image usually consists of piecewise smooth regions with the closed loop contour which can be represented by the Bendlet function proposed in recent years. Therefore, a novel image inpainting method based on Bendlet and interval Shannon–Cosine wavelet is proposed. Since the deformation of the locust slice image is flexible, it is necessary to obtain as many feature points as possible to ensure the accuracy of the inpainting, so we introduce the curvature as a new registration element for registration. First, the homography matrix is obtained by calculating the correct feature points by our proposed registration method. Second, the fold position is located by homography matrix and inpainted by Shannon–Cosine interval wavelet interpolation. Finally, the pixel difference is eliminated through adaptive fusion. The results indicate that, in comparison to the SURF and ORB algorithms, our registration method significantly enhances the extraction of feature points, achieving a more even distribution. Furthermore, when compared to four other methods (K-SVD, BSCB, TV and Criminisi), as well as various interpolation methods such as cubic polynomial interpolation, cubic spline interpolation, and nearest neighbor interpolation, our approach consistently achieves the highest PSNR and SSIM values. [ABSTRACT FROM AUTHOR]

Image inpainting aims to restore missing or damaged regions of an image with plausible visual content. Most existing methods always face challenges when dealing with large hole images, such as structural distortion and prominent artifacts, which mainly stem from the absence of adequate semantic guidance. To overcome this problem, this paper proposes a novel progressive image inpainting network driven by dynamic context, where the guided restoration dynamic semantic prior not only includes information from the known region but also considers the semantic inference during the filling process. Specifically, a multi-view cooperative strategy is proposed firstly to predict the inherent semantic information by estimating the distributions of the image, grayscale and edge of the masked image. In addition, to cope with potential semantic changes, an auxiliary generative unit is proposed for learning semantic information of the intermediate inpainting results, where the learned intermediate semantics are fused with the intrinsic semantics in a weighted manner to form a real-time updating dynamic context. Moreover, the dynamic semantic prior is propagated to various stages of inpainting to assist in constructing refined feature maps with multiple levels of resolution. Experiments on CelebA-HQ and Paris street view datasets demonstrate that the proposed approach can recover reasonable structures and realistic textures on large-scale masked images, achieving advanced performance. [ABSTRACT FROM AUTHOR]

Existing image inpainting networks may cause distorted image structures and blurred textures during restoration. Therefore, we propose an image inpainting algorithm based on Multi‐Scale Feature Fusion Generative Adversarial Network (MSFF). First, we use a Context Fusion Transformer Block (CFTB) to enlarge the receptive field of the defective image and reduce the feature loss caused by sampling. Second, we design an Aperture Attention Block (ATAB) to capture the shallow feature information and perform fine‐grained feature fusion. Finally, through the decoder, we generate high‐quality inpainted images step by step from coarse to fine. The results of multiple experiments show that our method has a better inpainting effect. [ABSTRACT FROM AUTHOR]

Image inpainting is the process of restoring damaged areas in an image using information available from neighboring regions. In this paper, we present a novel, efficient, and simple local image inpainting algorithm based on the Allen–Cahn (AC) equation with a fidelity term. We utilize the phase separation property of the AC equation and introduce a new phase-dependent fidelity parameter to preserve the original values in the neighboring regions of an inpainting region. The governing partial differential equation is solved using the finite difference method, with the values of the neighboring cells serving as the Dirichlet boundary condition. The proposed algorithm is both local and explicit, making it is fast and easy to implement. We demonstrate the performance of the proposed model through several numerical experiments. Furthermore, comparing this method to other image inpainting methods demonstrates its superiority in image inpainting. [ABSTRACT FROM AUTHOR]

This paper proposes a Cahn-Hilliard type inpainting model equipped with a nonlocal diffusion operator. A rigorous analysis of the well-posedness of the stationary solution is established using Schauder's fixed point theory. We construct a time stepping scheme based on the convex splitting method with the nonlocal term treated implicitly and the fidelity term treated explicitly. We prove the consistency, stability and convergence of the semidiscrete-in-time scheme. To the best of our knowledge, this is the first study to present such an analysis for semidiscrete-in-time problems of this model, which provides valuable guidance for parameter selection. Numerical experiments validate the effectiveness of the proposed nonlocal model, which shows superior performance compared to both local and classical total variation models in preserving fine textures and recovering image edges. [ABSTRACT FROM AUTHOR]

In this paper, a new thresholding operator, namely, designed thresholding operator, is designed to recover the low-rank matrices. With the change of parameter in designed thresholding operator, the designed thresholding operator can apply less bias to the larger singular values of a matrix compared with the classical soft thresholding operator. Based on the designed thresholding operator, an iterative thresholding algorithm for recovering the low-rank matrices is proposed. Numerical experiments on some image inpainting problems show that the proposed thresholding algorithm performs effectively in recovering the low-rank matrices. [ABSTRACT FROM AUTHOR]

Background and objective: Metallic magnetic resonance imaging (MRI) implants can introduce magnetic field distortions, resulting in image distortion, such as bulk shifts and signal‐loss artifacts. Metal Artifacts Region Inpainting Network (MARINet), using the symmetry of brain MRI images, has been developed to generate normal MRI images in the image domain and improve image quality. Methods: T1‐weighted MRI images containing or located near the teeth of 100 patients were collected. A total of 9000 slices were obtained after data augmentation. Then, MARINet based on U‐Net with a dual‐path encoder was employed to inpaint the artifacts in MRI images. The input of MARINet contains the original image and the flipped registered image, with partial convolution used concurrently. Subsequently, we compared PConv with partial convolution, and GConv with gated convolution, SDEdit using a diffusion model for inpainting the artifact region of MRI images. The mean absolute error (MAE) and peak signal‐to‐noise ratio (PSNR) for the mask were used to compare the results of these methods. In addition, the artifact masks of clinical MRI images were inpainted by physicians. Results: MARINet could directly and effectively inpaint the incomplete MRI images generated by masks in the image domain. For the test results of PConv, GConv, SDEdit, and MARINet, the masked MAEs were 0.1938, 0.1904, 0.1876, and 0.1834, respectively, and the masked PSNRs were 17.39, 17.40, 17.49, and 17.60 dB, respectively. The visualization results also suggest that the network can recover the tissue texture, alveolar shape, and tooth contour. Additionally, for clinical artifact MRI images, MARINet completed the artifact region inpainting task more effectively when compared with other models. Conclusions: By leveraging the quasi‐symmetry of brain MRI images, MARINet can directly and effectively inpaint the metal artifacts in MRI images in the image domain, restoring the tooth contour and detail, thereby enhancing the image quality. [ABSTRACT FROM AUTHOR]

The quality of veneer determines the grade of veneer wood-based panels and the treatment of defects on veneer becomes an important part of wood processing. In order to deal with veneer defects and improve wood utilization, an image inpainting method of venerr defect based on match attention is proposed. The method proposes a match attention module to acquire features at a distance to enhance the accuracy of the model and uses a double convolution module that captures multi-scale contextual information. Then region normalization is used throughout the network to avoid mean and variance bias. Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are used as evaluation indicators. The results show that the PSNR of the improved method reaches 28. 48, and the SSIM reaches 0. 91. Compared to the globally and locally consistent image completion (GL) method, the PSNR and SSIM are improved by 1. 03% and 0. 05%, respectively. This method can achieve consistent effect in structure and texture, which providing guidance for the inpainting of the veneer defects. [ABSTRACT FROM AUTHOR]

Diffusion models specialized in image-to-image translation tasks, like inpainting and colorization, have outperformed the state of the art, yet their computational requirements are exceptionally demanding. This study analyzes different strategies to train image-to-image diffusion models in a low-resource setting. The studied strategies include incremental learning and task/domain transfer learning. First, a base model for human face inpainting is trained from scratch with an incremental learning strategy. The resulting model achieves an FID score almost equivalent to that of its batch learning equivalent while significantly reducing the training time. Second, the base model is fine-tuned to perform a different task, image colorization, and, in a different domain, landscape images. The resulting colorization models showcase exceptional performances with a minimal number of training epochs. We examine the impact of different configurations and provide insights into the ability of image-to-image diffusion models for transfer learning across tasks and domains. [ABSTRACT FROM AUTHOR]

Image inpainting networks based on deep learning techniques have been widely used in many important fields. However, most inpainting networks fail to generate desirable repaired images. This may be due to their failure to extract effective features and accurately assign high weights to the undamaged regions. To alleviate these problems, an image inpainting network based on gated convolution and multi‐level attention mechanism (IIN‐GCMAM) is proposed in this paper. This network follows encoder–decoder architecture, consisting of the gated convolution encoder (GC‐encoder) and the multi‐level attention mechanism decoder (MAM‐decoder). The GC‐encoder weighs the extracted features with gated convolutions, which reduces the interference caused by the damaged regions. The multi‐level attention mechanism employed in the MAM‐decoder uses multi‐scale feature maps spatially and channel‐wise to improve the consistency in global structure and the fineness of repaired results. Extensive experiments are conducted on the common datasets, Paris StreetView and CelebA. Experimental results indicate that the proposed IIN‐GCMAM can achieve a good performance on the common evaluation metrics and visual effects. It can achieve 0.0408, 0.720, and 22.27 in MAE, SSIM, and PSNR at the mask ratio of 50%–60%, respectively. [ABSTRACT FROM AUTHOR]

In recent years, significant advancements in the field of machine learning have influenced the domain of image restoration. While these technological advancements present prospects for improving the quality of images, they also present difficulties, particularly the proliferation of manipulated or counterfeit multimedia information on the internet. The objective of this paper is to provide a comprehensive review of existing inpainting algorithms and forgery detections, with a specific emphasis on techniques that are designed for the purpose of removing objects from digital images. In this study, we will examine various techniques encompassing conventional texture synthesis methods as well as those based on neural networks. Furthermore, we will present the artifacts frequently introduced by the inpainting procedure and assess the state-of-the-art technology for detecting such modifications. Lastly, we shall look at the available datasets and how the methods compare with each other. Having covered all the above, the outcome of this study is to provide a comprehensive perspective on the abilities and constraints of detecting object removal via the inpainting procedure in images. [ABSTRACT FROM AUTHOR]

Very recently, with the widespread research of deep learning, its achievements are increasingly evident in image inpainting tasks. However, many existing multi-stage methods fail to effectively inpainting the larger missing areas, their common drawback is that the result of each stage is easily misguided by the wrong content generated in the previous stage. To solve this issue, in this paper, a novel one-stage generative adversarial network based on the progressive decoding architecture and gradient guidance. Firstly, gradient priors are extracted at the encoder stage to be passed to the decoding branch, and multiscale attention fusion group is used to help the network understand the image features. Secondly, multiple parallel decoding branches fill and refine the missing regions by top-down passing the reconstructed priors. This progressively guided repair avoids the detrimental effects of inappropriate priors. The joint guidance of features and gradient priors helps the restoration results contain the correct structure and rich details. And the progressive guidance is achieved by our fusion strategy, combining reimage convolution and design channel coordinate attention to fuse and reweight the features of different branches. Finally, we use the multiscale fusion to merge the feature maps at different scales reconstructed by the last decoding branch and map them to the image space, which further improves the semantic plausibility of the restoration results. Experiments on multiple datasets show that the qualitative and quantitative results of our computationally efficient model are competitive with those of state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Camera failure or loss of storage components in imaging equipment may result in the loss of important image information or random pulse noise interference. The low-rank prior is one of the most important priors in image optimization processing. This paper reviews and compares some low-rank constraint models for image matrices. Firstly, an overview of image-inpainting models based on nuclear norm, truncated nuclear norm, weighted nuclear norm, and matrix-factorization-based F norm is presented, and corresponding optimization iterative algorithms are provided. Then, we use different image matrix low-order constraint models to recover satellite images from three types of pulse interference and provide our experimental visual and numerical results. Finally, it can be concluded that the method based on the weighted nuclear norm can achieve the best image restoration effect. The F norm method based on matrix factorization has the shortest computational time and can be used for large-scale low-rank matrix calculations. Compared with nuclear norm-based methods, weighted nuclear norm-based methods and truncated nuclear norm-based methods can significantly improve repair performance. [ABSTRACT FROM AUTHOR]

Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area. Image feature extraction is the core of image restoration. Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting. However, in the process of feature extraction, it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time. In order to obtain more spatial information and a larger receptive field at the same time, we put forward a kind of image restoration based on space path and context path network. For the space path, we stack three convolution layers for 1/8 of the figure, the figure retained the rich spatial details. For the context path, we use the global average pooling layer, where the accept field is the maximum of the backbone network, and the pooling module can provide global context information for the maximum accept field. In order to better integrate the features extracted from the spatial and contextual paths, we study the fusion module of the two paths. Features fusionmodule first path output of the space and context path, and then through themass normalization to balance the scale of the characteristics, finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector. Features of images in order to extract context information, we add attention to the context path refinement module. Attention modules respectively from channel dimension and space dimension to weighted images, in order to obtain more effective information. Experiments show that our method is better than the existing technology in the quality and quantity of themethod, and further to expand our network to other inpainting networks, in order to achieve consistent performance improvements. [ABSTRACT FROM AUTHOR]

3D (Three-Dimensional) scene inpainting aims to remove objects from scenes and generate visually plausible regions to fill the hollows. Leveraging the foundation of NeRF (Neural Radiance Field), considerable advancements have been achieved in the realm of 3D scene inpainting. However, prevalent issues persist: primarily, the presence of inconsistent 3D details across different viewpoints and occlusion losses of real background details in inpainted regions. This paper presents a NeRF-based inpainting approach using uncertainty estimation that formulates mask and uncertainty branches for consistency enhancement. In the initial training, the mask branch learns a 3D-consistent representation from inaccurate input masks, and after background rendering, the background regions can be fully exposed to the views. The uncertainty branch learns the visibility of spatial points by modeling them as Gaussian distributions, generating variances to identify regions to be inpainted. During the inpainting training phase, the uncertainty branch measures 3D consistency in the inpainted views and calculates the confidence from the variance as dynamic weights, which are used to balance the color and adversarial losses to achieve 3D-consistent inpainting with both the structure and texture. The results were evaluated on datasets such as Spin-NeRF and NeRF-Object-Removal. The proposed approach outperformed the baselines in inpainting metrics of LPIPS and FID, and preserved more spatial details from real backgrounds in multi-scene settings, thus achieving 3D-consistent restoration. [ABSTRACT FROM AUTHOR]

Recovery or filling in of missing pixels in damaged images is a challenging problem known as image inpainting. Many currently used techniques still suffer from artifacts and other visual defects. In the proposed inpainting approach, the authors have combined wavelet patch-based and group-based sparse representation learning so as to exploit the benefits of (a) multiresolution decomposition using wavelets, (b) sparsity and (c) coherence. The proposed method creates multiple dictionaries employing adaptive K-SVD (K-singular value decomposition) on wavelet decomposed components. The method also creates another dictionary employing PCA (principal component analysis) on group-based image patches. Finally, to accomplish the operation of inpainting, dictionaries of both types are integrated using the ADMM (alternating direction method of multipliers). The proposed method has been tested on images with varying degrees of degradation in terms of the percentage of missing pixels or blocks. We have rated the performance and compared proposed method with other state-of-the-art inpainting methods based on measures like the peak signal-to-noise ratio, the structural similarity index measure, and the figure of merit. The high values of the performance measures establish the efficacy of the proposed method. [ABSTRACT FROM AUTHOR]

Recently, deep learning-based image inpainting methods have made great strides in reconstructing damaged regions. However, these methods often struggle to produce satisfactory results when dealing with missing images with large holes, leading to distortions in the structure and blurring of textures. To address these problems, we combine the advantages of transformers and convolutions to propose an image inpaintingmethod that incorporates edge priors and attention mechanisms. The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details. This method divides the inpainting task into two phases: edge prediction and image inpainting. Specifically, in the edge prediction phase, a transformer architecture is designed to combine axial attention with standard self-attention. This design enhances the extraction capability of global structural features and location awareness. It also balances the complexity of self-attention operations, resulting in accurate prediction of the edge structure in the defective region. In the image inpainting phase, a multi-scale fusion attention module is introduced. This module makes full use of multi-level distant features and enhances local pixel continuity, thereby significantly improving the quality of image inpainting. To evaluate the performance of our method, comparative experiments are conducted on several datasets, including CelebA, Places2, and Facade. Quantitative experiments show that our method outperforms the other mainstream methods. Specifically, it improves Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity IndexMeasure (SSIM) by 1.141-3.234 db and 0.083-0.235, respectively.Moreover, it reduces Learning Perceptual Image Patch Similarity (LPIPS) and Mean Absolute Error (MAE) by 0.0347-0.1753 and 0.0104-0.0402, respectively. Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details. Furthermore, ourmodel exhibits impressive performance in terms of the number of parameters, memory cost, and testing time. [ABSTRACT FROM AUTHOR]

Image inpainting refers to filling in unknown regions with known knowledge, which is in full flourish accompanied by the popularity and prosperity of deep convolutional networks. Current inpainting methods have excelled in completing small-sized corruption or specifically masked images. However, for large-proportion corrupted images, most attention-based and structure-based approaches, though reported with state-of-the-art performance, fail to reconstruct high-quality results due to the short consideration of semantic relevance. To relieve the above problem, in this paper, we propose a novel image inpainting approach, namely cascading blend network (CBNet), to strengthen the capacity of feature representation. As a whole, we introduce an adjacent transfer attention (ATA) module in the decoder, which preserves contour structure reasonably from the deep layer and blends structure-texture information from the shadow layer. In a coarse to delicate manner, a multi-scale contextual blend (MCB) block is further designed to felicitously assemble the multi-stage feature information. In addition, to ensure a high qualified hybrid of the feature information, extra deep supervision is applied to the intermediate features through a cascaded loss. Qualitative and quantitative experiments on the Paris StreetView, CelebA, and Places2 datasets demonstrate the superior performance of our approach compared with most state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we present a numerical linear algebra analytical study of some schemes for the Bertozzi–Esedoglu–Gillette–Cahn–Hilliard equation. Both 1D and 2D finite difference discretizations in space are proposed with semi-implicit and implicit discretizations on time. We prove that our proposed numerical solutions converge to continuous solutions. [ABSTRACT FROM AUTHOR]

Image inpainting with the large missing blocks is tremendous challenging to achieve visual consistency and realistic effect. In this paper, an Adaptive Visual field Multi-scale Generative Adversarial Networks (denoted as GANs) Image Inpainting based on Coordinate-attention (denoted as AVMGC) is proposed. Firstly, an encoder with deformable convolutional networks in the generator of multi-scale generative adversarial networks is designed to expand the local vision field of network sampling adaptively in the image inpainting, which improves the local visual consistency of the image inpainting. Secondly, in order to expand the receptive field of the deep network and the global visual field, AVMGC combines the coordinate-attention mechanism with the convolutional layers, aiming to capture the direction-aware and position-sensitive information by cross-channel, which helps models to more accurately locate and recognize the objects of interest and generate globally consistent geometric contour in the image inpainting. In particular, instance normalization is introduced to the mutil-scale discriminator for transferring the statistic information of the feature maps and aims to keep the style of the original images. Extensive experiments conducted on public datasets prove that the proposal algorithms have the qualitative performance and outperform the baselines. [ABSTRACT FROM AUTHOR]

The most image inpainting algorithms often have existed problems such as blurred image, texture distortion and semantic inaccuracy, and the image inpainting effect is limited for images with large missing regions and resolution level. To solve above problems, the paper proposes an improved two-stage image inpainting network based on parallel network and contextual attention. Firstly, the improved deep residual network is used to perform generative pixels filling on the missing area, and the first-stage adversarial network is used to complete the edges information. Then, the color features of the filling map are extracted, the edge map is fused and complemented, and the fusion map is used as the conditional label of the second-stage adversarial network. Finally, the image repairing result has obtained through the two-stage network with the contextual attention module. The experiments on public datasets can show that the proposed algorithm can obtain a more realistic repairing effect. [ABSTRACT FROM AUTHOR]

Objective: Inpainting is a technique for fixing or removing undesired areas of an image. Methods: In present scenario, image plays a vital role in every aspect such as business images, satellite images, and medical images and so on. Results and Conclusion: This paper presents a comprehensive review of past traditional image inpainting methods and the present state-of-the-art deep learning methods and also detailed the strengths and weaknesses of each to provide new insights in the field. [ABSTRACT FROM AUTHOR]