Search

Group sparse residual constraint model with nonlocal priors (GSRC-NLP) has made great success in image restoration and produce state-of-the-art performance, realized through reducing the group sparsity residual. In the GSRC-NLP model, L 1 norm is used to reduce the group sparsity residual. However, the L 1 norm penalty has been known to lead to the over-shrinkage of the large sparse coefficients. In this paper, we utilize the adaptive correction procedure to reduce excessive penalties on large coefficient values while improving group sparsity. The proposed corrected group sparse residual constraint model (CGSRC) can improve the sparsity of the group sparsity residual, which lead to better performance in image restoration. We apply the iterative shrinkage/thresholding and the alternating direction method of multipliers to solve the proposed models. In addition, we study the properties of the proposed CGSRC model including the existence and uniqueness of the solution as well as the convergence of the proposed algorithms. Experimental results on image denoising and image inpainting show that the proposed models outperform several state-of-the-art image denoising and image inpainting methods in terms of both objective and perceptual quality metrics. [ABSTRACT FROM AUTHOR]

In the domain of handwritten character recognition, inpainting occluded offline characters is essential. Relying on the remarkable achievements of transformers in various tasks, we present a novel framework called “Enhanced Inpainting with Multi-head Attention and stacked long short-term memory (LSTM) Network” (E-Inpaint). This framework aims to restore occluded offline handwriting while capturing its online signal counterpart, enriched with dynamic characteristics. The proposed approach employs Convolutional Neural Network (CNN) and Multi-Layer Perceptron (MLP) in order to extract essential hidden features from the handwriting image. These features are then decoded by stacked LSTM with Multi-head Attention, achieving the inpainting process and generating the online signal corresponding to the uncorrupted version. To validate our work, we utilize the recognition system Beta-GRU on Latin, Indian, and Arabic On/Off dual datasets. The obtained results show the efficiency of using stacked-LSTM network with multi-head attention, enhancing the quality of the restored image and significantly improving the recognition rate using the innovative Beta-GRU system. Our research mainly highlights the potential of E-Inpaint in enhancing handwritten character recognition systems. [ABSTRACT FROM AUTHOR]

Hyperspectral images are typically composed of hundreds of narrow and contiguous spectral bands, each containing information regarding the material composition of the imaged scene. However, these images can be affected by various sources of noise, distortions, or data loss, which can significantly degrade their quality and usefulness. This paper introduces a convergent guaranteed algorithm, LRS-PnP-DIP(1-Lip), which successfully addresses the instability issue of DHP that has been reported before. The proposed algorithm extends the successful joint low-rank and sparse model to further exploit the underlying data structures beyond the conventional and sometimes restrictive unions of subspace models. A stability analysis guarantees the convergence of the proposed algorithm under mild assumptions, which is crucial for its application in real-world scenarios. Extensive experiments demonstrate that the proposed solution consistently delivers visually and quantitatively superior inpainting results, establishing state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Maize leaf area offers valuable insights into physiological processes, playing a critical role in breeding and guiding agricultural practices. The Azure Kinect DK possesses the real-time capability to capture and analyze the spatial structural features of crops. However, its further application in maize leaf area measurement is constrained by RGB–depth misalignment and limited sensitivity to detailed organ-level features. This study proposed a novel approach to address and optimize the limitations of the Azure Kinect DK through the multimodal coupling of RGB-D data for enhanced organ-level crop phenotyping. To correct RGB–depth misalignment, a unified recalibration method was developed to ensure accurate alignment between RGB and depth data. Furthermore, a semantic information-guided depth inpainting method was proposed, designed to repair void and flying pixels commonly observed in Azure Kinect DK outputs. The semantic information was extracted using a joint YOLOv11-SAM2 model, which utilizes supervised object recognition prompts and advanced visual large models to achieve precise RGB image semantic parsing with minimal manual input. An efficient pixel filter-based depth inpainting algorithm was then designed to inpaint void and flying pixels and restore consistent, high-confidence depth values within semantic regions. A validation of this approach through leaf area measurements in practical maize field applications—challenged by a limited workspace, constrained viewpoints, and environmental variability—demonstrated near-laboratory precision, achieving an MAPE of 6.549%, RMSE of 4.114 cm2, MAE of 2.980 cm2, and R2 of 0.976 across 60 maize leaf samples. By focusing processing efforts on the image level rather than directly on 3D point clouds, this approach markedly enhanced both efficiency and accuracy with the sufficient utilization of the Azure Kinect DK, making it a promising solution for high-throughput 3D crop phenotyping. [ABSTRACT FROM AUTHOR]

Gaze tracking technology in HMDs (Head-Mounted Displays) suffers from decreased accuracy due to highlight reflection noise from users' glasses. To address this, we present a denoising method which directly pinpoints the noisy regions through advanced segmentation models and then fills the flawed regions through advanced image inpainting algorithms. In segmentation stage, we introduce a novel model based on the recently proposed segmentation large model SAM (Segment Anything Model), called SF-SAM-Adapter (Spatial and Frequency aware SAM Adapter). It injects prior knowledge regarding the strip-like shaped in spatial and high-frequency in frequency of reflection noise into SAM by integrating specially designed trainable adapter modules into the original structure, while retaining the expressive power of the large model and better adapting to the downstream task. We achieved segmentation metrics of IoU (Intersection over Union) = 0.749 and Dice = 0.853 at a memory size of 13.9 MB, outperforming recent techniques, including UNet, UNet++, BATFormer, FANet, MSA, and SAM2-Adapter. In inpainting, we employ the advanced inpainting algorithm LAMA (Large Mask inpainting), resulting in significant improvements in gaze tracking accuracy by 0.502°, 0.182°, and 0.319° across three algorithms. The code and datasets used in current study are available in the repository: https://github.com/leiting5297/SF-SAM-Adapter.git. • Network employs large model and Adapter technology. • Architecture integrates prior spatial and frequency Knowledge. • Reflection noise removed, improving eye tracking accuracy. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive study of the application of AI-driven inpainting techniques to the restoration of historical photographs of the Czech city Most, with a focus on restoration and reconstructing the lost architectural heritage. The project combines state-of-the-art methods, including generative adversarial networks (GANs), patch-based inpainting, and manual retouching, to restore and enhance severely degraded images. The reconstructed/restored photographs of the city Most offer an invaluable visual representation of a city that was largely destroyed for industrial purposes in the 20th century. Through a series of blind and informed user tests, we assess the subjective quality of the restored images and examine how knowledge of edited areas influences user perception. Additionally, this study addresses the technical challenges of inpainting, including computational demands, interpretability, and bias in AI models. Ethical considerations, particularly regarding historical authenticity and speculative reconstruction, are also discussed. The findings demonstrate that AI techniques can significantly contribute to the preservation of cultural heritage, but must be applied with careful oversight to maintain transparency and cultural integrity. Future work will focus on improving the interpretability and efficiency of these methods, while ensuring that reconstructions remain historically and culturally sensitive. [ABSTRACT FROM AUTHOR]

With the recent rise in the development of deep learning techniques, image inpainting—the process of restoring missing or corrupted regions in images—has witnessed significant advancements. Although state-of-the-art models are effective, they often fail to inpaint complex missing areas, especially when handwritten occlusions are present in the image. To address this issue, an image inpainting model based on a residual CycleGAN is proposed. The generator takes as input the image occluded by handwritten missing patches and generates a restored image, which the discriminator then compares with the original ground truth image to determine whether it is real or fake. An adversarial trade-off between the generator and discriminator motivates the model to improve its training and produce a superior reconstructed image. Extensive experiments and analyses confirm that the proposed method generates inpainted images with superior visual quality and outperforms state-of-the-art deep learning approaches. [ABSTRACT FROM AUTHOR]

The integration of Internet of Medical Things (IoMT) technology has revolutionized healthcare, allowing rapid access to medical images and enhancing remote diagnostics in telemedicine. However, this advancement raises serious cybersecurity concerns, particularly regarding unauthorized access and data integrity. This paper presents a novel, user-friendly, visible watermarking scheme for medical images—Visual and User-Friendly Medical Image Watermarking Scheme (VUF-MIWS)—designed to secure medical image ownership while maintaining usability for diagnostic purposes. VUF-MIWS employs a unique combination of inpainting and data hiding techniques to embed hospital logos as visible watermarks, which can be removed seamlessly once image authenticity is verified, restoring the image to its original state. Experimental results demonstrate the scheme's robust performance, with the watermarking process preserving critical diagnostic information with high fidelity. The method achieved Peak Signal-to-Noise Ratios (PSNR) above 70 dB and Structural Similarity Index Measures (SSIM) of 0.99 for inpainted images, indicating minimal loss of image quality. Additionally, VUF-MIWS effectively restored the ROI region of medical images post-watermark removal, as verified through test cases with restored watermarked regions matching the original images. These findings affirm VUF-MIWS's suitability for secure telemedicine applications. [ABSTRACT FROM AUTHOR]

Employing low tensor rank decomposition in image inpainting has attracted increasing attention. This study exploited novel tensor arrangement schemes to transform an image (a low-order tensor) to a higher-order tensor without changing the total number of pixels. The developed arrangement schemes enhanced the low rankness of images under three tensor decomposition methods: matrix SVD, tensor train (TT) decomposition, and tensor singular value decomposition (t-SVD). By exploiting the schemes, we solved the image inpainting problem with three low-rank constrained models that use the matrix rank, TT rank, and tubal rank as constrained priors. The tensor tubal rank and tensor train multi-rank were developed from t-SVD and TT decomposition, respectively. Then, ADMM algorithms were efficiently exploited for solving the three models. Experimental results demonstrate that our methods are effective for image inpainting and superior to numerous close methods. [ABSTRACT FROM AUTHOR]

The elastic bending energy model is commonly used to describe the shape transformation of biological lipid vesicles, making it a classical phase field model. In this paper, by coupling the elastic bending energy with the total variation (TV) regularization, we develop an elastic bending-TV model for image inpainting. By solving the energy minimization problem of this model, we obtain the results for image processing. We adopt an operator splitting method for the model and the numerical scheme involves the introduction of two vector- and scalar-valued functions to reconstruct this functional. The energy minimization problem is transformed into finding the steady state solution of artificial time-dependent PDE systems. At each fractional step, we can find either a closed-form solution or being solved by an efficient algorithm, which is a very robust and stable algorithm. Experimental results validate the superiority of our model and the effectiveness of the scheme for image inpainting. [ABSTRACT FROM AUTHOR]

ABSTRACT The task of tensor (matrix) completion has been widely used in the fields of computer vision and image processing, etc. To achieve the completion, the existing methods are mostly based on singular value decomposition of the real tensors and nuclear norm minimization. However, the real tensor completion methods cannot simultaneously maintain color channel correlation and evolution robustness of color video frames, and they need high computational costs to handle the high‐dimensional data. Hence they have some limitations in model generalization ability and computational efficiency. In this article, a new completion method for the quaternion tensor (matrix) is explored via the QR decomposition and the definition of novel quaternion tensor norm, which can well balance the model generalization ability and efficiency, and the performance of the completion method has been substantially improved. Numerical experiments on color images and videos prove the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

AbstractBackgroundObjectivesMaterials and methodsResultsConclusions and significanceOptimizing the educational experience of trainees in the operating room is important; however, ear anatomy and otologic surgery are challenging for trainees to grasp. Viewing otologic surgeries often involves limitations related to video quality, such as visual disturbances and instability.We aimed to (1) improve the quality of surgical videos (tympanomastoidectomy [TM]) by using artificial intelligence (AI) techniques and (2) evaluate the effectiveness of processed videos through a questionnaire-based assessment from trainees.We conducted prospective study using video inpainting and stabilization techniques processed by AI. In each study set, we enrolled 21 trainees and asked them to watch processed videos and complete a questionnaire.Surgical videos with the video inpainting technique using the implicit neural representation (INR) model were found to be the most helpful for medical students (0.79 ± 0.58) in identifying bleeding focus. Videos with the stabilization technique via point feature matching were more helpful for low-grade residents (0.91 ± 0.12) and medical students (0.78 ± 0.35) in enhancing overall visibility and understanding surgical procedures.Surgical videos using video inpainting and stabilization techniques with AI were beneficial for educating trainees, especially participants with less anatomical knowledge and surgical experience. [ABSTRACT FROM AUTHOR]

Nowadays, the Hough transform (HT) is a tool extensively used in image processing and computer vision. It finds applications ranging from line detection in images to tomographic reconstruction. Due to their numerous industrial applications, which comprise execution not only on remote servers but also on less powerful embedded devices and within the Internet of Things (IoT) paradigm, fast algorithms for computing the HT (fast HT, or FHT) that efficiently utilize computational memory are in high demand. In-place algorithms, which use memory allocated for the input data array and may employ a small amount of additional memory for intermediate calculations, are recognized to have such characteristics. For images with widths that are powers of two, such an algorithm is known; it is the in-place version of the de facto standard Brady–Yong algorithm. However, in practice, it is often needed to compute the FHT for images with arbitrary widths, to which the Brady–Yong algorithm is not applicable. In this paper, we propose an in-place algorithm. It is a modification of the out-of-place algorithm for computing the FHT for images of arbitrary width previously presented in the literature. We justify correctness of the algorithm and show that the result of processing an image of an arbitrary width matches the output of the algorithm. We demonstrate that the algorithm allocates a significantly smaller data array at each recursion step compared to the algorithm: at each recursion step, it requires an array of size no greater than , rather than , assuming that the input image has a shape. We also develop a nonrecursive version of the algorithm, which is called the - algorithm. The auxiliary space complexity of the - algorithm is proved to be , while the - algorithm, a nonrecursive version of the algorithm, demonstrates the auxiliary space complexity of , where and denote the dimensions of the input image. Experimental results show that the - algorithm implemented in C/C is 45% faster than its out-of-place counterpart, - . The and - algorithms are also implemented in Python within an open-access adrt library, which the readers may find useful in their research. [ABSTRACT FROM AUTHOR]

Image inpainting refers to restoring the damaged areas of an image using the remaining available information. In recent years, deep learning-based image inpainting has been extensively explored and shown remarkable performance, among which the parallel prior embedding methods have the advantages of few network parameters and relatively low training difficulty. However, most methods use a single prior that is unable to provide sufficient guidance information. Hence, they are unable to generate high-quality, realistic, and vivid images. Fusion labels are effective priors that could provide more meaningful guidance information for inpainting. Meanwhile, attention mechanisms can focus on effective features and establish long-range correlations, which is helpful to refine texture details. Therefore, this paper proposes a parallel prior embedding image inpainting method based on fusion structure information (FSI) and pixelwise attention. A FSI module using the color structure and edge information is designed to update structure features and image features alternately, pixelwise attention is utilized to refine image details, and a joint loss is applied to constrain model training. Extensive experiments are conducted on multiple public datasets, and the results show that the proposed method achieves generally superior performance over several compared methods in terms of several quantitative metrics and qualitative analysis. [ABSTRACT FROM AUTHOR]

Aiming at the problem of increasing image texture disorder and edge structure blurring and loss during the inpainting process of tile component images in Chinese historical buildings, this paper proposed a generative adversarial tile image in-painting method based on the interaction of high-order texture and structural features. Firstly, the method used an encoderdecoder as the basic architecture to encode and decode texture and structure features of the damaged image and its structure image. Secondly, this paper designed the recursive partial convolutional layer in the encoder and decoder to enhance the inter- action between the high-order and low-order features of the image, and to improve the model's ability of characterizing the texture and structural details of the tile image. Finally, this paper designed the feature fusion layer to realize the information fusion and detail enhancement of texture and structure feature maps. For typical tile components, this paper constructed a tile image dataset containing image type, pattern type and text type. This paper carried out experimental validation on the constructed dataset, and the experimental results show that the proposed method in this paper exhibits more excellent inpainting results in terms of both subjective feeling and objective evaluation indexes compared with commonly used algorithms. [ABSTRACT FROM AUTHOR]

Recent studies have shown that image structure can significantly facilitate image inpainting. However, current approaches mostly explore structure prior without considering its guidance to texture reconstruction, leading to performance degradation. To solve this issue, we propose a two-stream Tracking Structure Transformer (TSFormer), including structure target stream and image completion stream, to capture the synchronous and dynamic interplay between structure and texture. Specifically, we first design a structure enhancement module to restore the Histograms of Oriented Gradient (HOG) and the edge of an input image in a sketch space, which forms the input of the structure target stream. Meanwhile, in the image completion stream, we design a channel-space parallel-attention component to facilitate the efficient co-learning of channel and spatial visual cues. To build a bridge between the two streams, we further develop a structure-texture cross-attention module, wherein both structure and texture are synchronously extracted through self-attention, and texture extraction is implemented by dynamically tracking the structure in a cross-attention fashion, enabling the capture of the intricate interaction between structure and texture. Extensive experiments evaluated on three benchmark datasets, including CelebA, Places2, and Paris StreetView, demonstrate that the proposed TSFormer achieves state-of-the-art performance compared to its competitors. The code is available at https://github.com/GZHU-DVL/TSFormer. [ABSTRACT FROM AUTHOR]

The majority of existing face inpainting methods primarily focus on generating a single result that visually resembles the original image. The generation of diverse and plausible results has emerged as a new branch in image restoration, often referred to as "Pluralistic Image Completion". However, most diversity methods simply use random latent vectors to generate multiple results, leading to uncontrollable outcomes. To overcome these limitations, we introduce a novel architecture known as the Reference-Guided Directional Diverse Face Inpainting Network. In this paper, instead of using a background image as reference, which is typically used in image restoration, we have used a face image, which can have many different characteristics from the original image, including but not limited to gender and age, to serve as a reference face style. Our network firstly infers the semantic information of the masked face, i.e., the face parsing map, based on the partial image and its mask, which subsequently guides and constrains directional diverse generator network. The network will learn the distribution of face images from different domains in a low-dimensional manifold space. To validate our method, we conducted extensive experiments on the CelebAMask-HQ dataset. Our method not only produces high-quality oriented diverse results but also complements the images with the style of the reference face image. Additionally, our diverse results maintain correct facial feature distribution and sizes, rather than being random. Our network has achieved SOTA results in face diverse inpainting when writing. Code will is available at https://github.com/nothingwithyou/RDFINet. [ABSTRACT FROM AUTHOR]

In the process of gradually upgrading aquaculture to the information and intelligence industries, it is usually necessary to collect images of underwater fish. In practical work, the quality of underwater images is often affected by water clarity and light refraction, resulting in most fish images not fully displaying the entire fish body. Image inpainting helps infer the occluded fish image information based on known images, thereby better identifying and analyzing fish populations. When using existing image inpainting methods for underwater fish images, limited by the small datasets available for training, the results were not satisfactory. Lightweight Multi-scales Feature Diffusion (LMF-Diffusion) is proposed to achieve results closer to real images when dealing with image inpainting tasks from small datasets. LMF-Diffusion is based on guided diffusion and flexibly extracts features from images at different scales, effectively capturing remote dependencies among pixels, and it is more lightweight, making it more suitable for practical deployment. Experimental results show that our architecture uses only 48.7 % of the parameter of the guided diffusion model and produces inpainting results closer to real images in our dataset. Experimental results show that LMF-Diffusion enables the Repaint method to exhibit better performance in underwater fish image inpainting. Underwater fish image inpainting results obtained using our LMF-Diffusion model outperform those produced by current popular image inpainting methods. [ABSTRACT FROM AUTHOR]

Crafting a single, versatile physics-based controller that can breathe life into interactive characters across a wide spectrum of scenarios represents an exciting frontier in character animation. An ideal controller should support diverse control modalities, such as sparse target keyframes, text instructions, and scene information. While previous works have proposed physically simulated, scene-aware control models, these systems have predominantly focused on developing controllers that each specializes in a narrow set of tasks and control modalities. This work presents MaskedMimic, a novel approach that formulates physics-based character control as a general motion inpainting problem. Our key insight is to train a single unified model to synthesize motions from partial (masked) motion descriptions, such as masked keyframes, objects, text descriptions, or any combination thereof. This is achieved by leveraging motion tracking data and designing a scalable training method that can effectively utilize diverse motion descriptions to produce coherent animations. Through this process, our approach learns a physics-based controller that provides an intuitive control interface without requiring tedious reward engineering for all behaviors of interest. The resulting controller supports a wide range of control modalities and enables seamless transitions between disparate tasks. By unifying character control through motion inpainting, MaskedMimic creates versatile virtual characters. These characters can dynamically adapt to complex scenes and compose diverse motions on demand, enabling more interactive and immersive experiences. [ABSTRACT FROM AUTHOR]

We present a novel and flexible learning-based method for generating tileable image sets. Our method goes beyond simple self-tiling, supporting sets of mutually tileable images that exhibit a high degree of diversity. To promote diversity we decouple structure from content by foregoing explicit copying of patches from an exemplar image. Instead we leverage the prior knowledge of natural images and textures embedded in large-scale pretrained diffusion models to guide tile generation constrained by exterior boundary conditions and a text prompt to specify the content. By carefully designing and selecting the exterior boundary conditions, we can reformulate the tile generation process as an inpainting problem, allowing us to directly employ existing diffusion-based inpainting models without the need to retrain a model on a custom training set. We demonstrate the flexibility and efficacy of our content-aware tile generation method on different tiling schemes, such as Wang tiles, from only a text prompt. Furthermore, we introduce a novel Dual Wang tiling scheme that provides greater texture continuity and diversity than existing Wang tile variants. [ABSTRACT FROM AUTHOR]

While high-quality texture maps are essential for realistic 3D asset rendering, few studies have explored learning directly in the texture space, especially on large-scale datasets. In this work, we depart from the conventional approach of relying on pre-trained 2D diffusion models for testtime optimization of 3D textures. Instead, we focus on the fundamental problem of learning in the UV texture space itself. For the first time, we train a large diffusion model capable of directly generating high-resolution texture maps in a feed-forward manner. To facilitate efficient learning in high-resolution UV spaces, we propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. Leveraging this architectural design, we train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images. Once trained, our model naturally supports various extended applications, including text-guided texture inpainting, sparse-view texture completion, and text-driven texture synthesis. The code is available at https://github.com/CVMI-Lab/TEXGen. [ABSTRACT FROM AUTHOR]

Two-stage traffic marking digital diagnostics method based on deep learning is proposed, that is, traffic marking inpainting is performed first, and diagnosis is performed later, to ensure the data quality of digital diagnosis. Firstly, a arrow damaged traffic marking dataset is collected and created. In inpainting stage, Data-driven traffic marking inpainting model (TMIN-GAN) based on generative adversarial network is constructed. By inpainting, the damaged traffic marking and the corresponding repaired complete traffic marking composition data pairs are obtained. Subsequently, classification of the degree of impairment according to visual recognizability. And the data pairs are subjected to comparison using the Learned Perceptual Image Patch Similarity (LPIPS) indicator. For training of TMIN-GAN model, FE-Mask R-CNN is adopted to automatically label the dataset by relying on the mask generated by instance segmentation. The experimental results demonstrate that traffic marking inpainting by the TMIN-GAN, compared by hand, reduces inpainting time from 10 s to milliseconds. This provides an excellent foundation for damage diagnosis. In TMIN-GAN training, the difference between PSNR value of the mask based on FE-Mask R-CNN and that of the manual annotation is only 2.35%. This demonstrates the feasibility of automatic annotation based on FE-Mask R-CNN masks. Compared by PSNR and SSIM evaluation indicators, the rationality and superiority of using LPIPS for traffic marking damage diagnosis is demonstrated and get the range of divided damage levels. [ABSTRACT FROM AUTHOR]

With the rapid development of stereoscopic display technology, how to generate high-quality virtual view images has become the key in the applications of 3D video, 3D TV and virtual reality. The traditional virtual view rendering technology maps the reference view into the virtual view by means of 3D transformation, but when the background area is occluded by the foreground object, the content of the occluded area cannot be inferred. To solve this problem, we propose a virtual view acquisition technique for complex scenes of monocular images based on a layered depth image (LDI). Firstly, the depth discontinuities of the edge of the occluded area are reasonably grouped by using the multilayer representation of the LDI, and the depth edge of the occluded area is inpainted by the edge inpainting network. Then, the generative adversarial network (GAN) is used to fill the information of color and depth in the occluded area, and the inpainting virtual view is generated. Finally, GAN is used to optimize the color and depth of the virtual view, and the high-quality virtual view is generated. The effectiveness of the proposed method is proved by experiments, and it is also applicable to complex scenes. [ABSTRACT FROM AUTHOR]

In response to the inadequate utilization of prior information in current mural inpainting processes, leading to issues such as semantically unreliable inpaintings and the presence of artifacts in the inpainting area, a Dunhuang mural inpainting method based on reference guidance and multi‐scale feature fusion is proposed. First, the simulated broken mural, the mask image, and the reference mural are input into the model to complete the multi‐level embedding of patches and align the multi‐scale fine‐grained features of damaged murals and reference murals. Following the patch embedding module, a hybrid residual module is added based on hybrid attention to fully extract mural features. In addition, by continuing the residual concatenation of outputs of the hierarchical embedding module improves the ability of the model to represent deeper features, and improves the robustness and generalisation of the model. Second, the encoded features are fed into the decoder to generate decoded features. Finally, the convolutional tail is employed to propagate them and complete the mural painting. Experimental validation on the Dunhuang mural dataset demonstrates that, compared to other algorithms, this model exhibits higher evaluation metrics in the inpainting of extensively damaged murals and demonstrates overall robustness. In terms of visual effects, the results of this model in the inpainting process exhibit finer textures, richer semantic information, more coherent edge structures, and a closer resemblance to authentic murals. [ABSTRACT FROM AUTHOR]

Traditional dental prosthetics require a significant amount of work, labor, and time. To simplify the process, a method to convert teeth scan images, scanned using an intraoral scanner, into 3D images for design was developed. Furthermore, several studies have used deep learning to automate dental prosthetic processes. Tooth images are required to train deep learning models, but they are difficult to use in research because they contain personal patient information. Therefore, we propose a method for generating virtual tooth images using image-to-image translation (pix2pix) and contextual reconstruction fill (CR-Fill). Various virtual images can be generated using pix2pix, and the images are used as training images for CR-Fill to compare the real image with the virtual image to ensure that the teeth are well-shaped and meaningful. The experimental results demonstrate that the images generated by the proposed method are similar to actual images. In addition, only using virtual images as training data did not perform well; however, using both real and virtual images as training data yielded nearly identical results to using only real images as training data. [ABSTRACT FROM AUTHOR]

The solution of inverse problems is of fundamental interest in medical and astronomical imaging, geophysics as well as engineering and life sciences. Recent advances were made by using methods from machine learning, in particular deep neural networks. Most of these methods require a huge amount of data and computer capacity to train the networks, which often may not be available. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We focus on the combination of model‐based and data‐driven methods by approximating just the image prior, also known as regularizer in the variational model. We review two methodically different approaches, namely optimizing the maximum log‐likelihood of the patch distribution, and penalizing Wasserstein‐like discrepancies of whole empirical patch distributions. From the point of view of Bayesian inverse problems, we show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods. We demonstrate the power of the methods in computed tomography, image super‐resolution, and inpainting. Indeed, the approach provides also high‐quality results in zero‐shot super‐resolution, where only a low‐resolution image is available. The article is accompanied by a GitHub repository containing implementations of all methods as well as data examples so that the reader can get their own insight into the performance. [ABSTRACT FROM AUTHOR]

This study presents a new image inpainting model based on U-Net and incorporating the Wasserstein Generative Adversarial Network (WGAN). The model uses skip connections to connect every encoder block to the corresponding decoder block, resulting in a strictly symmetrical architecture referred to as Symmetric Connected U-Net (SC-Unet). By combining SC-Unet with a GAN, the study aims to reconstruct images more effectively and seamlessly. The traditional discriminators only differentiate the entire image as true or false. In this study, the discriminator calculated the probability of each pixel belonging to the hole and non-hole regions, which provided the generator with more gradient loss information for image inpainting. Additionally, every block of SC-Unet incorporated a Dilated Convolutional Neural Network (DCNN) to increase the receptive field of the convolutional layers. Our model also integrated Multi-Head Self-Attention (MHSA) into selected blocks to enable it to efficiently search the entire image for suitable content to fill the missing areas. This study adopts the publicly available datasets CelebA-HQ and ImageNet for evaluation. Our proposed algorithm demonstrates a 10% improvement in PSNR and a 2.94% improvement in SSIM compared to existing representative image inpainting methods in the experiment. [ABSTRACT FROM AUTHOR]

Since performing simultaneous localization and mapping in dynamic environments is a challenging problem, conventional approaches have used preprocessing to detect and then remove movable objects from images. However, those methods create many holes in the places, where the movable objects are located, reducing the reliability of the estimated pose. In this paper, we propose a model with detailed classification criteria for moving objects and point cloud restoration to handle hole generation and pose errors. Our model includes a moving object segmentation network and an inpainting network with a light detection and ranging sensor. By providing residual images to the segmentation network, the model can classify idle and moving objects. Moreover, we propose a smoothness loss to ensure that the inpainting result of the model naturally connects to the existing background. Our proposed model uses the movable object's information in an idle state and the inpainted background to accurately estimate the sensor's pose. To use a ground truth dataset for inpainting, we created a new dataset using the CARLA simulation environment. We use our virtual datasets and the KITTI dataset to verify our model's performance. In a dynamic environment, our proposed model demonstrates a notable enhancement of approximately 24.7% in pose estimation performance compared to the previous method. [ABSTRACT FROM AUTHOR]

The rapid advancement of image inpainting tools, especially those aimed at removing artifacts, has made digital image manipulation alarmingly accessible. This paper proposes several innovative ideas for detecting inpainting forgeries based on a low-level noise analysis by combining Dual-Tree Complex Wavelet Transform (DT-CWT) for feature extraction with convolutional neural networks (CNN) for forged area detection and localization, and lastly by employing an innovative combination of texture segmentation with noise variance estimations. The DT-CWT offers significant advantages due to its shift-invariance, enhancing its robustness against subtle manipulations during the inpainting process. Furthermore, its directional selectivity allows for the detection of subtle artifacts introduced by inpainting within specific frequency bands and orientations. Various neural network architectures were evaluated and proposed. Lastly, we propose a fusion detection module that combines texture analysis with noise variance estimation to give the forged area. Also, to address the limitations of existing inpainting datasets, particularly their lack of clear separation between inpainted regions and removed objects—which can inadvertently favor detection—we introduced a new dataset named the Real Inpainting Detection Dataset. Our approach was benchmarked against state-of-the-art methods and demonstrated superior performance over all cited alternatives. [ABSTRACT FROM AUTHOR]

Based on the comparison of three established inpainting techniques, namely Criminisi, Beltamio, and Galerne and Leclaire, our study aimed to identify the most effective method for road restoration after extraction and detection using a Mathematical Morphology operators combined with hybrid techniques of digital image processing in remote sensing images. While all techniques were evaluated based on both visual analysis and quantitative metrics, the Criminisi approach emerged as the better choice. Despite introducing some additional noise, this technique demonstrated superior performance in terms of Completeness and overall Quality, achieving approximately 95.23% and 94.56%, respectively. Its ability to accurately reconstruct linear geometries while effectively removing existing noise highlighted its suitability for road restoration tasks. [ABSTRACT FROM AUTHOR]

The issue of distortion or misalignment in 3D modeling resulting from moving objects such as people and vehicles during UAV-obtained oblique image acquisition was addressed. An enhanced GAN-based image restoration technique was proposed, which modifies the GAN network by incorporating a U-Net architecture in the generator, fortified with a dual-channel attention mechanism via CBAM in the connecting layers, thereby enhancing the technique's capability for restoring local image details. The discriminator was augmented with the VGG16 model and the SE-Net channel attention mechanism has been undertaken to ensure the high fidelity of generated images in the present approach. Image analysis and processing were carried out using Context Capture software, facilitating automatic 3D modeling. This methodology enables proactive removal of moving entities, such as people and vehicles, from high resolution, extensive oblique imagery, thus minimizing their detrimental effects on subsequent 3D modeling and enhancing model accuracy. The presented algorithm demonstrates superior performance compared to conventional GAN and WGAN-GP networks, exhibiting increases of 3. 329 96 dB and 0. 097 9 in PSNR values and 2. 288 94 dB and 0. 047 8 in SSIM indices, respectively. Moreover, through comparison with generated 3D models, the method effectively reduces geometric deformation and road texture mapping errors, leading to heightened model precision. [ABSTRACT FROM AUTHOR]

Many large-scale regularized inverse problems in imaging such as image restoration and reconstruction can be modeled as a generic objective function involves sum of nonsmooth but proximable terms, which are usually linear-operator-coupled. For the solution of these problems, a parallel linearized alternating direction method of multipliers (PLADMM) is proposed in this paper. At each step of the proposed algorithm, the proximity operators of the nondifferential terms are called individually. This leads to a highly parallel algorithm structure, where most sub-steps can be simultaneously solved. Profiting from the linearization step, the linear inverse operation is excluded. The convergence property of the proposed method is analyzed. The image deblurring, inpainting, and pMRI reconstruction experiments show that the proposed method has vast applicable vistas. Compared with the state-of-the-art methods, such as PADMM [21], FTVD-v4 [22], PPDS [33], FUSL [8], LADM [36], and ALADMM [27], it gains competitive results both in terms of quantitative indicators, such as PSNR or SSIM, and in terms of visual impression. [ABSTRACT FROM AUTHOR]

Visual aesthetics has always been an important area of computational vision, and researchers have continued exploring it. To further improve the performance of the image aesthetic evaluation task, we introduce a Transformer into the image aesthetic evaluation task. This paper pioneers a novel self-supervised image aesthetic evaluation model founded upon Transformers. Meanwhile, we expand the pretext task to capture rich visual representations, adding a branch for inpainting the masked images in parallel with the tasks related to aesthetic quality degradation operations. Our model’s refinement employs the innovative uncertainty weighting method, seamlessly amalgamating three distinct losses into a unified objective. On the AVA dataset, our approach surpasses the efficacy of prevailing self-supervised image aesthetic assessment methods. Remarkably, we attain results approaching those of supervised methods, even while operating with a limited dataset. On the AADB dataset, our approach improves the aesthetic binary classification accuracy by roughly 16% compared to other self-supervised image aesthetic assessment methods and improves the prediction of aesthetic attributes. [ABSTRACT FROM AUTHOR]

To enhance the appeal of residential real estate listings and captivate online customers, clean and visually convincing indoor scenes are highly desirable. In this research, we introduce an innovative image inpainting model designed to seamlessly replace undesirable elements within images of indoor residential spaces with realistic and coherent alternatives. While Generative Adversarial Networks (GANs) have demonstrated remarkable potential for removing unwanted objects, they can be resource-intensive and face difficulties in consistently producing high-quality outcomes, particularly when unwanted objects are scattered throughout the images. To empower small- and medium-sized businesses with a competitive edge, we present a novel GAN model that is resource-efficient and requires minimal training time using arbitrary mask generation and a novel half-perceptual loss function. Our GAN model achieves compelling results in removing unwanted elements from indoor scenes, demonstrating the capability to train within a single day using a single GPU, all while minimizing the need for extensive post-processing. [ABSTRACT FROM AUTHOR]

The authors propose Point'n Move, a method that achieves interactive scene object manipulation with exposed region inpainting. Interactivity here further comes from intuitive object selection and real‐time editing. To achieve this, Gaussian Splatting Radiance Field is adopted as the scene representation and its explicit nature and speed advantage are fully leveraged. Its explicit representation formulation allows to devise a 2D prompt points to 3D masks dual‐stage self‐prompting segmentation algorithm, perform mask refinement and merging, minimize changes, and provide good initialization for scene inpainting and perform editing in real‐time without per‐editing training; all lead to superior quality and performance. The method was tested by editing both forward‐facing and 360 scenes. The method is also compared against existing methods, showing superior quality despite being more capable and having a speed advantage. [ABSTRACT FROM AUTHOR]