Search

In recent years, Generative Adversarial Network (GAN) have shown great potential and achieved excellent performance on the task of generating optical images. However, when GAN is applied to Synthetic Aperture Radar (SAR) images, the differences in imaging mechanisms between SAR images and optical images make GAN susceptible to training instability and model collapse problems during training. In this paper, we propose a new end-to-end model called Dual Discriminator Modified Conditional Generative Adversarial Network (DDM-CGAN) to address these issues. First, two discriminators are designed to play an adversarial game against the generator in DDM-CGAN. One discriminator favors samples from real data, while the other rewards high scores for generated samples. Essentially, we designed a novel objective function by utilizing the dual discriminator to combine the respective advantages of alternative cost function and original cost function of the standard GAN. We theoretically prove that this objective function can optimize DDM-CGAN towards minimizing the Kullback–Leibler Divergence, thus avoiding the problem of non-convergence during model training. Second, we propose a modified gradient penalty, which makes the model training more stable. In addition, the introduction of a discriminative auxiliary classifier provides the generator with information about the target distribution, thus improving the diversity of the generated images. We perform comprehensive qualitative and quantitative experiments with the Gaofen-3 SAR image dataset. Our proposed DDM-CGAN is compared with the state-of-the-art SAR image generation methods. Experimental results demonstrate that the SAR images generated by DDM-CGAN achieves optimal results in terms of the similarity, statistical characteristics, and diversity. [ABSTRACT FROM AUTHOR]

In this paper we present a novel multi-attribute face manipulation method based on textual descriptions. Previous text-based image editing methods either require test-time optimization for each individual image or are restricted to single attribute editing. Extending these methods to multi-attribute face image editing scenarios will introduce undesired excessive attribute change, e.g., text-relevant attributes are overly manipulated and text-irrelevant attributes are also changed. In order to address these challenges and achieve natural editing over multiple face attributes, we propose a new decoupling training scheme where we use group sampling to get text segments from same attribute categories, instead of whole complex sentences. Further, to preserve other existing face attributes, we encourage the model to edit the latent code of each attribute separately via an entropy constraint. During the inference phase, our model is able to edit new face images without any test-time optimization, even from complex textual prompts. We show extensive experiments and analysis to demonstrate the efficacy of our method, which generates natural manipulated faces with minimal text-irrelevant attribute editing. Code and pre-trained model are available at https://github.com/hwang1996/ManiCLIP. [ABSTRACT FROM AUTHOR]

Although this research has made significant progress in image generation models, they still face issues such as insufficient diversity of generated images, poor quality of high-resolution images, and the need for a large amount of training data for model optimization. This paper studies poster design based on deep learning automatic image generation algorithm, using a recursive supervised image generation algorithm framework of generative adversarial networks for multi-view image generation and super-resolution generation tasks of small sample digital poster images. Various improvements have been proposed to enhance the performance of the GAN network model for poster design image generation tasks. Based on experimental research, this paper's model uses generative adversarial networks to distinguish randomly cropped low resolution and high-resolution poster blocks, ensuring that high-resolution posters maintain their original resolution canvas texture and brush strokes, effectively improving the automatic generation effect of poster images. The evaluation results show that the quantitative evaluation of the proposed algorithm model in knowledge management is distributed in a reasonable range, which indicates that the proposed algorithm model has good performance in knowledge management. The poster design model based on deep learning automatic image generation algorithm proposed in this paper has certain effects. In subsequent practice, the automatic image generation algorithm can be combined with practical needs to improve the efficiency and design effect of poster design. [ABSTRACT FROM AUTHOR]

Traditional computer vision techniques aim to extract meaningful information from images but often depend on manual feature engineering, making it difficult to handle complex real-world scenarios. Fractional calculus (FC), which extends derivatives to non-integer orders, provides a flexible way to model systems with memory effects and long-term dependencies, making it a powerful tool for capturing fractional rates of variation. Recently, neural networks (NNs) have demonstrated remarkable capabilities in learning complex patterns directly from raw data, automating computer vision tasks and enhancing performance. Therefore, the use of fractional calculus in neural network-based computer vision is a powerful method to address existing challenges by effectively capturing complex spatial and temporal relationships in images and videos. This paper presents a survey of fractional calculus neural network-based (FC NN-based) computer vision techniques for denoising, enhancement, object detection, segmentation, restoration, and NN compression. This survey compiles existing FFC NN-based approaches, elucidates underlying concepts, and identifies open questions and research directions. By leveraging FC's properties, FC NN-based approaches offer a novel way to improve the robustness and efficiency of computer vision systems. [ABSTRACT FROM AUTHOR]

The rapid and accurate classification of Panax notoginseng leaf diseases is vital for timely disease control and reducing economic losses. Recently, image classification algorithms have shown great promise for plant disease diagnosis, but dataset quantity and quality are crucial. Moreover, classifying P. notoginseng leaf diseases faces severe challenges, including the small features of anthrax and the strong similarity between round spot and melasma diseases. In order to address these problems, we have proposed an ECA-based diffusion model and Inception-SSNet for the classification of the six major P. notoginseng leaf diseases, namely gray mold, powdery mildew, virus infection, anthrax, melasma, and round spot. Specifically, we propose an image generation scheme, in which the lightweight attention mechanism, ECA, is used to capture the dependencies between channels for improving the dataset quantity and quality. To extract disease features more accurately, we developed an Inception-SSNet hybrid model with skip connection, attention feature fusion, and self-calibrated convolutional. These innovative methods enable the model to make better use of local and global information, especially when dealing with diseases with similar features and small targets. The experimental results show that our proposed ECA-based diffusion model FID reaches 42.73, compared with the baseline model, which improved by 74.71%. Further, we tested the classification model using the data set of P. notoginseng leaf disease generation, and the accuracy of 11 mainstream classification models was improved. Our proposed Inception-SSNet classification model achieves an accuracy of 97.04% on the non-generated dataset, which is an improvement of 0.11% compared with the baseline model. On the generated dataset, the accuracy reached 99.44%, which is an improvement of 1.02% compared to the baseline model. This study provides an effective solution for the monitoring of Panax notoginseng diseases. [ABSTRACT FROM AUTHOR]

Freehand sketch‐to‐image (S2I) is a challenging task due to the individualized lines and the random shape of freehand sketches. The multi‐class freehand sketch‐to‐image synthesis task, in turn, presents new challenges for this research area. This task requires not only the consideration of the problems posed by freehand sketches but also the analysis of multi‐class domain differences in the conditions of a single model. However, existing methods often have difficulty learning domain differences between multiple classes, and cannot generate controllable and appropriate textures while maintaining shape stability. In this paper, we propose a style‐guided multi‐class freehand sketch‐to‐image synthesis model, SMFS‐GAN, which can be trained using only unpaired data. To this end, we introduce a contrast‐based style encoder that optimizes the network's perception of domain disparities by explicitly modelling the differences between classes and thus extracting style information across domains. Further, to optimize the fine‐grained texture of the generated results and the shape consistency with freehand sketches, we propose a local texture refinement discriminator and a Shape Constraint Module, respectively. In addition, to address the imbalance of data classes in the QMUL‐Sketch dataset, we add 6K images by drawing manually and obtain QMUL‐Sketch+ dataset. Extensive experiments on SketchyCOCO Object dataset, QMUL‐Sketch+ dataset and Pseudosketches dataset demonstrate the effectiveness as well as the superiority of our proposed method. [ABSTRACT FROM AUTHOR]

Chinese landscape paintings, realistic landscape photographs, and oil paintings each possess unique artistic characteristics and painting features. Image-to-image translation between these three domains is an extremely challenging task. Existing image-to-image translation networks suffer from deficiencies in preserving content or conveying style, posing difficulties in achieving this task. To address this issue, we propose a novel two-stage multi-domain generative adversarial network approach (TMGAN). We add edge maps as additional guidance input and implement content control to better retain content information. In addition, we design the IOST (In/Out module for Style Transfer) module to better assist the style transfer task. By employing a clever design, we decompose the image translation task into two stages: content extraction and style injection. In the content extraction stage, TMGAN extracts high-resolution edge images from content images. In the style injection stage, TMGAN takes the high-resolution edge image as input and injects the specified style for generation. Notably, we accomplish this two-stage task using only a single multi-domain generator network. Extensive qualitative and quantitative experiments conducted against the baseline model validate the exceptional performance of TMGAN. Furthermore, to facilitate further research, we release MLHQ, a high-quality multi-domain landscape dataset. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence (AI), deepfakes, in which the face of one person is changed to another expression of the same person or a different person, have advanced. There is a need for countermeasures against crimes that exploit deepfakes. Methods to interfere with deepfake generation by adding an invisible weak adversarial signal to an image have been proposed. However, there is a problem: the weak signal can be easily removed by processing the image. In this paper, we propose trap signals that appear in response to a process that weakens adversarial signals. We also propose a new type of adversarial signal injection that allow us to reconstruct and change the original image as far as people do not feel strange by Denoising Diffusion Probabilistic Model (DDPM)-based Iterative Latent Variable Refinement. In our experiments with Star Generative Adversarial Network (StarGAN) trained with the CelebFaces Attributes (CelebA) Dataset, we demonstrate that the proposed approach achieves more robust proactive deepfake defense. [ABSTRACT FROM AUTHOR]

Stargardt atrophy and geographic atrophy (GA) represent pivotal endpoints in FDA-approved clinical trials. Predicting atrophy progression is crucial for evaluating drug efficacy. Fundus autofluorescence (FAF), the standard 2D imaging modality in these trials, has limitations in patient comfort. In contrast, spectral-domain optical coherence tomography (SD-OCT), a 3D imaging modality, is more patient friendly but suffers from lower image quality. This study has two primary objectives: (1) develop an efficient predictive modeling for the generation of future FAF images and prediction of future Stargardt atrophic (as well as GA) regions and (2) develop an efficient predictive modeling with advanced 3D OCT features at ellipsoid zone (EZ) for the comparative performance in the generation of future enface EZ maps and prediction of future Stargardt atrophic regions on OCT as on FAF. To achieve these goals, we propose two deep neural networks (termed ReConNet and ReConNet-Ensemble) with recurrent learning units (long short-term memory, LSTM) integrating with a convolutional neural network (CNN) encoder–decoder architecture and concurrent learning units integrated by ensemble/multiple recurrent learning channels. The ReConNet, which incorporates LSTM connections with CNN, is developed for the first goal on longitudinal FAF. The ReConNet-Ensemble, which incorporates multiple recurrent learning channels based on enhanced EZ enface maps to capture higher-order inherent OCT EZ features, is developed for the second goal on longitudinal OCT. Using FAF images at months 0, 6, and 12 to predict atrophy at month 18, the ReConNet achieved mean (±standard deviation, SD) and median Dice coefficients of 0.895 (±0.086) and 0.922 for Stargardt atrophy and 0.864 (±0.113) and 0.893 for GA. Using SD-OCT images at months 0 and 6 to predict atrophy at month 12, the ReConNet-Ensemble achieved mean and median Dice coefficients of 0.882 (±0.101) and 0.906 for Stargardt atrophy. The prediction performance on OCT images is comparably good to that on FAF. These results underscore the potential of SD-OCT for efficient and practical assessment of atrophy progression in clinical trials and retina clinics, complementing or surpassing the widely used FAF imaging technique. [ABSTRACT FROM AUTHOR]

The integration of Retrieval-Augmented Generation (RAG) with Large Language Models (LLMs) presents a transformative approach to generating personalized and contextually relevant images that cater to specific user preferences. This project aims to harness the synergistic potential of RAG and LLMs to develop a robust and scalable image generation pipeline that seamlessly blends state-of-the-art natural language processing with advanced computer vision techniques. The process begins by utilizing a RAG model, which combines the strengths of retrieval-based methods and generative models to produce high-quality images that are not only coherent with the input prompts but also enriched with context from external knowledge sources. Following the image generation, a dedicated preprocessing module is employed to resize and optimize the images, ensuring they meet the quality standards required for subsequent integration. The next critical phase involves the detection of human upper bodies in photographs using Haar Cascade classifiers, a machine learning-based approach known for its efficiency in real-time object detection. The accurate identification of the upper body regions is crucial for the next step, where the generated images are overlaid onto these detected regions using OpenCV, a powerful computer vision library. This integration ensures that the images are aligned precisely with the contours of the human body, creating a visually realistic and aesthetically pleasing effect. To facilitate user interaction and deployment, the entire process is encapsulated within a Flask framework, which serves as the backbone of the application's architecture. The Flask framework not only handles the backend processing, including API requests and image processing tasks, but also supports a userfriendly frontend interface, allowing users to interact with the system effortlessly. [ABSTRACT FROM AUTHOR]

Denoising diffusion probabilistic models (DDPM) have shown impressive performance in various domains as a class of deep generative models. In this article, we introduce the mixture noise-based DDPM (Mix-DDPM), which considers the Markov diffusion posterior as a Gaussian mixture model. Specifically, Mix-DDPM randomly selects a Gaussian component and then adds the chosen Gaussian noise, which can be demonstrated as a more efficient way to perturb the signals into a simple known distribution. We further define the reverse probabilistic model as a parameterized Gaussian mixture kernel. Due to the intractability in calculating the KL divergence between Gaussian mixture models, we derive a variational bound to maximize the likelihood, offering a concise formulation for optimizing the denoising model and valuable insights for designing the sampling strategies. Our theoretical derivation highlights that Mix-DDPM need only shift image which requires the inclusion of a global stochastic offset in both the diffusion and reverse processes, which can be efficiently implemented with just several lines of code. The global stochastic offset effectively fits a Gaussian mixture distribution enhancing the degrees of freedom of the entire diffusion model. Furthermore, we present three streamlined sampling strategies that interface with diverse fast dedicated solvers for diffusion ordinary differential equations, boosting the efficacy of image representation in the sampling phase and alleviating the issue of slow generation speed, thereby enhancing both efficiency and accuracy. Extensive experiments on benchmark datasets demonstrate the effectiveness of Mix-DDPM and its superiority over the original DDPM. [ABSTRACT FROM AUTHOR]

In the field of underwater perception and detection, side-scan sonar (SSS) plays an indispensable role. However, the imaging mechanism of SSS results in slow information acquisition and high complexity, significantly hindering the advancement of downstream data-driven applications. To address this challenge, we designed an SSS image generator based on diffusion models. We developed a data collection system based on Autonomous Underwater Vehicles (AUVs) to achieve stable and rich data collection. For the process of converting acoustic signals into image signals, we established an image compensation method based on nonlinear gain enhancement to ensure the reliability of remote signals. On this basis, we developed the first controllable category SSS image generation algorithm, which can generate specified data for five categories, demonstrating outstanding performance in terms of the Fréchet Inception Distance (FID) and the Inception Score (IS). We further evaluated our image generator in the task of SSS object detection, and our cross-validation experiments showed that the generated images contributed to an average accuracy improvement of approximately 10% in object detection. The experimental results validate the effectiveness of the proposed SSS image generator in generating highly similar sonar images and enhancing detection accuracy, effectively addressing the issue of data scarcity. [ABSTRACT FROM AUTHOR]

Copyright of Information & Control is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Abstract Background Image-based crop growth modeling can substantially contribute to precision agriculture by revealing spatial crop development over time, which allows an early and location-specific estimation of relevant future plant traits, such as leaf area or biomass. A prerequisite for realistic and sharp crop image generation is the integration of multiple growth-influencing conditions in a model, such as an image of an initial growth stage, the associated growth time, and further information about the field treatment. While image-based models provide more flexibility for crop growth modeling than process-based models, there is still a significant research gap in the comprehensive integration of various growth-influencing conditions. Further exploration and investigation are needed to address this gap. Methods We present a two-stage framework consisting first of an image generation model and second of a growth estimation model, independently trained. The image generation model is a conditional Wasserstein generative adversarial network (CWGAN). In the generator of this model, conditional batch normalization (CBN) is used to integrate conditions of different types along with the input image. This allows the model to generate time-varying artificial images dependent on multiple influencing factors. These images are used by the second part of the framework for plant phenotyping by deriving plant-specific traits and comparing them with those of non-artificial (real) reference images. In addition, image quality is evaluated using multi-scale structural similarity (MS-SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID). During inference, the framework allows image generation for any combination of conditions used in training; we call this generation data-driven crop growth simulation. Results Experiments are performed on three datasets of different complexity. These datasets include the laboratory plant Arabidopsis thaliana (Arabidopsis) and crops grown under real field conditions, namely cauliflower (GrowliFlower) and crop mixtures consisting of faba bean and spring wheat (MixedCrop). In all cases, the framework allows realistic, sharp image generations with a slight loss of quality from short-term to long-term predictions. For MixedCrop grown under varying treatments (different cultivars, sowing densities), the results show that adding these treatment information increases the generation quality and phenotyping accuracy measured by the estimated biomass. Simulations of varying growth-influencing conditions performed with the trained framework provide valuable insights into how such factors relate to crop appearances, which is particularly useful in complex, less explored crop mixture systems. Further results show that adding process-based simulated biomass as a condition increases the accuracy of the derived phenotypic traits from the predicted images. This demonstrates the potential of our framework to serve as an interface between a data-driven and a process-based crop growth model. Conclusion The realistic generation and simulation of future plant appearances is adequately feasible by multi-conditional CWGAN. The presented framework complements process-based models and overcomes their limitations, such as the reliance on assumptions and the low exact field-localization specificity, by realistic visualizations of the spatial crop development that directly lead to a high explainability of the model predictions.

Abstract To improving ship image generation and classification tasks, a deep convolution generative adversarial network based on attention mechanism (ADCGAN) model was constructed. The rectified linear unit (ReLU) activation function was adopted, and three Deconv layers and Conv layers were added to both the generator and discriminator. Subsequently, an attention mechanism was added to the generator, while spectral normalization (SN) was added to the discriminator. Mean squared error (MSE) was used as loss function to stabilize the training process. Furthermore, ship classification tasks were performed using the generated ship images by end-to-end training of the classification network, enabling ship data augmentation and co-learning with other tasks. Experimental results on the Ship700 and Seaship7000 datasets demonstrate that the ADCGAN model can generate clear and robust ship images, with PSNR, LIPIPS, MS-SSIM values of 20.279 and 27.523, 0.596 and 0.096, 0.781 and 0.947, respectively. The effectiveness of the proposed method in ship image classification tasks was also verified, providing a data foundation for other collaborative tasks.

Abstract The cast thin sections of tight oil reservoirs contain important parameters such as rock mineral composition and content, porosity, permeability and stratigraphic characteristics, which are of great significance for reservoir evaluation. The use of deep learning technology for intelligent identification of thin section images is a development trend of mineral identification. However, the difficulty of making cast thin sections, the complexity of the making process and the high cost of thin section annotation have led to a lack of cast thin section images, which cannot meet the training requirements of deep learning image recognition models. In order to increase the sample size and improve the training effect of deep learning model, we proposed a generation and annotation method of thin section images of tight oil reservoir based on deep learning, by taking Fuyu reservoir in Sanzhao Sag as the target area. Firstly, the Augmentor strategy space was used to preliminarily augment the original images while preserving the original image features to meet the requirements of the model. Secondly, the category attention mechanism was added to the original StyleGAN network to avoid the influence of the uneven number of components in thin sections on the quality of the generated images. Then, the SALM annotation module was designed to achieve semi-automatic annotation of the generated images. Finally, experiments on image sharpness, distortion, standard accuracy and annotation efficiency were designed to verify the advantages of the method in image quality and annotation efficiency.

AI painting, as a popular research direction in the field of computer vision, is expanding its application boundaries in the fields of art creation, film and media, industrial design, and art education through natural language processing, graphic pre-training models, and diffusion models. Two types of AI painting, namely, image-to-image and text-to-image, are taken as the main lines, and the representative models and their key technologies and methods are analyzed in depth. For the image-to-image, the development lineage, generation principle, and advantages and disadvantages of each model are explored from two types of models based on AE and GAN, and their effects on the public dataset are summarized. For the text-to-image, the structural differences of the three types of models based on diffusion model and other models, as well as the generation effects of various types of models on three datasets are summarized. It is pointed out that the text-to-image utilizing the diffusion model has become a hot topic nowadays, which predicts the diversified development of image generation in the future. And the current mainstream AI painting platforms are compared and summarized from the perspectives of usage and generation speed. Finally, on the basis of summarizing the problems and controversies faced by AI painting at the technical and social levels, future trends such as the complementary development of AI painting and human artists, the increased interactivity of the painting process, and the emergence of new professions and industries are envisioned.

While comprehensive knowledge networks can be instrumental in finding solutions to complex problems or supporting the development of detailed simulation models, their large number of nodes and edges can become a hindrance. When the representation of a network becomes opaque, they stop fulfilling their role as a shared representation of a system between participants and modelers; hence, participants are less engaged in the model-building process. Combating the information overload created by large conceptual models is not merely a matter of changing formats: shifting from an unwieldy diagram to enormous amounts of text does not promote engagement. Rather, we posit that participants need an environment that provides details on demand and where interactions with a model rely primarily on a familiar format (i.e., text). In this study, we developed a visual analytics environment where linked visualizations allow participants to interact with large conceptual models, as shown in a case study with hundreds of nodes and almost a thousand relationships. Our environment leverages several advances in generative AI to automatically transform (i) a conceptual model into detailed paragraphs, (ii) detailed text into an executive summary of a model, (iii) prompts about the model into a safe version that avoids sensitive topics, and (iv) a description of the model into a complementary illustration. By releasing our work open source along with a video of our case study, we encourage other modelers to use this approach with their participants. Their feedback and future usability studies are key to respond to the needs of participants by improving our environment given individual preferences, models, and application domains. [ABSTRACT FROM AUTHOR]

In this paper, we propose a few-shot method for pose transfer of anime characters—given a source image of an anime character and a target pose, we transfer the pose of the target to the source character. Despite recent advances in pose transfer on real people images, these methods typically require large numbers of training images of different person under different poses to achieve reasonable results. However, anime character images are expensive to obtain they are created with a lot of artistic authoring. To address this, we propose a meta-learning framework for few-shot pose transfer, which can well generalize to an unseen character given just a few examples of the character. Further, we propose fusion residual blocks to align the features of the source and target so that the appearance of the source character can be well transferred to the target pose. Experiments show that our method outperforms leading pose transfer methods, especially when the source characters are not in the training set. [ABSTRACT FROM AUTHOR]

Recent advancements in cognitive neuroscience, particularly in electroencephalogram (EEG) signal processing, image generation, and brain–computer interfaces (BCIs), have opened up new avenues for research. This study introduces a novel framework, Bridging Artificial Intelligence and Neurological Signals (BRAINS), which leverages the power of artificial intelligence (AI) to extract meaningful information from EEG signals and generate images. The BRAINS framework addresses the limitations of traditional EEG analysis techniques, which struggle with nonstationary signals, spectral estimation, and noise sensitivity. Instead, BRAINS employs Long Short-Term Memory (LSTM) networks and contrastive learning, which effectively handle time-series EEG data and recognize intrinsic connections and patterns. The study utilizes the MNIST dataset of handwritten digits as stimuli in EEG experiments, allowing for diverse yet controlled stimuli. The data collected are then processed through an LSTM-based network, employing contrastive learning and extracting complex features from EEG data. These features are fed into an image generator model, producing images as close to the original stimuli as possible. This study demonstrates the potential of integrating AI and EEG technology, offering promising implications for the future of brain–computer interfaces. [ABSTRACT FROM AUTHOR]

The article investigates the problem of consistent generation in diffusion models. Modern generative diffusion models are capable of creating high-precision images, but maintaining the consistency between the related generation results remains a challenging task. The key methods for ensuring generation consistency are analyzed. Additionally, a new type of consistency is introduced — conceptual consistency, which allows for assessing the models’ ability not only to reproduce existing styles and objects but also to generate entirely new visual ideas that the model has never encountered during training. The existing methods for ensuring consistency are analyzed, and their advantages and disadvantages are identified. The image-to-image generation method based on an input reference image has the advantage of simplicity in implementation. Fine-tuning methods like DreamBooth and LoRA DreamBooth provide broader control over object consistency. ControlNet models ensure shape consistency using a special input image that serves as a guide shape in the reverse diffusion process. Noise inversion methods allow for more precise control and iterative refinement of the resulting images through manipulations with the noise space, enabling the generation of more stylistically and conceptually consistent images. The StyleAligned method, using a shared attention mechanism, can ensure the stylistic consistency of generated images. Understanding the capabilities and limitations of methods for ensuring diffusion generation consistency allows for selecting the most effective set of tools according to the task at hand. Diffusion models continue to evolve and expand into new areas, so achieving reliable and universal consistency in diffusion models could pave the way for even more creative and effective solutions [ABSTRACT FROM AUTHOR]

Generative domain adaptation in a one-shot scenario involves transferring a pretrained generator from one domain to another using only a single reference image. To address the issue of extremely scarce data, existing methods resort to complex parameter constraints and leverage additional semantic knowledge from CLIP models to mitigate it. However, these methods still suffer from overfitting and underfitting issues due to the lack of prior knowledge about the domain adaptation task. In this paper, we firstly introduce the perspective of the frequency domain into the generative domain adaptation task to support the model in understanding the adaptation goals in a one-shot scenario and propose a method called frequency-auxiliary GAN (FAGAN). The FAGAN contains two core modules: a low-frequency fusion module (LFF-Module) and high-frequency guide module (HFG-Module). Specifically, the LFF-Module aims to inherit the domain-sharing information of the source module by fusing the low-frequency features of the source model. In addition, the HFG-Module is designed to select the domain-specific information of the reference image and guide the model to fit them by utilizing high-frequency guidance. These two modules are dedicated to alleviating overfitting and underfitting issues, thereby enchancing the diversity and fidelity of generated images. Extensive experimental results showed that our method leads to better quantitative and qualitative results than the existing methods under a wide range of task settings. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) with gaming abilities have been widely applied in image generation. However, gamistic generators and discriminators may reduce the robustness of the obtained GANs in image generation under varying scenes. Enhancing the relation of hierarchical information in a generation network and enlarging differences of different network architectures can facilitate more structural information to improve the generation effect for image generation. In this paper, we propose an enhanced GAN via improving a generator for image generation (EIGGAN). EIGGAN applies a spatial attention to a generator to extract salient information to enhance the truthfulness of the generated images. Taking into relation the context account, parallel residual operations are fused into a generation network to extract more structural information from the different layers. Finally, a mixed loss function in a GAN is exploited to make a tradeoff between speed and accuracy to generate more realistic images. Experimental results show that the proposed method is superior to popular methods, i.e. Wasserstein GAN with gradient penalty (WGAN-GP) in terms of many indexes, i.e. Frechet Inception Distance, Learned Perceptual Image Patch Similarity, Multi-Scale Structural Similarity Index Measure, Kernel Inception Distance, Number of Statistically-Different Bins, Inception Score and some visual images for image generation. [ABSTRACT FROM AUTHOR]

Background: Image-based crop growth modeling can substantially contribute to precision agriculture by revealing spatial crop development over time, which allows an early and location-specific estimation of relevant future plant traits, such as leaf area or biomass. A prerequisite for realistic and sharp crop image generation is the integration of multiple growth-influencing conditions in a model, such as an image of an initial growth stage, the associated growth time, and further information about the field treatment. While image-based models provide more flexibility for crop growth modeling than process-based models, there is still a significant research gap in the comprehensive integration of various growth-influencing conditions. Further exploration and investigation are needed to address this gap. Methods: We present a two-stage framework consisting first of an image generation model and second of a growth estimation model, independently trained. The image generation model is a conditional Wasserstein generative adversarial network (CWGAN). In the generator of this model, conditional batch normalization (CBN) is used to integrate conditions of different types along with the input image. This allows the model to generate time-varying artificial images dependent on multiple influencing factors. These images are used by the second part of the framework for plant phenotyping by deriving plant-specific traits and comparing them with those of non-artificial (real) reference images. In addition, image quality is evaluated using multi-scale structural similarity (MS-SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID). During inference, the framework allows image generation for any combination of conditions used in training; we call this generation data-driven crop growth simulation. Results: Experiments are performed on three datasets of different complexity. These datasets include the laboratory plant Arabidopsis thaliana (Arabidopsis) and crops grown under real field conditions, namely cauliflower (GrowliFlower) and crop mixtures consisting of faba bean and spring wheat (MixedCrop). In all cases, the framework allows realistic, sharp image generations with a slight loss of quality from short-term to long-term predictions. For MixedCrop grown under varying treatments (different cultivars, sowing densities), the results show that adding these treatment information increases the generation quality and phenotyping accuracy measured by the estimated biomass. Simulations of varying growth-influencing conditions performed with the trained framework provide valuable insights into how such factors relate to crop appearances, which is particularly useful in complex, less explored crop mixture systems. Further results show that adding process-based simulated biomass as a condition increases the accuracy of the derived phenotypic traits from the predicted images. This demonstrates the potential of our framework to serve as an interface between a data-driven and a process-based crop growth model. Conclusion: The realistic generation and simulation of future plant appearances is adequately feasible by multi-conditional CWGAN. The presented framework complements process-based models and overcomes their limitations, such as the reliance on assumptions and the low exact field-localization specificity, by realistic visualizations of the spatial crop development that directly lead to a high explainability of the model predictions. [ABSTRACT FROM AUTHOR]

The virtual try-on technology based on 2D images aims to seamlessly transfer provided garments onto target person images. Prior methods mainly concentrated on warping garments and generating images, overlooking the influence of feature alignment on the try-on results. In this study, we initially analyze the distortions present by existing methods and elucidate the critical role of feature alignment in the extraction stage. Building on this, we propose a novel feature alignment-based model (FA-VTON). Specifically, FA-VTON aligns the upsampled higher-level features from both person and garment images to acquire precise boundary information, which serves as guidance for subsequent garment warping. Concurrently, the Efficient Channel Attention mechanism (ECA) is introduced to generate the final result in the try-on generation module. This mechanism enables adaptive adjustment of channel feature weights to extract important features and reduce artifact generation. Furthermore, to make the student network focus on salient regions of each channel, we utilize channel-wise distillation (CWD) to minimize the Kullback–Leibler (KL) divergence between the channel probability maps of the two networks. The experiments show that our model achieves better results in both qualitative and quantitative analyses compared to current methods on the popular virtual try-on datasets. [ABSTRACT FROM AUTHOR]

Human pose image generation is a challenging task in computer vision with applications in animation, gaming, and virtual reality. This paper presents a novel approach for Human pose image generation GAN using a combination of U-Net architecture, Decomposed Component encoder, Laplacian image enhancement technique in the generator, and Siamese Networks with Alignment Loss in the discriminator. The generator network leverages the U-Net architecture to capture and reconstruct intricate details of human poses. Additionally, a Decomposed Component encoder is incorporated to learn disentangled representations of pose attributes, such as body position, joint angles, and limb orientations. Furthermore, a Laplacian image enhancement technique is applied to enhance the visual quality and sharpness of the generated pose images. To ensure the alignment and similarity of the generated pose images with real poses, Siamese Networks with Alignment Loss are integrated into the discriminator. The Siamese Networks enable pairwise comparison between generated and real poses, guiding the discriminator to learn features that capture the alignment relationship. The alignment loss is computed based on the similarity of feature representations extracted by the Siamese Networks. This proposed model focuses on enhancing the level of detail and realism in the pose representation rather than improving the resolution or sharpness of the image itself so the U-Net architecture and image sharpening algorithm helps the proposed model to generate more realistic images in comparison to the other state-of-art approaches. This manuscript also contains a comparative analysis of experimental results on various physical conditions with the previously existing state-of-the-art approaches. This analysis shows that the proposed algorithm generates 15–20% more realistic images in comparison to other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The cast thin sections of tight oil reservoirs contain important parameters such as rock mineral composition and content, porosity, permeability and stratigraphic characteristics, which are of great significance for reservoir evaluation. The use of deep learning technology for intelligent identification of thin section images is a development trend of mineral identification. However, the difficulty of making cast thin sections, the complexity of the making process and the high cost of thin section annotation have led to a lack of cast thin section images, which cannot meet the training requirements of deep learning image recognition models. In order to increase the sample size and improve the training effect of deep learning model, we proposed a generation and annotation method of thin section images of tight oil reservoir based on deep learning, by taking Fuyu reservoir in Sanzhao Sag as the target area. Firstly, the Augmentor strategy space was used to preliminarily augment the original images while preserving the original image features to meet the requirements of the model. Secondly, the category attention mechanism was added to the original StyleGAN network to avoid the influence of the uneven number of components in thin sections on the quality of the generated images. Then, the SALM annotation module was designed to achieve semi-automatic annotation of the generated images. Finally, experiments on image sharpness, distortion, standard accuracy and annotation efficiency were designed to verify the advantages of the method in image quality and annotation efficiency. [ABSTRACT FROM AUTHOR]

Generative adversarial network (GAN) can generate diverse and high-resolution images for data augmentation. However, when GAN is applied to the synthetic aperture radar (SAR) dataset, the generated categories are not of the same quality. The unrealistic category will affect the performance of the subsequent automatic target recognition (ATR). To overcome the problem, we propose a reinforced constraint filtering with compensation afterwards GAN (RCFCA-GAN) algorithm to generate SAR images. The proposed algorithm includes two stages. We focus on improving the quality of easily generated categories in Stage 1. Then, we record the categories that are hard to generate and compensate by using traditional augmentation methods in Stage 2. Thus, the overall quality of the generated images is improved. We conduct experiments on the moving and stationary target acquisition and recognition (MSTAR) dataset. Recognition accuracy and Fréchet inception distance (FID) acquired by the proposed algorithm indicate its effectiveness. [ABSTRACT FROM AUTHOR]

Digital rock analysis is a promising approach for visualizing geological microstructures and understanding transport mechanisms for underground geo-energy resources exploitation. Accurate image reconstruction methods are vital for capturing the diverse features and variability in digital rock samples. Stable diffusion, a cutting-edge artificial intelligence model, has revolutionized computer vision by creating realistic images. However, its application in digital rock analysis is still emerging. This study explores the applications of stable diffusion in digital rock analysis, including enhancing image resolution, improving quality with denoising and deblurring, segmenting images, filling missing sections, extending images with outpainting, and reconstructing three-dimensional rocks from twodimensional images. The powerful image generation capability of diffusion models shed light on digital rock analysis, showing potential in filling missing parts of rock images, lithologic discrimination, and generating network parameters. In addition, limitations in existing stable diffusion models are also discussed, including the lack of real digital rock images, and not being able to fully understand the mechanisms behind physical processes. Therefore, it is suggested to develop new models tailored to digital rock images for further progress. In sum, the integration of stable diffusion into digital core analysis presents immense research opportunities and holds the potential to transform the field, ushering in groundbreaking advances. [ABSTRACT FROM AUTHOR]

The breakthrough brought by generative adversarial networks (GANs) in computer vision (CV) applications has gained a lot of attention in different fields due to their ability to capture the distribution of a dataset and generate high-quality similar images. From one side, this technology has been rapidly adopted as an alternative to traditional applications and introduced novel perspectives in data augmentation, domain transfer, image expansion, image restoration, image segmentation, and super-resolution. From another side, we found that due to the lack of industrial datasets and the limitation for acquiring and accurately annotating new images, GANs form an exciting solution to generate new industrial image datasets or to restore and augment existing ones. Therefore, we introduce a review of the latest trend in GANs applications and project them in industrial use cases. We conducted our experiments with synthetic images and analyzed most of GAN's failures and image artifacts to provide training's best practices. [ABSTRACT FROM AUTHOR]

In light of growing concerns about the misuse of personal data resulting from the widespread use of artificial intelligence technology, it is necessary to implement robust privacy-protection methods. However, existing methods for protecting facial privacy suffer from issues such as poor visual quality, distortion and limited reusability. To tackle this challenge, we propose a novel approach called Diffusion Models for Face Privacy Protection (DIFP). Our method utilizes a face generator that is conditionally controlled and reality-guided to produce high-resolution encrypted faces that are photorealistic while preserving the naturalness and recoverability of the original facial information. We employ a two-stage training strategy to generate protected faces with guidance on identity and style, followed by an iterative technique for improving latent variables to enhance realism. Additionally, we introduce diffusion model denoising for identity recovery, which facilitates the removal of encryption and restoration of the original face when required. Experimental results demonstrate the effectiveness of our method in qualitative privacy protection, achieving high success rates in evading face-recognition tools and enabling near-perfect restoration of occluded faces. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Recent text-to-image generative models have enabled us to transform our words into vibrant, captivating imagery. The surge of personalization techniques that has followed has also allowed us to imagine unique concepts in new scenes. However, an intriguing question remains: How can we generate a new, imaginary concept that has never been seen before? In this article, we present the task of creative text-to-image generation, where we seek to generate new members of a broad category (e.g., generating a pet that differs from all existing pets). We leverage the under-studied Diffusion Prior models and show that the creative generation problem can be formulated as an optimization process over the output space of the diffusion prior, resulting in a set of "prior constraints." To keep our generated concept from converging into existing members, we incorporate a question-answering Vision-Language Model that adaptively adds new constraints to the optimization problem, encouraging the model to discover increasingly more unique creations. Finally, we show that our prior constraints can also serve as a strong mixing mechanism allowing us to create hybrids between generated concepts, introducing even more flexibility into the creative process. [ABSTRACT FROM AUTHOR]

The paper demonstrates a novel methodology for Content-Based Image Retrieval (CBIR), which shifts the focus from conventional domain-specific image queries to more complex text-based query processing. Latent diffusion models are employed to interpret complex textual prompts and address the requirements of effectively interpreting the complex textual query. Latent Diffusion models successfully transform complex textual queries into visually engaging representations, establishing a seamless connection between textual descriptions and visual content. Custom triplet network design is at the heart of our retrieval method. When trained well, a triplet network will represent the generated query image and the different images in the database. The cosine similarity metric is used to assess the similarity between the feature representations in order to find and retrieve the relevant images. Our experiments results show that latent diffusion models can successfully bridge the gap between complex textual prompts for image retrieval without relying on labels or metadata that are attached to database images. This advancement sets the stage for future explorations in image retrieval, leveraging the generative AI capabilities to cater to the ever-evolving demands of big data and complex query interpretations. [ABSTRACT FROM AUTHOR]

Chinese shadow puppetry has been recognized as a world intangible cultural heritage. However, it faces substantial challenges in its preservation and advancement due to the intricate and labor-intensive nature of crafting shadow puppets. To ensure the inheritance and development of this cultural heritage, it is imperative to enable traditional art to flourish in the digital era. This paper presents an Interactive Collaborative Creation System for shadow puppets, designed to facilitate the creation of high-quality shadow puppet images with greater ease. The system comprises four key functions: Image contour extraction, intelligent reference recommendation, generation network, and color adjustment, all aimed at assisting users in various aspects of the creative process, including drawing, inspiration, and content generation. Additionally, we propose an enhanced algorithm called Smooth Generative Adversarial Networks (SmoothGAN), which exhibits more stable gradient training and a greater capacity for generating high-resolution shadow puppet images. Furthermore, we have built a new dataset comprising high-quality shadow puppet images to train the shadow puppet generation model. Both qualitative and quantitative experimental results demonstrate that SmoothGAN significantly improves the quality of image generation, while our system efficiently assists users in creating high-quality shadow puppet images, with a SUS scale score of 84.4. This study provides a valuable theoretical and practical reference for the digital creation of shadow puppet art. [ABSTRACT FROM AUTHOR]

The modern QD backplane production line deploys deep learning‐based defect classifier, which uses backplane images, to the auto repair system. However, one key challenge in this application is the lack of defect images available for training when a new product is first introduced. To address this issue, generative models like GAN have been used to generate defect images for new products but the image quality and diversity are not optimal due to unstable training and mode collapse. In this paper, the newest diffusion model is applied for defect image generation. It is trained on both normal (defect‐free) and defect images from new Product A and existing Product B. This allows the model to learn defects from Product B and generate synthetic defect images of Product A from normal images. Using an innovative method to automatically apply masks in diffusion sampling, the generated synthetic images are demonstrated to achieve high image quality and good defect diversity while preserving image fidelity outside the defective region when compared to the input normal image. Experiments are also conducted to train classifiers with generated defect images from different sampling methods. The proposed automatic masking method outperforms other sampling methods in the experiments. [ABSTRACT FROM AUTHOR]

Chinese paper-cutting, as an ancient folk art, is facing difficulties in preserving and passing down its traditions due to a lack of skilled paper-cut artists. In contrast to other image generation tasks, paper-cut images not only necessitate symmetry and exaggeration but also demand a certain level of resemblance to human facial features. To address these issues, this paper proposes a dual-branch generative adversarial network model for automatically generating paper-cut images, referred to as CutGAN. Specifically, we first construct a paper-cut dataset consisting of 891 pairs of facial images and handcrafted paper-cut images to train and evaluate CutGAN. Next, during the pre-training phase, we utilize gender and eyeglasses recognition tasks to train the fixed encoder. In the fine-tuning phase, we design a flexible encoder based on the modified U-net structure without skip connections. Furthermore, we introduce an average face loss to augment the diversity and improve the quality of the generated paper-cut images. We conducted extensive qualitative and quantitative experiments, as well as ablation experiments, comparing CutGAN with state-of-the-art baseline models on the test set. The experimental results indicate that CutGAN outperforms other image translation models by generating paper-cut images that more accurately capture the essence of Chinese paper-cut art and closely resemble actual facial images. [ABSTRACT FROM AUTHOR]

The generation of images from electroencephalography (EEG) signals has become a popular research topic in recent research because it can bridge the gap between brain signals and visual stimuli and has wide application prospects in neuroscience and computer vision. However, due to the high complexity of EEG signals, the reconstruction of visual stimuli through EEG signals continues to pose a challenge. In this work, we propose an EEG-ConDiffusion framework that involves three stages: feature extraction, fine-tuning of the pretrained model, and image generation. In the EEG-ConDiffusion framework, classification features of EEG signals are first obtained through the feature extraction block. Then, the classification features are taken as conditions to fine-tune the stable diffusion model in the image generation block to generate images with corresponding semantics. This framework combines EEG classification and image generation means to enhance the quality of generated images. Our proposed framework was tested on an EEG-based visual classification dataset. The performance of our framework is measured by classification accuracy, 50-way top-k accuracy, and inception score. The results indicate that the proposed EEG-Condiffusion framework can extract effective classification features and generate high-quality images from EEG signals to realize EEG-to-image conversion. [ABSTRACT FROM AUTHOR]

Image generation has always been one of the important research directions in the field of computer vision. It has rich applications in virtual reality, image design, and video synthesis. Our experiments proved that the proposed multi-style image generative network can efficiently generate high-quality images with different artistic styles based on the semantic images. Compared with the current state-of-the-art methods, the result generation speed of our proposed method is the fastest. In this paper, we focus on implementing arbitrary style transfer based on semantic images with high resolution ( 512 × 1024 ). We propose a new multi-channel generative adversarial network which uses fewer parameters to generate multi-style images. The network framework consists of a content feature extraction network, a style feature extraction network, and a content-stylistic feature fusion network. Our qualitative experiments show that the proposed multi-style image generation network can efficiently generate semantic-based, high-quality images with multiple artistic styles and with greater clarity and richer details. We adopt a user preference study, and the results show that the results generated by our method are more popular. Our speed study shows that our proposed method has the fastest result generation speed compared to the current state-of-the-art methods. We publicly release the source code of our project, which can be accessed at https://github.com/JuanMaoHSQ/Multi-style-image-generation-based-on-semantic-image. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Fundus image generation can serve training data diversity as well as clinical and medical education. To solve the problem of the controllability and flexibility of fundus image generation with multiple lesion features, we propose a controllable fundus image generation model (CFIGGAN) based on conditional generative adversarial networks (GAN) for medical data augmentation. The least square loss and the perceptual loss term are added to the final loss to make the generated images more realistic, and the spectral normalization is used as the normalization method of the discriminator to make the training process more stable. In the two-stage training of the model, the vascular tree image concatenates the real and generated images as positive and negative samples to train the model. CFIGGAN can generate diseased fundus images by using the annotations of vascular tree, field of vision(FOV), DR-related lesions as input and controlling the morphology of four types of lesions. Qualitative experimental evaluation shows that the fundus images generated by our model are clear and realistic and close to the real image data distribution. Quantitative experimental evaluation shows that the combination of the spectral norm and the perceptual loss can improve visual observation and quantitative indices, and data augmentation by image generation can further increase the classification accuracy. More importantly, CFIGGAN achieve the controllability of fundus image generation corresponding to DR-related lesions, and the proposed method can be extended to medical images generation of other diseases for broader prospects. [ABSTRACT FROM AUTHOR]

Instructional image editing has received a significant surge of attention recently. In this work, we are interested in the challenging problem of instructional image editing within the particular fashion realm, a domain with significant potential demand in both commercial and personal contexts. This specific domain presents heightened challenges owing to the stringent quality requirements. It necessitates not only the creation of vivid details in alignment with instructions, but also the preservation of precise attributes unrelated to the text guidance. Naive extensions of existing image editing methods produce noticeable artifacts. In order to achieve high-fidelity fashion editing, we propose a novel framework, leveraging the generative prior of a pre-trained human generator and performing edit in the latent space. In addition, we introduce a novel CLIP-based loss to better align the generated target with the instruction. Extensive experiments demonstrate that our approach outperforms prior works including GAN-based editing as well as diffusion-based editing by a large margin, showing impressive visual quality.