Search

AI painting, as a popular research direction in the field of computer vision, is expanding its application boundaries in the fields of art creation, film and media, industrial design, and art education through natural language processing, graphic pre-training models, and diffusion models. Two types of AI painting, namely, image-to-image and text-to-image, are taken as the main lines, and the representative models and their key technologies and methods are analyzed in depth. For the image-to-image, the development lineage, generation principle, and advantages and disadvantages of each model are explored from two types of models based on AE and GAN, and their effects on the public dataset are summarized. For the text-to-image, the structural differences of the three types of models based on diffusion model and other models, as well as the generation effects of various types of models on three datasets are summarized. It is pointed out that the text-to-image utilizing the diffusion model has become a hot topic nowadays, which predicts the diversified development of image generation in the future. And the current mainstream AI painting platforms are compared and summarized from the perspectives of usage and generation speed. Finally, on the basis of summarizing the problems and controversies faced by AI painting at the technical and social levels, future trends such as the complementary development of AI painting and human artists, the increased interactivity of the painting process, and the emergence of new professions and industries are envisioned.

Abstract The cast thin sections of tight oil reservoirs contain important parameters such as rock mineral composition and content, porosity, permeability and stratigraphic characteristics, which are of great significance for reservoir evaluation. The use of deep learning technology for intelligent identification of thin section images is a development trend of mineral identification. However, the difficulty of making cast thin sections, the complexity of the making process and the high cost of thin section annotation have led to a lack of cast thin section images, which cannot meet the training requirements of deep learning image recognition models. In order to increase the sample size and improve the training effect of deep learning model, we proposed a generation and annotation method of thin section images of tight oil reservoir based on deep learning, by taking Fuyu reservoir in Sanzhao Sag as the target area. Firstly, the Augmentor strategy space was used to preliminarily augment the original images while preserving the original image features to meet the requirements of the model. Secondly, the category attention mechanism was added to the original StyleGAN network to avoid the influence of the uneven number of components in thin sections on the quality of the generated images. Then, the SALM annotation module was designed to achieve semi-automatic annotation of the generated images. Finally, experiments on image sharpness, distortion, standard accuracy and annotation efficiency were designed to verify the advantages of the method in image quality and annotation efficiency.

Abstract Background Image-based crop growth modeling can substantially contribute to precision agriculture by revealing spatial crop development over time, which allows an early and location-specific estimation of relevant future plant traits, such as leaf area or biomass. A prerequisite for realistic and sharp crop image generation is the integration of multiple growth-influencing conditions in a model, such as an image of an initial growth stage, the associated growth time, and further information about the field treatment. While image-based models provide more flexibility for crop growth modeling than process-based models, there is still a significant research gap in the comprehensive integration of various growth-influencing conditions. Further exploration and investigation are needed to address this gap. Methods We present a two-stage framework consisting first of an image generation model and second of a growth estimation model, independently trained. The image generation model is a conditional Wasserstein generative adversarial network (CWGAN). In the generator of this model, conditional batch normalization (CBN) is used to integrate conditions of different types along with the input image. This allows the model to generate time-varying artificial images dependent on multiple influencing factors. These images are used by the second part of the framework for plant phenotyping by deriving plant-specific traits and comparing them with those of non-artificial (real) reference images. In addition, image quality is evaluated using multi-scale structural similarity (MS-SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID). During inference, the framework allows image generation for any combination of conditions used in training; we call this generation data-driven crop growth simulation. Results Experiments are performed on three datasets of different complexity. These datasets include the laboratory plant Arabidopsis thaliana (Arabidopsis) and crops grown under real field conditions, namely cauliflower (GrowliFlower) and crop mixtures consisting of faba bean and spring wheat (MixedCrop). In all cases, the framework allows realistic, sharp image generations with a slight loss of quality from short-term to long-term predictions. For MixedCrop grown under varying treatments (different cultivars, sowing densities), the results show that adding these treatment information increases the generation quality and phenotyping accuracy measured by the estimated biomass. Simulations of varying growth-influencing conditions performed with the trained framework provide valuable insights into how such factors relate to crop appearances, which is particularly useful in complex, less explored crop mixture systems. Further results show that adding process-based simulated biomass as a condition increases the accuracy of the derived phenotypic traits from the predicted images. This demonstrates the potential of our framework to serve as an interface between a data-driven and a process-based crop growth model. Conclusion The realistic generation and simulation of future plant appearances is adequately feasible by multi-conditional CWGAN. The presented framework complements process-based models and overcomes their limitations, such as the reliance on assumptions and the low exact field-localization specificity, by realistic visualizations of the spatial crop development that directly lead to a high explainability of the model predictions.

Abstract To improving ship image generation and classification tasks, a deep convolution generative adversarial network based on attention mechanism (ADCGAN) model was constructed. The rectified linear unit (ReLU) activation function was adopted, and three Deconv layers and Conv layers were added to both the generator and discriminator. Subsequently, an attention mechanism was added to the generator, while spectral normalization (SN) was added to the discriminator. Mean squared error (MSE) was used as loss function to stabilize the training process. Furthermore, ship classification tasks were performed using the generated ship images by end-to-end training of the classification network, enabling ship data augmentation and co-learning with other tasks. Experimental results on the Ship700 and Seaship7000 datasets demonstrate that the ADCGAN model can generate clear and robust ship images, with PSNR, LIPIPS, MS-SSIM values of 20.279 and 27.523, 0.596 and 0.096, 0.781 and 0.947, respectively. The effectiveness of the proposed method in ship image classification tasks was also verified, providing a data foundation for other collaborative tasks.

In the field of underwater perception and detection, side-scan sonar (SSS) plays an indispensable role. However, the imaging mechanism of SSS results in slow information acquisition and high complexity, significantly hindering the advancement of downstream data-driven applications. To address this challenge, we designed an SSS image generator based on diffusion models. We developed a data collection system based on Autonomous Underwater Vehicles (AUVs) to achieve stable and rich data collection. For the process of converting acoustic signals into image signals, we established an image compensation method based on nonlinear gain enhancement to ensure the reliability of remote signals. On this basis, we developed the first controllable category SSS image generation algorithm, which can generate specified data for five categories, demonstrating outstanding performance in terms of the Fréchet Inception Distance (FID) and the Inception Score (IS). We further evaluated our image generator in the task of SSS object detection, and our cross-validation experiments showed that the generated images contributed to an average accuracy improvement of approximately 10% in object detection. The experimental results validate the effectiveness of the proposed SSS image generator in generating highly similar sonar images and enhancing detection accuracy, effectively addressing the issue of data scarcity. [ABSTRACT FROM AUTHOR]

Copyright of Information & Control is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Recent advancements in cognitive neuroscience, particularly in electroencephalogram (EEG) signal processing, image generation, and brain–computer interfaces (BCIs), have opened up new avenues for research. This study introduces a novel framework, Bridging Artificial Intelligence and Neurological Signals (BRAINS), which leverages the power of artificial intelligence (AI) to extract meaningful information from EEG signals and generate images. The BRAINS framework addresses the limitations of traditional EEG analysis techniques, which struggle with nonstationary signals, spectral estimation, and noise sensitivity. Instead, BRAINS employs Long Short-Term Memory (LSTM) networks and contrastive learning, which effectively handle time-series EEG data and recognize intrinsic connections and patterns. The study utilizes the MNIST dataset of handwritten digits as stimuli in EEG experiments, allowing for diverse yet controlled stimuli. The data collected are then processed through an LSTM-based network, employing contrastive learning and extracting complex features from EEG data. These features are fed into an image generator model, producing images as close to the original stimuli as possible. This study demonstrates the potential of integrating AI and EEG technology, offering promising implications for the future of brain–computer interfaces. [ABSTRACT FROM AUTHOR]

In this paper, we propose a few-shot method for pose transfer of anime characters—given a source image of an anime character and a target pose, we transfer the pose of the target to the source character. Despite recent advances in pose transfer on real people images, these methods typically require large numbers of training images of different person under different poses to achieve reasonable results. However, anime character images are expensive to obtain they are created with a lot of artistic authoring. To address this, we propose a meta-learning framework for few-shot pose transfer, which can well generalize to an unseen character given just a few examples of the character. Further, we propose fusion residual blocks to align the features of the source and target so that the appearance of the source character can be well transferred to the target pose. Experiments show that our method outperforms leading pose transfer methods, especially when the source characters are not in the training set. [ABSTRACT FROM AUTHOR]

While comprehensive knowledge networks can be instrumental in finding solutions to complex problems or supporting the development of detailed simulation models, their large number of nodes and edges can become a hindrance. When the representation of a network becomes opaque, they stop fulfilling their role as a shared representation of a system between participants and modelers; hence, participants are less engaged in the model-building process. Combating the information overload created by large conceptual models is not merely a matter of changing formats: shifting from an unwieldy diagram to enormous amounts of text does not promote engagement. Rather, we posit that participants need an environment that provides details on demand and where interactions with a model rely primarily on a familiar format (i.e., text). In this study, we developed a visual analytics environment where linked visualizations allow participants to interact with large conceptual models, as shown in a case study with hundreds of nodes and almost a thousand relationships. Our environment leverages several advances in generative AI to automatically transform (i) a conceptual model into detailed paragraphs, (ii) detailed text into an executive summary of a model, (iii) prompts about the model into a safe version that avoids sensitive topics, and (iv) a description of the model into a complementary illustration. By releasing our work open source along with a video of our case study, we encourage other modelers to use this approach with their participants. Their feedback and future usability studies are key to respond to the needs of participants by improving our environment given individual preferences, models, and application domains. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) with gaming abilities have been widely applied in image generation. However, gamistic generators and discriminators may reduce the robustness of the obtained GANs in image generation under varying scenes. Enhancing the relation of hierarchical information in a generation network and enlarging differences of different network architectures can facilitate more structural information to improve the generation effect for image generation. In this paper, we propose an enhanced GAN via improving a generator for image generation (EIGGAN). EIGGAN applies a spatial attention to a generator to extract salient information to enhance the truthfulness of the generated images. Taking into relation the context account, parallel residual operations are fused into a generation network to extract more structural information from the different layers. Finally, a mixed loss function in a GAN is exploited to make a tradeoff between speed and accuracy to generate more realistic images. Experimental results show that the proposed method is superior to popular methods, i.e. Wasserstein GAN with gradient penalty (WGAN-GP) in terms of many indexes, i.e. Frechet Inception Distance, Learned Perceptual Image Patch Similarity, Multi-Scale Structural Similarity Index Measure, Kernel Inception Distance, Number of Statistically-Different Bins, Inception Score and some visual images for image generation. [ABSTRACT FROM AUTHOR]

This paper explores the potential of artificial intelligence (AI) technologies in the field of digital image processing, analyzing current applications and future prospects. AI-based image processing techniques, such as noise reduction, super-resolution, inpainting, and image generation, are transforming traditional approaches, offering new levels of accuracy, efficiency, and creative potential. The paper examines AI applications across various fields, including healthcare for medical diagnostics, cultural heritage for art restoration, industry for quality control, and marketing for content creation [ABSTRACT FROM AUTHOR]

The article investigates the problem of consistent generation in diffusion models. Modern generative diffusion models are capable of creating high-precision images, but maintaining the consistency between the related generation results remains a challenging task. The key methods for ensuring generation consistency are analyzed. Additionally, a new type of consistency is introduced — conceptual consistency, which allows for assessing the models’ ability not only to reproduce existing styles and objects but also to generate entirely new visual ideas that the model has never encountered during training. The existing methods for ensuring consistency are analyzed, and their advantages and disadvantages are identified. The image-to-image generation method based on an input reference image has the advantage of simplicity in implementation. Fine-tuning methods like DreamBooth and LoRA DreamBooth provide broader control over object consistency. ControlNet models ensure shape consistency using a special input image that serves as a guide shape in the reverse diffusion process. Noise inversion methods allow for more precise control and iterative refinement of the resulting images through manipulations with the noise space, enabling the generation of more stylistically and conceptually consistent images. The StyleAligned method, using a shared attention mechanism, can ensure the stylistic consistency of generated images. Understanding the capabilities and limitations of methods for ensuring diffusion generation consistency allows for selecting the most effective set of tools according to the task at hand. Diffusion models continue to evolve and expand into new areas, so achieving reliable and universal consistency in diffusion models could pave the way for even more creative and effective solutions [ABSTRACT FROM AUTHOR]

Generative domain adaptation in a one-shot scenario involves transferring a pretrained generator from one domain to another using only a single reference image. To address the issue of extremely scarce data, existing methods resort to complex parameter constraints and leverage additional semantic knowledge from CLIP models to mitigate it. However, these methods still suffer from overfitting and underfitting issues due to the lack of prior knowledge about the domain adaptation task. In this paper, we firstly introduce the perspective of the frequency domain into the generative domain adaptation task to support the model in understanding the adaptation goals in a one-shot scenario and propose a method called frequency-auxiliary GAN (FAGAN). The FAGAN contains two core modules: a low-frequency fusion module (LFF-Module) and high-frequency guide module (HFG-Module). Specifically, the LFF-Module aims to inherit the domain-sharing information of the source module by fusing the low-frequency features of the source model. In addition, the HFG-Module is designed to select the domain-specific information of the reference image and guide the model to fit them by utilizing high-frequency guidance. These two modules are dedicated to alleviating overfitting and underfitting issues, thereby enchancing the diversity and fidelity of generated images. Extensive experimental results showed that our method leads to better quantitative and qualitative results than the existing methods under a wide range of task settings. [ABSTRACT FROM AUTHOR]

Background: Image-based crop growth modeling can substantially contribute to precision agriculture by revealing spatial crop development over time, which allows an early and location-specific estimation of relevant future plant traits, such as leaf area or biomass. A prerequisite for realistic and sharp crop image generation is the integration of multiple growth-influencing conditions in a model, such as an image of an initial growth stage, the associated growth time, and further information about the field treatment. While image-based models provide more flexibility for crop growth modeling than process-based models, there is still a significant research gap in the comprehensive integration of various growth-influencing conditions. Further exploration and investigation are needed to address this gap. Methods: We present a two-stage framework consisting first of an image generation model and second of a growth estimation model, independently trained. The image generation model is a conditional Wasserstein generative adversarial network (CWGAN). In the generator of this model, conditional batch normalization (CBN) is used to integrate conditions of different types along with the input image. This allows the model to generate time-varying artificial images dependent on multiple influencing factors. These images are used by the second part of the framework for plant phenotyping by deriving plant-specific traits and comparing them with those of non-artificial (real) reference images. In addition, image quality is evaluated using multi-scale structural similarity (MS-SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID). During inference, the framework allows image generation for any combination of conditions used in training; we call this generation data-driven crop growth simulation. Results: Experiments are performed on three datasets of different complexity. These datasets include the laboratory plant Arabidopsis thaliana (Arabidopsis) and crops grown under real field conditions, namely cauliflower (GrowliFlower) and crop mixtures consisting of faba bean and spring wheat (MixedCrop). In all cases, the framework allows realistic, sharp image generations with a slight loss of quality from short-term to long-term predictions. For MixedCrop grown under varying treatments (different cultivars, sowing densities), the results show that adding these treatment information increases the generation quality and phenotyping accuracy measured by the estimated biomass. Simulations of varying growth-influencing conditions performed with the trained framework provide valuable insights into how such factors relate to crop appearances, which is particularly useful in complex, less explored crop mixture systems. Further results show that adding process-based simulated biomass as a condition increases the accuracy of the derived phenotypic traits from the predicted images. This demonstrates the potential of our framework to serve as an interface between a data-driven and a process-based crop growth model. Conclusion: The realistic generation and simulation of future plant appearances is adequately feasible by multi-conditional CWGAN. The presented framework complements process-based models and overcomes their limitations, such as the reliance on assumptions and the low exact field-localization specificity, by realistic visualizations of the spatial crop development that directly lead to a high explainability of the model predictions. [ABSTRACT FROM AUTHOR]

The virtual try-on technology based on 2D images aims to seamlessly transfer provided garments onto target person images. Prior methods mainly concentrated on warping garments and generating images, overlooking the influence of feature alignment on the try-on results. In this study, we initially analyze the distortions present by existing methods and elucidate the critical role of feature alignment in the extraction stage. Building on this, we propose a novel feature alignment-based model (FA-VTON). Specifically, FA-VTON aligns the upsampled higher-level features from both person and garment images to acquire precise boundary information, which serves as guidance for subsequent garment warping. Concurrently, the Efficient Channel Attention mechanism (ECA) is introduced to generate the final result in the try-on generation module. This mechanism enables adaptive adjustment of channel feature weights to extract important features and reduce artifact generation. Furthermore, to make the student network focus on salient regions of each channel, we utilize channel-wise distillation (CWD) to minimize the Kullback–Leibler (KL) divergence between the channel probability maps of the two networks. The experiments show that our model achieves better results in both qualitative and quantitative analyses compared to current methods on the popular virtual try-on datasets. [ABSTRACT FROM AUTHOR]

Human pose image generation is a challenging task in computer vision with applications in animation, gaming, and virtual reality. This paper presents a novel approach for Human pose image generation GAN using a combination of U-Net architecture, Decomposed Component encoder, Laplacian image enhancement technique in the generator, and Siamese Networks with Alignment Loss in the discriminator. The generator network leverages the U-Net architecture to capture and reconstruct intricate details of human poses. Additionally, a Decomposed Component encoder is incorporated to learn disentangled representations of pose attributes, such as body position, joint angles, and limb orientations. Furthermore, a Laplacian image enhancement technique is applied to enhance the visual quality and sharpness of the generated pose images. To ensure the alignment and similarity of the generated pose images with real poses, Siamese Networks with Alignment Loss are integrated into the discriminator. The Siamese Networks enable pairwise comparison between generated and real poses, guiding the discriminator to learn features that capture the alignment relationship. The alignment loss is computed based on the similarity of feature representations extracted by the Siamese Networks. This proposed model focuses on enhancing the level of detail and realism in the pose representation rather than improving the resolution or sharpness of the image itself so the U-Net architecture and image sharpening algorithm helps the proposed model to generate more realistic images in comparison to the other state-of-art approaches. This manuscript also contains a comparative analysis of experimental results on various physical conditions with the previously existing state-of-the-art approaches. This analysis shows that the proposed algorithm generates 15–20% more realistic images in comparison to other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Digital rock analysis is a promising approach for visualizing geological microstructures and understanding transport mechanisms for underground geo-energy resources exploitation. Accurate image reconstruction methods are vital for capturing the diverse features and variability in digital rock samples. Stable diffusion, a cutting-edge artificial intelligence model, has revolutionized computer vision by creating realistic images. However, its application in digital rock analysis is still emerging. This study explores the applications of stable diffusion in digital rock analysis, including enhancing image resolution, improving quality with denoising and deblurring, segmenting images, filling missing sections, extending images with outpainting, and reconstructing three-dimensional rocks from twodimensional images. The powerful image generation capability of diffusion models shed light on digital rock analysis, showing potential in filling missing parts of rock images, lithologic discrimination, and generating network parameters. In addition, limitations in existing stable diffusion models are also discussed, including the lack of real digital rock images, and not being able to fully understand the mechanisms behind physical processes. Therefore, it is suggested to develop new models tailored to digital rock images for further progress. In sum, the integration of stable diffusion into digital core analysis presents immense research opportunities and holds the potential to transform the field, ushering in groundbreaking advances. [ABSTRACT FROM AUTHOR]

In light of growing concerns about the misuse of personal data resulting from the widespread use of artificial intelligence technology, it is necessary to implement robust privacy-protection methods. However, existing methods for protecting facial privacy suffer from issues such as poor visual quality, distortion and limited reusability. To tackle this challenge, we propose a novel approach called Diffusion Models for Face Privacy Protection (DIFP). Our method utilizes a face generator that is conditionally controlled and reality-guided to produce high-resolution encrypted faces that are photorealistic while preserving the naturalness and recoverability of the original facial information. We employ a two-stage training strategy to generate protected faces with guidance on identity and style, followed by an iterative technique for improving latent variables to enhance realism. Additionally, we introduce diffusion model denoising for identity recovery, which facilitates the removal of encryption and restoration of the original face when required. Experimental results demonstrate the effectiveness of our method in qualitative privacy protection, achieving high success rates in evading face-recognition tools and enabling near-perfect restoration of occluded faces. [ABSTRACT FROM AUTHOR]

Generative adversarial network (GAN) can generate diverse and high-resolution images for data augmentation. However, when GAN is applied to the synthetic aperture radar (SAR) dataset, the generated categories are not of the same quality. The unrealistic category will affect the performance of the subsequent automatic target recognition (ATR). To overcome the problem, we propose a reinforced constraint filtering with compensation afterwards GAN (RCFCA-GAN) algorithm to generate SAR images. The proposed algorithm includes two stages. We focus on improving the quality of easily generated categories in Stage 1. Then, we record the categories that are hard to generate and compensate by using traditional augmentation methods in Stage 2. Thus, the overall quality of the generated images is improved. We conduct experiments on the moving and stationary target acquisition and recognition (MSTAR) dataset. Recognition accuracy and Fréchet inception distance (FID) acquired by the proposed algorithm indicate its effectiveness. [ABSTRACT FROM AUTHOR]

The breakthrough brought by generative adversarial networks (GANs) in computer vision (CV) applications has gained a lot of attention in different fields due to their ability to capture the distribution of a dataset and generate high-quality similar images. From one side, this technology has been rapidly adopted as an alternative to traditional applications and introduced novel perspectives in data augmentation, domain transfer, image expansion, image restoration, image segmentation, and super-resolution. From another side, we found that due to the lack of industrial datasets and the limitation for acquiring and accurately annotating new images, GANs form an exciting solution to generate new industrial image datasets or to restore and augment existing ones. Therefore, we introduce a review of the latest trend in GANs applications and project them in industrial use cases. We conducted our experiments with synthetic images and analyzed most of GAN's failures and image artifacts to provide training's best practices. [ABSTRACT FROM AUTHOR]

The cast thin sections of tight oil reservoirs contain important parameters such as rock mineral composition and content, porosity, permeability and stratigraphic characteristics, which are of great significance for reservoir evaluation. The use of deep learning technology for intelligent identification of thin section images is a development trend of mineral identification. However, the difficulty of making cast thin sections, the complexity of the making process and the high cost of thin section annotation have led to a lack of cast thin section images, which cannot meet the training requirements of deep learning image recognition models. In order to increase the sample size and improve the training effect of deep learning model, we proposed a generation and annotation method of thin section images of tight oil reservoir based on deep learning, by taking Fuyu reservoir in Sanzhao Sag as the target area. Firstly, the Augmentor strategy space was used to preliminarily augment the original images while preserving the original image features to meet the requirements of the model. Secondly, the category attention mechanism was added to the original StyleGAN network to avoid the influence of the uneven number of components in thin sections on the quality of the generated images. Then, the SALM annotation module was designed to achieve semi-automatic annotation of the generated images. Finally, experiments on image sharpness, distortion, standard accuracy and annotation efficiency were designed to verify the advantages of the method in image quality and annotation efficiency. [ABSTRACT FROM AUTHOR]

Chinese shadow puppetry has been recognized as a world intangible cultural heritage. However, it faces substantial challenges in its preservation and advancement due to the intricate and labor-intensive nature of crafting shadow puppets. To ensure the inheritance and development of this cultural heritage, it is imperative to enable traditional art to flourish in the digital era. This paper presents an Interactive Collaborative Creation System for shadow puppets, designed to facilitate the creation of high-quality shadow puppet images with greater ease. The system comprises four key functions: Image contour extraction, intelligent reference recommendation, generation network, and color adjustment, all aimed at assisting users in various aspects of the creative process, including drawing, inspiration, and content generation. Additionally, we propose an enhanced algorithm called Smooth Generative Adversarial Networks (SmoothGAN), which exhibits more stable gradient training and a greater capacity for generating high-resolution shadow puppet images. Furthermore, we have built a new dataset comprising high-quality shadow puppet images to train the shadow puppet generation model. Both qualitative and quantitative experimental results demonstrate that SmoothGAN significantly improves the quality of image generation, while our system efficiently assists users in creating high-quality shadow puppet images, with a SUS scale score of 84.4. This study provides a valuable theoretical and practical reference for the digital creation of shadow puppet art. [ABSTRACT FROM AUTHOR]

The paper demonstrates a novel methodology for Content-Based Image Retrieval (CBIR), which shifts the focus from conventional domain-specific image queries to more complex text-based query processing. Latent diffusion models are employed to interpret complex textual prompts and address the requirements of effectively interpreting the complex textual query. Latent Diffusion models successfully transform complex textual queries into visually engaging representations, establishing a seamless connection between textual descriptions and visual content. Custom triplet network design is at the heart of our retrieval method. When trained well, a triplet network will represent the generated query image and the different images in the database. The cosine similarity metric is used to assess the similarity between the feature representations in order to find and retrieve the relevant images. Our experiments results show that latent diffusion models can successfully bridge the gap between complex textual prompts for image retrieval without relying on labels or metadata that are attached to database images. This advancement sets the stage for future explorations in image retrieval, leveraging the generative AI capabilities to cater to the ever-evolving demands of big data and complex query interpretations. [ABSTRACT FROM AUTHOR]

Recent text-to-image generative models have enabled us to transform our words into vibrant, captivating imagery. The surge of personalization techniques that has followed has also allowed us to imagine unique concepts in new scenes. However, an intriguing question remains: How can we generate a new, imaginary concept that has never been seen before? In this article, we present the task of creative text-to-image generation, where we seek to generate new members of a broad category (e.g., generating a pet that differs from all existing pets). We leverage the under-studied Diffusion Prior models and show that the creative generation problem can be formulated as an optimization process over the output space of the diffusion prior, resulting in a set of "prior constraints." To keep our generated concept from converging into existing members, we incorporate a question-answering Vision-Language Model that adaptively adds new constraints to the optimization problem, encouraging the model to discover increasingly more unique creations. Finally, we show that our prior constraints can also serve as a strong mixing mechanism allowing us to create hybrids between generated concepts, introducing even more flexibility into the creative process. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The modern QD backplane production line deploys deep learning‐based defect classifier, which uses backplane images, to the auto repair system. However, one key challenge in this application is the lack of defect images available for training when a new product is first introduced. To address this issue, generative models like GAN have been used to generate defect images for new products but the image quality and diversity are not optimal due to unstable training and mode collapse. In this paper, the newest diffusion model is applied for defect image generation. It is trained on both normal (defect‐free) and defect images from new Product A and existing Product B. This allows the model to learn defects from Product B and generate synthetic defect images of Product A from normal images. Using an innovative method to automatically apply masks in diffusion sampling, the generated synthetic images are demonstrated to achieve high image quality and good defect diversity while preserving image fidelity outside the defective region when compared to the input normal image. Experiments are also conducted to train classifiers with generated defect images from different sampling methods. The proposed automatic masking method outperforms other sampling methods in the experiments. [ABSTRACT FROM AUTHOR]