Search

In recent years traditional numerical methods for accurate weather prediction have been increasingly challenged by deep learning methods. Numerous historical datasets used for short and medium-range weather forecasts are typically organized into a regular spatial grid structure. This arrangement closely resembles images: each weather variable can be visualized as a map or, when considering the temporal axis, as a video. Several classes of generative models, comprising Generative Adversarial Networks, Variational Autoencoders, or the recent Denoising Diffusion Models have largely proved their applicability to the next-frame prediction problem, and is thus natural to test their performance on the weather prediction benchmarks. Diffusion models are particularly appealing in this context, due to the intrinsically probabilistic nature of weather forecasting: what we are really interested to model is the probability distribution of weather indicators, whose expected value is the most likely prediction. In our study, we focus on a specific subset of the ERA-5 dataset, which includes hourly data pertaining to Central Europe from the years 2016 to 2021. Within this context, we examine the efficacy of diffusion models in handling the task of precipitation nowcasting, with a lead time of 1 to 3 hours. Our work is conducted in comparison to the performance of well-established U-Net models, as documented in the existing literature. An additional comparative analysis has been done with the forecasting capabilities of the CERRA system, part of the Copernicus Climate Change Service. The novelty of our approach, Generative Ensemble Diffusion (GED), lies in its innovative use of a diffusion model to generate a diverse set of possible weather scenarios. These scenarios are then amalgamated into a single prediction in a post-processing phase. This approach mimics the usual weather forecasting technique consisting in running an ensemble of numerical simulations under slightly different initial conditions by exploiting instead the intrinsic stochasticity of the generative model. In comparison to recent deep learning models addressing the same problem, our approach results in approximately a 25% reduction in the mean squared error. Reverse diffusion is a core concept in our GED approach, is particularly relevant to weather forecasting. In the context of diffusion models, reverse diffusion refers to the process of iteratively refining a noisy initial prediction into a coherent and realistic forecast. By leveraging reverse diffusion, our model effectively simulates the complex temporal dynamics of weather systems, mirroring the inherent uncertainty and variability in weather patterns. [ABSTRACT FROM AUTHOR]

In this work, we present and study Continuous Generative Neural Networks (CGNNs), namely, generative models in the continuous setting: the output of a CGNN belongs to an infinite-dimensional function space. The architecture is inspired by DCGAN, with one fully connected layer, several convolutional layers and nonlinear activation functions. In the continuous L2 setting, the dimensions of the spaces of each layer are replaced by the scales of a multiresolution analysis of a compactly supported wavelet. We present conditions on the convolutional filters and on the nonlinearity that guarantee that a CGNN is injective. This theory finds applications to inverse problems, and allows for deriving Lipschitz stability estimates for (possibly nonlinear) infinite-dimensional inverse problems with unknowns belonging to the manifold generated by a CGNN. Several numerical simulations, including signal deblurring, illustrate and validate this approach. [ABSTRACT FROM AUTHOR]

This paper describes how computational generative models can describe aspects of the artistic process, and how these generative models can provide tools for formulating and testing psychological theories of art. The term "generative models" here refers to algorithms that can generate artistic imagery, video, text, or other artistic media, including techniques developed in both computer graphics and AI research. Generative models can both describe artistic processes and offer useful experimental tools. This paper first outlines different ways to understand the types of research in generative models. It then surveys several recent examples of using generative models to develop theories and to perform experiments. The paper then discusses misleading uses of the concept of "AI-generated art" in psychological studies, and the need for study of our relationship with new artistic technologies. Finally, the paper offers a few remarks on pursuing interdisciplinary research across psychology and computer graphics. [ABSTRACT FROM AUTHOR]

Machine-learning-based predictive maintenance models, i.e. models that predict breakdowns of machines based on condition information, have a high potential to minimize maintenance costs in industrial applications by determining the best possible time to perform maintenance. Modern machines have sensors that can collect all relevant data of the operating condition and for legacy machines which are still widely used in the industry, retrofit sensors are readily, easily and inexpensively available. With the help of this data it is possible to train such a predictive maintenance model. The main problem is that most data is obtained from normal operating conditions, whereas only limited data are from failures. This leads to highly unbalanced data sets, which makes it very difficult, if not impossible, to train a predictive maintenance model that can detect faults reliably and timely. Another issue is the lack of available real data due to privacy concerns. To address these problems, a suitable data generation strategy is needed. In this work, a literature review is conducted to identify a solution approach for a suitable data augmentation strategy that can be applied to our specific use case of hydrogen combustion engines in the automotive field. This literature review shows that, among the different state-of-the-art proposals, the most promising for the generation of reliable synthetic data are the ones based on generative models. The analysis of the different metrics used in the state of the art allows to identify the most suitable ones to evaluate the quality of generated signals. Finally, an open problem in research in this area is identified and it is the need to validate the plausibility of the data generated. The generation of results in this area will contribute decisively to the development of predictive maintenance models. [ABSTRACT FROM AUTHOR]

Recent breakthroughs in generative neural networks have paved the way for transformative capabilities, particularly in their capacity to generate novel data, notably in the realm of images. The integration of these models with the increasingly popular technique of transfer learning, designed for proficient feature extraction, holds the promise of enhancing overall performance. This paper delves into the exploration of employing generative models in conjunction with transfer learning methods for feature extraction, with a specific focus on image classification tasks. Our investigation aims to scrutinize the effectiveness of leveraging generative models alongside pre-trained models as feature extractors in the context of image classification. To the best of our knowledge, our investigation is the first to link transfer learning and generative models for a discriminative task under one roof. The proposed approach undergoes rigorous evaluation on two distinct datasets, employing specific metrics to gauge the model’s performance. The results exhibit a notable nearly 10% enhancement achieved through the integration of generative models, underscoring their potential for achieving heightened accuracy in image classification. These findings highlight significant advancements in image classification accuracy, surpassing the performance of conventional Artificial Neural Network (ANN) models. [ABSTRACT FROM AUTHOR]

The generation and enhancement of satellite imagery are critical in remote sensing, requiring high-quality, detailed images for accurate analysis. This research introduces a two-stage diffusion model methodology for synthesizing high-resolution satellite images from textual prompts. The pipeline comprises a low-resolution diffusion model (LRDM) that generates initial images based on text inputs and a super-resolution diffusion model (SRDM) that refines these images into high-resolution outputs. The LRDM merges text and image embeddings within a shared latent space, capturing essential scene content and structure. The SRDM then enhances these images, focusing on spatial features and visual clarity. Experiments conducted using the Remote Sensing Image Captioning Dataset demonstrate that our method outperforms existing models, producing satellite images with accurate geographical details and improved spatial resolution. [ABSTRACT FROM AUTHOR]

We present a mathematical framework for analyzing fractal patterns in AI-generated images using persistent homology. Given a text-to-image mapping M : T → I , we demonstrate that the persistent homology groups H k (t) of sublevel set filtrations { f − 1 ((− ∞ , t ]) } t ∈ R characterize multi-scale geometric structures, where f : M (p) → R is the grayscale intensity function of a generated image. The primary challenge lies in quantifying self-similarity in scales, which we address by analyzing birth–death pairs (b i , d i) in the persistence diagram P D (M (p)) . Our contribution extends beyond applying the stability theorem to AI-generated fractals; we establish how the self-similarity inherent in fractal patterns manifests in the persistence diagrams of generated images. We validate our approach using the Stable Diffusion 3.5 model for four fractal categories: ferns, trees, spirals, and crystals. An analysis of guidance scale effects γ ∈ [ 4.0 , 8.0 ] reveals monotonic relationships between model parameters and topological features. Stability testing confirms robustness under noise perturbations η ≤ 0.2 , with feature count variations Δ μ f < 0.5 . Our framework provides a foundation for enhancing generative models and evaluating their geometric fidelity in fractal pattern synthesis. [ABSTRACT FROM AUTHOR]

In today's digital age, ensuring the appropriateness of content for children is crucial for their cognitive and emotional development. The rise of automated text generation technologies, such as Large Language Models like LLaMA, Mistral, and Zephyr, has created a pressing need for effective tools to filter and classify suitable content. However, the existing methods often fail to effectively address the intricate details and unique characteristics of children's literature. This study aims to bridge this gap by developing a robust framework that utilizes fine-tuned language models, classification techniques, and contextual story generation to generate and classify children's stories based on their suitability. Employing a combination of fine-tuning techniques on models such as LLaMA, Mistral, and Zephyr, alongside a BERT-based classifier, we evaluated the generated stories against established metrics like ROUGE, METEOR, and BERT Scores. The fine-tuned Mistral-7B model achieved a ROUGE-1 score of 0.4785, significantly higher than the base model's 0.3185, while Zephyr-7B-Beta achieved a METEOR score of 0.4154 compared to its base counterpart's score of 0.3602. The results indicated that the fine-tuned models outperformed base models, generating content more aligned with human standards. Moreover, the BERT Classifier exhibited high precision (0.95) and recall (0.97) for identifying unsuitable content, further enhancing the reliability of content classification. These findings highlight the potential of advanced language models in generating age-appropriate stories and enhancing content moderation strategies. This research has broader implications for educational technology, content curation, and parental control systems, offering a scalable approach to ensuring children's exposure to safe and enriching narratives. [ABSTRACT FROM AUTHOR]

Electronic health records (EHR) data in the medical field contain a wealth of valuable biomedical knowledge and provide a crucial resource for healthcare data analysis. However, privacy protection and data sharing constraints have become significant bottlenecks for researchers, hindering the application and development of data analysis and machine learning techniques in healthcare. To address these challenges, researchers have been exploring the use of generative modeling methods to generate EHR data. Firstly, this paper introduced and summarized the origins and evolution of diffusion models. Next, it delved into various existing diffusion model methods, providing a detailed analysis of each approach. Then it listed and compared different generative modeling methods applied in EHR data generation, emphasized the advantages and limitations of diffusion models. Finally, it summarized the current applications of diffusion models in EHR data generation, discussed the limitations of current research, and presented an outlook on the future development and application of diffusion models in this field [ABSTRACT FROM AUTHOR]

Artificial Intelligence (AI) is pivotal in advancing science, including nanomaterial studies. This review explores AI‐based image processing in nanoscience, focusing on algorithms to enhance characterization results from instruments like scanning electron microscopy, transmission electron microscopy, X‐ray diffraction, atomic force microscopy etc. It addresses the significance of AI in nanoscience, challenges in advancing AI‐based image processing for nano material characterization, and AI's role in structural analysis, property prediction, deriving structure‐property relations, dataset augmentation, and improving model robustness. Key AI techniques such as Graph Neural Networks, adversarial training, transfer learning, generative models, attention mechanisms, and federated learning are highlighted for their contributions to nano science studies. The review concludes by outlining persisting challenges and thrust areas for future research, aiming to propel nanoscience with AI. This comprehensive analysis underscores the importance of AI‐powered image processing in nanomaterial characterization, offering valuable insights for researchers. [ABSTRACT FROM AUTHOR]

In anomaly detection applications, anomalies might come from multiple sources and there might be many reasons why a sample is considered to be anomalous. However, most novel anomaly detection methods do not consider this. In our work, we describe a novel approach that is demonstrated on the problem of detection of anomalies in image data. We propose the SGVAEGAN model, which decomposes the image into three independent components—the shape of an object and its foreground and background textures—and provides anomaly scores for each of those factors separately. The overall anomaly score of an image is a weighted combination of the individual factor scores. The anomaly scores are learned in an unsupervised manner, and the weights are considered as hyperparameters that can be learned in the validation stage. The approach allows the identification of the source of the anomaly using factor scores, as well as the detection of semantic anomalies where the semantic meaning is encoded in the weights and learned from very few samples of validation anomalies. On classical anomaly detection benchmarks, the proposed model outperforms all baseline models. This is shown in a rigorous experimental study that covers the behavior of the model under a varying range of conditions. [ABSTRACT FROM AUTHOR]

The exponential growth in data from technological advancements has created opportunities across fields like healthcare, finance, and social media, but sensitive data raise security and privacy challenges. Generative models offer solutions by modeling complex data and generating synthetic data, making them useful for the analysis of large private datasets. This article is a review of data analysis techniques based on generative models, with a focus on large language models (LLMs). It covers the strengths, limitations, and applications of methods like the fine-tuning of LLMs and retrieval-augmented generation (RAG). This study consolidates, analyzes, and interprets the findings from the literature to provide a coherent overview of the current research landscape on this topic, aiming to guide effective, privacy-conscious data analysis and exploring future improvements, especially for low-resource languages. [ABSTRACT FROM AUTHOR]

Modern 3D content creation heavily relies on procedural assets. In particular, procedural materials are ubiquitous in the industry, but their manipulation remains challenging. Previous work [Hu et al. 2023] conditionally generates procedural graphs that match a given input image. However, the parameter generation step limits how accurately the generated graph matches the input image, due to a reliance on supervision with scarcely available procedural data. We propose to improve parameter prediction accuracy for image-conditioned procedural material generation by leveraging reinforcement learning (RL) and present the first RL approach for procedural materials. RL circumvents the limited availability of procedural data, the domain gap between real and synthetic materials, and the need for end-to-end differentiable loss functions. Given a target image, we retrieve a procedural material and use an RL-trained transformer model to predict a set of parameters that reconstruct the target image as closely as possible. We show that using RL significantly improves parameter prediction to match a given target image compared to supervised methods on both synthetic and real target images. [ABSTRACT FROM AUTHOR]

We introduce ToonCrafter, a novel approach that transcends traditional correspondence-based cartoon video interpolation, paving the way for generative interpolation. Traditional methods, that implicitly assume linear motion and the absence of complicated phenomena like dis-occlusion, often struggle with the exaggerated non-linear and large motions with occlusion commonly found in cartoons, resulting in implausible or even failed interpolation results. To overcome these limitations, we explore the potential of adapting live-action video priors to better suit cartoon interpolation within a generative framework. ToonCrafter effectively addresses the challenges faced when applying live-action video motion priors to generative cartoon interpolation. First, we design a toon rectification learning strategy that seamlessly adapts live-action video priors to the cartoon domain, resolving the domain gap and content leakage issues. Next, we introduce a dual-reference-based 3D decoder to compensate for lost details due to the highly compressed latent prior spaces, ensuring the preservation of fine details in interpolation results. Finally, we design a flexible sketch encoder that empowers users with interactive control over the interpolation results. Experimental results demonstrate that our proposed method not only produces visually convincing and more natural dynamics, but also effectively handles dis-occlusion. The comparative evaluation demonstrates the notable superiority of our approach over existing competitors. Code and model weights are available at https://doubiiu.github.io/projects/ToonCrafter [ABSTRACT FROM AUTHOR]

Normalizing flows (NF) use a continuous generator to map a simple latent (e.g. Gaussian) distribution, towards an empirical target distribution associated with a training data set. Once trained by minimizing a variational objective, the learnt map provides an approximate generative model of the target distribution. Since standard NF implement differentiable maps, they may suffer from pathological behaviors when targeting complex distributions. For instance, such problems may appear for distributions on multi-component topologies or characterized by multiple modes with high probability regions separated by very unlikely areas. A typical symptom is the explosion of the Jacobian norm of the transformation in very low probability areas. This paper proposes to overcome this issue thanks to a new Markov chain Monte Carlo algorithm to sample from the target distribution in the latent domain before transporting it back to the target domain. The approach relies on a Metropolis adjusted Langevin algorithm whose dynamics explicitly exploits the Jacobian of the transformation. Contrary to alternative approaches, the proposed strategy preserves the tractability of the likelihood and it does not require a specific training. Notably, it can be straightforwardly used with any pre-trained NF network, regardless of the architecture. Experiments conducted on synthetic and high-dimensional real data sets illustrate the efficiency of the method. [ABSTRACT FROM AUTHOR]

Copyright of Science & Technology of Food Industry is the property of Science & Technology of Food Industry Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)