Search

Generative models in the field of artificial intelligence and their applications and deployment have demonstrated their great strength in the past few years. Of the vast spectrum of generative models, diffusion probabilistic models have proven to be particularly powerful and productive, transforming notions such as text-to-image and text-to-video generation from ideas into practical applications. In our previous works, we proposed a thermal metamaterial-based periodic interparticle interaction mechanism for heat management, with a specific application in thermal transparency. To address the challenging problems associated with the inverse design of thermal metamaterial structures, we employed an autoencoder-based machine learning approach and a reinforcement learning-based approach successfully. In this work, we demonstrate that our particular problems with the inverse design of thermal metamaterial-based periodic lattices for the realization of thermal transparency can also be reframed and efficiently solved by training a generative diffusion probabilistic model that can generate the design parameters corresponding to the desired response. Furthermore, we show that for a specific response, multiple sets of design parameters can be obtained by simply performing multiple inferences with the generative diffusion probabilistic model, enabling us to select the ones that can be more economical to fabricate and implement. Our work is among the first to use a diffusion model for the inverse design of thermal metamaterial-based structures and demonstrates the effectiveness of generating low-dimensional design parameters through a diffusion model. [ABSTRACT FROM AUTHOR]

The structural characterization of proteins with a disorder requires a computational approach backed by experiments to model their diverse and dynamic structural ensembles. The selection of conformational ensembles consistent with solution experiments of disordered proteins highly depends on the initial pool of conformers, with currently available tools limited by conformational sampling. We have developed a Generative Recurrent Neural Network (GRNN) that uses supervised learning to bias the probability distributions of torsions to take advantage of experimental data types such as nuclear magnetic resonance J-couplings, nuclear Overhauser effects, and paramagnetic resonance enhancements. We show that updating the generative model parameters according to the reward feedback on the basis of the agreement between experimental data and probabilistic selection of torsions from learned distributions provides an alternative to existing approaches that simply reweight conformers of a static structural pool for disordered proteins. Instead, the biased GRNN, DynamICE, learns to physically change the conformations of the underlying pool of the disordered protein to those that better agree with experiments. [ABSTRACT FROM AUTHOR]

Background: Consider a setting where multiple parties holding sensitive data aim to collaboratively learn population level statistics, but pooling the sensitive data sets is not possible due to privacy concerns and parties are unable to engage in centrally coordinated joint computation. We study the feasibility of combining privacy preserving synthetic data sets in place of the original data for collaborative learning on real-world health data from the UK Biobank. Methods: We perform an empirical evaluation based on an existing prospective cohort study from the literature. Multiple parties were simulated by splitting the UK Biobank cohort along assessment centers, for which we generate synthetic data using differentially private generative modelling techniques. We then apply the original study's Poisson regression analysis on the combined synthetic data sets and evaluate the effects of 1) the size of local data set, 2) the number of participating parties, and 3) local shifts in distributions, on the obtained likelihood scores. Results: We discover that parties engaging in the collaborative learning via shared synthetic data obtain more accurate estimates of the regression parameters compared to using only their local data. This finding extends to the difficult case of small heterogeneous data sets. Furthermore, the more parties participate, the larger and more consistent the improvements become up to a certain limit. Finally, we find that data sharing can especially help parties whose data contain underrepresented groups to perform better-adjusted analysis for said groups. Conclusions: Based on our results we conclude that sharing of synthetic data is a viable method for enabling learning from sensitive data without violating privacy constraints even if individual data sets are small or do not represent the overall population well. Lack of access to distributed sensitive data is often a bottleneck in biomedical research, which our study shows can be alleviated with privacy-preserving collaborative learning methods. [ABSTRACT FROM AUTHOR]

Conventional and electron microscopy visualize structures in the micrometer to nanometer range, and such visualizations contribute decisively to our understanding of biological processes. Due to different factors in recording processes, microscopy images are subject to noise. Especially at their respective resolution limits, a high degree of noise can negatively effect both image interpretation by experts and further automated processing. However, the deteriorating effects of strong noise can be alleviated to a large extend by image enhancement algorithms. Because of the inherent high noise, a requirement for such algorithms is their applicability directly to noisy images or, in the extreme case, to just a single noisy image without a priori noise level information (referred to as blind zero-shot setting). This work investigates blind zero-shot algorithms for microscopy image denoising. The strategies for denoising applied by the investigated approaches include: filtering methods, recent feed-forward neural networks which were amended to be trainable on noisy images, and recent probabilistic generative models. As datasets we consider transmission electron microscopy images including images of SARS-CoV-2 viruses and fluorescence microscopy images. A natural goal of denoising algorithms is to simultaneously reduce noise while preserving the original image features, e.g., the sharpness of structures. However, in practice, a tradeoff between both aspects often has to be found. Our performance evaluations, therefore, focus not only on noise removal but set noise removal in relation to a metric which is instructive about sharpness. For all considered approaches, we numerically investigate their performance, report their denoising/sharpness tradeoff on different images, and discuss future developments. We observe that, depending on the data, the different algorithms can provide significant advantages or disadvantages in terms of their noise removal vs. sharpness preservation capabilities, which may be very relevant for different virological applications, e.g., virological analysis or image segmentation. Author summary: For any image (e.g., recorded using digital photography or electron microscopy), there is a limiting resolution for which the image still looks sharp. Beyond this resolution (by further zooming in), an image becomes increasingly noisy. Resolution limits may be improved technically (bigger lenses, more light, longer exposure times etc.). Alternatively, or additionally, modern denoising algorithms can be applied after the image is recorded. Here we investigate state-of-the-art such algorithms with many of them being developed in field of Machine Learning. The algorithms work by first learning representations of images (e.g., edges, textures etc.), which they then use to remove the noise from an image. To be applicable at high resolutions, algorithms need to be able to learn from noisy images. If just one noisy image is available for learning without a priori noise level information, the task is also referred to as blind zero-shot denoising. Importantly, we argue that the implicit goal of denoising is noise removal without majorly diminishing image sharpness. All the here investigated blind zero-shot algorithms are consequently evaluated based on both: noise removal and sharpness preservation. We observed salient strengths and weaknesses of different approaches for different image types, which can significantly impact different tasks. [ABSTRACT FROM AUTHOR]

This paper proposes a method called Meta Generative Data Augmentation Optimization (MGDAO) to overcome limited types of operations for the policy-based automatic data augmentation method. While traditional data augmentation methods have relied on expert intuition to determine effective transformations, recent approaches have attempted to generate data augmentation strategies automatically. However, these automatic methods can still suffer from limited operation sets and difficulty training conditional generative models. To address these issues, MGDAO replaces the limited operations space in the AutoAugment series with a deep-style generator and replaces the discriminator in a generative adversarial model with the validation loss of the target model. These replacements released fixed image operations and made MGDAO useful for sequential data. The generator learns to transform the data from the training domain to the validation data domain. It is further used to generate augmented samples to train the target model and reduce the validation loss. Experiments on classification benchmarks of few-shot image and text-based datasets show that MGDAO achieves competitive results compared to data auto-augmentation methods. [ABSTRACT FROM AUTHOR]

Smart grids commonly rely on analyzing sensor data to monitor power equipment. However, these sensor data can suffer varying levels of loss or corruption due to complex interferences, leading to a pressing need for precise missing value imputation in power equipment data. We propose a data imputation model for power equipment based on generative adversarial networks with an auxiliary encoder, named PEDI-GAN. In particular, the auxiliary encoder is designed and integrated into the GAN structure to optimize random vectors for the generator. Through data masking, we pinpoint missing data locations, enabling the generator to focus on generating accurate values for those points. Additionally, we address gradient disappearance and model collapse in GAN training by using the gradient penalty to redesign the loss function for PEDI-GAN. Experimental results demonstrate PEDI-GAN's superiority in accuracy and generalization compared to baseline methods, with notable reductions in mean absolute error and root mean square error by an average of 16.75% and 11.09%, respectively. [ABSTRACT FROM AUTHOR]

Statistical depth, a commonly used analytic tool in nonparametric statistics, has been extensively studied for multivariate and functional observations over the past few decades. Although various forms of depth were introduced, they are mainly procedure based whose definitions are independent of the generative model for observations. To address this problem, we introduce a generative model-based approach to define statistical depth for both multivariate and functional data. The proposed model-based depth framework permits simple computation via a bootstrap sampling and improves the depth estimation accuracy. When applied to functional data, the proposed depth can capture important features such as continuity, smoothness or phase variability, depending on the defining criteria. We propose efficient algorithms to compute the proposed depths and establish estimation consistency. Through simulations and real data, we demonstrate that the proposed functional depths reveal important statistical information such as those captured by the median and quantiles, and detect outliers. [ABSTRACT FROM AUTHOR]

Although information theory resolves the inconsistencies (known in the form of famous enigmas) of the traditional approach of thermostatistics, its place in the corresponding literature is not what it deserves. This article supports the idea that this is mainly due to epistemological rather than scientific reasons: the subjectivity introduced into physics is perceived as a problem. Here is an attempt to expose and clarify where exactly this subjectivity lies: in the representation of reality and in probabilistic inference, two aspects that have been integrated into the practice of science for a long time and which should no longer frighten anyone but have become explicit with information theory. [ABSTRACT FROM AUTHOR]

Motivation Single-cell profiling has become a common practice to investigate the complexity of tissues, organs, and organisms. Recent technological advances are expanding our capabilities to profile various molecular layers beyond the transcriptome such as, but not limited to, the genome, the epigenome, and the proteome. Depending on the experimental procedure, these data can be obtained from separate assays or the very same cells. Yet, integration of more than two assays is currently not supported by the majority of the computational frameworks avaiable. Results We here propose a Multi-Omic data integration framework based on Wasserstein Generative Adversarial Networks suitable for the analysis of paired or unpaired data with a high number of modalities (>2). At the core of our strategy is a single network trained on all modalities together, limiting the computational burden when many molecular layers are evaluated. Availability and implementation Source code of our framework is available at https://github.com/vgiansanti/MOWGAN [ABSTRACT FROM AUTHOR]

The Bayesian approach to solving inverse problems relies on the choice of a prior. This critical ingredient allows expert knowledge or physical constraints to be formulated in a probabilistic fashion and plays an important role for the success of the inference. Recently, Bayesian inverse problems were solved using generative models as highly informative priors. Generative models are a popular tool in machine learning to generate data whose properties closely resemble those of a given database. Typically, the generated distribution of data is embedded in a low-dimensional manifold. For the inverse problem, a generative model is trained on a database that reflects the properties of the sought solution, such as typical structures of the tissue in the human brain in magnetic resonance imaging. The inference is carried out in the low-dimensional manifold determined by the generative model that strongly reduces the dimensionality of the inverse problem. However, this procedure produces a posterior that does not admit a Lebesgue density in the actual variables and the accuracy attained can strongly depend on the quality of the generative model. For linear Gaussian models, we explore an alternative Bayesian inference based on probabilistic generative models; this inference is carried out in the original high-dimensional space. A Laplace approximation is employed to analytically derive the prior probability density function required, which is induced by the generative model. Properties of the resulting inference are investigated. Specifically, we show that derived Bayes estimates are consistent, in contrast to the approach in which the low-dimensional manifold of the generative model is employed. The MNIST data set is used to design numerical experiments that confirm our theoretical findings. It is shown that the approach proposed can be advantageous when the information contained in the data is high and a simple heuristic is considered for the detection of this case. Finally, the pros and cons of both approaches are discussed. [ABSTRACT FROM AUTHOR]

Cross-project defect prediction (CPDP) aims to predict software defects in a target project domain by leveraging information from different source project domains, allowing testers to identify defective modules quickly. However, CPDP models often underperform due to different data distributions between source and target domains, class imbalances, and the presence of noisy and irrelevant instances in both source and target projects. Additionally, standard features often fail to capture sufficient semantic and contextual information from the source project, leading to poor prediction performance in the target project. To address these challenges, this research proposes Smote Correlation and Attention Gated recurrent unit based Long Short-Term Memory optimization (SCAG-LSTM), which first employs a novel hybrid technique that extends the synthetic minority over-sampling technique (SMOTE) with edited nearest neighbors (ENN) to rebalance class distributions and mitigate the issues caused by noisy and irrelevant instances in both source and target domains. Furthermore, correlation-based feature selection (CFS) with best-first search (BFS) is utilized to identify and select the most important features, aiming to reduce the differences in data distribution among projects. Additionally, SCAG-LSTM integrates bidirectional gated recurrent unit (Bi-GRU) and bidirectional long short-term memory (Bi-LSTM) networks to enhance the effectiveness of the long short-term memory (LSTM) model. These components efficiently capture semantic and contextual information as well as dependencies within the data, leading to more accurate predictions. Moreover, an attention mechanism is incorporated into the model to focus on key features, further improving prediction performance. Experiments are conducted on apache_lucene, equinox, eclipse_jdt_core, eclipse_pde_ui, and mylyn (AEEEM) and predictor models in software engineering (PROMISE) datasets and compared with active learning-based method (ALTRA), multi-source-based cross-project defect prediction method (MSCPDP), the two-phase feature importance amplification method (TFIA) on AEEEM and the two-phase transfer learning method (TPTL), domain adaptive kernel twin support vector machines method (DA-KTSVMO), and generative adversarial long-short term memory neural networks method (GB-CPDP) on PROMISE datasets. The results demonstrate that the proposed SCAG-LSTM model enhances the baseline models by 33.03%, 29.15% and 1.48% in terms of F1-measure and by 16.32%, 34.41% and 3.59% in terms of Area Under the Curve (AUC) on the AEEEM dataset, while on the PROMISE dataset it enhances the baseline models' F1-measure by 42.60%, 32.00% and 25.10% and AUC by 34.90%, 27.80% and 12.96%. These findings suggest that the proposed model exhibits strong predictive performance. [ABSTRACT FROM AUTHOR]

In this paper we investigate the application of Bing Chat in stock-market-related news headlines and evaluate whether it can offer accurate predictions to enhance investors' returns. We first perform an analysis using web data scraping techniques to gather information. We then employ a text classification method to score the replied content of Bing Chat. Finally, we compare these scores with financial data for further evaluation. The experimental results demonstrate that Bing Chat can provide practical assistance in the analysis of stock-market-related news headlines. It can be considered a potential financial investment tool that enhances the accuracy and efficacy for decision-making in investment. [ABSTRACT FROM AUTHOR]

Image inpainting aims to restore missing or damaged regions of an image with plausible visual content. Most existing methods always face challenges when dealing with large hole images, such as structural distortion and prominent artifacts, which mainly stem from the absence of adequate semantic guidance. To overcome this problem, this paper proposes a novel progressive image inpainting network driven by dynamic context, where the guided restoration dynamic semantic prior not only includes information from the known region but also considers the semantic inference during the filling process. Specifically, a multi-view cooperative strategy is proposed firstly to predict the inherent semantic information by estimating the distributions of the image, grayscale and edge of the masked image. In addition, to cope with potential semantic changes, an auxiliary generative unit is proposed for learning semantic information of the intermediate inpainting results, where the learned intermediate semantics are fused with the intrinsic semantics in a weighted manner to form a real-time updating dynamic context. Moreover, the dynamic semantic prior is propagated to various stages of inpainting to assist in constructing refined feature maps with multiple levels of resolution. Experiments on CelebA-HQ and Paris street view datasets demonstrate that the proposed approach can recover reasonable structures and realistic textures on large-scale masked images, achieving advanced performance. [ABSTRACT FROM AUTHOR]

In the realm of targeted advertising, the demand for precision is paramount, and the traditional centralized machine learning paradigm fails to address this necessity effectively. Two critical challenges persist in the current advertising ecosystem: the data privacy concerns leading to isolated data islands and the complexity in handling non-Independent and Identically Distributed (non-IID) data and concept drift due to the specificity and diversity in user behavior data. Current federated learning frameworks struggle to overcome these hurdles satisfactorily. This paper introduces Fed-GANCC, an innovative federated learning framework that synergizes Generative Adversarial Networks (GANs) and Group Clustering. The framework incorporates a user data augmentation algorithm predicated on adversarial generative networks to enrich user behavior data, curtail the impact of non-uniform data distribution, and enhance the applicability of the global machine learning model. Unlike traditional approaches, our framework offers user data augmentation algorithms based on adversarial generative networks, which not only enriches user behavior data but also reduces the challenges posed by non-uniform data distribution, thereby enhancing the applicability of the global machine learning (ML) model. The effectiveness of Fed-GANCC is distinctly showcased through experimental results, outperforming contemporary methods like FED-AVG and FED-SGD in terms of accuracy, loss value, and receiver operating characteristic (ROC) indicators within the same computing time. Experimental results vindicate the effectiveness of Fed-GANCC, revealing substantial enhancements in accuracy, loss value, and receiver operating characteristic (ROC) metrics compared to FED-AVG and FED-SGD given the same computational time. These outcomes underline Fed-GANCC's exceptional prowess in mitigating issues such as isolated data islands, non-IID data, and concept drift. With its novel approach to addressing the prevailing challenges in targeted advertising such as isolated data islands, non-IID data, and concept drift, the Fed-GANCC framework stands as a benchmark, paving the way for future advancements in federated learning solutions tailored for the advertising domain. The Fed-GANCC framework promises to offer pivotal insights for the future development of efficient and advanced federated learning solutions for targeted advertising. [ABSTRACT FROM AUTHOR]

Tidal turbines play a critical role in converting the kinetic energy of water into electricity, contributing significantly to energy conversion. However, the current optimization design of these turbines involves computationally intensive simulations, leading to higher design costs. Additionally, traditional parameterized modeling methods, constrained by predefined design parameters, limit the exploration of innovative designs. In response, this study introduces an innovative data-driven "generative–predictive" design approach comprising a generative model and a predictive model. The generative model autonomously learns feature representations from existing turbines and leverages this knowledge to generate a novel set of turbines with superior hydrodynamic performance. Subsequently, an efficient performance evaluation is conducted using a predictive model for the generated turbines. Compared to the current parameterized modeling approaches, the proposed approach is combined with multi-objective optimization algorithm to optimize the tidal turbine hydrodynamic performance. Research results demonstrate that the generative model, trained on gradients, can generate highly complex turbines with minimal latent vectors. Through transfer learning, the predictive model exhibits robustness and accuracy, effectively guiding the design process. In the final optimization comparison, the proposed generative–predictive design approach requires only 4% of the optimization time while achieving results similar to or surpassing traditional design approaches. This approach proves to be a powerful tool for guiding the efficient and optimized design of turbines. [ABSTRACT FROM AUTHOR]

The convolutional neural network-based deep generative model (DGM) is a powerful tool for handling image datasets that opens up strategies for fast synthesis of optimum designs at unseen boundary conditions. Existing DGMs for design synthesis are typically based on O (10000) training data, which limits the engineering applications. This paper explores the feasibility of improving DGM data efficiency with O (100) training data through prior constraints. A two-stage data-efficient deep generative model (DE-DGM) is proposed which leverages the first-stage design synthesis from probabilistic proper orthogonal decomposition and the second-stage enhancement from encoder-decoder convolutional neural network. Four topology optimization cases have been adopted, including compliance minimization, heat conduction, airplane bearing bracket design, and three-dimensional machine tool column structure design. The proposed DE-DGMs could be trained with 100–200 data and synthesize the main features of the design at unseen boundary conditions. The overall computation cost of warm-start topology optimization leveraging DE-DGM predictions reduces to 36 %–58 % of the standard cases. [ABSTRACT FROM AUTHOR]

Class imbalance remains a large problem in high-throughput omics analyses, causing bias towards the over-represented class when training machine learning-based classifiers. Oversampling is a common method used to balance classes, allowing for better generalization of the training data. More naive approaches can introduce other biases into the data, being especially sensitive to inaccuracies in the training data, a problem considering the characteristically noisy data obtained in healthcare. This is especially a problem with high-dimensional data. A generative adversarial network-based method is proposed for creating synthetic samples from small, high-dimensional data, to improve upon other more naive generative approaches. The method was compared with 'synthetic minority over-sampling technique' (SMOTE) and 'random oversampling' (RO). Generative methods were validated by training classifiers on the balanced data. [ABSTRACT FROM AUTHOR]

In Taiwan, traditional production equipment for the mainframe panels is imported from overseas, and the parameters are adjusted through the operation panel for automated manufacturing. However, these parameters are slightly different from those used in Taiwan, requiring manual recording of warning parameters to ensure quality control. Currently, the only way to capture the operation records of these machines is by manually recording the system panel information, which is a time-consuming and laborious process that is expensive for production line personnel. In this work, we use image recognition to capture and analyze data externally without damaging the machine. This research improves the ESRGAN network to restore the image details and textures of the mainframe panels. It also captures relevant data with Google OCR to convert panel images into parameter values. The captured results are combined with data analysis to provide more accurate standard mainframe panel information. Even under different interference source conditions, data extraction and analysis can still be carried out to output the text files, effectively assisting the production line in recording the machine panel parameters and reducing personnel's workload. [ABSTRACT FROM AUTHOR]

Recent studies on watermarking techniques based on image carriers have demonstrated new approaches that combine adversarial perturbations against steganalysis with embedding distortions. However, while these methods successfully counter convolutional neural network-based steganalysis, they do not adequately protect the data of the carrier itself. Recognizing the high sensitivity of Deep Neural Networks (DNNs) to small perturbations, we propose HAG-NET, a method based on image carriers, which is jointly trained by the encoder, decoder, and attacker. In this paper, the encoder generates Adversarial Steganographic Examples (ASEs) that are adversarial to the target classification network, thereby providing protection for the carrier data. Additionally, the decoder can recover secret data from ASEs. The experimental results demonstrate that ASEs produced by HAG-NET achieve an average success rate of over 99% on both the MNIST and CIFAR-10 datasets. ASEs generated with the attacker exhibit greater robustness in terms of attack ability, with an average increase of about 3.32%. Furthermore, our method, when compared with other generative stego examples under similar perturbation strength, contains significantly more information according to image information entropy measurements. [ABSTRACT FROM AUTHOR]

This article delves into the debate between Predictive Processing (PP) models of cognition and enactivist views on representation. The author argues against the enactivist critique of PP's non-semantic representational content, suggesting that a distinction between different senses of 'use' can resolve this issue. The article also explores the question of whether iconic representations have content or are merely characterized by covariance. The author proposes that the adequacy of iconic representations can be understood in terms of their function and properties as evolutionarily entrenched orientational maps. The article concludes by suggesting that the author's view can be labeled as a version of 'contentless representationalism'. Overall, the article provides insights into the debate between enactivism and predictive processing and offers a nuanced perspective on the nature of representation. [Extracted from the article]

In randomized experiments, adjusting for observed features when estimating treatment effects has been proposed as a way to improve asymptotic efficiency. However, among parametric methods, only linear regression has been proven to form an estimate of the average treatment effect that is asymptotically no less efficient than the treated-minus-control difference in means regardless of the true data generating process. Randomized treatment assignment provides this do-no-harm property, with neither truth of a linear model nor a generative model for the outcomes being required. We present a general calibration method that confers the same no-harm property onto estimators leveraging a broad class of nonlinear models. This recovers the usual regression-adjusted estimator when ordinary least squares is used, and further provides noninferior treatment effect estimators using methods such as logistic and Poisson regression. The resulting estimators are noninferior to both the difference-in-means estimator and to treatment effect estimators that have not undergone calibration. We show that our estimator is asymptotically equivalent to an inverse-probability-weighted estimator using a logit link with predicted potential outcomes as covariates. In a simulation study, we demonstrate that common nonlinear estimators without our calibration procedure may perform markedly worse than both the calibrated estimator and the unadjusted difference in means. [ABSTRACT FROM AUTHOR]

A gray market emerges when some distributors divert products to unauthorized distributors/retailers to make sneaky profits from the manufacturers' differential channel incentives, such as quantity discounts. Traditionally, manufacturers rely heavily on internal audits to periodically investigate the flows of products and funds so as to deter the gray market; however, this is too costly given the large number of distributors and their huge volumes of orders. Owing to the advances in data analytics techniques, the ordering quantities of a distributor over time, which form multivariate time series, can help reveal suspicious product diversion behaviors and narrow the audit scope drastically. To that end, in this paper, we build on the recent advancement of representation learning for time series and adopt a sequence autoencoder to automatically characterize the overall demand patterns. To cope with the underlying entangled factors and interfering information in the multivariate time series of ordering quantities, we develop a disentangled learning scheme to construct more effective sequence representations. An interdistributor correlation regularization is also proposed to ensure more reliable representations. Finally, given the highly scarce anomaly labels for the detection task, an unsupervised deep generative model based on the learned representations of the distributors is developed to estimate the densities of distributions, which enables the anomaly scores generated through end-to-end learning. Extensive experiments on a real-world distribution channel data set and a larger simulated data set empirically validate our model's superior and robust performances compared with several state-of-the-art baselines. Additionally, our illustrative economic analysis demonstrates that the manufacturers can launch more targeted and cost-effective audits toward the suspected distributors recommended by our model so as to deter the gray market. History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning. Funding: This work was supported by the National Natural Science Foundation of China [Grants 72031001, 72301017, 72371011, and 72242101]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information (https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0155) as well as from the IJOC GitHub software repository (https://github.com/INFORMSJoC/2022.0155). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/. [ABSTRACT FROM AUTHOR]

Generative models have recently received a lot of attention. However, a challenge with such models is that it is usually not possible to compute the likelihood function, which makes parameter estimation or training of the models challenging. The most commonly used alternative strategy is called likelihood-free estimation, based on finding values of the model parameters such that a set of selected statistics have similar values in the dataset and in samples generated from the model. However, a challenge is how to select statistics that are efficient in estimating unknown parameters. The most commonly used statistics are the mean vector, variances, and correlations between variables, but they may be less relevant in estimating the unknown parameters. We suggest utilizing Tukey depth contours (TDCs) as statistics in likelihood-free estimation. TDCs are highly flexible and can capture almost any property of multivariate data, in addition, they seem to be as of yet unexplored for likelihood-free estimation. We demonstrate that TDC statistics are able to estimate the unknown parameters more efficiently than mean, variance, and correlation in likelihood-free estimation. We further apply the TDC statistics to estimate the properties of requests to a computer system, demonstrating their real-life applicability. The suggested method is able to efficiently find the unknown parameters of the request distribution and quantify the estimation uncertainty. [ABSTRACT FROM AUTHOR]

There have been recent calls for wider application of generative modelling approaches in applied social network analysis. At present, however, it remains difficult for typical end users—for example, field researchers—to implement generative network models, as there is a dearth of openly available software packages that make application of such models as simple as other, permutation‐based approaches.Here, we outline the STRAND R package, which provides a suite of generative models for Bayesian analysis of animal social network data that can be implemented using simple, base R syntax.To facilitate ease of use, we provide a tutorial demonstrating how STRAND can be used to model proportion, count or binary network data using stochastic block models, social relation models or a combination of the two modelling frameworks.STRAND facilitates the application of generative network models to a broad range of data found in the animal social networks literature. [ABSTRACT FROM AUTHOR]

Extracting the rules of real-world multi-agent behaviors is a current challenge in various scientific and engineering fields. Biological agents independently have limited observation and mechanical constraints; however, most of the conventional data-driven models ignore such assumptions, resulting in lack of biological plausibility and model interpretability for behavioral analyses. Here we propose sequential generative models with partial observation and mechanical constraints in a decentralized manner, which can model agents' cognition and body dynamics, and predict biologically plausible behaviors. We formulate this as a decentralized multi-agent imitation-learning problem, leveraging binary partial observation and decentralized policy models based on hierarchical variational recurrent neural networks with physical and biomechanical penalties. Using real-world basketball and soccer datasets, we show the effectiveness of our method in terms of the constraint violations, long-term trajectory prediction, and partial observation. Our approach can be used as a multi-agent simulator to generate realistic trajectories using real-world data. [ABSTRACT FROM AUTHOR]

The precise forecasting of wind speeds is critical to lessen the harmful impacts of wind fluctuations on power networks and aid in merging wind energy into the grid system. However, prior research has predominantly focused on point forecasts, often overlooking the uncertainties inherent in the prediction accuracy. For this research, we suggest a new approach for forecasting wind speed intervals (PI). Specifically, the actual wind speed series are initially procured, and the complete ensemble empirical mode decomposition coupled with adaptive noise (CEEMDAN) method decomposes the actual wind speed series into constituent numerous mode functions. Furthermore, a generative adversarial network (GAN) is utilized to achieve the wind speed PI in conjunction with the multivariate linear regression method. To confirm the effectiveness of the suggested model, four datasets are selected. The validation results suggest that this suggested model attains a superior PI accuracy compared with those of numerous benchmark techniques. In the context of PI of dataset 4, the PINAW values show improvements of 68.06% and 32.35% over the CEEMDAN-CNN and VMD-GRU values in single-step forecasting, respectively. In conclusion, the proposed model excels over the counterpart models by exhibiting diminished a PINAW and CWC, while maintaining a similar PICP. [ABSTRACT FROM AUTHOR]

The purpose of imbalanced data classification is to solve the problem of unfair learning caused by the large difference in data distribution. Traditional classifiers are designed on the basis of balanced data, but the performance of imbalanced data will decline sharply. Therefore, balancing the majority class and minority class samples before classification is a popular strategy for solving imbalanced learning. Current methods for data balance mainly include oversampling and undersampling. However, the existing undersampling will face the problem of losing important sample information, while oversampling cannot effectively fit the global distribution and generate noise. In recent years, generative adversarial network (GAN) has shown great potential in fitting real sample distributions. Based on this, this paper proposes an improved GAN and biased loss combined model, namely VGAN-BL, to solve the learning problem under imbalanced conditions. In the improvement based on GAN, VAE is used to generate latent vectors with posterior distribution as the input of GAN, and KL similarity measurement loss is introduced into the generator to improve the quality of minority samples generated by GAN. In addition, we propose a biased loss definition method based on the discriminator to improve the performance of classifier. Experiments on 20 real datasets show that the classification performance of the proposed method is significantly improved compared with other advanced methods. The source code can be found here: https://github.com/universuen/VGAN-BL. [ABSTRACT FROM AUTHOR]

A key metric to determine the performance of a stock in a market is its return over different investment horizons (τ). Several works have observed heavy-tailed behavior in the distributions of returns in different markets, which are observable indicators of underlying complex dynamics. Such prior works study return distributions that are marginalized across the individual stocks in the market, and do not track statistics about the joint distributions of returns conditioned on different stocks, which would be useful for optimizing inter-stock asset allocation strategies. As a step towards this goal, we study emergent phenomena in the distributions of returns as captured by their pairwise correlations. In particular, we consider the pairwise (between stocks i, j) partial correlations of returns with respect to the market mode, ci,j(τ), (thus, correcting for the baseline return behavior of the market), over different time horizons (τ), and discover two novel emergent phenomena: (i) the standardized distributions of the ci,j(τ)'s are observed to be invariant of τ ranging from from 1000min (2.5 days) to 30000min (2.5 months); (ii) the scaling of the standard deviation of ci,j(τ)'s with τ admits good fits to simple model classes such as a power-law τ−λ or stretched exponential function e-τβ (λ, β > 0). Moreover, the parameters governing these fits provide a summary view of market health: for instance, in years marked by unprecedented financial crises—for example 2008 and 2020—values of λ (scaling exponent) are substantially lower. Finally, we demonstrate that the observed emergent behavior cannot be adequately supported by existing generative frameworks such as single- and multi-factor models. We introduce a promising agent-based Vicsek model that closes this gap. [ABSTRACT FROM AUTHOR]

Nowadays, most trajectory prediction algorithms have difficulty simulating actual traffic behavior, and there is still a problem of large prediction errors. Therefore, this paper proposes a multi-object trajectory prediction algorithm based on lane information and foresight information. A Hybrid Dilated Convolution module based on the Channel Attention mechanism (CA-HDC) is developed to extract features, which improves the lane feature extraction in complicated environments and solves the problem of poor robustness of the traditional PINet. A lane information fusion module and a trajectory adjustment module based on the foresight information are developed. A socially acceptable trajectory with Generative Adversarial Networks (S-GAN) is developed to reduce the error of the trajectory prediction algorithm. The lane detection accuracy in special scenarios such as crowded, shadow, arrow, crossroad, and night are improved on the CULane dataset. The average F1-measure of the proposed lane detection has been increased by 4.1% compared to the original PINet. The trajectory prediction test based on D2-City indicates that the average displacement error of the proposed trajectory prediction algorithm is reduced by 4.27%, and the final displacement error is reduced by 7.53%. The proposed algorithm can achieve good results in lane detection and multi-object trajectory prediction tasks. [ABSTRACT FROM AUTHOR]

(1) Background: In order to solve the problem of missing time-series data due to the influence of the acquisition system or external factors, a missing time-series data interpolation method based on random forest and a generative adversarial interpolation network is proposed. (2) Methods: First, the position of the missing part of the data is calibrated, and the trained random forest algorithm is used for the first data interpolation. The output value of the random forest algorithm is used as the input value of the generative adversarial interpolation network, and the generative adversarial interpolation network is used to calibrate the position. The data are interpolated for the second time, and the advantages of the two algorithms are combined to make the interpolation result closer to the true value. (3) Results: The filling effect of the algorithm is tested on a certain bearing data set, and the root mean square error (RMSE) is used to evaluate the interpolation results. The results show that the RMSE of the interpolation results based on the random forest and generative adversarial interpolation network algorithms in the case of single-segment and multi-segment missing data is only 0.0157, 0.0386, and 0.0527, which is better than the random forest algorithm, generative adversarial interpolation network algorithm, and K-nearest neighbor algorithm. (4) Conclusions: The proposed algorithm performs well in each data set and provides a reference method in the field of data filling. [ABSTRACT FROM AUTHOR]

The image‐to‐image translation (I2IT) task aims to transform images from the source domain into the specified target domain. State‐of‐the‐art CycleGAN‐based translation algorithms typically use cycle consistency loss and latent regression loss to constrain translation. In this work, it is demonstrated that the model parameters constrained by the cycle consistency loss and the latent regression loss are equivalent to optimizing the medians of the data distribution and the generative distribution. In addition, there is a style bias in the translation. This bias interacts between the generator and the style encoder and visually exhibits translation errors, e.g. the style of the generated image is not equal to the style of the reference image. To address these issues, a new I2IT model termed high‐quality‐I2IT (HQ‐I2IT) is proposed. The optimization scheme is redesigned to prevent the model from optimizing the median of the data distribution. In addition, by separating the optimization of the generator and the latent code estimator, the redesigned model avoids error interactions and gradually corrects errors during training, thereby avoiding learning the median of the generated distribution. The experimental results demonstrate that the visual quality of the images produced by HQ‐I2IT is significantly improved without changing the generator structure, especially when guided by the reference images. Specifically, the Fréchet inception distance on the AFHQ and CelebA‐HQ datasets are reduced from 19.8 to 10.2 and from 23.8 to 17.0, respectively. [ABSTRACT FROM AUTHOR]

Generative models have gained significant attention in recent years. They are increasingly used to estimate the underlying structure of high-dimensional data and artificially generate various kinds of data similar to those from the real world. The performance of generative models depends critically on a good set of hyperparameters. Yet, finding the right hyperparameter configuration can be an extremely time-consuming task. In this paper, we focus on speeding up the hyperparameter search through adaptive resource allocation, early stopping underperforming candidates quickly and allocating more computational resources to promising ones by comparing their intermediate performance. The hyperparameter search is formulated as a non-stochastic best-arm identification problem where resources like iterations or training time constrained by some predetermined budget are allocated to different hyperparameter configurations. A procedure which uses hypothesis testing coupled with Successive Halving is proposed to make the resource allocation and early stopping decisions and compares the intermediate performance of generative models by their exponentially weighted Maximum Means Discrepancy (MMD). The experimental results show that the proposed method selects hyperparameter configurations that lead to a significant improvement in the model performance compared to Successive Halving for a wide range of budgets across several real-world applications. [ABSTRACT FROM AUTHOR]

Recently, large pretrained language models (PLMs) have led a revolution in the information retrieval community. In most PLMs‐based retrieval frameworks, the ranking performance broadly depends on the model structure and the semantic complexity of the input text. Sequence‐to‐sequence generative models for question answering or text generation have proven to be competitive, so we wonder whether these models can improve ranking effectiveness by enhancing input semantics. This article introduces SE‐BERT, a semantically enhanced bidirectional encoder representation from transformers (BERT) based ranking framework that captures more semantic information by modifying the input text. SE‐BERT utilizes a pretrained generative language model to summarize both sides of the candidate passage and concatenate them into a new input sequence, allowing BERT to acquire more semantic information within the constraints of the input sequence's length. Experimental results from two Text Retrieval Conference datasets demonstrate that our approach's effectiveness increasing as the length of the input text increases. [ABSTRACT FROM AUTHOR]

The applications of modern large language models are diverse and new at the same time. It is forecasted that the scientific society and businesses may experience a long period of time exploring all opportunities and challenges for using them which will allow analysis of the impact on how advanced generative artificial intelligence is changing work occupation activities and performance efficiency. Undoubtedly, today’s question that every business must answer is not if but how to implement large language models, due to their ability to transform numerous business processes. This study aims to give a better understanding on how large language models are contributing to the process of exploratory data analysis as they are not here to replace the traditional methods but to add generative artificial intelligence capabilities to the well-established ones. The results of this paper reveal high level of accuracy of the paired output between operation prompts in OpenAI’s large language model and human-mediated entry. However, such output comparison highlighs the need for more informative and specific input prompts to ascertain this accuracy. Further caveats that need to be placed in consideration refer to possible system downtimes, as well as the expenses incurred with every prompt execution. Nevertheless, the comparative speed of operation of large language models remains their most substantial competitive advantage. Overall, the findings in this paper contribute to understanding that large language models streamline with ease the desired extraction of insightful information which may further be used for better decision-making, good data management, and design of winning growth strategy as is the case of the gaming industry. [ABSTRACT FROM AUTHOR]

Given the often enormous effort required to train GANs, both computationally as well as in dataset collection, the re-use of pretrained GANs largely increases the potential impact of generative models. Therefore, we propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs. This is done using a miner network that identifies which part of the generative distribution of each pretrained GAN outputs samples closest to the target domain. Mining effectively steers GAN sampling towards suitable regions of the latent space, which facilitates the posterior finetuning and avoids pathologies of other methods, such as mode collapse and lack of flexibility. Furthermore, to prevent overfitting on small target domains, we introduce sparse subnetwork selection, that restricts the set of trainable neurons to those that are relevant for the target dataset. We perform comprehensive experiments on several challenging datasets using various GAN architectures (BigGAN, Progressive GAN, and StyleGAN) and show that the proposed method, called MineGAN, effectively transfers knowledge to domains with few target images, outperforming existing methods. In addition, MineGAN can successfully transfer knowledge from multiple pretrained GANs. MineGAN. [ABSTRACT FROM AUTHOR]

In this study, facing the condition of structural and informational defects and complex correlation in data of fault feature gases for transformers, a transformer fault diagnosis (TFD) method based on the two-dimensional cloud model (2D-CM) under defective data is investigated. Firstly, to solve the problem that the accuracy of TFD is impacted by the completeness of the priori rule set caused by the incompleteness, imbalance and feature redundancy of priori transformer fault information, a data processing framework based on the strengthened Mahalanobis distance self-organizing map (SMSOM) and data generation algorithm is proposed. By applying the data generation algorithm based on the Wasserstein conditional generative adversarial network with gradient penalty and the filtering algorithm of the generated data based on SMSOM, the data set can be expanded evenly and the diversity of feature information in the limited training set can be maximized. Secondly, fault data is converted to cloud concept combination by constructing 2D-CM and adopting the computing method of cloud membership degree under offset space. Finally, TFD is realized by multi-rule fusion reasoning under the complete prior rule knowledge obtained by rule mining from all concept combinations. The experimental results indicate that the proposed data processing framework can make the TFD model based on 2D-CM achieve a diagnostic accuracy of 83.1%, which is superior to other data processing schemes. Moreover, the proposed TFD model based on 2D-CM increases the diagnosis accuracy by 22.8% compared with the traditional 1D-CM. [ABSTRACT FROM AUTHOR]

Background: With the development of single-cell technology, many cell traits can be measured. Furthermore, the multi-omics profiling technology could jointly measure two or more traits in a single cell simultaneously. In order to process the various data accumulated rapidly, computational methods for multimodal data integration are needed. Results: Here, we present inClust+, a deep generative framework for the multi-omics. It's built on previous inClust that is specific for transcriptome data, and augmented with two mask modules designed for multimodal data processing: an input-mask module in front of the encoder and an output-mask module behind the decoder. InClust+ was first used to integrate scRNA-seq and MERFISH data from similar cell populations, and to impute MERFISH data based on scRNA-seq data. Then, inClust+ was shown to have the capability to integrate the multimodal data (e.g. tri-modal data with gene expression, chromatin accessibility and protein abundance) with batch effect. Finally, inClust+ was used to integrate an unlabeled monomodal scRNA-seq dataset and two labeled multimodal CITE-seq datasets, transfer labels from CITE-seq datasets to scRNA-seq dataset, and generate the missing modality of protein abundance in monomodal scRNA-seq data. In the above examples, the performance of inClust+ is better than or comparable to the most recent tools in the corresponding task. Conclusions: The inClust+ is a suitable framework for handling multimodal data. Meanwhile, the successful implementation of mask in inClust+ means that it can be applied to other deep learning methods with similar encoder-decoder architecture to broaden the application scope of these models. [ABSTRACT FROM AUTHOR]

In machine learning, a generative model is responsible for generating new samples of data in terms of a probabilistic model. Generative adversarial network (GAN) has been widely used to generate realistic samples in different domains and outperforms its peers in the generative models family. However, producing a robust GAN model is not a trivial task because many challenges face the GAN during the training process and impact its performance, affecting the quality and diversity of the generated samples. In this article, we conduct a comprehensive review of GANs to present the fundamentals of GAN, including its components, types, and objective functions. Also, we present an overview of the evaluation matrices used to evaluate GAN models. Moreover, we list the applications of GANs and research work in various domains. Finally, we present the challenges that face GANs and highlight two significant issues, representing mode collapse and training instability, in addition to those research efforts that tackle these challenges. This article is categorized under:Statistical Learning and Exploratory Methods of the Data Sciences > Deep LearningStatistical Learning and Exploratory Methods of the Data Sciences > Neural Networks [ABSTRACT FROM AUTHOR]

Deep generative models with latent variables have been used lately to learn joint representations and generative processes from multi-modal data, which depict an object from different viewpoints. These two learning mechanisms can, however, conflict with each other and representations can fail to embed information on the data modalities. This research studies the realistic scenario in which all modalities and class labels are available for model training, e.g. images or handwriting, but where some modalities and labels required for downstream tasks are missing, e.g. text or annotations. We show, in this scenario, that the variational lower bound limits mutual information between joint representations and missing modalities. We, to counteract these problems, introduce a novel conditional multi-modal discriminative model that uses an informative prior distribution and optimizes a likelihood-free objective function that maximizes mutual information between joint representations and missing modalities. Extensive experimentation demonstrates the benefits of our proposed model, empirical results show that our model achieves state-of-the-art results in representative problems such as downstream classification, acoustic inversion, and image and annotation generation. [ABSTRACT FROM AUTHOR]

Background: The rapid emergence of single-cell RNA-seq (scRNA-seq) data presents remarkable opportunities for broad investigations through integration analyses. However, most integration models are black boxes that lack interpretability or are hard to train. Results: To address the above issues, we propose scInterpreter, a deep learning-based interpretable model. scInterpreter substantially outperforms other state-of-the-art (SOTA) models in multiple benchmark datasets. In addition, scInterpreter is extensible and can integrate and annotate atlas scRNA-seq data. We evaluated the robustness of scInterpreter in a variety of situations. Through comparison experiments, we found that with a knowledge prior, the training process can be significantly accelerated. Finally, we conducted interpretability analysis for each dimension (pathway) of cell representation in the embedding space. Conclusions: The results showed that the cell representations obtained by scInterpreter are full of biological significance. Through weight sorting, we found several new genes related to pathways in PBMC dataset. In general, scInterpreter is an effective and interpretable integration tool. It is expected that scInterpreter will bring great convenience to the study of single-cell transcriptomics. [ABSTRACT FROM AUTHOR]

Road agencies commonly face high level complexity of comprehensive pavement maintenance scheduling complications. This could be due to uncertainty of some parameters that substance in the predicting models. Disregarding this uncertainty of such parameters in the road maintenance scheduling problems may lead to imprecise solutions and unreliable road conditions. Frequently time series data that have a hierarchical structure are used for predicting road surface or pavement structure distresses models. However, further realistic and accurate predicted models are those consider supplementary inconsistency between time series data with utilising a predictive approach that imposes knowledge of the input parameters. The purpose of this study is to test this theory by comparing the predictions of models developed using two approaches and their accuracies in utilising the same data. These approaches are the traditional regression model (TRM) with only one level of heterogeneity approach and the hierarchical regression model (HRM) with many levels of heterogeneities approach. A Metro North Western (MNW) urban road network in Victoria (Australia) was considered as the case study for this study investigation where covering the condition of approximately 305 Km of asphalt surfaced pavements. By considering the structure of time series data, the study revealed that more accurate models could predict the pavement behaviour, the deterioration rate, and significantly influenced parameters. In addition, the study highlighted that it is essential to deliberate and process the hierarchical structure of datasets to consider the observed and unobserved heterogeneities in time series data for developing any pavement performance model using either deterministic or probabilistic prediction approaches. Multiple simulation scenarios have been performed for both approaches and the HRM approach shows more reasonable outputs. For predicted deterministic SIR models, road intervention (SIR ≥ 20) are triggered at age 12.5 when using TRM approach and at age 10.5 when using HRM approach. For predicted probabilistic SIR models, the surface condition of a section with SIR weighted average condition values of 2 is triggered at age 17.5 when using TRM approach and at age 10.5 when using HRM approach. This also refers to earlier maintenance scheduling is required when using the later approach because it deals confidentially with each road sections time series data. In that regard, ignoring the uncertainty in time series data of such factors in the road surfacing maintenance scheduling problems may lead to suboptimal solutions and unstable pavement conditions. [ABSTRACT FROM AUTHOR]

This paper proposes an iterative score-based generative model for solving the automatic colorization problem. Although unsupervised learning methods have shown the capability to generate plausible color, inadequate exploration of detailed information and data dimensions still limit the performance of the colorization model. Considering that the number of samples in score-based generative model has influence on estimating the target gradients and the gradient map possesses important latent information of the image, the inference process of the generative modeling is conducted in joint intensity–gradient domain for colorization. Specifically, a set of intensity–gradient formed high-dimensional tensors are trained, via the score matching, to attain the gradient of data distribution in joint intensity–gradient domain. As the score function is determined, data samples are generated by means of annealed Langevin dynamics, forming an iterative colorization procedure. Furthermore, the joint intensity–gradient constraint in data-fidelity term is proposed to limit the degree of freedom within generative model at the iterative colorization stage, thus being conducive to edge-preserving colorization effect. Experimental results conveyed the remarkable performance and diversity of our proposed method. [ABSTRACT FROM AUTHOR]

Subsurface earth models (referred as geomodels) are crucial for characterizing complex subsurface systems. Multiple-point statistics is commonly used to generate geomodels. In this paper, a deep-learning-based generative method is developed as an alternative to traditional geomodel generation procedure. The generative method comprises two deep-learning models, namely hierarchical vector-quantized variational autoencoder (VQ-VAE-2) and PixelSNAIL autoregressive model. Based on the principle of neural discrete representation learning, the VQ-VAE-2 learns to massively compress the geomodels to extract the low-dimensional, discrete latent representation corresponding to each geomodel. Following that, PixelSNAIL uses deep autoregressive network to learn the prior distribution of the latent codes. For the purpose of geomodel generation, PixelSNAIL samples from the newly learnt prior distribution of latent codes, and then the decoder of the VQ-VAE-2 converts the newly sampled latent code to a newly constructed geomodel. PixelSNAIL can be used for unconditional or conditional geomodel generation. In unconditional generation, the generative workflow generates an ensemble of geomodels without any constraint. On the other hand, in the conditional geomodel generation, the generative workflow generates an ensemble of geomodels similar to a user-defined source image, which ultimately facilitates the control and manipulation of the generated geomodels. To better construct the fluvial channels in the geomodels, perceptual loss is implemented in the VQ-VAE-2 model instead of the traditional 'mean squared error' loss. At a specific compression ratio, the quality of multi-attribute geomodel generation is better than that of single-attribute geomodel generation. [ABSTRACT FROM AUTHOR]

Cross-modal hashing which embeds data to binary codes is an efficient tool for retrieving heterogeneous but correlated multimedia data. In real applications, the sizes of queries are much larger than that of the training set and the queries may be dissimilar to training data, which lays bare the shortage of generalization of deterministic models, such as cross-encoder and autoencoder. In this paper, we design a variational cross-encoder (VCE), a generative model, to tackle this problem. At the bottleneck layer, the VCE outputs distributions parameterized by means and variances. As VCE can generate diversified data using noises, the proposed model can perform better in testing data. Ideally, each distribution is expected to describe a category of data and samples of this distribution are expected to generate data in the same category. Under this expectation, the means and variances can be used as real codes for input data. However, the generated data generally are not belonging to the same category as the input data. Hence, we add a penalty term on variance output of VCE and use means as real codes for further generating hashing codes. Experiments on three widely used datasets validate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Humans excel at predictively synchronizing their behavior with external rhythms, as in dance or music performance. The neural processes underlying rhythmic inferences are debated: whether predictive perception relies on high-level generative models or whether it can readily be implemented locally by hard-coded intrinsic oscillators synchronizing to rhythmic input remains unclear and different underlying computational mechanisms have been proposed. Here we explore human perception for tone sequences with some temporal regularity at varying rates, but with considerable variability. Next, using a dynamical systems perspective, we successfully model the participants behavior using an adaptive frequency oscillator which adjusts its spontaneous frequency based on the rate of stimuli. This model better reflects human behavior than a canonical nonlinear oscillator and a predictive ramping model–both widely used for temporal estimation and prediction–and demonstrate that the classical distinction between absolute and relative computational mechanisms can be unified under this framework. In addition, we show that neural oscillators may constitute hard-coded physiological priors–in a Bayesian sense–that reduce temporal uncertainty and facilitate the predictive processing of noisy rhythms. Together, the results show that adaptive oscillators provide an elegant and biologically plausible means to subserve rhythmic inference, reconciling previously incompatible frameworks for temporal inferential processes. Author summary: Real-life sequence processing requires prediction in uncertain contexts. Temporal predictions are thought to be manifest by a dual timing mechanism: a relative timing mechanism measuring timing relative to an expected rhythm, and an absolute timing mechanism which measures absolute interval differences. We show that these mechanisms can be unified into a Bayesian Framework, reformulating the two components into the combination of prior and sensory measurement, and demonstrate that this framework can be implemented by an adaptive neural oscillator. [ABSTRACT FROM AUTHOR]

Generative models of protein sequence families are an important tool in the repertoire of protein scientists and engineers alike. However, state-of-the-art generative approaches face inference, accuracy, and overfitting- related obstacles when modeling moderately sized to large proteins and/or protein families with low sequence coverage. Here, we present a simple to learn, tunable, and accurate generative model, GENERALIST: GENERAtive nonLInear tenSor-factorizaTion for protein sequences. GENERALIST accurately captures several high order summary statistics of amino acid covariation. GENERALIST also predicts conservative local optimal sequences which are likely to fold in stable 3D structure. Importantly, unlike current methods, the density of sequences in GENERALIST-modeled sequence ensembles closely resembles the corresponding natural ensembles. Finally, GENERALIST embeds protein sequences in an informative latent space. GENERALIST will be an important tool to study protein sequence variability. Author summary: Protein sequence families show tremendous sequence variation. Yet, it is thought that a large portion of the functional sequence space remains unexplored. Generative models are machine learning methods that allow us to learn what makes proteins functional using sequences of naturally occurring proteins. Here, we present a new type of generative model GENERALIST: GENERAtive nonLInear tenSor-factorizaTion for protein sequences that is accurate, easy to implement, and works with very small datasets. We believe that GENERALIST will be an important tool in the repertoire of protein scientists and engineers alike. [ABSTRACT FROM AUTHOR]

Discovering novel magnetic materials is essential for advancing the spintronic technology with significant applications in data communication, data storage, quantum computing, etc. While density functional theory (DFT) has been widely used for designing materials, its high computational demand for estimating the magnetic ground states of even a single material limits its ability to explore the vast chemical design space to find the right materials for spintronic applications. In this work, we developed a computational framework combining generative adversarial networks (GANs), machine learning (ML) classifiers, and DFT for de novo magnetic material discovery. We used the CubicGAN generative crystal structure design model for creating new ternary cubic structures. Machine learning classifiers were developed with around 90% accuracy to screen candidate ternary magnetic materials, which were then subjected to DFT based stability validation. Our calculations discovered and confirmed that Na6TcO6, K6TcO6, and BaCuF6 are stable ferromagnetic compounds, while Rb6IrO6 is a stable antiferromagnetic material. All these materials have zero energy above hull. Moreover, Na6TcO6 and BaCuF6 are found to be half metals that are highly favorable for spintronic applications. Due to the structural differences, the A6MO6 materials have a higher thermal capacity (Cv) compared to BaCuF6. At 300 K temperature, the Cv of the A6MO6 materials is around 1100 J K−1 mol−1 and that of BaCuF6 is about 176 J K−1 mol−1. This work demonstrates the promising potential of deep generative design for discovering novel functional materials. [ABSTRACT FROM AUTHOR]

While a multitude of deep generative models have recently emerged there exists no best practice for their practically relevant validation. On the one hand, novel de novo-generated molecules cannot be refuted by retrospective validation (so that this type of validation is biased); but on the other hand prospective validation is expensive and then often biased by the human selection process. In this case study, we frame retrospective validation as the ability to mimic human drug design, by answering the following question: Can a generative model trained on early-stage project compounds generate middle/late-stage compounds de novo? To this end, we used experimental data that contains the elapsed time of a synthetic expansion following hit identification from five public (where the time series was pre-processed to better reflect realistic synthetic expansions) and six in-house project datasets, and used REINVENT as a widely adopted RNN-based generative model. After splitting the dataset and training REINVENT on early-stage compounds, we found that rediscovery of middle/late-stage compounds was much higher in public projects (at 1.60%, 0.64%, and 0.21% of the top 100, 500, and 5000 scored generated compounds) than in in-house projects (where the values were 0.00%, 0.03%, and 0.04%, respectively). Similarly, average single nearest neighbour similarity between early- and middle/late-stage compounds in public projects was higher between active compounds than inactive compounds; however, for in-house projects the converse was true, which makes rediscovery (if so desired) more difficult. We hence show that the generative model recovers very few middle/late-stage compounds from real-world drug discovery projects, highlighting the fundamental difference between purely algorithmic design and drug discovery as a real-world process. Evaluating de novo compound design approaches appears, based on the current study, difficult or even impossible to do retrospectively. Scientific Contribution This contribution hence illustrates aspects of evaluating the performance of generative models in a real-world setting which have not been extensively described previously and which hopefully contribute to their further future development. [ABSTRACT FROM AUTHOR]

The monitoring of rapidly changing land surface processes requires remote sensing images with high spatiotemporal resolution. As remote sensing satellites have different satellite orbits, satellite orbital velocities, and sensors, it is challenging to acquire remote sensing images with high resolution and dense time series within a reasonable temporal interval. Remote sensing spatiotemporal fusion is one of the effective ways to acquire high-resolution images with long time series. Most of the existing STF methods use artificially specified fusion strategies, resulting in blurry images and poor generalization ability. Additionally, some methods lack continuous time change information, leading to poor performance in capturing sharp changes in land covers. In this paper, we propose an adaptive multiscale network for spatiotemporal fusion (AMS-STF) based on a generative adversarial network (GAN). AMS-STF reconstructs high-resolution images by leveraging the temporal and spatial features of the input data through multiple adaptive modules and multiscale features. In AMS-STF, for the first time, deformable convolution is used for the STF task to solve the shape adaptation problem, allowing for adaptive adjustment of the convolution kernel based on the different shapes and types of land use. Additionally, an adaptive attention module is introduced in the networks to enhance the ability to perceive temporal changes. We conducted experiments comparing AMS-STF to the most widely used and innovative models currently available on three Landsat-MODIS datasets, as well as ablation experiments to evaluate some innovative modules. The results demonstrate that the adaptive modules significantly improve the fusion effect of land covers and enhance the clarity of their boundaries, which proves the effectiveness of AMS-STF. [ABSTRACT FROM AUTHOR]

A novel method for clustering functional data is introduced that utilizes information maximization. The method employs unsupervised learning to develop a probabilistic classifier that maximizes the mutual information or squared loss mutual information between data points and their corresponding cluster assignments. A significant advantage of this method is that it involves only continuous optimization of model parameters, which is simpler than discrete optimization of cluster assignments and avoids the drawbacks of generative models. Unlike existing methods, this method does not necessitate the estimation of probability densities of Karhunen-Loeve expansion scores under different clusters and does not require the common eigenfunction assumption. The efficacy of the proposed method is demonstrated through simulation studies and real data analysis. Additionally, the technique permits out-of-sample clustering, and its performance is comparable to that of supervised classifiers. [ABSTRACT FROM AUTHOR]