Search

Abstract Deep learning-based computer vision systems have become powerful tools for automated and cost-effective pavement distress detection, essential for efficient road maintenance. Current methods focus primarily on developing supervised learning architectures, which are limited by the scarcity of annotated image datasets. The use of data augmentation with synthetic images created by generative models to improve these supervised systems is not widely explored. The few studies that do focus on generative architectures are mostly non-conditional, requiring extra labeling, and typically address only road crack defects while aiming to improve classification models rather than object detection. This study introduces AsphaltGAN, a novel class-conditional Generative Adversarial Network with attention mechanisms, designed to augment datasets with various rare road defects to enhance object detection. An in-depth analysis evaluates the impact of different loss functions and hyperparameter tuning. The optimized AsphaltGAN outperforms state-of-the-art generative architectures on public datasets. Additionally, a new workflow is proposed to improve object detection models using synthetic road images. The augmented datasets significantly improve the object detection metrics of You Only Look Once version 8 by 33.0 $$\%$$ , 3.8 $$\%$$ , 46.3 $$\%$$ , and 51.8 $$\%$$ on the Road Damage Detection 2022 dataset, Crack Dataset, Asphalt Pavement Detection Dataset, and Crack Surface Dataset, respectively.

The perception systems of autonomous vehicles face significant challenges under adverse conditions, with issues such as obscured objects and false detections due to environmental noise. Traditional approaches, which typically focus on noise removal, often fall short in such scenarios. Addressing the lack of diverse adverse weather data in existing automotive datasets, we propose a novel data augmentation method that integrates realistically simulated adverse weather effects into clear condition datasets. This method not only addresses the scarcity of data but also effectively bridges domain gaps between different driving environments. Our approach centers on a conditional generative model that uses segmentation maps as a guiding mechanism to ensure the authentic generation of adverse effects, which greatly enhances the robustness of perception and object detection systems in autonomous vehicles, operating under varied and challenging conditions. Besides the capability of accurately and naturally recreating over 90% of the adverse effects, we demonstrate that this model significantly improves the performance and accuracy of deep learning algorithms for autonomous driving, particularly in adverse weather scenarios. In the experiments employing our augmented approach, we achieved a 2.46% raise in the 3D average precision, a marked enhancement in detection accuracy and system reliability, substantiating the model's efficacy with quantifiable improvements in 3D object detection compared to models without augmentation. This work not only serves as an enhancement of autonomous vehicle perception systems under adverse conditions but also marked an advancement in deep learning models in adverse condition research. [ABSTRACT FROM AUTHOR]

The perception systems of autonomous vehicles face significant challenges under adverse conditions, with issues such as obscured objects and false detections due to environmental noise. Traditional approaches, which typically focus on noise removal, often fall short in such scenarios. Addressing the lack of diverse adverse weather data in existing automotive datasets, we propose a novel data augmentation method that integrates realistically simulated adverse weather effects into clear condition datasets. This method not only addresses the scarcity of data but also effectively bridges domain gaps between different driving environments. Our approach centers on a conditional generative model that uses segmentation maps as a guiding mechanism to ensure the authentic generation of adverse effects, which greatly enhances the robustness of perception and object detection systems in autonomous vehicles, operating under varied and challenging conditions. Besides the capability of accurately and naturally recreating over 90% of the adverse effects, we demonstrate that this model significantly improves the performance and accuracy of deep learning algorithms for autonomous driving, particularly in adverse weather scenarios. In the experiments employing our augmented approach, we achieved a 2.46% raise in the 3D average precision, a marked enhancement in detection accuracy and system reliability, substantiating the model’s efficacy with quantifiable improvements in 3D object detection compared to models without augmentation. This work not only serves as an enhancement of autonomous vehicle perception systems under adverse conditions but also marked an advancement in deep learning models in adverse condition research.

Automatic detection and classification of thoracic diseases using deep learning algorithms have many applications supporting radiologists’ diagnosis and prognosis. However, in the medical field, the class-imbalanced problem is extremely common due to the differences in prevalence among diseases, making it difficult to develop these applications. Many GAN-based methods have been proposed to solve the class-imbalance problem on chest X-ray (CXR) data. However, these models have not been trained well for small-sized diseases because it is challenging to extract sufficient information with only a few pixels. In this paper, we propose a novel deep generative model called a class activation region influence maximization conditional generative adversarial network (CARIM-cGAN). The proposed network can control the target disease’s presence, location, and size with a controllable conditional mask. We newly introduced class activation region influence maximization (CARIM) loss to maximize the probability of disease occurrence in the bounded region represented by a conditional mask. To demonstrate an enhanced generative performance, we conducted numerous qualitative and quantitative evaluations with the samples generated using a CARIM-cGAN. The results showed that our method has a better performance than other methods. In conclusion, because the CARIM-cGAN can generate high-quality samples based on information on the location and size of the disease, we can contribute to solving problems such as disease classification, -detection, and -localization, requiring a higher annotation cost.

We propose a fully convolutional conditional generative neural network, the latent transformation neural network, capable of rigid and non-rigid object view synthesis using a lightweight architecture suited for real-time applications and embedded systems. In contrast to existing object view synthesis methods which incorporate conditioning information via concatenation, we introduce a dedicated network component, the conditional transformation unit. This unit is designed to learn the latent space transformations corresponding to specified target views. In addition, a consistency loss term is defined to guide the network toward learning the desired latent space mappings, a task-divided decoder is constructed to refine the quality of generated views of objects, and an adaptive discriminator is introduced to improve the adversarial training process. The generalizability of the proposed methodology is demonstrated on a collection of three diverse tasks: multi-view synthesis on real hand depth images, view synthesis of real and synthetic faces, and the rotation of rigid objects. The proposed model is shown to be comparable with the state-of-the-art methods in structural similarity index measure and L 1 metrics while simultaneously achieving a 24% reduction in the compute time for inference of novel images. [ABSTRACT FROM AUTHOR]

Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the governing equations of the physical model in the loss/likelihood functions. The resulting physics-constrained, deep learning models are trained without any labeled data (e.g. employing only input data) and provide comparable predictive responses with data-driven models while obeying the constraints of the problem at hand. This work employs a convolutional encoder-decoder neural network approach as well as a conditional flow-based generative model for the solution of PDEs, surrogate model construction, and uncertainty quantification tasks. The methodology is posed as a minimization problem of the reverse Kullback-Leibler (KL) divergence between the model predictive density and the reference conditional density, where the later is defined as the Boltzmann-Gibbs distribution at a given inverse temperature with the underlying potential relating to the PDE system of interest. The generalization capability of these models to out-of-distribution input is considered. Quantification and interpretation of the predictive uncertainty is provided for a number of problems. • Physics-constrained surrogates achieve comparable accuracy with data-driven ones without labeled data and generalize better. • Flow-based conditional generative model trained with reverse KL-divergence without labels captures the predictive uncertainty. • The developed models are used to solve PDEs, as surrogate models and for uncertainty propagation and calibration tasks. • Convolutional neural nets capture multiscale features of PDE solution fields much more effectively than fully-connected ones. [ABSTRACT FROM AUTHOR]

Highlights • A method for radar HRRP target recognition is proposed based on Conditional VAE. • The model is regularized through the average HRRP for robust representations. • The label information is introduced by a three-way weight conditional model. • Weight factorization is utilized to effectively reduce model parameters. • Our method performs well under the low SNR and small amount of training data. Abstract In this paper, we propose a conditional generative model for radar high resolution range profile (HRRP) target recognition to learn the discriminative representations and sufficiently encode the observed feature variability by taking the multi-layer perception (MLP) as the sufficient statistics of posterior approximation distribution, thus offering the potential to improve the overall recognition performance. Considering the target-aspect sensitivity of HRRP, the model is regularized through reconstructing the average profiles. Then we introduce three-way weight tensors for MLPs to capture the multiplicative interactions between label information and HRRP samples, which are then further factorized to effectively reduce model parameters. The extensive experimental results on the measured HRRP data demonstrate that the proposed algorithm achieves the promising target recognition and reconstruction performance. [ABSTRACT FROM AUTHOR]

As a branch of machine learning, generative models are widely used in supervised and unsupervised learning. To speedup certain machine learning tasks, quantum generative adversarial networks, quantum circuit Born machine (QCBM), and quantum Boltzmann machine have been proposed. These generative models can implement some specific generative tasks but have no control over the modes of the generated data. To make the generative model more intelligent and controllable, additional conditional information (such as category labels for MNIST digits) can be added to the model to guide the generation of data. A more in-depth study was carried out based on the QCBM, and a conditional quantum circuit Born machine (CQCBM) based on a hybrid quantum–classical (HQC) framework was proposed. The conditional information was encoded by adding extra qubits to guide the model training process. Experiments were conducted on both mixed Gaussian distribution and MNIST handwritten digit dataset. Numerical and experimental results show that the proposed CQCBM is able to generate the target distribution while satisfying the conditional constraints well. Compared to other conditional quantum generative models only applied to Bars and Stripes (BAS) or Chessboard datasets, the proposed model also performed well on more difficult image-generating tasks. • The conditional quantum circuit Born machine is proposed to control the modes of the generated data. • The model is easy to expand via introducing conditional information into one-hot encoding. • The proposed model with hybrid quantum–classical framework has good generative ability. [ABSTRACT FROM AUTHOR]

• Use the VAE and GAN models to produce the ECG signals and boost the ECG classifier. • Generate various heartbeat signals based on the conditional generative framework. • Compare ECG generators based on the GAN and VAE models from different perspectives. Due to the high cost of labeling medical data such as electrocardiogram (ECG) signals, the performance of classifiers suffers significantly from the lack of annotated data. In recent years, generative models have achieved great success in image and natural language synthesis. However, for ECG synthesis, the method is still in its infancy. So far, researchers in this field mainly focus on Generative Adversarial Networks (GAN) or its variants. Besides, the model generally produces data separately for different classes. This strategy leads to a cumbersome generative process and bias to classes with few instances. We propose two unique ECG generators to address the above problems: Conditional Variational Auto-Encoder (CVAE) and Conditional Wasserstein Generative Adversarial Networks (CWGAN), which haven't been covered in previous works. We build simple networks for the encoder and decoder to demonstrate the strong potential of VAE for ECG synthesis. Additionally, we extend GAN-based ECG generators to a more practical version by conditioning the generative process. We use the MIT-BIH arrhythmia database for performance evaluation. The results show that the conditional generative framework can shorten the training time and simplify the generation process without significant performance loss for ECG generators. In particular, the VAE-based ECG generator shows similar potential to the GAN-based model. [ABSTRACT FROM AUTHOR]

We propose a conditional generative model, named Part Geometry Network (PG-Net), which synthesizes realistic objects and can be used as a robust feature descriptor for object reconstruction and classification. Surface and volumetric representations of objects have complementary properties of three-dimensional objects. Combining these modalities is more informative than using one modality alone. Therefore, PG-Net utilizes complementary properties of surface and volumetric representations by estimating curvature, surface area, and occupancy in voxel grids of objects with a single decoder as a multi-task learning. Objects are combinations of multiple parts, and therefore part geometry (PG) is essential to synthesize each part of the objects. PG-Net employs a part identifier to learn the part geometry. Additionally, we augmented a dataset by interpolating individual functional parts such as wings of an airplane, which helps learning part geometry and finding local/global minima of PG-Net. To demonstrate the capability of learning object representations of PG-Net, we performed object reconstruction and classification tasks on two standard large-scale datasets. PG-Net outperformed the state-of-the-art methods in object synthesis, classification, and reconstruction in a large margin. • Learning surface and volumetric geometry for more effective model training and shape prediction. • Synthesizing objects with the parametric model given conditional information. • Developed a feature descriptor for shape classification and reconstruction. • Learning shape distribution with a conditional generative model with multi-task learning. [ABSTRACT FROM AUTHOR]

Accurate information of traffic regulators at junctions is important for navigating and driving in cities. However, such information is often missing, incomplete or not up-to-date in digital maps due to the high cost, e.g., time and money, for data acquisition and updating. In this study we propose a crowdsourced method that harnesses the light-weight GPS tracks from commuting vehicles as Volunteered Geographic Information (VGI) for traffic regulator detection. We explore the novel idea of detecting traffic regulators by learning the movement patterns of vehicles at regulated locations. Vehicles' movement behavior was encoded in the form of speed-profiles, where both speed values and their sequential order during movement development were used as features in a three-class classification problem for the most common traffic regulators: traffic-lights, priority-signs and uncontrolled junctions. The method provides an average weighting function and a majority voting scheme to tolerate the errors in the VGI data. The sequence-to-sequence framework requires no extra overhead for data processing, which makes the method applicable for real-world traffic regulator detection tasks. The results showed that the deep-learning classifier Conditional Variational Autoencoder can predict regulators with 90% accuracy, outperforming a random forest classifier (88% accuracy) that uses the summarized statistics of movement as features. In our future work images and augmentation techniques can be leveraged to generalize the method's ability for classifying a greater variety of traffic regulator classes. [ABSTRACT FROM AUTHOR]