Search

Generative models have emerged as potential tools for creating high-quality images, videos, and text. This paper explores the application of generative models in automating vlog content creation. It addresses both static and dynamic visual elements, eliminating the need for human intervention. Traditional vlogs often require specific environmental conditions and proper lighting for the vlog creation. To streamline this process, an automated system utilizing the generative models is proposed here. Generative models excel at generating realistic content that seamlessly integrates with real-world content. They enhance overall video quality and introduce creative elements by generating new scenes and backgrounds. This paper categorizes various generative modeling techniques based on frame elements and foreground-background conditions. It offers a comparative analysis of different generative model variants tailored for specific objectives. Furthermore, the paper reviews existing research on generative models for video and image content generation, visual quality enhancement, diversity, and coherence outcomes. Additionally, the paper highlights practical uses of the generative model for content creation in various contexts, such as face swapping, scene translation, and virtual content insertion. The paper also examines the public datasets used to train generative models. These datasets contain diverse visual content such as celebrity images, urban landscapes, and everyday scenes. [ABSTRACT FROM AUTHOR]

The maturity of 5G technology provides a guarantee for increasingly large communication networks, while the resources required for communication and computation are also increasing, and reasonable resource allocation can improve the efficiency of network communication and reduce the consumption of communication resources. Existing deep learning methods have been able to predict network traffic to a certain extent, so as to solve the communication efficiency and resource consumption problems in the field of integrated sensing, communication and computation (ISCC) through rational resource allocation. However, the following problems still exist: (1) The feature learning ability of the prediction model is insufficient, and the prediction accuracy needs to be improved. (2) Powerful and complex deep learning methods lead to an increase in the prediction cost of the model. To address these problems, this paper proposes a deep learning method based on a lightweight hybrid attention network. In order to capture the key features of 5G data more effectively, an efficient hybrid attention mechanism (EHA) is proposed. After this attention is applied to convolution, the key information can be well enhanced. We use depthwise separable convolution in feature extraction, which greatly improves the efficiency of lightweight convolution layer (LC) in feature extraction. Combined with the efficient hybrid attention mechanism (EHA), the proposed model has better lightweight properties. Experimental results show that the model proposed in this paper has lower RMSE and MAE values on the three datasets, as well as fewer parameters and computational effort compared to the baseline scheme. [ABSTRACT FROM AUTHOR]

• The paper presents a lightweight model with satisfactory performance in complex industrial noise environments. • Data fusion using data from multiple sensors for more complete fault information. • Design a unique grayscale feature map based on IMF components to obtain multi-sensor and multi-frequency band fault features. • Design a UL-GoogLeNet lightweight fault diagnosis model based on GoogLeNet and ULSAM. As artificial intelligence and sensor technology develop rapidly, intelligent fault diagnosis methods based on deep learning are widely used in industrial production. However, in practical industrial applications, the complex noise environment affects the performance of the diagnostic model, and the huge model parameters cannot meet the requirements of low cost and high performance in industrial production. To address the above problems, this paper proposes a lightweight intelligent fault diagnosis model using multi-sensor data fusion that not only meets the lightweight requirements of "small, light, and fast", but also realizes high accuracy diagnosis in noisy environments. Firstly, the vibration signals from different sensors of rolling bearings are processed using the variational mode decomposition (VMD) to design a unique method of constructing grayscale feature maps based on each intrinsic modal function (IMF) component. Then, the ultra-lightweight GoogLeNet model (UL-GoogLeNet) is constructed to adjust the traditional GoogLeNet structure, while the Ultra-lightweight subspace attention module (ULSAM) is introduced to reduce the model parameters and enhance the feature extraction capability. UL-GoogLeNet is trained and tested by dividing the grayscale feature maps into training and testing sets to realize the intelligent recognition of different fault types in rolling bearings. Experiments are conducted on two datasets and compared with multiple methods, and the final experimental results prove the effectiveness and superiority of the proposed method in this paper. [ABSTRACT FROM AUTHOR]

Brain-computer interface (BCI) provides a platform for humans to communicate using Electroencephalogram (EEG) signals by converting them into commands that can be used by the output device to perform the desired tasks. This paper focuses on the identification of vowels from EEG signals. First, a dataset of EEG signals has been created for the identification of vowels by collecting data using a 14-channel EEG device Emotiv- epoc+ from 16 subjects. Then, a deep learning-based model is proposed using a multi-headed Convolutional Neural Network for feature extraction and classification of imagined speech of vowels. Butterworth lowpass and bandpass filter of order five are implemented for denoising and sub-banding of the EEG signals which are further pre-processed using Hilbert Huang Transform. The model has achieved an average accuracy of 97.67% with a five-fold cross-validation technique using all six sub-bands of the EEG signals. The model has achieved an average precision and recall of 95.54% and 95.11% respectively. The proposed model is statistically tested using the Mann-Whitney U test and paired t-test with a p-value less than 0.05. [ABSTRACT FROM AUTHOR]

Recently, deep learning methods have achieved excellent performance in single image super-resolution (SISR), but most existing methods suffer from heavy computational costs and memory storage. To address this problem, some lightweight SR methods have been proposed, among which convolutional neural network (CNN) with attention mechanisms has received increasing attention. However, the existing CNN-based lightweight network and attention mechanism still have redundancy in feature extraction and enhancement. This paper presents Efficient Self-calibrated and Hierarchical Refinement Network (ESHR) for Lightweight Super-Resolution that includes three effective designs. First, we propose a more effective feature extraction basic module called Self-calibrated Residual Block (SRB), which can achieve superior performance when replacing other common convolution modules such as traditional convolution, BSConv and DSConv. Second, we propose a more lightweight spatial attention mechanism called Squeeze-Excitation Spatial Attention (SESA), which can interact and enhance features more effectively in spatial dimensions to improve the representation ability of the model. Third, we design an Efficient Hierarchical Refinement Block (EHRB) to integrate and utilize multi-level features, which can achieve better performance than single-level refinement without increasing the number of parameters. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art CNN-based methods quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Anomaly detection is distinguishing unusual objects from normal patterns. It is a complex task due to unpredictable nature of anomalies, which can appear in many forms or they can be hidden by mimicking normal behaviors in a graph structure. Such diversity makes this Deep learning approaches can solve these problems by extracting complex patterns from networks. However, addressing different forms of anomalous instances is essential for successfully implementing these approaches, as different anomaly types require further analysis. Additionally, it is challenging to interpret anomalies beforehand without focusing on every aspect of anomalies. Our objective is to propose an architecture capable of handling all types of anomalous entities by tackling challenges across various domains. In this paper, we introduce ARNAD, a novel framework that integrates three deep models to identify anomalies in graphs: graph neural network, autoencoder, and adversarial autoencoder. ARNAD approaches graph anomaly detection by utilizing the features of the deep parts, and four key elements stand out: (1) the autoencoder learns the overall graph structure and identifies highly deviated ones, (2) the graph neural network exploits graph structure to detect anomalies among the communities, (3) a fixed-size randomized neighborhood that prevents overfitting while reducing complexity (4) the adversarial autoencoder improves the robustness of the framework and discriminates anomalies. To detect anomalies, four receptive components assign risk scores to objects in the attributed network. We evaluated the framework with three synthetic datasets that simulate different behaviors of anomalies and six widely used real attributed networks. Our experimental results show that ARNAD performs competitively with other state-of-the-art models in detecting anomalous entities while minimizing false positives, demonstrating ARNAD's effectiveness in detecting graph anomalies. [ABSTRACT FROM AUTHOR]

Most learned image compression methods focus on reducing the average code length of the image, which is unreasonable for the human visual system. Thus, appropriate bit allocation gains significance. This paper presents a novel end-to-end method for perceptual image compression, emphasizing content prioritization and a region-based hierarchical strategy. Firstly, our modified class activation mapping (CAM) can serve as the visual perception network for simulating human visual perception. On this basis, we develop an efficient compression system that leverages a discretized hybrid entropy model to allocate bits automatically according to the perceptual prioritization of image content. In our compression system, an image is compressed jointly and hierarchically with different standards, guided by the importance map extracted by the visual perception network. We introduce content-weighted perceptual metrics to evaluate visual quality more reasonably and objectively. Experiments on publicly available datasets demonstrate that our method outperforms traditional codecs and recent content-oriented learned methods in overall performance. Compared with other methods, the proposed method reconstructs images that are more friendly to the human visual system. [ABSTRACT FROM AUTHOR]

Currently, weapon detection through video surveillance has been extensively studied using deep learning techniques by researchers. However, limited attention has been given to the detection of weapons in night time or dark scenarios. This paper aims to address this gap by proposing a novel approach for weapon detection specifically modified for low-light conditions. The authors demonstrate accurate and robust detection of weapons in challenging nighttime environments by modifying the deep learning model YOLOv7. The YOLOv7-DarkVision model has been developed by combining a brightening algorithm with the advanced image processing techniques and architecture of YOLOv7. A dataset of 15,367 images were collected for training of the model, along with five dark videos from various sources for performance evaluation. The derived detection model, which has a precision score of 95.50% and an F1-Score of 93.41%, performs absolutely well as a weapon detector. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) signals contains a major role in examining the behavior of brain activity. Moreover, these signals are contaminated with artifacts, which may affect the EEG signal analysis. The EEG performs as the low spatial resolution in the unimodal analysis. Owing to the highly time-consumption procedure, the removal of artifact techniques cannot be applied in practical applications. In existing works, automatic artifact removal techniques are ineffective for eliminating the artifacts in the EEG signals. Thus, the core of this concept is to design a low complexity aware, intelligent artifacts removal model using deep learning. This paper attempts to introduce a 1-Dimensional-Convolutional Neural Network with a Weighted Deep Feature (1D-CNN-WDF), which acts as a filter to remove the unnecessary noise in EEG signals. The WDF extraction in 1D-CNN-WDF is developed by combining the Coyote Optimization Algorithm (COA), and Bird Swarm Algorithm (BSA) named Hybrid Bird-COA (HB-COA). The same hybrid algorithm is used for the tuning of model parameters in 1D-CNN-WDF. The analysis results confirm that the recommended method is more enriched than recently introduced methods when considering the improved Signal to Noise Ratio (SNR), less Mean Squared Error (MSE), and reduced computational complexity, which shows that the suggested method is attained the superior reconstruction quality. [ABSTRACT FROM AUTHOR]

Glaucoma is a prevalent eye disease that is responsible for causing blindness worldwide. To diagnose glaucoma, the cup-to-disc ratio (CDR) is an important factor. We introduce MRSNet, a novel segmentation network that incorporates encoding and decoding structures. The key innovation of this network is the application of convolutional block with large kernel convolutional attention to the task of medical image segmentation for optic cup and disc. By combining the benefits of residual and self-attention, our network achieves improved performance. The coding region of the network utilizes convolutional block with large kernel convolutional attention, enabling the extraction of multi-scale features with lower computational resources while also enhancing spatial attention. The self-attention layer acts as a transition between the encoding and decoding regions, capturing long connection information and providing additional image details. To further enhance segmentation performance, we employ a multi-resolution image combination approach and adaptively extract the input form using the compression and excitation module. Additionally, we propose a novel approach that combines the principle of consistency of deep supervision mechanisms with cross-entropy and Dice loss to guide the network towards accurate segmentation. In this study, we utilized a five-fold cross-validation method to train our network model. We then performed experimental validation and evaluation on three widely-used datasets, namely REFUGE, DRISHTI-GS, and RIM-ONE-r3. Our model achieved impressive results in the cup-to-disc ratio metric, which accurately reflects the segmentation effect. Specifically, we achieved scores of 0.0242, 0.0941, and 0.0158 for the aforementioned datasets, respectively. These scores outperformed some current classical algorithms. The experimental results demonstrate that the method proposed in this paper has the capability to extract more comprehensive information about the optic cup and disc, with the ability to generalize across different datasets. Furthermore, it shows that the convolutional block with large kernel convolutional attention module can be effectively utilized for the segmentation task of optic cup and disc. These findings provide a valuable research foundation for future researchers. [ABSTRACT FROM AUTHOR]

Near-field multiple-input multiple-output (MIMO) radar imaging systems have recently gained significant attention. These systems generally reconstruct the three-dimensional (3D) complex-valued reflectivity distribution of the scene using sparse measurements. Consequently, imaging quality highly relies on the image reconstruction approach. Existing analytical reconstruction approaches suffer from either high computational cost or low image quality. In this paper, we develop novel non-iterative deep learning-based reconstruction methods for real-time near-field MIMO imaging. The goal is to achieve high image quality with low computational cost at compressive settings. The developed approaches have two stages. In the first approach, physics-based initial stage performs adjoint operation to back-project the measurements to the image-space, and deep neural network (DNN)-based second stage converts the 3D backprojected measurements to a magnitude-only reflectivity image. Since scene reflectivities often have random phase, DNN processes directly the magnitude of the adjoint result. As DNN, 3D U-Net is used to jointly exploit range and cross-range correlations. To comparatively evaluate the significance of exploiting physics in a learning-based approach, two additional approaches that replace the physics-based first stage with fully connected layers are also developed as purely learning-based methods. The performance is also analyzed by changing the DNN architecture for the second stage to include complex-valued processing (instead of magnitude-only processing), 2D convolution kernels (instead of 3D), and ResNet architecture (instead of U-Net). Moreover, we develop a synthesizer to generate large-scale dataset for training the neural networks with 3D extended targets. We illustrate the performance through experimental data and extensive simulations. The results show the effectiveness of the developed physics-based learned reconstruction approach compared to commonly used approaches in terms of both runtime and image quality at highly compressive settings. Our source codes and dataset are made available at https://github.com/METU-SPACE-Lab/Efficient-Learned-3D-Near-Field-MIMO-Imaging upon publication to advance research in this field. • 3D image reconstruction methods are developed for near-field MIMO radar imaging using deep learning-based direct inversion and 3D CNNs. • A synthesizer to generate 3D scenes with extended targets is developed to obtain large data for training the neural networks. • Resolution analysis is performed for compressive imaging settings with sparse arrays. • Performance is comprehensively evaluated with simulated and experimental data, using both magnitude-only and complex-valued processing. • The results show the effectiveness of the developed physics-based learned reconstruction method in terms of image quality and runtime. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach to audio-visual consistency judgment (AVCJ), in which vowel-like regions exhibiting significant changes in lip shape are used as key pronunciations. Unlike conventional methods that typically apply speech information to the whole utterance and corresponding visual features, the proposed technique involves key pronunciation analysis and frontal lip reconstruction for non-frontal lip frames corresponding to these key pronunciations. This is done to eliminate the influence of variations in video viewing angle on consistency judgment and address the limitations of whole-sentence analysis in a process consisting of four steps. First, key pronunciations are detected in speech utterance. Second, view angle classification is performed on lip frames corresponding to the key pronunciations and then frontal reconstruction is performed by a proposed SL-CycleGAN. Third, deep correlation analysis is applied to the covariance of the deep features obtained from both audio and visual features. Fourth, time-delay and correlation differences are combined to estimate the joint score of each key pronunciation. The results of AVCJ are then acquired by calculating a sentence level score from each joint score. Experimental results from the OuluVS2 bimodal multi-view database showed that the proposed method outperformed several state-of-the-art algorithms, including quadratic mutual information (QMI), space-time canonical correspondence analysis, and multiple variations of SyncNet. [ABSTRACT FROM AUTHOR]

Accurate exchange rate forecasting is of great importance for foreign exchange investment, hedging foreign exchange risk and international economic transactions. However, it is extremely challenging to make accurate forecasts for exchange rates because of their high volatility, nonlinearity, and non-stationarity. Based on this, this paper proposes a multifactor clustering integration paradigm for exchange rate prediction. From the classical theory of exchange rate determination, a comprehensive library of factors affecting the exchange rate is constructed. A two-stage feature engineering is constructed to capture the stable structure of features. In order to enable features to be learned adequately and to improve algorithmic efficiency and predictive performance, a novel clustering integration paradigm is constructed to improve the stability of the model. The framework builds different predictive sub-models for different data, and then embeds Bayesian optimization algorithms into the bidirectional deep neural networks. Finally, the output results of different sub-models are integrated using nonlinear integration techniques. In addition, the superiority of the proposed model is verified using eight comparative models. The results of the empirical analysis show that the average percentage error of our proposed model is the lowest among all the comparative models (0.412648 %, 0.515348 %, and 0.329892 % on the three datasets, respectively). Compared to the standard LSTM, the average percentage error is at least 88 % lower, proving the effectiveness of the proposed model. It can help investors to make better decisions in the international financial markets. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning-based method for image restoration given an inaccurate knowledge of the degradation. We first show how the impulse response of a Wiener filter can approximate the Moore-Penrose pseudo-inverse of the blur convolution operator. The deconvolution problem is then cast as the learning of a residual in the null space of the blur kernel, which, when added to the Wiener restoration, will satisfy the image formation model. This approach is expected to make the network capable of dealing with different blurs since only residuals associated with the Wiener filter have to be learned. Artifacts caused by inaccuracies in the blur estimation and other image formation model inconsistencies are removed by a Dynamic Filter Network. The extensive experiments carried out on several synthetic and real image datasets assert the proposed method's performance and robustness and demonstrate the advantage of the proposed method over existing ones. [ABSTRACT FROM AUTHOR]

Intelligent fault prognosis plays an essential role in ensuring the stability and safety of power systems. However, the prediction of power transformer faults encounters significant challenges, including ultra-high noise interference, imbalanced samples, and difficulties in extracting sensitive fault features. To address these problems, this paper proposes SD2SDFNet, a novel deep fusion network that utilizes statistical denoising and a dual-dimension self-attention network. The proposed approach incorporates a statistical denoising approach called successive variational mode decomposition and Cramer Von Misses (SVMD-CVM). This technique aims to decompose signals into intrinsic mode functions (IMFs) and eliminate noise from each corresponding mode. By employing SVMD-CVM, the scheme effectively mitigates the impact of noise on fault prediction. Furthermore, a dual-dimension self-attention model is developed to selectively capture salient features from different IMF and time step dimensions. This adaptive feature selection process enhances the network's ability to focus on relevant information. To integrate the extracted features from both dimensions, a deep fusion network is adopted. This fusion process combines the weighted features, resulting in a more comprehensive set of features that facilitates accurate fault prediction. Experimental evaluations conducted on hybrid-space datasets demonstrate that SD2SDFNet achieves outstanding performance in predicting inter-turn faults. The network exhibited a Mean Squared Error (MSE) of 3.25E-05 and a Relative Absolute Error (RAE) of 1.58% for sensor 1, surpassing state-of-the-art prediction methods. [ABSTRACT FROM AUTHOR]

The future wireless networks are expected to be data-driven integrated sensing, communication and computing (ISCC) systems and benefit from employing deep learning to process the large amount of data with less resources and computational limit. As a result, deep learning based learning MIMO communication has been widely discussed in recent years and show fascinating performance advantages over human designed MIMO systems. However, we find that the autoencoder frameworks proposed in current deep learning based methods are not able to be implemented under multipath channels, which is the most common channel model in actual environment. To overcome this problem, a modified autoencoder framework is proposed in this paper. This proposed autoencoder framework introduces well designed convolutional neural layers into the widely used feedforward neural networks, and demonstrates significant performance improvements over traditional Turbo code method adopted in existing wireless MIMO systems. Moreover, computer simulations are performed and the proposed autoencoder framework is also demonstrated to be effective in SISO and MISO systems under AWGN and Rayleigh fading channels, which verifies the generality of the proposed autoencoder framework for end-to-end learning communication systems. [ABSTRACT FROM AUTHOR]

Digital images are playing a progressively important role in almost all the fields such as computer science, medicine, communications, transmission, security, surveillance, and many more. Digital images are susceptible to a number of distortions due to faulty imaging instruments, transmission channels, atmospheric and environmental conditions, etc. resulting in degraded images. Degradation can be of different types such as noise, backscattering, low saturation, low contrast, tilt, spectral absorption, blurring, etc. The degradation reduces digital images' effectiveness and therefore needs to be restored. In this paper, we present an extensive review of image restoration tasks. It addresses problems like image deblurring, denoising, dehazing and super-resolution. Image restoration is fundamentally an image processing problem, but deep learning techniques, based mainly on convolutional neural networks have received a lot of attention in almost all areas of computer science. Along with deep learning, other machine learning methods have also been tried for restoring digital images. In this review, we have therefore categorized digital image restoration techniques as either image processing-based, machine learning-based or deep learning-based. For each category, a variety of approaches presented in recent years have been reviewed. This review also includes a summary of the data sets used for image restoration along with a baseline reference that can be used by future researchers to compare and improve their results. We also suggest some interesting research directions for future work in this area. [ABSTRACT FROM AUTHOR]

Motion Estimation is used many video processing algorithms for video compression, moving object detection, 3D reconstruction, video segmentation, etc. Video standards use motion estimation in the encoder and decoder to compress the video. It becomes difficult to determine motion vectors when the video is prone to weather conditions like haze. So, there is a necessity to remove haze for video analysis. In this paper, hardware and software implementation of Motion Estimation (ME) and Haze removal techniques are reviewed. Architectures like low power and parallelism Motion Estimation techniques for Block Matching algorithms (BMA) in video coding standards and Deep Learning based methods Pixel-based Methods are studied and discussed. The atmospheric model for hazy images and videos is discussed. Techniques such as single-image dehazing are analyzed. Performance metrics carried out for video analysis are also mentioned. Based on the analysis and observation the advantages and drawbacks of these techniques are addressed. [ABSTRACT FROM AUTHOR]

As is widely accepted, mobile edge computing (MEC) is a promising technology to enable wireless devices (WDs) to process computation-intensive tasks. Due to the mutual influence among different WDs, collaborative task offloading is needed in multi-agent environments. In this paper, a multi-agent MEC network with delay-sensitive and non-partitionable tasks is considered, as well as the load on MEC servers. The joint optimization problem of offloading decision and resource allocation is formulated to minimize the average delay. To realize the collaborative decision-making, a multi-agent deep reinforcement learning based algorithm is proposed based on the framework of centralized training and decentralized execution. The centralized deep neural networks (DNN) learn from the past experience and the WDs learn policies from the evaluation of the actions from the centralized DNNs. Based on the learned policies, WDs can make offloading decisions with only local information. Simulation results show that the proposed algorithm achieves near-optimal performance and has the advantage of high stability in varying multi-agent environments. [ABSTRACT FROM AUTHOR]

Computed tomography (CT) provides a three-dimensional view of a patient's internal organs. Compared to CT volumes, X-ray imaging can significantly reduce the patient's exposure to ionizing radiation. Moreover, X-ray images are more economical and widely applied in surgical procedures. However, X-ray images can only provide two-dimensional information. In this paper, an end-to-end GAN network, named TRCT-GAN, is proposed to reconstruct chest CT volumes from biplane X-ray images. In the GAN network, the Transformer network module is employed to enhance the feature representation of X-ray images. Moreover, a dynamic attention module is added to exploit some 2D feature maps and 3D feature maps to enhance the contextual association. The experimental results demonstrate that the proposed network can effectively produce high-quality CT reconstructions from X-ray images. • Add Transformer module to the feature conversion of 2D X-ray images to 3D CT volumes. • Our proposed 2D dynamic attention module can encode complex 2D X-ray image features very well. • 3D dynamic attention helps reconstruct more accurate CT images. [ABSTRACT FROM AUTHOR]

Deep learning based compressed sensing imaging algorithms usually adopt supervised learning mechanism, which generally require massive high-quality images paired with the measured data in the training process. However, clear images are difficult to obtain in some specific application scenarios. In view of this, we propose a self-attention powered multi-scale compressed sensing imaging network named BNN-SANet with the Bayesian self-supervised learning manner in this paper. BNN-SANet can automatically learn effective feature representation from the measured data and reconstruct high-quality images without large-scale training samples. In particular, the self-attention based Transformer employed in BNN-SANet is effective to exploit global and long-range dependencies. Multi-scale dual path design in BNN-SANet performs well in extracting the semantic information at different scales to further improve the CS reconstruction quality. Experimental results illustrate that the proposed approach is substantially superior to previous compressed sensing imaging methods in terms of Peak Signal to Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM) and visual metrics. [ABSTRACT FROM AUTHOR]

With the popularity of online shopping and the rise of various brands, logo detection is gradually coming to the forefront of researchers' minds. However, accurately detecting multiscale, similar, diverse and shape-shifting logos poses a challenge for this technology. The Swin transformer has created a milestone as a high-performance deep learning method across modalities, domains and tasks in various tasks of computer vision. We improve the Swin transformer for better image feature extraction and enhance the robustness for its application to logo detection. In this paper, we propose the PMST as the backbone of logo detection, and design a bypass-parallelizable shift module and a miniature window tandem shift strategy to further enhance image feature fusion and transfer between windows. The PMST achieves a 79.2 box AP on the LogoData dataset, surpassing most detection models. According to existing known methods, in logo detection, the PMST is the first backbone to put transformer to the detection. The results achieved state-of-the-art on the FlickrLogos-32 and FoodLogoDet-1500 datasets. It also achieves excellent efficiency on the ImageNet-1K, OpenBrands-80 and MS-COCO datasets. The code is available at https://github.com/blowhen/PMST. [ABSTRACT FROM AUTHOR]

As a new type of surveillance technology, Automatic Dependent Surveillance-Broadcast (ADS-B) has been regarded as the cornerstone of the next generation air surveillance. However, a potential problem is that multiple ADS-B signals may overlap when they come from different aircraft simultaneously. This paper explores the possibility of using deep learning to separate pulsed ADS-B overlapping signals. Most existing deep learning algorithms separate the overlapping signals by utilizing the difference in time-frequency domains. But all ADS-B signals are transmitted at the same frequency, those deep learning methods based on time-frequency analysis are ineffective. This paper separates pulsed ADS-B overlapping signals directly in the time domain. Firstly, an encoder is used to extract the mixed features of ADS-B overlapping signals in the time domain. Then, Temporal Convolutional Network (TCN) is used to extract the non-overlapping ADS-B signal feature from the output of encoder. Finally, the separated time-domain signals are reconstructed by a decoder. Simulation results show that the separation accuracy of the algorithm based on deep learning is higher than that of the traditional algorithm. • The effectiveness of deep learning technology to separate pulsed ADS-B overlapping signals is verified. • The training data set of ADS-B overlapping signal is constructed from the parameters in the actual situation. • Simulation results show that the deep learning algorithm performs better than the traditional algorithm. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has demonstrated tremendous potential in the real world. Object detection is a critical real-world task for deep learning. You Only Look Once (YOLO) object detection model recognizes interesting regions in images with impressive accuracy and real-time performance. The objective of this paper is to apply object detection to the field of security and counter-terrorism. Individuals are protected from violence by recognizing and locating the guns on closed-circuit television (CCTV). This paper presents a real-time detection approach for CCTV autonomous weapons based on YOLO v4. For the characteristics of CCTV scenarios, we propose the YOLO v4 backbone with Spatial Cross Stage Partial-ResNet (SCSP-ResNet). Meanwhile, the receptive field enhancement module is shown to capture fine semantic features of high-dimensional small objects. The Fusion-PANet (F-PaNet) module has been used to fuse multi-scale information to improve the model's perceptive power on the region of interest. Furthermore, we merge synthetic and real-world datasets to comprehensively investigate the effects of synthetic datasets on detectors. The experimental results reveal that our suggested detection model improves mAP (mean Accuracy Precision) and inference time by 7.37% and 4.2%, respectively. The model's parameter is reduced by 0.349 BFLOP/s(billion floating point operations per second). The proposed detector outperforms the baseline model in terms of accuracy, real-time, and robustness. [ABSTRACT FROM AUTHOR]

This paper introduces an approach aiming to determine head pose healthiness of computer users. The main contributions of this paper are: 1) Image Quality Assessment (IQA) based Stacked Autoencoder (referred to as IQASAE) which adjusts the value of learning rate based on the quality of images; 2) Head Pose Healthiness Prediction (HPHP) framework which leverages the proposed IQASAE algorithm in combination with image processing operations; 3) A set of features suitable for face analysis applications; 4) Ontology-driven semantic framework which enables further exploiting pose estimation results within applications in synergy with healthcare expert domain knowledge about pose healthiness. Our framework was evaluated on both offline (BIWI and AFLW) and online (our own, collected using Arduino) datasets. Furthermore, it was compared to several state-of-art methods, including Multi-Layer Perceptron (MLP), CART, Random Forest, Convolutional Neural Networks (CNN), Temporal Deep Learning Model (TDLM), hybrid CNN with Support Vector Machine (SVM), Quatnet and Trinet. According to the achieved experimental results, it reaches accuracy up to 79.63% outperforming all of them, except Quatnet and Trinet. However, the main advantages of IQASAE compared to state-of-art methods are: 1) it does not require selection of features, so the processing time is reduced, 2) utilizing angle between chin and mouth reduces training time for SAE, 3) leveraging vector-based feature set to create training data resulted in a significant improvement, especially in offline facial images. [ABSTRACT FROM AUTHOR]

Different from the single low-light image enhancement, the key for video enhancement is to improve temporal stability in spatial and temporal varying manner. However, most methods fail to effectively align images or features of adjacent frames to fuse spatio-temporal information and achieve limited enhancement performance, especially in dynamic videos with scene changes and moving targets. In this paper, we propose a spatio-temporal propagation and reconstruction (STPR) network to enable video representation learning for low-light video enhancement. Specifically, we develop a recursively supervised-based pyramid residual dense structure for extracting expressive context features from video sequences, the unique structural design enlarges the receptive field in multi-scale space with limited training parameters. Moreover, to strengthen the interaction of across-frame features, a feature propagation subnet is present to achieve precise alignment and effective fusion of spatio-temporal features. After obtaining the aggregated features of adjacent frames, the spatio-temporal feature reconstruction subnet exploits temporal dependence to improve the quality of target frames and enforce temporal consistency. Experimental results on several low-light video datasets demonstrate that our network outperforms state-of-the-art methods both in quantitative and visual comparisons. • Low-light video enhancement in spatial and temporal varying is proposed. • A recursively supervised pyramid residual dense structure is present. • Alignment and fusion of spatio-temporal features using feature propagation subnet. • The proposed method shows its superior performance and temporal consistency. [ABSTRACT FROM AUTHOR]

Signal unmixing is a class of complex, ill-posed inverse problems, which includes blind source separation or underdetermined signal separation to cite only two. Retrieving signals from their mixtures generally relies on adapted representations allowing to disentangle them. When dealing with real-world scientific data, the main challenge is to further build meaningful signal representations, which generally means capturing the underlying low-dimensional manifold structure of the signals to be recovered. Since the latter is generally unknown, this calls for a learning-based approach, which is a challenging task, especially when available training samples are scarce. The objective of the paper is to investigate a new learning model to build low-dimensional signal representations from few training samples. Based on an encoder-decoder architecture, the proposed approach aims to learn a non-linear interpolating scheme from examples. Extensive numerical experiments have been carried out to evaluate the performances of the proposed approach. We further illustrate how the learned representations can be conveniently deployed to tackle challenging semi-blind unmixing problems in the field of γ -ray spectroscopy. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of effective and robust fusion representation for foreground moving object detection. Many deep learning-based approaches pay attention to network architecture based on a single modality image, ignoring the complementary mechanism of cross-modal images or only considering single-frame prediction without temporal association. We tackle these problems by proposing a fusion representation learning method for the foreground moving object detection task, which consists of two major modules: the upstream fusion representation module (FRM) and the downstream foreground moving object detection module (FODM). Unlike traditional feature aggregate methods, the FRM module is a quality-aware and online learnable fusion module which can aggregate valuable features while rejecting the harmful information in the source images. Specifically, the FODM module is a siamese convolutional neural network to detect foreground moving objects by aggregating the time-sequence images generated by FRM. Moreover, a new aligned foreground moving object detection dataset of infrared and visible images is constructed to provide a new option for benchmark evaluation. Experimental results and comparisons with the state-of-the-art on three public datasets validate the effectiveness, robustness, and overall superiority of our method. [ABSTRACT FROM AUTHOR]

With the vast development of deep learning, many deep learning-based approaches have demonstrated their outstanding performance on the task of fine-grained visual categorization (FGVC). However, existing fine-grained datasets mainly focus on simple images (i.e., objects tend to occupy a significantly larger portion of the image and appear in a relatively clear background). This seriously restricts the application of FGVC in real-world scenarios. In this paper, we construct a fine-grained dataset named AIBD-Cars, which contains 28 , 471 car images with complex backgrounds belonging to 196 fine-grained classes. Furthermore, we propose a Location-Aware Channel-Spatial Attention Network (LCSANet), which considers both locating object regions and mining discriminative information to achieve better fine-grained visual categorization in complex scenes. We evaluate popular fine-grained visual categorization algorithms to build a benchmark. Extensive experiments show that our proposed method achieves a new state of the art on AIBD-Cars and FGVC Aircraft, and competitive results on CUB-200-2011 and Stanford Cars. [ABSTRACT FROM AUTHOR]

With the aid of image nonlocal self-similarity (NSS), recent studies have indicated that structural sparse representation (SSR) models with patch groups as processing units have a great potential in various image inverse problems. Most of existing works, however, only use the dictionary learned from internal or external patch groups for sparse representation. Consequently, such methods generally overfit degradation or do not adapt to the given image. In this paper, we propose a novel sparse representation model, termed joint group dictionary-based SSR (JGD-SSR) model, to alleviate the above problem. Compared with traditional SSR models, the proposed JGD-SSR provides an effective mechanism to exploit internal and external dictionaries jointly. Moreover, to make the proposed model more robust, the connection between JGD-SSR and maximum a posteriori (MAP) estimation is revealed to adaptively estimate regularization parameters. When we apply the proposed JGD-SSR to image restoration tasks, an alternating minimization scheme is developed to solve the resulting optimization problems. The experimental results, including image denoising, deblocking, and deblurring, demonstrate that the proposed JGD-SSR-based restoration algorithm outperforms many state-of-the-art competing methods in terms of both objective and visual qualities. • A novel JGD-SSR model is proposed which provides a simple yet effective mechanism to exploit internal and external group dictionaries jointly for sparse representation. • Applying the proposed JGD-SSR model to image restoration, a general but effective restoration algorithm is developed via alternating minimization. • Extensive experimental results show that the proposed JGD-SSR-based algorithm outperforms not only state-of-the-art traditional SSR-based methods, but also popular deep learning-based methods. [ABSTRACT FROM AUTHOR]

Signal recognition technology is a currently active area in both civilian and military applications. Recently, deep learning has aroused extensive attempts in radar signal recognition due to its remarkable capability of automatic feature extraction. However, existing radar signal recognition networks overly depend on the probability-based decision model, resulting in poor robustness. This paper develops a novel deep metric learning frame to enhance the robustness of the recognition system. First, a multiscale atrous pyramid network (MAPNet) is proposed to efficiently learn high-resolution and distinct feature representation. Second, a variance loss is designed to constrain the intra-class feature distribution in metric space. Third, according to the distribution of training signals in metric space, recognition results are recalibrated to provide explicit rejection probabilities for unknowns. Extensive experiments and evaluations demonstrate that the proposed model can accurately classify known signals while robustly identifying unknown signals. The signal database and model can be freely accessed at https://github.com/bryantky/MAPNet. [ABSTRACT FROM AUTHOR]

The downlink channel state information (CSI) must be available on the base station (BS) side to take advantage of all the features of massive multiple-input multiple-output (MIMO) systems. The channel estimation in massive MIMO systems is a challenging task because of the huge pilot and feedback overhead due to the large size of antennas. In this paper, we have proposed a multi task deep network for the channel estimation with the aim of decreasing the pilot and feedback overhead. An encoder network is designed to compress the received signal and reduce the feedback overhead. Furthermore, a decoder network is developed to reconstruct the compressed feedback. The estimator network is suggested to provide the channel estimation from the reconstructed feedback. The performance of the presented scheme has been evaluated in various simulation scenarios. The results confirm that the proposed method is capable of estimating the channel more accurately than the contemporary works. Moreover, this method has reduced the feedback and pilot overhead to a great extent. [ABSTRACT FROM AUTHOR]

The conventional detectors that require high computational complexity can not be applied in complicated 5G systems, where many new techniques (massive MIMO and mmWave) to be employed. This paper proposes a new detector for 5G systems, called tensor decomposition deep learning (3-D DL). We show that the 5G systems can be expressed as a deep learning network in an equivalent tensor form, where the affine transformation is replaced by the multi-linear and multi-way. Such multi-way information then is preserved through layer wise factorization, where the tensor decomposition and nonlinear activation are performed in each hidden layer. Finally, the tensor-decomposed error backpropagation is developed to train the established network. This 3-D DL fully exploits the advantages of the multi-dimensional structural information of signals, to accomplish the estimations with higher resolution. It also avoids the disregard of structural information across different ways and high complexity found in the traditional methods. [ABSTRACT FROM AUTHOR]

As an essential step in the early diagnosis of retinopathy, the blood vessels morphological attributes assist specialists to obtain pathological information efficiently. Most existing deep learning methods are based on U-shaped convolutional neural networks for the segmentation of blood vessels and have made substantial progress. However, the variance between vessel images remains challenging for segmentation algorithms, as demonstrated by poor cross-validation performance between different datasets. In this paper, a novel U-shaped deep convolutional network is proposed for retinal vessel segmentation, namely spiking neural P-type Dual-channel dilated convolutional network (SDDC-Net). We redesign the classical U-shaped convolution network based on the spiking neural P system computational mechanism for the first time. Distinct from the conventional convolutional neural network, SDDC-Net integrates the spiking neural P system convolutional neurons into the classic encoder-decoder architecture. We employ dilated convolution into an encoder, which improves both capabilities of perceiving more contexts and perceptual sensitivity of thin blood. We evaluate this model on three public datasets (DRIVE, STARE, CHASE_DB1), which indicates the more sensitive detection of thin vessels compared to most existing methods, showing higher sensitivity and F1 metrics. Compared to the baseline U-Net, our sensitivity, accuracy, and F1 score metrics on DRIVE dataset surpass by 10.66%, 1.73%, and 1.47% respectively. We also evaluate the effectiveness of spiking neural P system and dilated convolution in the ablation experiments, which demonstrates that accuracy increases with few drops in specificity. The cross-validation experiments show that our model has not only effective segmentation ability but also excellent generalization ability. [ABSTRACT FROM AUTHOR]

Recent deep learning-based approaches on facial expression recognition face challenges on achieving optimal solution due to ineffective extraction of semantic expression features and difficulty in perceiving the positional relationship among facial expression features. In this paper we propose a novel framework, named Ventral-Dorsal Attention Capsule Network (VDACaps) to address the challenges. VDACaps adopts ResNet18 with a ventral channel attention and dilated spatial attention mechanism to strengthen the feature extraction. A novel attention layer containing multiple groups of ventral and dorsal attention is also designed to focus on the key features of facial expressions at the channel and spatial levels, exploring optimal sizes of receptive fields. Experiments on three well-known public expression datasets (CK+, JAFFE, and SFEW) have demonstrated that VDACaps achieves better or competitive performance on facial expression recognition against the state-of-the-art, with the accuracy of 98.98%, 98.44% and 54.07% respectively. [ABSTRACT FROM AUTHOR]

Deep learning-based voiceprint recognition methods rely heavily on adequate datasets, especially those closer to the natural environment and more complex under unconstrained conditions. Yet, the data types of open-source speech datasets are too homogeneous nowadays, and there are some differences with the address collected in natural application environments. For few Chinese datasets used, this paper proposes and produces an unconstrained Chinese speech dataset with richer data types closer to those collected in a natural environment. To address the inadequate extraction of acoustic features in the unconstrained speech dataset, a new two-dimensional convolutional residual network structure based on the attention mechanism is designed and applied to acoustic feature extraction. The residual block structure in the residual network is improved by the SE module and the CBAM module to obtain the SE-Cov2d and CSA-Cov2d models respectively. Finally, it is experimentally demonstrated that the attention mechanism can help the network focus on more critical feature information and fuse more differentiated features in feature extraction. • An unconstrained Chinese speech dataset proposed in a natural environment. • A new two-dimensional convolutional residual network structure designed and applied to acoustic feature extraction. • SE-Cov2d obtains the smallest EER values of 2.72% and 6.76% on both VoxCeleb and CN-Human datasets. [ABSTRACT FROM AUTHOR]

In this paper, we propose an improved face recognition framework where the training is started with a small set of human annotated face images and then new images are incorporated into the training set with minimum human annotation effort. In order to minimize the human annotation effort for new images, the proposed framework combines three different strategies, namely self-paced learning (SPL), active learning (AL), and minimum sparse reconstruction (MSR). As in the recently proposed ASPL framework [1] , SPL is used for automatic annotation of easy images, for which the classifiers are highly confident and AL is used to request the help of an expert for annotating difficult or low-confidence images. In this work, we propose to use MSR to subsample the low-confidence images based on diversity using minimum sparse reconstruction in order to further reduce the number of images that require human annotation. Thus, the proposed framework provides an improvement over the recently proposed ASPL framework [1] by employing MSR for eliminating "similar" images from the set selected by AL for human annotation. Experimental results on two large-scale datasets, namely CASIA-WebFace-Sub and CACD show that the proposed method called ASPL-MSR can achieve similar face recognition performance by using significantly less expert-annotated data as compared to the state-of-the-art. In particular, ASPL-MSR requires manual annotation of only 36.10% and 54.10% of the data in CACD and CASIA-WebFace-Sub datasets, respectively, to achieve the same face recognition performance as the case when the whole training data is used with ground truth labels. The experimental results indicate that the number of manually annotated samples have been reduced by nearly 4% and 2% on the two datasets as compared to ASPL [1]. [ABSTRACT FROM AUTHOR]

Due to the variety of documents and people's different stamping habits, it is difficult for current models to achieve satisfactory results in the process of extracting and removing rubber stamps. Most current methods cannot extract or remove rubber stamps without destroying the background text information. To solve the above problems, we propose a neural network that uses a double-cycle structure, named stamp extraction and removal generative adversarial network (SERGAN). In detail, we improve SERGAN's cycle-consistency by simultaneously recombining the generated rubber stamp image and unstamped image into a stamped image, which enables SERGAN to have multi-domain image conversion capability. To prove the model's generalization ability, we compare it with current mainstream methods on the public watermark dataset CLWD, and the experimental results show the excellence of our model. In addition, there is no publicly available dataset for rubber stamp extraction and removal experiments. To address the scarcity of rubber stamp datasets, we construct a large-scale rubber stamp dataset named LRSD. • A network with strong cycle-consistency which can simultaneously extract and remove rubber stamps from documents is proposed. • A multi-domain image conversion structure which can reduce the number of parameters is proposed. • A large-scale rubber stamp dataset for stamp extraction and removal task is constructed. [ABSTRACT FROM AUTHOR]

Recent developments in the field of deep learning have led to the widespread integration of artificial neural networks in various domains of application. Prominent contemporary artificial neural network training techniques are based on first-order gradient computation. The emphasis on algorithmic performance has driven the emergence of variant artificial neural network training methodologies. Estimation theory, traditionally considered a sub-field of statistics and signal processing, has been explored by various researchers for the development of non-gradient based training methods. Articles published with the aim of utilizing estimation-based artificial neural network training techniques have shown promising results. We identify the integration of estimation theory within the artificial neural network training procedure as intelligent estimation. In this paper, the field of intelligent estimation is analyzed in greater depth with emphasis on the algorithmic performance of novel implementations. Intelligent estimation with applications in the professional domain is also considered, and will help lay the foundation for future research in the literature. [ABSTRACT FROM AUTHOR]

Radar specific emitter identification (SEI) plays important roles in spectrum resource management and Internet of Things (IoT) security. This term refers to the identification of different individual devices of the same type by analyzing the unintentional modulation features caused by hardware imperfections (i.e., fingerprint features). Increasingly many researchers are using deep learning (DL)-based methods for SEI. However, DL-based methods still have many problems, including model generalization, recognition efficiency, and adaptability in open electromagnetic environments. This paper proposes a one-dimensional convolutional neural network (1D CNN)—called a selective kernel residual network (SKResNet)—capable of autonomously adjusting the receptive field, to improve the feature extraction capability. The transient part is cut efficiently from a radar signal sequence as the input of SKResNet using a universal preprocessing method. In addition, considering that most existing methods obtain incorrect results while identifying new emitters without any prior knowledge, SKResNet was improved using OpenMax—an open-set recognition method—to develop Open-SKResNet. As this network could identify unknown emitters effectively, generating reliable classification results of known emitters simultaneously, it could be of great importance and have broad application prospects. [ABSTRACT FROM AUTHOR]

Deep learning has achieved successful results in semantic segmentation of images. However, semantic segmentation of images still faces challenges in real-world applications. On the one hand, due to the repeated stacking of convolution layer and pooling layer, each feature information is localized, eliminating long-term dependence and content deviation. On the other hand, it requires large number of annotated dataset for training, and can not classify unknown categories. To address the above issues, a dual-stream reinforcement network (DRNet) is proposed for few-shot image semantic segmentation in the paper. The dual-stream branches are designed, including a prototype enhancement branch and a query-guided branch. The former branch is designed to compute query vectors based on feature similarity. The latter branch is presented to extract the expression of target features and feature consistency information between images by utilizing additional memory units. Moreover, a matrix of feature correlations is generated by a graph attention mechanism between support vectors and query vectors, which allows query information to be propagated to support vectors. Finally, additional memory units are used to mine the representation of target features. In this way, images with consistent features and information about common features can be effectively fused. The experimental results show that the proposed model DRNet can achieve state-of-the-art results compared to other models. Notably, the mean-IoU scores of our model are 53.34% and 62.03% for PASCAL-5 i at 1-shot and 5-shot settings, respectively, which are more competitive than other methods. [ABSTRACT FROM AUTHOR]

1-bit compressive sensing (1-bit CS) refers to reconstructing a sparse signal from the sign of its measurements. Unlike compressive sensing measurements, the binary measurements in 1-bit CS can be realized using a simple comparator, thereby reducing the hardware complexity and the cost of the receiving array. Many recovery algorithms have been proposed for 1-bit CS, including recently methods based on deep learning. In this paper, we design a new method for underwater source localization by combining the matched field processing method (MFP) with 1-bit CS. We use the Fixed Point Continuation (FPC) method and a deep neural network designed by unfolding its iterations to solve the 1-bit recovery problem and evaluate their performance in the associated source localization problem. Furthermore, in order to improve the robustness of the signal recovery to noise added in the binary measurements, we propose to preprocess the received data with a simple average technique; this formulates the proposed Average FPC- ℓ 1 (AVG-FPC- ℓ 1) method. Our experiments show that an underwater target can be successfully located by using the sign of the measurements. Moreover, with the aid of the noise immunity method and the deep neural network (DNN) structure, the accuracy of the location estimates is greatly improved. [ABSTRACT FROM AUTHOR]

Recent studies on deep metric learning have taken efforts to learn a common feature representation for each category, which is conflict to the intrinsic intra-class variance caused by the fact that latent attributes of the subcategories enlarge the variability of samples from the same class, leading to difficulties of inferring more fine-gained similarity retrieval. This paper proposes a hierarchical multi-proxy representation to establish both the common feature representation of the main-proxy and the complementary representations of their sub-proxies. Specifically, the main-proxy represents the intrinsic characteristics of a certain category for inter-class distinction. Meanwhile, sub-proxies belonging to each main-proxy provide additional capacity for describing intra-class discrepancy. Therefore, a hierarchical multi-proxy representation naturally captures both inter-class and intra-class diversity, yielding a more complete and distinguishable data representation. Significantly, the instances from distinct categories with similar main-proxy attributes can be further separated by the multiple proxy representation due to distinguishable difference among sub-proxy attributes. Correspondingly, we propose Hierarchical Multi-proxy loss as a reliable guidance for deep metric learning. Performance improvement of around 0.5% in precision on the benchmarks as well as the theoretical analysis demonstrate the effectiveness of the proposed method, especially on datasets with high intra-class variance. [ABSTRACT FROM AUTHOR]

In the realm of computer vision, Deep Convolutional Neural Networks (DCNNs) have demonstrated excellent performance. Video Processing, Object Detection, Image Segmentation, Image Classification, Speech Recognition and Natural Language Processing are some of the application areas of CNN. Object detection is the most crucial and challenging task of computer vision. It has numerous applications in the field of security, military, transportation and medical sciences. In this review, object detection and its different aspects have been covered in detail. With the gradual increase in the evolution of deep learning algorithms for detecting objects, a significant improvement in the performance of object detection models has been observed. However, this does not imply that the conventional object detection methods, which had been evolving for decades prior to the emergence of deep learning, had become outdated. There are some cases where conventional methods with global features are superior choice. This review paper starts with a quick overview of object detection followed by object detection frameworks, backbone convolutional neural network, and an overview of common datasets along with the evaluation metrics. Object detection problems and applications are also studied in detail. Some future research challenges in designing deep neural networks are discussed. Lastly, the performance of object detection models on PASCAL VOC and MS COCO datasets is compared and conclusions are drawn. [ABSTRACT FROM AUTHOR]

The immersive viewing environment of Virtual Reality (VR) system inevitably brings motion sickness to users, and restricts the development of VR system to a certain extent. In order to solve the problem of motion sickness when users view, it is necessary to predicate the motion sickness degree of the visual content. However, the methods based on subjective questionnaires or physiological signals will become cumbersome and infeasible with the growth of demand and supply of VR content. Most of the existing prediction methods for motion sickness are designed for 2D panoramic video. In this paper, a motion sickness prediction method based on content perception and binocular characteristics is proposed for stereoscopic panoramic video. The proposed method mainly consists of two modules: feature extraction based on content analysis and time pooling. The feature extraction module takes into account the attention mechanism of human visual system and the multi-channel characteristics of retina, and simulates the sensory conflict and the three-stage process of binocular stereoscopic perception in virtual environment. The time pooling module takes into account the time memory effect of human eyes. The proposed method has achieved excellent prediction performance on stereoscopic panoramic video comfort database SPVCD, its predication results have a good correlation with the subjective scores, which shows that the proposed method can effectively predict the motion sickness degree of VR content. [ABSTRACT FROM AUTHOR]

Ptychography is a coherent diffractive imaging technique that surpasses the resolution limitations of typical X-ray optics. It generates label-free, high-contrast quantitative amplitude and phase images without requiring prior knowledge. How to quickly recover high-fidelity amplitude and phase contrast images of an authentic sample is a challenge, however, due to the expensive computation that precludes real-time imaging. This paper proposes a framework with a double-branch network, Π-Net, to solve this complex problem. It can simultaneously and robustly reconstruct the object's phase and amplitude information from a degraded image. By analyzing the performance of various activation functions, we find that the activation function plays a crucial role in phase retrieval. Numerical experiments demonstrate the robustness of Π-Net, and our reconstruction results are superior to those of state-of-the-art methods. In addition, ablation studies have revealed that the negative output of neurons can contribute to improved reconstruction outcomes. • We proposed excellent double-branch network architecture with potential and wide application. • It has strong robustness and tidy processing. • We study the vital role of negative output in architecture. [ABSTRACT FROM AUTHOR]

We propose an infrared image super-resolution method with meta-transfer learning and a lightweight network. We design a lightweight network to learn the map between low-resolution and high-resolution infrared images. We train the network with an external dataset and use meta-transfer learning with an internal dataset that makes the network drop to a sensitive and transferable point. We build an infrared imaging system with an infrared module. The designed network is implemented on a personal computer and the SR image is reconstructed by the trained network. The main contribution of this paper is to adopt a lightweight network and meta-transfer learning method, which obtains infrared super-resolution images with better visual effects. Both numerical and experimental results show that the proposed method achieves the infrared image super-resolution, and the performance of the proposed method is superior to four state-of-art image super-resolution methods. The proposed method has practical application in the image super-resolution of mobile infrared devices. [ABSTRACT FROM AUTHOR]

• Explicitly modeling coding features helps prediction of TIS. • Integrating coding features with consensus motifs improves prediction of TIS. • CNN is sensitive to the first reading frame. Translation initiation site (TIS) prediction is one of the most crucial subtasks for gene annotation. Many computational methods have been proposed and achieved acceptable accuracy in transcripts (e.g., cDNA, mRNA). However, the prediction of TIS at the genome level is far more challenging and the computational methods for TIS prediction in genomic sequences so far reach modest performance. Recently, we proposed a method that improves the prediction of TIS in mRNA sequences and demonstrated the significance of explicitly modeling coding features. In this paper, we extend the same results to genomic sequence and present a two-stage deep learning model for TIS prediction in genomic sequence: the first stage to extract coding contrast features around TIS by a hybrid Convolutional Neural Network-Bidirectional Recurrent Neural Network architecture (Content-RCNN), and the second stage to integrate coding contrast features around TIS with TIS sequence encoded by one-hot encoding to jointly predict TIS by a CNN (Integrated-CNN). Four-fold cross validation tests on genome-wide human and mouse datasets demonstrate that the proposed model yields an improved prediction performance of TIS over existing state-of-the-art methods. The source code and the dataset used in the paper are publicly available at: https://github.com/xdcwei/DeepTIS/. [ABSTRACT FROM AUTHOR]

Existing fully convolutional networks (FCNs) - based salient object detection (SOD) methods have achieved great performance by integrating diverse multi-scale context information. However, the performance of context information directly obtained by single dilated convolution has limitations because the introduction of dilated convolution with different filling rates will cause the problem of local information loss, which limits the prediction accuracy of the model. For that, in this paper, a novel Aggregating Dense and Attentional Multi-scale Feature Network (DAMFNet) is designed to generate high-quality feature representations for accurate SOD task. More specifically, we first propose a dense-depth feature exploration (DDFE) module to adequately capture the robust multi-scale and multi-receptive field context information by utilizing parallel integrated convolution (PIC) blocks and dense connections for improving the model ability of locating salient objects and refining object details. Afterwards, we develop a multi-scale channel attention enhancement (MCAE) module to further enhance the selection of the salient objects information in the feature channels by integrating multiple attentional features with diverse perspectives. The proposed DAMFNet method has been broadly evaluated on five public SOD benchmark datasets and the extensive experimental results demonstrate that our DAMFNet method has superior advantages compared to 18 state-of-the-art SOD methods under different evaluation metrics. [ABSTRACT FROM AUTHOR]

Automatic modulation recognition (AMR) plays an essential role in wireless communication systems. Our paper proposes a novel multi-stream neural network (MSNN) to extract the features in parallel from the amplitude, phase, frequency, and raw data of the modulated signal. The framework integrates convolutional neural networks (CNN) and bidirectional gated recurrent units (Bi-GRU) to extract features more effectively from the spatial and temporal characteristics with the assistance of two different attention mechanisms, convolutional block attention module (CBAM) and multi-head self-attention (MHSA). Simulation experiments show that the performance of our proposed algorithm is better than that of other state-of-the-art (SOTA) recognition algorithms on four widely used DeepSig datasets. The recognition accuracy of our proposed model exceeds 99% (18 dB), 93% (16 dB), 95% (20 dB), and 97% (20 dB) respectively on the datasets RML2016.04C, RML2016.10A, RML2016.10B, and RML2018.01A. [ABSTRACT FROM AUTHOR]