Search

Using 3D reconstruction techniques within computer vision frameworks can result in more robust and accurate solutions. However, the main challenge lies in the high computation and memory resources required by such methods. To reduce the complexity of these frameworks, a practical solution is to use geometric landmarks instead of the entire image. Therefore, in this paper the problem of 3D shape recovery of a set of standard 2D landmarks, on a single human face image is faced using multi-dimensional scaling (MDS) approach to find a 3D embedding for a set of standard 2D points. Hence, MDS approach is used for the first time in this study to establish an unbiased mapping from 2D landmark space to the corresponding 3D shape space. A deep neural network learns the pairwise 3D dissimilarity among standard 2D landmarks. This scheme leads to find a symmetric dissimilarity matrix, to be fed into the MDS approach to appropriately recovering the 3D shape of corresponding 2D landmarks. In the case of complex input image formations like posedness or perspective projection causing occlusion in the input image, an autoencoder component is used in the proposed framework, as an occlusion removal part, which turns different input views of the human face into a profile view. The results of performance evaluation using variety of synthetic and real-world human face datasets, including NoW dataset, Besel Face Model (BFM), CelebA, CoMA - FLAME, and CASIA-3D, indicates the superiority of the proposed framework, despite its small number of training parameters, with the related state-of-the-art and recent 3D shape recovery of landmark methods from the literature, in terms of unbiasedness, independent projection, efficiency and accuracy. Further, ablation study is performed to find the best training scheme of the deep learning components. All codes and public data of our paper are publicly available for research purposes at https://github.com/s2kamyab/DeepMDS. [ABSTRACT FROM AUTHOR]

Convolutional neural networks are data-driven Image Quality Assessment (IQA) models based on the convolutional operation in a rectangular window. Deformable convolutions with learnable receptive fields can efficiently extract structural features of irregular objects in an image. By superimposing deformable convolutions, a plug-and-play module is designed to obtain information for irregular geometric shapes. To selectively fuse shallow and deep features, we propose a weighted non-local attention (WNLA) module with the input and output of self-attention in a weighted manner. This paper proposes a dual branch residual full-reference IQA network that combines weighted non-local attention and stacked deformable convolution. The proposed network was trained on PIPAL dataset and tested on LIVE and TID2013. The cross-dataset evaluation shows that the network has a competitive generalization ability. Ablation experiments indicate that the proposed modules can effectively improve the performance of the network. Comparative experiments reveal that our network is superior to existing excellent networks. The codes for training, test and visualization are available at: https://github.com/Pengchang-haha/SDCN.git. • BRB was designed to compensate for the information loss from deep convolutions. • WNLA is proposed to capture the interaction information between patches. • Deformable convolutions were stacked to extract more comprehensive features. [ABSTRACT FROM AUTHOR]

Highlights • We propose a novel framework to person re-identification. • We combine Gaussian features with deep semantic features to enhance the discrimination of overall features. • Two identification losses and a verification loss are used to our fusion model. • Experimental results on benchmarks achieve the state-of-the-art performance of person re-identification. Abstract Person re-identification is an emerging research field in computer vision. Our paper aims to study how to improve the discrimination of person features. We find that some peculiarities of people have not been better attention in the semantic features of deep learning. However, some features obtained by traditional methods can better express the color, and these features are an important clue for re-identification. Therefore, in this paper, we combine traditional Gaussian features with deep semantic features to enhance the discrimination of overall features. At last, we have achieved good performance on two public datasets (Market1501 and VIPeR) in three main distance method learning (DML). In addition, we applied this model to the task of vehicle re-identification. Experiments show that our method has a great improvement on the VeRi vehicle dataset. We compare the results with the current high level results, which indicates the effectiveness of our model. [ABSTRACT FROM AUTHOR]

Highlights • Our method can model various temporal dynamics embedded in videos. • The representations of our method are endowed with discriminative ability. • The dimension of the learned representation is as same as the original one. The aim of this paper is to model long-term evolution of an action video with temporal multi-scale representation. This task is tough due to huge intra-class variations in motion speed. Most of the existing methods consider evolution modeling and multi-scale feature fusion in two separated phases, which generates sub-optimal representation. To address this issue, this paper proposes a novel method to integrate the evolution modeling and multi-scale representation into a unified framework. The core idea is to introduce a temporal multi-scale smoothing vector, which is used to define how the representations at different temporal scales are combined together for frame smoothing. By formulating the smoothing vector learning, evolution modeling and classifier training jointly, our method can learn a discriminative and flexible representation of multi-scale rather than a single scale or a fixed multi-scale smoothing. Experimental results on three datasets demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

• Proposing a shift-insensitive perceptual feature based on GM and LoG signals, and theoretically selecting a ratio parameter to balance them. • Building FR-IQA models based on the proposed QGL feature maps that achieve good performance on three benchmark databases. • Achieving the best shift-insensitive performance on images with spatial changes compared with other metrics. • Combining QGL with deep neural networks and validating that QGL can help improving the accuracy in texture classification. Most existing full-reference (FR) Image quality assessment (IQA) models work in the premise of that the two images should be well registered. Shifting an image would lead to an inaccurate evaluation of image quality, because small spatial shifts are far less noticeable than structural distortion for human observers. To this regard, we propose to study an IQA feature that is shift-insensitive to the basic primitive structure of images, i.e., image edge. According to previous studies, the image gradient magnitude (GM) and the Laplacian of Gaussian (LoG) operator that depict the edge profiles of natural images are highly efficient structural features in IQA tasks. In this paper, we find that the Quadratic sum of the normalized GM and the LoG signals (QGL) has excellent shift-insensitive property in representing image edges after theoretically solving the selection problem of a ratio parameter to balance the GM and LoG signals. Based on the proposed QGL feature, two FR-IQA models can be built directly by measuring the similarity map with mean and standard deviation pooling strategies, named mQGL and sQGL, respectively. Experimental results show that the proposed sQGL and mQGL work robustly on four benchmark IQA databases, and QGL-based models show great shift-insensitive property to spatial translation and image rotation while judging the image quality. In addition, we explore the feasibility of combining QGL feature with deep neural networks, and verify that it can help to promote image pattern recognition in texture classification tasks. [ABSTRACT FROM AUTHOR]

Accurate 3D human pose estimation is a challenging task due to occlusion and depth ambiguity. In this paper, we introduce a multi-hop graph transformer network designed for 2D-to-3D human pose estimation in videos by leveraging the strengths of multi-head self-attention and multi-hop graph convolutional networks with disentangled neighborhoods to capture spatio-temporal dependencies and handle long-range interactions. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix, and a multi-hop graph convolutional block comprised of multi-hop convolutional and dilated convolutional layers. The combination of multi-head self-attention and multi-hop graph convolutional layers enables the model to capture both local and global dependencies, while the integration of dilated convolutional layers enhances the model's ability to handle spatial details required for accurate localization of the human body joints. Extensive experiments demonstrate the effectiveness and generalization ability of our model, achieving competitive performance on benchmark datasets. [ABSTRACT FROM AUTHOR]

In this paper, a semi supervised method for classification of satellite images based on Genetic Algorithm (GA) and Radial Basis Function Neural Network (RBFNN) is proposed. Satellite image classification problem has two major concerns to be addressed. The first issue is mixed pixel problem and the second issue is handling large amount of data present in these images. RBFNN function is an efficient network with a large set of tunable parameters. This network is able to generalize the results and is immune to noise. A RBFNN has learning ability and can appropriately react to unseen data. This makes the network a good choice for satellite images. The efficiency of RBFNN is greatly influenced by the learning algorithm and seed point selection. Therefore, in this paper spectral indices are used for seed selection and GA is used to train the network. The proposed method is used to classify the Landsat 8 OLI images of Dongting Lake in South China. The application of this method is shown for detection of flooded area over this region. The performance of the proposed method was analyzed and compared with three existing methods and the error matrix was computed to test the performance of the method. The method yields high producer’s accuracy, consumer’s accuracy and kappa coefficient value which indicated that the proposed classifier is highly effective and efficient. [ABSTRACT FROM AUTHOR]

To overcome challenges in multiple-object tracking (MOT) tasks, recent algorithms use interaction cues alongside motion and appearance features. These algorithms use graph neural networks or transformers to extract interaction features that lead to high computation costs. In this paper, a novel interaction cue based on geometric features is presented aiming to detect occlusion and reidentify lost targets with low computational costs. Moreover, in the majority of algorithms, camera motion is considered negligible, which is a strong assumption that is not always true and can lead to identity (ID) switching or mismatching of targets. In this paper, a method for measuring camera motion is presented that efficiently reduces its effect on tracking. The proposed algorithm is evaluated on MOT17 and MOT20 datasets and achieves state-of-the-art performance on MOT17 with comparable results on MOT20. The code is also publicly available. 1 1 https://github.com/mhnasseri/for_tracking. • Cascade association based on the detection score. • A new similarity metric that considers detection's size. • A novel interaction model for occlusion handling and target re-identification. • A novel algorithm for estimating camera motion and removing it. • State-of-the-art results on MOT17 and comparable results on MOT20. [ABSTRACT FROM AUTHOR]

Due to the absorption and scattering of light on the water surface, underwater images often face challenges such as low contrast, color deviation, insufficient exposure, and blurred details, exacerbating the difficulty of underwater tasks. In recent years, underwater image enhancement has become increasingly crucial in marine applications. Among existing underwater enhancement methods, the focus has often been on pixel-level learning, which may lead to image noise and an inability to finely adjust the image. In this paper, we propose a dual-branch Transformer-CNN Parameter Filtering network for underwater image enhancement, referred to as DTCPF. Specifically, to better aggregate window information, we introduce an overlapping window self-attention module to enhance interaction between windows. Additionally, we employ an improved Transformer encoder and decoder, utilizing long-distance attention and reversible neural networks to extract low-frequency and high-frequency information from the image. Moreover, we introduce a regression parameter filtering group for regression prediction, using the predicted parameters to enhance the image and obtain a reliable underwater enhancement model. Our approach undergoes qualitative and quantitative analyses on four real underwater datasets, demonstrating outstanding performance. [ABSTRACT FROM AUTHOR]

Convolutional neural networks have achieved remarkable success in single image dehazing tasks, and previous studies verified the dehazing performance of the U-shaped framework. However, most existing U-shaped architecture dehazing networks still face challenges in sufficiently dealing with a large area of haze with low visibility. In this paper, we propose a novel dehazing network named Double U-Net(DU-Net). Specifically, to reduce the interference of haze features in the encoder to the recovery stage when skip-connecting to the decoder directly, we develop a new architecture firstly, which is composed of an extended encoder–decoder. Besides, the hierarchical depth-wise convolution block(HDCB) is designed to gradually increase the receptive field by leveraging the depth-wise convolution, enriching the global information. Moreover, we propose a multi-branch interactive fusion(MIF) which achieves efficient cross-branch and cross-channel interaction through parallel multiple 1D convolutions. Extensive experiments on both synthetic and real-world hazy images demonstrate the effectiveness of our proposed method. • We construct a Double U-Net for image dehazing, by extending the encoder–decoder. • We build hierarchical convolution blocks to obtain a multi-scale receptive field. • A branch fusion is proposed to enhance cross-branch and cross-channel interaction. [ABSTRACT FROM AUTHOR]

Gait recognition is an important biometric technology that allows for the remote collection of stakeholders' characteristics, without requiring their explicit cooperation. It has gained considerable attention in the fields of criminal investigation and intelligent security. Previous studies have shown that local gait features can enhance gait recognition performance by improving robustness to disturbances. However, global gait features also play a crucial role in gait recognition. Many researchers have utilized convolutional operations to extract global features, but these operations tend to focus on features within the receptive field, neglecting those outside of it. Therefore, the potential of global gait features has not been fully explored. In this paper, we propose a gait recognition framework based on vision transformers, aiming to enhance the extraction of global gait features. We introduce an adaptive multi-frame global feature mapping (AMGM) method to address the challenge of inconsistent feature dimensions caused by variations in the number of gait frames when fusing global and local features. We evaluate our model on the latest datasets, and the experimental results demonstrate a significant breakthrough. Notably, our model achieves state-of-the-art recognition accuracy, particularly in scenarios where subjects are wearing coats. Additionally, our model achieves remarkable improvements in recognition accuracy through training with small sample sets. [ABSTRACT FROM AUTHOR]

Seam carving is a method of resizing images based on content awareness. It can realize image retargeting while retaining the main content of the image. However, it may also be maliciously used to tamper with images, such as changing image semantic content by object removal. Therefore, seam carving detection has become important in image forensics. In this paper, a noise residual-based deep learning method is proposed to detect seam carving images. We try to learn the local noise in-consistency of images to recognize them. Firstly, a noise residual extraction segment is used to learn local noise features in the image, and a noise augmentation module is designed to enrich the features, which leverages the noise features extracted from a steganalysis rich model filter to discover the noise in-consistency between authentic and tampered regions. Then, through the feature dimensionality reduction section, the features are further learned and the size of feature maps are reduced. Finally, the output is obtained through global average pooling and a fully-connected layer. A careful testing strategy is further proposed, which greatly improves the detection performance, especially for seam carving with small scaling ratios. The experimental results demonstrate that our method achieves state-of-the-art performance at various scales, and has good robustness and generalization compared with other methods. [ABSTRACT FROM AUTHOR]

Recent self-supervised learning methods, where instance discrimination task is a fundamental way of pretraining convolutional neural networks (CNN), excel in transfer learning performance. Even though instance discrimination task is a well suited pretraining method for classification with its image-level learning, lack of dense representation learning makes it sub-optimal for localization tasks such as object detection. In this paper, we aim to mitigate this shortcoming of instance discrimination task by extending it to jointly learn dense representations alongside image-level representations. We add a segmentation branch parallel to the image-level learning to predict class-agnostic masks, enhancing location-awareness of the representations. We show the effectiveness of our pretraining approach on localization tasks by transferring the learned representations to object detection and segmentation tasks, providing relative improvements by up to 1.7% AP on PASCAL VOC and 0.8% AP on COCO object detection, 0.8% AP on COCO instance segmentation and 3.6% mIoU on PASCAL VOC semantic segmentation respectively. • Convolutional Neural Networks lose localization info with instance discrimination task. • Augmenting instance discrimination with an extra dense task enhances localization. • Unsupervised foreground segmentation boosts localization in (non-)contrastive methods. • Improved localization in SSL enhances transfer learning in dense downstream tasks. [ABSTRACT FROM AUTHOR]

Deep learning methods utilizing Convolutional Neural Networks (CNNs) and Transformers have achieved remarkable success in volumetric medical image analysis. While successful, the symmetrical structure of numerous networks pays insufficient attention to the encoding phase, and the large amount of memory occupied by voxels leads to unnecessary redundancy in the network. In this paper, we present a novel approach to handle volumetric medical images by converting them into point cloud and introduce a new asymmetrical segmentation architecture. We propose a dual-path encoder that fully captures both dense and sparse representations of the input point cloud sampled from volumes. Moreover, the two obtained representations are subtracted at the skip connection as a complementary feature during the decoding stage. Experimental results on the Brain Tumor Segmentation (BraTS) and the Multi-sequence Cardiac MR Segmentation tasks demonstrate the great potential of our point-based method for volumetric medical image segmentation. [ABSTRACT FROM AUTHOR]

Video anomaly detection aims to identify a set of abnormal events in videos. Deep reconstruction and prediction-based models have been employed to detect anomalies. Deep reconstruction models sometimes recreate the abnormal events along with the normal ones. However, the prediction-based approaches have demonstrated encouraging results. This paper presents a video vision transformer (ViViT) based generative adversarial network (GAN), TransGANomaly, a novel approach for detecting anomalies. The proposed framework is a video frame predictor and trained only on normal video data adversarially. The generator of the GAN is a ViViT network that receives 3D input tokens from the video snippets. The generator aims to predict the future frame based on past sequences. After that, the predicted and original frames are given to the model's discriminator for binary classification. Extensive experiments have been performed on UCSD Pedestrian, CUHK Avenue, and ShanghiaTech datasets to validate the efficacy of the proposed method. [ABSTRACT FROM AUTHOR]

The images in foggy weather are often degraded which may affect the object detection process. Hence, in recent times, several schemes have been designed to eliminate the haze effect and enhance the performance of computer vision systems. Although these methods reduce the haze effect, they also produce over-saturation and over-degradation on most occasions. These problems occur due to inadequate utilization of the regional haze properties during the dehazing process. Therefore, a new dehazing model is proposed in this paper to address the issues faced by existing dehazing algorithms. The residual convolutional neural network (RCNN) with 14 layers is developed in the present work. In this model, features to indicate the extent of haze are extracted using the regional characteristics of a hazy image and are used as the fourth channel along with the three colour channels of a hazy image. The proposed RCNN is trained with 4-channel hazy images as input and their corresponding haze-free images as output. The results of our model are obtained with several images taken from three different datasets. Through the experimentation, quantitative and qualitative improvements are observed when compared to state-of-the-art dehazing methods. • The local-haze characteristics driven dehazing method is developed in this work. • The regional haze properties are extracted using pixel-wise local max-min difference and pixel intensities of the hazy image. • A 4-channel hazy image is developed by concatenating the regional haze properties map to the hazy image. • A 14-layered RCNN is developed with the 4-channel hazy image as input and the haze-free image as output. [ABSTRACT FROM AUTHOR]

In this paper, an eye-tracking dataset of movie viewing in the immersive environment is developed, which contains 256 movie clips with 2K QHD resolution and corresponding movie genre labels from IMDb (Internet Movie Database). The dataset provides the audio-visual clues for studying the human visual attention when watching movie using a VR headset, by recording the eye movements using integrated eye tracker. To provide benchmarks for a saliency prediction for movie viewing in the immersive environment, fifteen computational models are evaluated on the dataset, including a newly developed multi-stream audio-visual saliency prediction model based on deep neural networks, named as MSAV. Detailed quantitative and qualitative comparisons and analyses are also provided. The developed dataset and benchmarks could help to facilitate the studies of visual saliency prediction for movie viewing in the immersive environments. [ABSTRACT FROM AUTHOR]

Domain generalizable person re-identification (DG ReID) aims to obtain a model that can be applied directly to unseen domains once trained on a set of source domains (datasets collected from different camera networks). Among current DG ReID methods, instance normalization is a promising solution to reduce the effect of domain bias, but it inevitably filters out some discriminative information. Besides, since most pioneering approaches cannot effectively address the loss of discriminative information, this paper proposes a Style Elimination and Information Restitution (SEIR) module and constructs a generalizable framework. Specifically, we utilize instance normalization to reinforce the generalization capability on unseen domains. Then, the instance-specific mean and variance are employed to construct a feature vector that includes discriminative and style information. Besides, an encoder–decoder structure is designed to restitute the discriminative information to ensure a high recognition rate. Finally, a dual optimization strategy is devised to update the proposed module by simulating a real train-test process to enhance the generalization robustness further and prevent SEIR from overfitting to the source domain. Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art methods by a large margin on the public DG ReID benchmarks. • A Style Elimination and Information Restitution module for handling domain bias. • An optimization strategy for improving the generalization ability. • We propose a framework for generalizable person re-identification. • Significant results were achieved for addressing the generalizable person re-identification. [ABSTRACT FROM AUTHOR]

Accurate segmentation of nasopharyngeal carcinoma (NPC) in magnetic resonance (MR) images is crucial for radiotherapy planning. However, vanilla 2D/3D deep convolutional networks fail to gain satisfying NPC segmentation results, since 3D methods suffer from inter-slice discontinuities and 2D methods lack inter-slice context learning. To address this problem, this paper proposes a 2.5D learning Selective Inter-slice Context Network (SICNet) for NPC segmentation. The proposed SICNet comprises a main encoder that extracts target features from the middle slice and an auxiliary encoder that learns inter-slice context from adjacent slices. The features from two encoders are fused by the Inter-slice Context Fusion (ICF) module and then sent to the decoder for predictions. Furthermore, a knowledge distillation module named Mask-Guided Self-knowledge Distillation (MGSD) is proposed to help the auxiliary encoder learn context selectively. Specifically, with the guidance of mask labels, MGSD tells the auxiliary encoder where to look and how to learn, thus achieving selective inter-slice context learning and boosting segmentation accuracy. The proposed SICNet achieves a DSC of 74.38 ± 11.99, an HD95 of 9.31 ± 2.68, and an ASSD of 1.46 ± 0.76 on an in-house NPC dataset, outperforming other state-of-the-art segmentation methods. [ABSTRACT FROM AUTHOR]

Lossy compression introduces artifacts, and many conventional in-loop filters have been adopted in the AV1 standard to reduce these artifacts. Researchers have explored deep learning-based filters to remove artifacts in the compression loop. However, the high computational complexity of CNN-based filters remains a challenge. In this paper, a Texture- and Motion-Aware Perception (TMAP) in-loop filter is proposed to addresses this issue by selectively applying CNNs to texture-rich and high-motion regions, while utilizing non-learning methods to detect these regions. The proposed method introduces a new CNN structure, the Dense-Dual-Field Network (DDFN), which leverages a larger receptive field to enhance the quality of reconstructed frames by incorporating more contextual information. Furthermore, to improve perceptual quality, a novel loss function integrating wavelet-based perceptual information is presented. Experimental results demonstrate the superiority of our proposed models over other lightweight CNN models, and the effectiveness of the perceptual loss function is validated using the VMAF metric. [ABSTRACT FROM AUTHOR]

Recent improvements in dynamic hand gesture recognition have seen a shift from traditional convolutional architectures to attention-based networks. These attention networks have been proven to outclass CNN + LSTM architectures, showing higher accuracy as well as reduced model parameters. Especially, skeleton-based attention networks have been shown to outperform visual-based networks due to the rich information from skeleton-based hand features. However, there is an opportunity to introduce complementary features from other modalities like RGB, depth, and optical flow images to enhance the recognition capability of skeleton-based networks. This paper aims to explore the addition of a multimodal fusion network to a skeleton-based Hierarchical Self-Attention Network (MF-HAN) and test for increased model effectiveness. Unlike traditional fusion techniques, this fusion network uses features derived from other sources of multimodal data in a reduced feature space using a cross-attention layer. The model outperforms its root model and other state-of-the-art models on the SHREC'17 track dataset, especially in the 28 gestures setting by more than 1 % in gesture classification accuracy. The experimentation was tested on the DHG dataset as well. [ABSTRACT FROM AUTHOR]

In the last decade, the exploration of deep-sea ecosystems has surged, offering exciting prospects for discovering untapped resources such as medical drugs, food and energy sources, and renewable energy products. Consequently, research in underwater image processing has witnessed substantial growth. However, underwater imaging poses significant challenges, particularly without sophisticated, specialized cameras. Traditional cameras are impacted by absorption and scattering in the aquatic environment, producing hazy images with a blue–green tint. This phenomenon holds implications for marine research and other disciplines that rely on underwater imaging. While hardware advancements have been made over the years, image processing remains a valuable, cost-effective, and practical approach for underwater enhancement. Despite the existence of state-of-the-art techniques for underwater enhancement and restoration, their performance is often inconsistent. While some methods excel in contrast restoration, color restoration remains a pervasive challenge. In this paper, we introduce Sea-Pix-GAN, a Generative Adversarial Network (GAN)-based model that addresses these issues in underwater image enhancement. We redefine the problem as an image-to-image translation task and tailor the objective and loss functions to achieve color, content, and style transfer. The model is trained on a large dataset of underwater scenes, encompassing the diverse color dynamics of underwater subjects. Sea-Pix-GAN demonstrates promising results in restoring color, contrast, texture, and saturation. To validate its effectiveness, we compare the performance of Sea-Pix-GAN quantitatively based on metrics like PSNR, SSIM, and UIQM and qualitatively against several existing techniques. [ABSTRACT FROM AUTHOR]

In social networks, people usually communicate online by posting and passing images, people will find that there are many similar or even visually identical images. When the images determine whether the published information is true or false, original images became crucial, and at the same time, deep learning excelled in image identification. Combining the original image identification and deep learning methods, this paper proposes the original image identification method based on the deep network models, aiming to use the deep network models to identify the original images between the original images and a bunch of variant images of the original images, and obtain satisfactory results. In this way, through the trained network models, the original images can be identified and can help the viewers to determine the correctness of the information. Experimental results verify the feasibility, effectiveness and security of the proposed method. • The method can prevent the images from being stolen during transmission. • The method can achieve good identification results on the image databases. • The method is feasible and flexible in actual operation. [ABSTRACT FROM AUTHOR]

• An image fusion method is proposed that can effectively highlight infrared target features and retain visible visual information. • Infrared targets are analyzed based on the saliency weight map. • The target contour and background texture are refined by a method based on weight refinement of guided filtering. • A multi-layer fusion strategy is used to fuse saliency weight maps at multiple scales. Image fusion is the fusion of multiple images from the same scene to produce a more informative image, and infrared and visible image fusion is an important branch of image fusion. To tackle the issues of diminished luminosity in the infrared target, inconspicuous target features, and blurred texture of the fused image after the fusion of infrared and visible images. This paper introduces a novel effective fusion framework that merges multi-scale Convolutional Neural Networks (CNN) with saliency weight maps. First, the method measures the source image features to estimate the initial saliency weight map. Then, the initial weight map is segmented and optimized using a guided filter before being further processed by CNN. Next, a trained Siamese convolutional network is used to solve the two key problems of activity measure and weight assignment. Meanwhile, a multi-layer fusion strategy is designed to effectively retain the luminance of the infrared target and the texture information in the visible background. Finally, adaptive adjustment of the fusion coefficients is achieved by employing saliency. The experimental results show that the method outperforms the state-of-the-art algorithms in terms of both subjective visual quality and objective evaluation effects. [ABSTRACT FROM AUTHOR]

• A novel attention module is introduced to capture important action features in a video both at pix level and channel level. • Another novel attention module is proposed to highlight discriminative convolutional features at channel level. • Based on the above two attention modules, a new network architecture is proposed for action classification in videos. Human action recognition in videos is a fundamental and important topic in computer vision, and modeling spatial–temporal dynamics in a video is crucial for action classification. In this paper, a novel attention module named Channel-wise Non-local Attention Module (CNAM) is proposed to highlight the important features both spatially and temporally. Besides, another new attention module named Channel-wise Attention Recalibration Module (CARM) is developed to focus on capturing discriminative features at channel level. Based on these two attention modules, a novel convolutional neural network named Residual Attention Fusion Network (RAFN) is proposed to model long-range temporal structure and capture more discriminative action features at the same time. More specifically, first, a sparse temporal sampling strategy is adopted to uniformly sample video data as input to RAFN along the temporal dimension. Secondly, the attention modules CNAM and CARM are plugged into residual network for highlighting important action regions around actors. Finally, the classification scores of four streams of RAFN are combined by late fusion. The experimental results on HMDB51 and UCF101 demonstrate the effectiveness and excellent recognition performance of our proposed method. [ABSTRACT FROM AUTHOR]

In contrast to Convolutional Neural Networks (CNNs), Vision Transformers (ViT) cannot capture sequence ordering of input tokens and require position embeddings. As a learnable fixed-dimension vector, the position embedding improves accuracy while limiting the migration of the model between different input sizes. Hence, this paper conducts an empirical study on position embeddings of pre-trained models, which mainly focuses on two questions: (1) What do the position embeddings learn from training? (2) How do the position embeddings affect the self-attention modules? This paper analyzes the pattern of position embedding in pre-trained models and finds that the linear combination of Gabor filters and edge markers can fit the learned position embeddings well. The Gabor filters and edge markers can occupy some channels to append the position information, and the edge markers have flowed to values in self-attention modules. The experimental results can guide future work to choose suitable position embeddings. • Position Embeddings are the linear combination of Gabor Filters and Edge Markers. • The main part of the positional embeddings is input-independent. • Relative position embeddings requires adding edge markers to the value matrix. [ABSTRACT FROM AUTHOR]

Dynamic hand gesture recognition is still an interesting topic for the computer vision community. A set of feature vectors can represent any hand gesture. A Recurrent Neural Network (RNN) can recognize these feature vectors as a hand gesture that analyzes the temporal and contextual information of the gesture sequence. Thus, we proposed a hybrid deep learning framework to recognize dynamic hand gestures. In the Hybrid model GoogleNet is pipelined with a Bidirectional GRU unit to recognize the dynamic hand gesture. Dynamic hand gestures consist of many frames, and features of each frame need to be extracted to get the temporal and dynamic information of the performed gesture. As RNN takes input as a sequence of feature vectors, we extract features from videos using pretrained GoogleNet. As Gated Recurrent Unit is one of the variants of RNN to classify the sequential data, we created a feature vector that corresponds to each video and passed it to the bidirectional GRU (BGRU) network to classify the gestures. We evaluate our model on four publicly available hand gesture datasets. The proposed method performs well and is comparable with the existing methods. For instance, we achieved 98.6% accuracy on Northwestern University Hand Gesture(NWUHG), 99.6% on SKIG, 99.4% on Cambridge Hand Gesture (CHG) datasets respectively. We performed our experiments on DHG14/28 dataset and achieved an accuracy of 97.8% with 14-gesture classes and 92.1% on 28-gesture classes. DHG14/28 dataset contains skeleton and depth data, and our proposed model used depth data and achieved comparable accuracy. • The paper presented a two-stream hybrid model that uses a convolutional neural network to extract the frame-level features and used a BGRU to classify the dynamic hand gesture. • A deeply coupled hybrid pipeline is used to extract spatial and temporal features. • The paper presented a comprehensive review of the BLSTM and BGRU model with different batch sizes. • Experimental results shows that the BGRU model with optical flow and RGB videos performs better than state-of-the-art. [ABSTRACT FROM AUTHOR]

Previous deep learning-based single image dehazing methods are implemented by using real-valued convolutional neural network (RV-CNN), which ignore the phase information of the image, and tend to cause models to perform well on synthetic datasets but low generalization on real-world datasets. In this paper, we propose a novel Complex-Valued convolutional Dehazing Network (CVD-Net), which considers both amplitude and phase information of image for haze removal in real-world scenes. Specifically, we construct complex-valued transformation module, including complex-valued convolutional layer and residual block embedded into the middle of the network to extract features of image efficiently, and also design a complex-valued selected fusion module (CVSF) and complex-valued attention module (CVAM) to promote the interaction between different scale features for preserving image detailed information. Both qualitative and quantitative experimental results show that the proposed CVD-Net can effectively remove the haze, and has good generalization in real-world hazy images. [ABSTRACT FROM AUTHOR]

Image quality assessment (IQA) is an indispensable technique in computer vision, which is widely applied in image classification, image clustering. With the development of deep learning, deep neural network (DNN)-based methods have shown impressive performance. Thus, in this paper, we propose a novel method for mechanical equipment fault diagnosis based on IQA. More specifically, we first conduct data acquisition base on our practice. Afterwards, we leverage image processing method for removing noise. Subsequently, we leverage CNN-based method for image classification. Finally, different mechanical equipment images will be grouped into different categories and fault detection can be achieved. Extensive experiments demonstrate the effectiveness and robustness of our method. [ABSTRACT FROM AUTHOR]

• A novel approach to gender classification with state-of-the-art accuracy. • Combination between local and contextual features extracted from face images. • Employing deep learning and convolutional neural networks (CNNs). • Easy detection of single or multiple face regions. • A challenging dataset of 42,592 face images for research purposes with annotations. Gender classification aims at recognizing a person's gender. Despite the high accuracy achieved by state-of-the-art methods for this task, there is still room for improvement in generalized and unrestricted datasets. In this paper, we advocate a new strategy inspired by the behavior of humans in gender recognition. Instead of dealing with the face image as a sole feature, we rely on the combination of isolated facial components and a contextual feature which we call the foggy face. Then, we use these features to train deep convolutional neural networks followed by an AdaBoost-based score fusion to infer the final gender class. We evaluate our method on four challenging datasets to demonstrate its efficacy in achieving better or on-par accuracy with state-of-the-art methods. In addition, we present a new face dataset that intensifies the challenges of occluded faces and illumination changes, which we believe to be a much-needed resource for gender classification research. [ABSTRACT FROM AUTHOR]

• A new method for non-blind correlated noise removal. • Convolutional neural network based. • Fully automatic once the autocovariance function of the noise and the noisy image are given. • Performs better than several methods which in addition may require careful parameter tuning. In this paper, removal of additive, signal-independent, correlated noise from images is considered. We consider the non-blind case, meaning that the stationary autocovariance function of the noise is assumed to be known. The denoising method is based on unrolled optimization where in the half-quadratic energy minimization each proximal step is replaced by a learnt convolutional neural network. The proximal steps take place in learnt transform domains. Functions producing the regularization parameters are also learnt. We assume that we have a distribution for autocovariance functions in order to be able to draw samples. For simplicity, we assume that the noise has low-pass spectral character and a typical autocovariance function has a relatively simple form. The experimental results demonstrate that in terms of PSNR values, the method performs better than two classical methods and a method based on a learnt patch prior. [ABSTRACT FROM AUTHOR]

This paper introduces a novel video saliency model for salient object detection in videos. Firstly, we generate multi-level deep features via a symmetrical convolutional neural network, in which the inputs are the current frame and the optical flow image. Then, the multi-level deep features are integrated in a hierarchical manner using a fusion network, which deploys attention module to make a selection for deep features. Lastly, the integrated deep feature is combined with the boundary information originated from shallow layer of the feature extraction networks, and the saliency map is generated by the saliency prediction step. The key advantages of our model lie on the attention module, the hierarchical integration and the boundary information, in which the former one acts as weight filter and is used to select the most salient regions in deep features, the middle one gives an effective integration manner for features from different layers and the last one provides well-defined boundaries for saliency map. Extensive experiments are performed on two challenging video dataset, and the experimental results show that our model consistently outperforms the state-of-the-art saliency models in a large margin. [ABSTRACT FROM AUTHOR]

• A novel top-down framework is presented to address multi-person pose estimation. • We improved the faster R-CNN detector by infusing the soft-NMS and ASDN items. • Heatmaps instead of activation map are used to pinpoint the precise joints location. Human motion recognition based on computer vision plays an important role in many fields, such as video surveillance, virtual reality, and medical care. To solve the inaccurate multi-person pose estimation problem and improve the generalizability of the extracted features, this paper proposes a multi-person pose estimation method based on a deep convolutional neural network. This method mainly relies on a top-down structure which includes two stages. In the first stage, the bounding boxes that are likely to contain people are first detected by an improved faster R-CNN. Individuals in the complex scenario are then tailored by box cropping. In the second stage, we combine heatmap detection with coordinate regression to address the single person pose estimation problem. Specially, a deep convolutional ResNet is employed to produce heatmaps of human body. The precise location of each joint is achieved by the fully connected conditional random field. Experimental results demonstrate our method achieves comparable performance with the state-of-the-art ones. [ABSTRACT FROM AUTHOR]

Deep learning has been widely applied in image processing and computer vision due to its powerful learning capability. Although some learning models have been proposed to suppress noise in images, most of them are developed for Gaussian noise and few are for impulse noise. This paper proposes an image recovery method based on deep convolutional neural networks for impulse noise removal. The proposed framework falls into two components: a classifier network which divides image pixels into noisy and noise-free, and a regression network which is trained for image reconstruction. In the regression network, the noise-free pixels identified by the classifier network together with the original noisy image are used for recovery of the noisy image. Furthermore, batch normalization is embedded to the network to improve denoising performance. Experimental results show that the proposed method can excellently remove impulse noise, providing clear performance improvements over other state-of-the-art denoising methods. [ABSTRACT FROM AUTHOR]

• We present a novel discriminative model to learn similarity measurement between source domain and target domain. • We freeze part of the network layers to extract well-defined representations from source domain to target domain. • We use auxiliary images labelled by perspective-class to improve categorization performance. Fine-grained categorization is challenging due to its small inter-class and large intra-class variance. Moreover, requiring domain expertise makes fine-grained labelled data much more expensive to acquire. Existing models predominantly require extra information such as bounding box and part annotation in addition to the image category labels, which involves heavy manual labor. In this paper, we propose a novel hierarchical deep transfer learning model, based on a compression convolutional neural network. Our model transfers the learned image representation from large-scale labelled fine-grained datasets to micro fine-grained datasets, which avoids using expensive annotations and realizes visual categorization task effectively. Firstly, we introduce a cohesion domain to measure correlation degree between source domain and target domain. Secondly, the source-domain convolutional neural network is adjusted according to its metrical feedback, in order to select task-specific features that are suitable for transferring to the target domain. Finally, we make most of perspective-class labels, which are inherent attributes of fine-grained data for multi-task learning and learn all the attributes through joint learning to extract more discriminative representations. The proposed model not only economizes training time effectively and achieves high categorization accuracy, but also verifies that the inter-domain feature transition can accelerate learning and optimization. [ABSTRACT FROM AUTHOR]

• Well-defined patches can sufficiently classify facial expressions and reduce computation cost. • Well-defined patches have better interference reduction than cropped face. • The proposed network can extract robust features under fast speed and cope with small scale dataset. • The proposed loss can adjust training by network output accordingly and boost up FER performance. One of the key challenge issues of deep-learning-based facial expression recognition (FER) is learning effective and robust features from variant samples. In this paper, Region-based Convolutional Fusion Network (RCFN) is proposed to solve this issue via three aspects. Firstly, a muscle movement model is built to segment out crucial regions of frontal face, providing well-unified patches with benefits of removing unrepresentative regions and greatly reducing interference caused by facial organs with varied sizes and positions among individuals. Secondly, a fast and practical network is constructed to extract robust triple-level features from low level to semantic level in each crucial region and fuse them for FER. Thirdly, constrained punitive loss is introduced to leverage the network training for boosting up FER performance. The experiment results show that RCFN is effective in commonly used datasets like KDEF, CK+, and Oulu-CASIA, and can achieve comparable performance with other state-of-the-art FER methods. [ABSTRACT FROM AUTHOR]

Illustration of the VST-Net where the channel number of the last Conv layer at SubNet1 and SubNet2 is 1. "C", "B" and "R" stands for the Conv, BN and ReLU layers, respectively. • A novel Poisson denoising network based on convolutional neural network is proposed. • The proposed VST-Net inherits and extends the structure and strengths of the traditional VST. • Two-stage learning strategy and joint learning strategy are exploited to investigate the network. Poisson noise occurs in various applications including medical imaging and night vision. There exist many traditional Poisson denoising algorithms, particularly, one class of impressive algorithm is based on variance-stabilizing transformation (VST). Inspired by the traditional VST scheme, in this paper we propose a novel Poisson denoising model based on convolutional neural network, called variance-stabilizing transform network (VST-Net). VST-Net inherits the structures and strengths of the traditional VST scheme via optimizing the nonlinear transformation by means of network design and supervised learning. The whole VST-Net network contains three sub-networks. The first and third sub-networks simulate the forward and inverse Anscombe transforms, respectively. Meanwhile, the second sub-network is devoted to playing the role of approximate Gaussian denoising. Joint learning strategy and two-stage progressive learning strategy are exploited to investigate the rationality and strength of the VST scheme. Experimental results verify the great potential of the VST-driven network. Code is available at: https://github.com/yqx7150/VST-Net. [ABSTRACT FROM AUTHOR]

• Radial feature descriptors improve cell classification and image recommendation. • Textural variations along cellular extra-nuclear narrow band describe cytoplasm. • Hybrid method contains unsupervised and supervised real-time machine learning. • Nuclei segmentation based on hierarchical clustering handles cell clumps and overlap. • Random Forest for classification and CNN for recommendation improve cell analytics. This paper introduces computational tools for cell classification into normal and abnormal, as well as content-based-image-retrieval (CBIR) for cell recommendation. It also proposes the radial feature descriptors (RFD), which define evenly interspaced segments around the nucleus, and proportional to the convexity of the nuclear boundary. Experiments consider Herlev and CRIC image databases as input to classification via Random Forest and bootstrap; we compare 14 different feature sets by means of False Negative Rate (FNR) and Kappa (κ), obtaining FNR = 0.02 and κ = 0.89 for Herlev, and FNR = 0.14 and κ = 0.78 for CRIC. Next, we sort and rank cell images using convolutional neural networks and evaluate performance with the Mean Average Precision (MAP), achieving MAP = 0.84 and MAP = 0.82 for Herlev and CRIC, respectively. Cell classification show encouraging results regarding RFD, including its sensitivity to intensity variation around the nuclear membrane as it bypasses cytoplasm segmentation. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) is the interesting research area that enables us to recognize the expression of the human face in the day-to-day life. Most of the traditional methods fail to recognize the expressions accurately as the expressions are based on the movements of the parts in the human face. The paper proposes the effective method of FER using the proposed Whale- Grasshopper Optimization algorithm based Multi-Support Vector Neural Network (W-GOA-based MultiSVNN). The features from the facial image is extracted using the Scale-Invariant Feature Transform (SIFT) and the proposed Scatter Local Directional Pattern (SLDP). The extracted features are classified using the proposed classifier to recognize the expression of the face. The proposed method of facial recognition enhances the recognition accuracy. The experimentation of the proposed algorithm is performed using the databases, such as Cohn-Kanade AU-Coded Expression Database and The Japanese Female Facial Expression (JAFFE) Database. The proposed algorithm outperforms the existing methods in terms of the accuracy, TPR, and FPR and the values are found to be 0.96, 0.96, and 0.009, respectively. [ABSTRACT FROM AUTHOR]

• A new distance metric method is proposed to improve the retrieval performance. • We build a deep framework to extract advanced image features for large-scale. • We extend the original deep framework to be suitable for multi-label datasets. • Extensive experimental results demonstrate the advantages of our method. Similarity-preserving hashing has become the mainstream of approximate nearest neighbor (ANN) search for large-scale image retrieval. Recent research shows that deep neural networks can produce efficient feature representation. Most existing deep hashing schemes simply utilize the middle-layer features of the deep neural networks to measure the similarity between query images and database images. However, these visual features are suboptimal for discriminating the semantic information of images, especially for complex images that contain multiple objects. In this paper, a deep framework is employed to learn multi-level non-linear transformations to obtain advanced image features, and then we combine these intermediate features and top layer visual information to implement image retrieval. Three criterions are enforced on these compact codes: (1) minimal quantization loss; (2) evenly distributed binary; (3) independent bits. The experimental results on five public large-scale datasets demonstrate the superiority of our method compared with several other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a depth-aware saliency model which consists of three networks. • Color saliency network, depth saliency network and saliency fusion network are jointly trained. • The depth saliency network exploits weight initialization to learn effective depth features. • The saliency fusion network can effectively integrate depth saliency map with color saliency map. • Results on five public datasets demonstrate the better performance of our model. This paper proposes a new end-to-end depth-aware saliency model using three convolutional neural networks including color saliency network, depth saliency network and saliency fusion network, for saliency detection in RGBD images and stereoscopic images. Firstly, the color image is fed to the color saliency network to generate the color saliency map. Then, by sharing the weights of some layers in the color saliency network, the depth saliency network exploits the weight initialization and multi-layer pyramid feature fusion to learn effective depth features from the three-channel depth image, which is converted from the original depth map, and generates the depth saliency map. Finally, the saliency fusion network integrates the color saliency map with the depth saliency map into the final saliency map, which can highlight salient object regions and suppress background regions more effectively. Experimental results on five public datasets demonstrate that our model achieves the better performance compared with the state-of-the-art depth-aware saliency models. [ABSTRACT FROM AUTHOR]

• A carrier-less text steganography based on CNN-LSTM network is proposed. • SSH scheme based on beam search can be applied in model sharing situation. • HH scheme for model-free extraction with hash function is proposed. • The tradeoff between text quality and embedded capacity can be balanced. • The proposed schemes achieve high text quality, invisibility, and security. It is a challenge to conduct natural language steganography on Online interactive platforms such as photo-sharing websites since the stego texts should be consistent with the content of the images. In this paper, a novel natural language steganographic framework based on an end-to-end generative network is proposed. A Convolution Neural Network (CNN) combined with Long Short-Term Memory (LSTM) is trained to generate stego descriptions. Word by Word Hiding (WWH) and Sentence by Sentence Hiding (SSH) schemes are proposed to achieve various embedding capacity under the premise of sharing model between the sender and the receiver. Furthermore, a blind extraction scheme called Hash Hiding (HH) is proposed in case that the model is unavailable for data extraction. Comparative experiments show the superiority of the proposed framework. It is verified that the proposed framework is an effective carrier-less steganographic framework with competitive embedding capacity, considerable text quality, and good reversibility. [ABSTRACT FROM AUTHOR]

Abstract Perceptual quality evaluation of multiply distorted images has become a very challenging research topic. In this paper, we present a novel and efficient deep blind quality evaluator for multiply distorted images based on monogenic binary coding (MBC). Local complementary structural information and a deep learning method are employed to blindly evaluate the quality of multiply distorted images. First, a monogenic signal representation is utilized to decompose a multiply distorted image into three complementary components: orientation, phase, and magnitude. The quality-predictive features are then determined from the complementary components. Finally, the features are mapped to the human quality score of the multiply distorted image based on the deep neural network. The results on two newly established multiply distorted image subjective databases confirm that our metric has a better prediction performance than existing state-of-the-art full-reference and classical blind metrics. [ABSTRACT FROM AUTHOR]

Highlights • A structure representation based contrast-distorted quality metric is proposed. • The statistical structural features are extracted on the S-CIELAB color space. • Experiments have shown the superiority and applications of the proposed method. Abstract In many real-world applications, images are prone to be degraded by contrast distortions during image acquisition. Quality assessment for contrast-distorted images is vital for benchmarking and optimizing the contrast-enhancement algorithms. To this end, this paper proposes a no-reference quality metric for contrast-distorted images based on Multifaceted Statistical representation of Structure (MSS). The "Multifaceted" has two meanings, namely (1) not only the luminance information, but also the chromatic information is used for structure representation. This is inspired by the fact that the chromatic information on the one hand affects the perception of image quality as well, and on the other hand it changes along with the contrast distortions. Therefore, the chromatic information should be integrated with the luminance information for quality assessment of contrast-distorted images, a fact most existing quality metrics overlook; (2) regarding structure representation, three aspects, i.e. spatial intensity, spatial distribution, and orientation of structures are calculated, which is enlightened by the fact that the human visual system (HVS) is sensitive to the three aspects of structures. Specifically, the image is first transformed from RGB to the S-CIELAB color space to obtain a representation that is more consistent with the characteristics of the HVS, as well as to separate the chromatic information from the luminance. Then the statistical structural features are extracted from both luminance and chromatic channels. Finally, the back propagation (BP) neural network is adopted to train a quality prediction model. Experimental results conducted on four public contrast-distorted image databases demonstrate the superiority of the proposed method to the relevant state-of-the-arts. [ABSTRACT FROM AUTHOR]

Abstract The goal of super-resolution (SR) is to recover a high-resolution (HR) image from its corresponding low-resolution (LR) image. It is an ill-posed problem. Most recent methods are based on external training data. They can reconstruct pleasing HR results, especially when the input patch has a similar counterpart within the training dataset. Other methods are driven by self-similarity and are called internal methods. They can produce visually plausible HR images when the input images contain abundant regular structures. In this paper, we propose a hybrid method for image SR that exploits the complementary advantages of external and internal SR methods. Each input LR patch is first super-resolved using convolutional neural network (CNN) for external SR and self-similarity for internal SR. Then, we calculate the perceptual similarity between the feature representations from the pre-trained VGG network to learn an adaptive weight. Finally, our algorithm automatically selects the optimal method on the basis of the calculated adaptive weight. The experimental results of our visual and quantitative evaluations verify the effectiveness of the proposed method, by comparing it with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract The rapid development of social media services has spawned abundant user generated contents (UGC), such as Sina Weibo, which is one of the biggest Chinese microblogging platforms. In order to enhance the quality and popularity of the posted weibo (the microblog), Weibo users usually embed some social information and images or micro-videos, namely the multimodal weibo, and we assume that there is a close correlation among the multimodal weibo data, especially between the visual data (image/micro-video) and its corresponding text, for a multimodal weibo of high quality. Hence, we try to evaluate the quality of multimodal weibo via analyzing the correlation in the multimodal weibo. This paper constructs the classification model based on back propagation (BP) neural network with genetic algorithm (GA), to automatically identify the correlation within the multimodal weibo, and investigates three kinds of features from multimodal weibo to uncover their contributions to the correlation. The experimental results verify the superiority of the GA-BP based classification model over the traditional BP neural network. [ABSTRACT FROM AUTHOR]

Abstract Collective activity recognition, which analyses the behavior of groups of people in videos, is an important goal of video surveillance systems. In this paper, we focus on collective activity recognition problem and propose a new multi-stream convolutional neural network architecture that utilizes information extracted from multiple regions. The proposed method is the first work that uses a multi-stream network and multiple regions in this problem. Various strategies to fuse multiple spatial and temporal streams are explored. We evaluate the proposed method on two benchmark datasets, the Collective Activity Dataset and the Volleyball Dataset. Our experimental results show that the proposed method improves collective activity recognition performance when compared to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Abstract Prediction of visual saliency in images and video is needed for video understanding, search and retrieval, coding, watermarking and other applications. The majority of prediction models are founded only on "bottom-up" features. Nevertheless, the "top-down" component of human visual attention becomes prevalent as human observers explore the visual scene. Visual saliency which is always a mix of bottom-up and top-down cues can be predicted on the basis of seen data. In this paper, a model of prediction of visual saliency in video on the basis of Deep convolutional neural networks (CNNs) is proposed. A Deep CNN architecture is designed. Various input channels for a CNN architecture are studied: using the known sensitivity of human visual system to residual motion, pixel colour values are completed with residual motion map. The latter is a normalized energy of residual motion in video frames with regard to the estimated global affine motion model. The experiments show that the choice of the input features for the Deep CNN depends on visual task: for highly dynamic content, the proposed model with residual motion is more efficient and gives decent results with relatively shallow Deep architecture. [ABSTRACT FROM AUTHOR]

Abstract How to correctly recognize and extract named entities, such as disease names, medical measurements and therapies, from online medical diagnosis data remains challenging. For the one hand, conventional natural language processing methods cannot be directly applied in the field of online medical diagnosis (OMD). Although existing supervised or unsupervised learning algorithms have offered strategies for OMD on open websites, such methods might extensively rely on specific knowledge sources or manually designed features. For the other hand, due to the large size of the data and the sophistication of the data structure, it is difficult to establish a robust NLP model for recognition and extraction of clinical named entities in paragraphs. Therefore, in the paper, we try to establish a new deep neural network (DNN), combine the bi-directional long short-term memory (Bi-LSTM) and the conditional random field (CRF), and utilize online medical diagnosis data for recognition and extraction of clinical named entities. Different from existing artificial neural networks (ANN), the proposed neural network well functions even without manual rules or features. The word representations input into the DNN is connected by character-based representations and continuous bags of word clusters (CBOWC) in an embedding way. We test the new DNN on different online medical diagnosis datasets obtained from a scalable web crawler, and compare it with the long short-term memory (LSTM) neural network linguistic model, the convolutional neural network (CNN) model and the most advanced Bi-LSTM-CRF. The fundamentals for the method selection is that they have been suggested to generate acceptable results. According to the comparison analyses, the DNN is proved to be a reliable tool and improves every benchmark performance of OMD. [ABSTRACT FROM AUTHOR]

Highlights • The first to embed features in a hypersphere manifold for person re-identification. • A new simple but efficient Sphere Loss and SphereReID network. • New balanced sampling strategy and warming-up strategy to boost the training. • We achieve state-of-the-art performance on four popular ReID datasets. • Our source code will be publicly released. Abstract Many current successful Person Re-Identification (ReID) methods train a model with the softmax loss function to classify images of different persons and obtain the feature vectors at the same time. However, the underlying feature embedding space is ignored. In this paper, we use a modified softmax function, termed Sphere Softmax, to solve the classification problem and learn a hypersphere manifold embedding simultaneously. A balanced sampling strategy is also introduced. Finally, we propose a convolutional neural network called SphereReID adopting Sphere Softmax and training a single model end-to-end with a new warming-up learning rate schedule on four challenging datasets including Market-1501, DukeMTMC-reID, CHHK-03, and CUHK-SYSU. Experimental results demonstrate that this single model outperforms the state-of-the-art methods on all four datasets without fine-tuning or re-ranking. For example, it achieves 94.4% rank-1 accuracy on Market-1501 and 83.9% rank-1 accuracy on DukeMTMC-reID. The code and trained weights of our model will be released. [ABSTRACT FROM AUTHOR]