Search

Despite pedestrian detectors having made significant progress by introducing convolutional neural networks, their performance still suffers degradation, especially in occlusion scenes with more false positives (FPs) and false negatives (FNs). To alleviate the problem, we propose a novel Sequential Attention-based Distinct Part Modeling (SA-DPM) for balanced pedestrian detection. It takes one step further in constructing more robust representation that supports detection with fewer FNs and FPs. Specifically, the Sequential Attention serves as one internal perception process that captures several distinct part areas step by step from each pedestrian proposal (full-body). Different from the previous either-or feature selection, the following Joint Learning attempts to seek a reasonable trade-off between part and full-body features, and combines both features for more accurate classification and regression. Evaluation on the widely used pedestrian datasets including Caltech and Citypersons shows that the proposed SA-DPM achieves promising performance for both non-occluded and occluded pedestrian detection tasks, especially on Caltech Heavy Occlusion set, which yields a new state-of-the-art miss rate by 30.18% and outperforms the second best detector by 6.32%. [ABSTRACT FROM AUTHOR]

As part of video transmission, video quality assessment plays an important role in improving the user’s visual quality and reducing transmission bitrate. However, the existing assessment schemes for the tile-based transmission mechanism cannot accurately predict the video quality. In this paper, we propose a tile-based panoramic video quality assessment method to optimize the transmission strategy for tile-based streaming. Considering the distortion of panoramic video projection, the inverse of gnomonic projection and bilinear interpolation are utilized to convert the planar tiles to sphere. Video Multimethod Assessment Fusion (VMAF) is applied to obtain the tile quality score. Besides, the user’s head movement and eye movement are combined to generate a tile weight map, and generate the final score of the panoramic video through weighted averaging. In addition, considering the impact of the quality fluctuation of the tiles within field of view, the tile quality loss model is established from multiple dimensions including the encoding parameter, viewport position and video content. Experimental results show that there is a high correlation between the predicted results and actual subjective scores, under two video quality assessment datasets. Furthermore, we integrate the method into the panoramic video transmission system. Compared with the tile-based adaptive rate algorithm, the proposed method degrades the video quality by only 3%, while reducing the bandwidth consumption by 16.99%. [ABSTRACT FROM AUTHOR]

Recently, many researches on convolution neural network (CNN) based in-loop filters have been proposed to improve coding efficiency. However, most existing CNN based filters tend to train and deploy multiple networks for various quantization parameters (QP) and frame types (FT), which drastically increases resources in training these models and the memory burdens for video codec. In this paper, we propose a novel variable CNN (VCNN) based in-loop filter for VVC, which can effectively handle the compressed videos with different QPs and FTs via a single model. Specifically, an efficient and flexible attention module is developed to recalibrate features according to QPs or FTs. Then we embed the module into the residual block so that these informative features can be continuously utilized in the residual learning process. To minimize the information loss in the learning process of the entire network, we utilize a residual feature aggregation module (RFA) for more efficient feature extraction. Based on it, an efficient network architecture VCNN is designed that can not only effectively reduce compression artifacts, but also can be adaptive to various QPs and FTs. To address training data imbalance on various QPs and FTs and improve the robustness of the model, a focal mean square error loss function is employed to train the proposed network. Then we integrate the VCNN into VVC as an additional tool of in-loop filters after the deblocking filter. Extensive experimental results show that our VCNN approach obtains on average 3.63%, 4.36%, 4.23%, 3.56% under all intra, low-delay P, low-delay, and random access configurations, respectively, which is even better than QP-Separate models. [ABSTRACT FROM AUTHOR]

Reflection caused by glass often degrades the quality of an image and further makes it difficult to estimate depth. In this article, we propose joint reflection removal and depth estimation from a single image. We perform reflection removal (transmission recovery) and depth estimation jointly using a collaborative neural network that consists of four blocks: 1) encoder for feature extraction; 2) reflection removal subnetwork (RRN); 3) depth estimation subnetwork (DEN); and 4) depth refinement guided by the transmission layer. We achieve collaboration between reflection removal and depth estimation by concatenating intermediate features of DEN with RRN. Since the recovered transmission layer contains accurate edges of objects behind glass, we refine the estimated depth with its guidance by guided image filtering. The experimental results demonstrate that the proposed method achieves both reflection removal and depth estimation even for images with dominant reflections. Besides, this article offers a new way of treating reflections in images to introduce depth estimation into reflection removal and achieve reflection removal and depth estimation simultaneously. [ABSTRACT FROM AUTHOR]

Glioblastoma (GBM) tumor is the most common primary brain malignant tumor. The precise identification of GBM tumors is very important for diagnosis and treatment. Hyperspectral imaging is a fast, noncontact, accurate, and safe modern medical detection technology, which is expected to be a new tool of intraoperative diagnosis. In order to make full use of the spectral and spatial information of hyperspectral images (HSIs) to achieve accurate GBM tumor identification, a method based on the fusion of multiple deep models is proposed for in vivo human brain HSI classification. The proposed method includes the following major steps: 1) spectral phasor analysis and data oversampling; 2) 1-D deep neural network (1D-DNN)-based spectral HSI feature extraction and classification; 3) 2-D convolution neural network (2D-CNN)-based spectral–spatial HSI feature extraction and classification; 4) edge-preserving filtering-based classification result fusion and optimization; and 5) fully convolutional network (FCN)-based background segmentation. To verify the capabilities of the proposed method, experiments are performed on two real human brain hyperspectral datasets, including 36 in vivo HSIs captured from 16 different patients. The proposed method can achieve an overall accuracy of 96.69% for four-class classification and overall accuracy of 96.34% for GBM tumor identification. Experimental results demonstrate that the proposed method exhibits competitive classification performance and can generate satisfactory thematic maps of the location of the GBM tumor, which can provide the surgeon with guidance on successful and precise tumor resection. [ABSTRACT FROM AUTHOR]

Developmental dysplasia of the hip (DDH) is one of the most common orthopedic disorders in infants and young children. Accurately detecting and identifying the misshapen anatomical landmarks plays a crucial role in the diagnosis of DDH. However, the diversity during the calcification and the deformity due to the dislocation lead it a difficult task to detect the misshapen pelvis landmarks for both human expert and computer. Generally, the anatomical landmarks exhibit stable morphological features in part regions and rigid structural features in long ranges, which can be strong identification for the landmarks. In this paper, we investigate the local morphological features and global structural features for the misshapen landmark detection with a novel Pyramid Non-local UNet (PN-UNet). Firstly, we mine the local morphological features with a series of convolutional neural network (CNN) stacks, and convert the detection of a landmark to the segmentation of the landmark’s local neighborhood by UNet. Secondly, a non-local module is employed to capture the global structural features with high-level structural knowledge. With the end-to-end and accurate detection of pelvis landmarks, we realize a fully automatic and highly reliable diagnosis of DDH. In addition, a dataset with 10,000 pelvis X-ray images is constructed in our work. It is the first public dataset for diagnosing DDH and has been already released for open research. To the best of our knowledge, this is the first attempt to apply deep learning method in the diagnosis of DDH. Experimental results show that our approach achieves an excellent precision in landmark detection (average point to point error of 0.9286mm) and illness diagnosis over human experts. Project is available at http://imcc.ustc.edu.cn/project/ddh/. [ABSTRACT FROM AUTHOR]

Accurate prediction of the host phenotype from a metagenomic sample and identification of the associated microbial markers are important in understanding potential host-microbiome interactions related to disease initiation and progression. We introduce PopPhy-CNN, a novel convolutional neural network (CNN) learning framework that effectively exploits phylogenetic structure in microbial taxa for host phenotype prediction. Our approach takes an input format of a 2D matrix representing the phylogenetic tree populated with the relative abundance of microbial taxa in a metagenomic sample. This conversion empowers CNNs to explore the spatial relationship of the taxonomic annotations on the tree and their quantitative characteristics in metagenomic data. We show the competitiveness of our model compared to other available methods using nine metagenomic datasets of moderate size for binary classification. With synthetic and biological datasets, we show the superior and robust performance of our model for multi-class classification. Furthermore, we design a novel scheme for feature extraction from the learned CNN models and demonstrate improved performance when the extracted features. PopPhy-CNN is a practical deep learning framework for the prediction of host phenotype with the ability of facilitating the retrieval of predictive microbial taxa. [ABSTRACT FROM AUTHOR]

In order to effectively identify and classify weld defects of thin-walled metal canisters, a weld defect detection and classification algorithm based on machine vision is proposed in this paper. With the weld defects categorized, a modified background subtraction method based on Gaussian mixture models, is proposed to extract the feature areas of the weld defects. Then, we design an algorithm for weld detection and classification according to the extracted features. Next, by using the weld images sampled by the constructed weld defect detection system on a real-world production line, the parameters of the weld defect classifiers are determined empirically. Experimental results show that the proposed methods can identify and classify the weld defects with more than 95% accuracy rate. Moreover, the weld detection results obtained in the actual production line show that the detection and classification accuracy can reach more than 99%, which means that the system enhanced with the proposed method can meet the requirements for the best real-time and continuous weld defect detection systems available nowadays. [ABSTRACT FROM AUTHOR]

Cloud and cloud shadow detection in remote sensing imagery is important for its wide range of applications. Traditionally, the detection is usually based on the manually designed thresholds from multiband, which is complicated and of multistage. To simplify the process of cloud and cloud shadow detection and improve the performance, we propose a multilevel feature fused segmentation network (MFFSNet), which can be trained end-to-end without any hand-tuned parameters. Specifically, a fully convolutional network is proposed for cloud and cloud shadow features learning. Then, we utilize a novel pyramid pooling module to extract contextual relation between cloud and shadow. Furthermore, a special multilevel feature fused structure is designed to combine semantic information with spatial information from different levels, so that we can better handle the multiscale objects and produce detailed segmentation boundaries. Experiments show that the MFFSNet outperforms the state-of-the-art methods and achieves high accuracies of 98.69% and 98.92% for cloud and cloud shadow detection. [ABSTRACT FROM AUTHOR]

Akin to human transfer of experiences, transfer learning as a subfield of machine learning adapts knowledge acquired in one domain to a new domain. The authors systematically investigate how this concept might be applied to the study of users on emerging Web platforms, proposing a transfer learning–based approach, TraNet. [ABSTRACT FROM AUTHOR]

Steganography based on bitrate modulation is a new type of information hiding approach for Voice over IP (VoIP), which can naturally resist state-of-the-art steganalysis schemes, due to inducing no modifications on the involved protocols and payloads of VoIP packets. To fill this gap, this letter presents a new corresponding detection scheme. Specifically, for the existing three methods based on bitrate modulation, we first reveal obvious regularities for the change intervals of bitrates in steganographic frame streams, which are totally different from the normal streams. Furthermore, exploiting the statistical properties of change intervals, we design a support-vector-machine-based steganalysis scheme. We evaluate the detection performance of the proposed scheme, respectively, with three state-of-the-art rate adaptation algorithms as the generators of normal VoIP streams. The results demonstrate that the proposed scheme can effectively detect the existing three bitrate modulation-based steganographic methods, even at relatively low covert bandwidths. [ABSTRACT FROM AUTHOR]

Text detection in a natural environment plays an important role in many computer vision applications. While existing text detection methods are focused on English characters, there are strong application demands on text detection in other languages, such as Chinese. In this paper, we present a novel text detection algorithm for Chinese characters based on a specific designed convolutional neural network (CNN). The CNN contains a text structure component detector layer, a spatial pyramid layer, and a multi-input-layer deep belief network (DBN). The CNN is pre-trained via a convolutional sparse auto-encoder, specifically designed for extracting complex features from Chinese characters. In particular, the text structure component detectors enhance the accuracy and uniqueness of feature descriptors by extracting multiple text structure components in various ways. The spatial pyramid layer enhances the scale invariability of the CNN for detecting texts in multiple scales. Finally, the multi-input-layer DBN replaces the fully connected layers in the CNN to ensure features from multiple scales are comparable. A multilingual text detection dataset, in which texts in Chinese, English, and digits are labeled separately, is set up to evaluate the proposed text detection algorithm. The proposed algorithm shows a significant performance improvement over the baseline CNN algorithms. In addition the proposed algorithm is evaluated over a public multilingual benchmark and achieves state-of-the-art result under multiple languages. Furthermore, a simplified version of the proposed algorithm with only general components is evaluated on the ICDAR 2011 and 2013 datasets, showing comparable detection performance to the existing general text detection algorithms. [ABSTRACT FROM PUBLISHER]