Search

In this paper, a new full-reference image quality assessment (IQA) method for performing perceptual quality evaluation on the light field (LF) image is proposed, called the contourlet transform-based model (CTM). The LF image consists of a set of sub-aperture images (SAIs) that have similar image content but with small angular deviations due to different viewing angles. Hence, the abundant image details can be extracted from the SAIs. To fulfil this goal, the contourlet transform is used to extract the multi-scale spatial features of the reference and distorted SAIs, respectively. Based on our proposed IQA metric, the degree of similarity can be computed based on the above-measured quantities for arriving at the final IQA score of the distorted LF image under evaluation. Experimental simulation results obtained from the dense light fields datasets clearly show that the proposed CTM algorithm is more in line with the quality assessment of the LF images perceived by the human visual system (HVS) when compared with that of using other state-of-the-art IQA algorithms.

Vehicle re-identification matching vehicles captured by different cameras has great potential in the field of public security. However, recent vehicle re-identification approaches exploit complex networks, causing large computations in their testing phases. In this paper, we propose a matching behavior difference learning (MBDL) method to compress vehicle re-identification models for saving testing computations. In order to represent the matching behavior evolution across two different layers of a deep network, a matching behavior difference (MBD) matrix is designed. Then, our MBDL method minimizes the L1 loss function among MBD matrixes from a small student network and a complex teacher network, ensuring the student network use less computations to simulate the teacher network’s matching behaviors. During the testing phase, only the small student network is utilized so that testing computations can be significantly reduced. Experiments on VeRi776 and VehicleID datasets show that MBDL outperforms many state-of-the-art approaches in terms of accuracy and testing time performance.

The light field (LF) images have received significant attentions from both academia and industry, which allow a wide range of applications in the areas of computer vision and computer graphics, such as three-dimensional reconstruction, virtual reality, and so on. Considering that the human eyes are the final receivers of the LF images and the LF images are inevitably suffered from various distortions, image quality assessment (IQA) becomes an important issue for the LF images with the goal of evaluating the image quality of the LF images in accordance with the human visual system. This paper presents an up-to-date overview of the IQA specifically for the LF images, focusing on the formation of the LF images, the LF IQA databases and the existing LF IQA models.

The homework tidiness assessment aims to auto evaluate the writing tidiness of homework, playing an important role in daily teaching. However, there is still no comprehensive basis for homework tidiness assessment. For this, a benchmark for homework tidiness assessment (HTA) is proposed. Firstly, a database named HTA 1.0 containing 1000 homework images is collected. Each image is manually annotated by multiple volunteers. Secondly, a comprehensive evaluation protocol is designed, using mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE) and accuracy (Acc) as performance indicators. Finally, three deep learning models (i.e., LeNet, AlexNet and VGGNet) are applied as baseline methods and the results are reported and analyzed.

The speech recognition technology makes it possible for people to communicate with intelligent electronic devices. However, existing speech recognition algorithms are overly complex for small intelligent electronic devices (e.g., mini speakers, intelligent toys, intelligent remote controls, etc.). For this, an efficient speech recognition algorithm is proposed. Firstly, the Mel-scale Frequency Cepstral Coefficients (MFCC) is applied to extract features of voices. Secondly, the Support Vector Machines (SVM) is used to train speech classification models. Finally, a speech database is collected to validate the proposed algorithm. The speech database contains 500 audio files of 10 speech commands for an electric motor car driving assistant and 550 audio files of 11 speech commands for a intelligent remote control. The proposed method is evaluated via a 5-fold cross-validation, and experiments show that the propose method acquires 94.20% and 88.73% average accuracy rates for the electric motor car driving assistant and the intelligent remote control, respectively.

In this paper, an image quality assessment (IQA) model for conducting objective evaluations of screen content images (S-CIs) is proposed, called the log-Gabor feature-based model (LGFM). From the standpoint of signal representation, the log-Gabor filters outperform the classical Gabor filters since the outputs of the log-Gabor filters are more consistent with the perception of visual cortex in human visual system (HVS). Furthermore, the following two remarkable characteristics of the log-Gabor filters are highly beneficial to develop a more accurate IQA model; i.e., (i) zero response at the DC, and (ii) stronger response at high frequencies. In our proposed L-GFM, the log-Gabor filters are used to extract features from the luminance of the reference SCIs and that of the distorted SCIs for measuring their degree of similarity. Together with the measurements from the other two chrominance components, the final LGFM score will be arrived at the output of the pooling stage. Extensive simulation results have shown that our proposed LGFM is highly consistent with the human perception, compared to other state-of-the-art IQA models.

The Vehicle Logo Recognition (VLR) is becoming more and more important in the Automated Vehicle Classification (AVC) research area since it can effectively assist vehicle classification. However, existing open-access vehicle logo datasets have many issues, which are hard to represent real-world conditions such as occlusions, low image resolutions and varying lighting conditions, thus heavily hindering the VLR application in reality. For this, a large-scale vehicle logo benchmark is proposed in this paper. Firstly, a large-scale vehicle logo dataset namely VLD 1.0 is collected, which contains 25,189 images of 66 classes. Each type of cars is collected with multiple images from various conditions to comprehensively represent real-world conditions. Secondly, a vehicle logo recognition evolution protocol is designed to fully evaluate performance, i.e., Mean Average Precision (MAP) under different Intersection Over Union (IOU) values and Speed. Thirdly, the YOLOv3 deep learning model is applied as a baseline method and its performance on the VLD 1.0 is reported.

Fabric image retrieval aims to match a probe fabric image from a gallery fabric image set, playing a great role in the textile industry. However, there is a lack of one comprehensive benchmark for fabric image retrieval. For this, a large benchmark for fabric image retrieval is proposed in this paper. Firstly, a dataset namely Fabric 1.0 is proposed, which contains 46,656 fabric images of 972 subjects. Each subject is collected with multiple images from various angles and both front and back sides. Secondly, a evaluation protocol is designed to evaluate the fabric image retrieval accuracy, which uses the cumulative match characteristic (CMC) curve and the mean average precision (MAP) as performance indicators. Finally, the baseline performance resulted from a non deep learning method (i.e., bag of feature) and three deep learning methods (i.e., LeNet, AlexNet and VGGNet) are reported and analyzed.

Just noticeable difference (JND), which reflects the minimum visibility threshold of human visual system, is usually utilized to remove the perceptual redundancies for image/video coding. However, most of the existing DCT-based JND models failed to effectively estimate contrast masking (CM) of image blocks with different complexity. In this paper, we propose an improved CM model via frequency energy of DCT block, which is calculated based on DCT coefficient distribution. Then, by integrating luminance adaptation and contrast sensitivity function, an effective JND model in the DCT domain is realized. Experimental results have confirmed that the proposed method can produce more accurate JND threshold and remove more visual redundancies than the state-of-the-art JND models.

In this paper, we propose a shortly and densely connected convolutional neural network (SDC-CNN) for vehicle re-identification. The proposed SDC-CNN mainly consists of short and dense units (SDUs), necessary pooling and normalization layers. The main contribution lies at the design of short and dense connection mechanism, which would effectively improve the feature learning ability. Specifically, in the proposed short and dense connection mechanism, each SDU contains a short list of densely connected convolutional layers and each convolutional layer is of the same appropriate channels. Consequently, the number of connections and the input channel of each convolutional layer are limited in each SDU, and the architecture of SDC-CNN is simple. Extensive experiments on both VeRi and VehicleID datasets show that the proposed SDC-CNN is obviously superior to multiple state-of-the-art vehicle re-identification methods.

In this paper, we propose an intelligent network surveillance system to protect the urban underground pipelines from external damages caused by excavation devices. At each monitoring site, a microphone array is implemented for real-time acoustic collection and an intelligent excavation device recognition algorithm is embedded. A surveillance platform built on the fusion of multi monitoring sites is designed for a whole city. A novel statistical feature extraction method is first developed to mining the useful and representative information for the collected acoustic signals. Then, an artificial neural network trained by the popular extreme learning machine (ELM) and the regularized ELM (RELM) is used to perform the recognition of excavation devices in each monitoring site. To show the efficiency of the proposed system, experiments are conducted in this paper. Recognition performance on four most destructive devices is studied.

In this paper, a novel image quality assessment (IQA) metric for the multi-exposure image fusion (MEF) is proposed by using contrast information. Specifically, the proposed approach firstly performs the measurements of contrast structure similarity and contrast saturation similarity based on the observation that human perception is sensitive to contrast information inherited in the MEF and reference images. Then, considering that different reference images contribute differently to the MEF image, the weights are adaptively assigned to each reference image according to its relevance to the MEF image. A standard deviation based pooling strategy and multi-scale scheme are subsequently used to generate the final MEF image quality score. Experimental results have shown that the proposed metric produces high consistency with human perception of the MEF image quality and outperforms the state-of-the-art quality metric.

Perceptual quality assessment of screen content images (S-CIs) has become a new challenging topic in the recent research of image quality assessment (IQA). In this work, we construct a new SCI database (called as SCID) for subjective quality evaluate of SCIs and investigate whether existing IQA models can effectively assess the perceptual quality of distorted SCIs. The proposed SCID, which is currently the largest one, containing 1,800 distorted SCIs generated from 40 reference SCIs with 9 types of distortions and 5 degradation levels for each distortion type. The double-stimulus impairment scale (DSIS) method is then employed to rate the perceptual quality, in which each image is evaluated by at least 40 assessors. After processing, each distorted SCI is accompanied with one mean opinion score (MOS) value to indicate its perceptual quality as ground truth. Based on the constructed S-CID, we evaluate the performances of 14 state-of-the-art IQA metrics. Experimental results show that the existing IQA metrics do not be able to evaluate the perceptual quality of SCIs well and an IQA metric specifically for SCIs is thus desirable. The proposed SCID will be made publicly available to the research community for further investigation on the perceptual processing of SCIs.

In many cities in the world, underground pipe network always suffers from serious external damage. The detection method of excavation devices based on acoustic signal has been extensively studied in the past research. It is important for preventing the destruction of urban underground pipeline network during disorderly underground excavation. However, the existing excavation devices detection methods have little attention to distance estimation, but only focus on the recognition results or direction-of-arrival (DOA) estimation. In the actual monitoring system, there may be frequent misinformation alarm, because of the lack of convincing distance-of-arrival (DisOA) estimation to achieve accurate source localization. In this paper, a new intelligent distance estimation algorithm, based on acoustic attenuation property for excavation devices localization, is brought forward. Specially, the extreme learning machine-based auto-encoder is used to obtain more robust feature representation from the acoustic signal frequency domain amplitude spectrum, and the regularized extreme learning machine (RELM) is utilized to train the regression model. Experimental results show that the proposed algorithm is effective.

Discrete cosine transform is an ideal method to decorrelate the signal. In this paper, we proposed a novel entropy coding based on 1D-DCT to compress the 3D point clouds. Different from the former method which is based on Laplacian distribution, we encoding the coefficients by count the nonzero coefficients. We count the number, index, value of the nonzero coefficients of each block and then use arithmetic encoder to compress the coefficient. Experimental results on a number of point clouds show our method is more efficient than the former method based on DCT.

As the coding standard for multi-view video plus depth format, 3D-High Efficiency Video Coding (3D-HEVC) can achieve superior coding efficiency while with high complexity. Because of the large areas with nearly constant values and sharp edges, the coding process for depth map can be optimized. A fast intra depth map algorithm based on sum-of-gradient and segment-wise direct component coding (FID-SOG-SDC) for 3D-HEVC is proposed in this paper. The sum of gradient information of each coding unit (CU) can be utilized to predict the intra prediction mode, decide CTU partition size and skip the SDC decision when CU belongs to a smooth region. Experimental results show that the proposed FID-SOG-SDC algorithm can achieve encoding time reduction by 23.07% with an impact of 0.06% in the rate-distortion(RD) performance compared to the original 3D-HEVC.

Since the object occlusion, view variation, arbitrary shape of pedestrian, etc., are inherited in the pedestrian images, the pedestrian gender classification is an extremely challenging task in the object recognition field. To address this problem, an effective multi-view joint learning network (MJLN) is proposed for pedestrian gender classification. Based on the observation that the view variation of pedestrian will affect the judgment of pedestrian gender, the proposed MJLN simultaneously performs pedestrian gender learning and pedestrian view learning. Consequently, the determination of pedestrian view (i.e., frontal, rear, and left/right profile) obtained by the pedestrian view learning module will effectively assist the pedestrian gender learning module to yield a more robust and distinguishable feature so as to improve the gender classification performance. Extensive experiments on multiple challenging pedestrian datasets, which includes CUHK, VIPeR, GRID, PRID, and MIT, have demonstrated that the proposed MJLN can effectively promote the gender classification performance, and outperforms multiple state-of-the-art methods.

Handwritten numeral recognition is a challenging problem due to large variation in the writing styles of different persons and high similarity in the contour of different digits. Based on the observation that the decision of scratchy/non-scratchy in the writing style could play a complementary role on the classification of handwritten numeral. In this paper, an effective multi-task learning network for handwritten numeral recognition is proposed to enhance the recognition performance. The proposed multi-task learning network consists of two tasks, which can simultaneously learn handwritten numeral recognition and the scratchy/non-scratchy decision. Furthermore, the two tasks can promote each other during training and achieve a better recognition performance. Extensive experiments on the MNIST database demonstrate that the proposed multi-task network can effectively improve the recognition accuracy and achieve a superior performance of 0.40% error rate, which outperforms most methods that take experiments on the M-NIST database.

With the development of urbanization, the underground pipelines play an important role in ensuring the basic conditions of citizen life. However, construction activities are continuously increasing in the mainland of China nowadays, which lead to serious damages and threatens to the underground pipelines by excavation equipment. To this end, we focus on developing an automatically monitoring system for underground pipeline network protection in this paper. The acoustic characteristics of excavation equipment are first analyzed. Then, a recognition framework combining with the feature extraction algorithm of Mel-Frequency Linear Prediction Cepstral Coefficients (MF-PLPCC) and the classification algorithm of Regularized Extreme Learning Machine (RELM) is developed for excavation equipment. The MF-PLPCC processing includes the use of Mel filter bank that simulates the acoustic perception of human ear to different frequency, and the use of a power-law nonlinearity that replaces the traditional log nonlinearity used in MFCC. Experimental results demonstrate that the method of this paper provides substantial improvements in recognition accuracy comparing to other state-of-the-art methods.

In this paper, a perceptual-based rate control algorithm for the high efficiency video coding (HEVC) is proposed. Since the human visual system (HVS) is highly sensitive to edge information inherited in image and video contents, gradient magnitude is considered a very effective attribute on the computation of the spatial perceptual characteristics (SPC) and the temporal perceptual characteristic (TPC). The latter is further computed based on the gradient magnitude similarity deviation (GMSD). The combined perceptual characteristic of SPC and TPC is then used to guide the bit allocation process for each CTU. Experimental results have clearly shown that the proposed perceptual rate control scheme can achieve superior perceptual coding performance, compared with the original R-A rate control adopted in the HEVC.

This paper proposes a multiple description image coding scheme based on 2D dual-tree transform and the enhanced x-tree encoding method. The input image is firstly mapped into 2D dual-tree discrete wavelet domain to form 2 wavelet coefficient trees. A sparse algorithm is then used to remove the most redundant wavelet coefficients resulted from dual-tree discrete wavelet transform (DDWT), forming the basic component of both descriptions, respectively. In order to improve the quality of the side reconstruction, a side sparse algorithm is then imposed on two sparse coefficient trees to produce the additional for both side decoding. The basic information from one tree and additional information from the other are sent to an enhanced x-tree encoder, which is proposed to exploit the strong correlation between two wavelet trees resulted from DDWT, forming the bitstream of a description. Since each description includes the basic information and part of details of the input image, even one of the descriptions gets lost, the reconstructed image can still keep acceptable quality. Simulation results have verified that the proposed algorithm has good coding performance and error resilient ability.

The latest high efficiency video coding-based three dimensional video coding (3D-HEVC) exploits sophisticated intra prediction scheme to improve the coding performance of the depth video, but incurring heavy computational complexity. To address this problem, a low complexity depth intra coding method is presented for 3D-HEVC based on depth classification. Firstly, a database of depth prediction units (PUs) with three kinds of complexities is collected based on their optimal intra prediction mode. Then, the histogram of oriented gradient (HOG) features are extracted on these established database to train the classifier using support vector machine (SVM). For the current depth PU, the trained classifier is applied to determine its most possible complexity class so as to select the corresponding modes for involving the mode decision process. Experimental results show that the proposed method is able to significantly reduce the computational complexity while keeping almost the same coding performance of depth video and video quality of the synthesized view, compared with the exhaustive mode decision in 3D-HEVC.

We propose a new method for robust learning Laplacian matrices from observed smooth graph signals in the presence of both Gaussian noise and random-valued impulse noise (i.e., outliers). Using the recently developed factor analysis model for representing smooth graph signals in [1], we formulate our learning process as a constrained optimization problem, and adopt the £i-norm for measuring the data fidelity in order to improve robustness. Computational results on three types of synthetic graphs demonstrate that the proposed method outperforms the state-of-the-art methods in terms of commonly used information retrieval metrics, such as F-measure, precision, recall and normalized mutual information. In particular, we observed that F-measure is improved by up to 16%.

As the concept of block level adaptivity becomes an important feature in recent video CODECs, block level adaptive quantization (BLAQ) is being considered in the High Efficiency Video Coding (HEVC) standard. The BLAQ is based on the assumption that each block should have its own quantization parameter (QP), which can adapt to the local content of video sequences much better, and hence the video encoder with adaptive QP can perform a better perceptual quality. However, in the HEVC reference software, the BLAQ is required to obtain a proper QP for each block by the rate distortion optimization (RDO) scheme and so the computational complexity of the encoder increases significantly. In this paper, an improved BLAQ algorithm is proposed to obtain the adaptive QP for each block. The simulation results show that the proposed method can save more bits as well as require lower computational complexity, compared to the traditional method.

The intra prediction in HEVC exploits the new techniques, such as the tree coding structure and up to 35 intra prediction modes, etc., to achieve high coding efficiency but incur heavy computational complexity as well. To reduce the computational complexity, a novel fast mode decision algorithm for intra prediction in HEVC is proposed in this paper. In the proposed algorithm, by using the texture characteristic of the current prediction unit (PU) and the similarity between neighboring PUs, the proposed rough and refine decision schemes are sequentially utilized to reduce the number of candidate prediction modes required for checking so that the computational complexity can be greatly saved. Experimental results have demonstrated that the proposed algorithm can reduce, on average, 25% computational complexity with negligible loss of coding efficiency, compared with the fast intra mode decision method implemented in HEVC reference software.

Perceptual quality evaluation of the retargeting image plays an important role in benchmarking different retargeting methods, as well as guiding or optimizing the retargeting process. The distortions introduced during the retargeting process are mainly categorized into shape distortion and content information loss [1]. The shape distortion measurement is critical to the evaluation of retargeting image perceptual quality. In this paper, the performances of different shape descriptors, such as PHOW [2], GIST [3], MPEG-7 descriptors [4], EMD [5], for evaluating the perceptual quality of the retargeting image are examined based on the public image retargeting subjective quality database [6]. Experimental results demonstrated that most of the shape descriptors can hardly capture the characteristics representing the quality of the retargeting image, but the global shape descriptor GIST [3] presents significant performance gains. Moreover, by incorporating with the measurements from the perspective of content information loss, a better performance is further obtained.

In high efficiency video coding (HEVC), Lagrangian rate distortion optimization (RDO) technique is used to optimize the rate distortion (RD) performance. However, the corresponding Lagrangian multiplier does not consider the perceptual characteristic of the input video and thus is not effective for perceptual video coding. To address this problem, an efficient perceptual adaptive Lagrangian multiplier for HEVC is proposed. Based on the human visual system (HVS) observation that the region with less perceptual sensitivity can tolerate more distortion, the Lagrangian multiplier is adaptively adjusted for each coding tree unit (CTU) based on its perceptual sensitivity so that the perceptual quality of the reconstructed video can be improved. The above-mentioned perceptual sensitivity for each CTU is measured according to two perceptual features-spatial energy ratio and temporal motion activity. Experimental results have shown that the proposed method is able to significantly improve the perceptual RD performance, compared with the original HEVC.

In this paper, we propose a rate distortion optimization based content adaptive transform method for motion compensation residuals. The proposed method utilizes pixel rearrangement to dynamically adjust the transform kernels to adapt to the residual content. Comparing with the traditional adaptive transforms, the highlight of this work is that it obtains the transform kernels from the decoded block, and hence it consumes only one overhead bit for each transform unit. Moreover, rate distortion optimization scheme is used to choose the best candidate kernels. Experimental results show that the proposed method achieves an average 0.35 dB gain of PSNR in comparison with the key technical areas (KTA) encoder.

Motion estimation and disparity estimation using variable block sizes have been exploited in multi-view video coding to effectively improve the coding efficiency, but at the expense of yielding higher computational complexity. In this paper, a fast block size decision algorithm, called motion activity-based block size decision (MABSD), is proposed. In our approach, the various motion estimation and disparity estimation block sizes are classified into four classes, and only one of them will be chosen to further identify the optimal block size within that class according to the measured motion activity of the current macroblock. The above-mentioned motion activity can be measured by the maximum city-block distance of a set of motion vectors taken from the adjacent macroblocks in the current view and its neighboring view. Experimental results have shown that compared with exhaustive block size decision, which is a default approach set in the JMVM reference software, the proposed MABSD algorithm achieves a reduction of computational complexity by 42% on average, while incurring only 0.01 dB loss in peak signal-to-noise ratio (PSNR) and 1% increment on the total bit rate.

The intra- and inter-prediction techniques have been effectively exploited in the H.264/AVC for significantly improving the coding efficiency. However, exhaustively searching for the optimal prediction mode vastly increases its computational complexity. To decrease the computational complexity of the mode decision process, a novel fast mode decision algorithm is proposed in this paper to speed up the encoding process by predicting the mode of the current macroblock based on the motion activity of the neighboring macroblocks. The proposed algorithm has two main stages. First, the SKIP mode is utilized as an early termination mechanism to terminate any further checking of other modes. This is quite effective for those regions with low-motion activity. If the SKIP mode is not met, the local macroblocks' motion activities will be identified in terms of motion-vector city-block length. Experimental results have shown that the proposed algorithm achieves a reduction of 56% computational complexity on average, while introducing only 0.059 dB loss in PSNR and 0.2% increment on the total bit rate compared with that of exhaustive trying out all possible modes, being set as the default in the original JM reference software.

Motion estimation (ME) algorithm is used in H.264 to improve the coding efficiency. However, it results in very high computational load. In this paper, a novel fast ME algorithm with reduced computational complexity is proposed. First, a predictive mode reduction method is used to free the encoder from checking the small block types in the homogeneous region. Second, a selective pixel matching scheme and a simplified optimal reference frame discrimination strategy are employed to find the best full-pixel motion vector. Finally, direction based selection rule is utilized to diminish the searching points in sub-pixel ME process. Experimental results have shown that the proposed algorithm achieves a reduction of 87% ME process time on average, while incurring only 0.033 dB loss in PSNR and 0.56% increment on the total bit rate compared with that of exhaustive ME process, which is a default approach set in the JM reference software.