Search

Dynamic scene classification has been extensively studied in computer vision due to its widespread applications. The key to dynamic scene classification lies in jointly characterizing spatial appearance and temporal dynamics to achieve informative representation, which remains an outstanding task in the literature. In this paper, we propose a unified framework to extract spatial and temporal features for dynamic scene representation. More specifically, we deploy two variants of deep convolutional neural networks to encode spatial appearance and short-term dynamics into short-term deep features (STDF). Based on STDF, we propose using the autoregressive moving average model to extract long-term frequency features (LTFF). By combining STDF and LTFF, we establish the long–short-term feature (LSTF) representations of dynamic scenes. The LSTF characterizes both spatial and temporal patterns of dynamic scenes for comprehensive and information representation that enables more accurate classification. Extensive experiments on three-dynamic scene classification benchmarks have shown that the proposed LSTF achieves high performance and substantially surpasses the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Action recognition has been extensively researched in computer vision due to its potential applications in a broad range of areas. The key to action recognition lies in modeling actions and measuring their similarity, which however poses great challenges. In this paper, we propose learning match kernels between actions on Grassmann manifold for action recognition. Specifically, we propose modeling actions as a linear subspace on the Grassmann manifold; the subspace is a set of convolutional neural network (CNN) feature vectors pooled temporally over frames in semantic video clips, which simultaneously captures local discriminant patterns and temporal dynamics of motion. To measure the similarity between actions, we propose Grassmann match kernels (GMK) based on canonical correlations of linear subspaces to directly match videos for action recognition; GMK is learned in a supervised way via kernel target alignment, which is endowed with a great discriminative ability to distinguish actions from different classes. The proposed approach leverages the strengths of CNNs for feature extraction and kernels for measuring similarity, which accomplishes a general learning framework of match kernels for action recognition. We have conducted extensive experiments on five challenging realistic data sets including Youtube, UCF50, UCF101, Penn action, and HMDB51. The proposed approach achieves high performance and substantially surpasses the state-of-the-art algorithms by large margins, which demonstrates the great effectiveness of proposed approach for action recognition. [ABSTRACT FROM AUTHOR]

Multi-target regression has recently regained great popularity due to its capability of simultaneously learning multiple relevant regression tasks and its wide applications in data mining, computer vision and medical image analysis, while great challenges arise from jointly handling inter-target correlations and input-output relationships. In this paper, we propose Multi-layer Multi-target Regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general framework via robust low-rank learning. Specifically, the MMR can explicitly encode inter-target correlations in a structure matrix by matrix elastic nets (MEN); the MMR can work in conjunction with the kernel trick to effectively disentangle highly complex nonlinear input-output relationships; the MMR can be efficiently solved by a new alternating optimization algorithm with guaranteed convergence. The MMR leverages the strength of kernel methods for nonlinear feature learning and the structural advantage of multi-layer learning architectures for inter-target correlation modeling. More importantly, it offers a new multi-layer learning paradigm for multi-target regression which is endowed with high generality, flexibility and expressive ability. Extensive experimental evaluation on 18 diverse real-world datasets demonstrates that our MMR can achieve consistently high performance and outperforms representative state-of-the-art algorithms, which shows its great effectiveness and generality for multivariate prediction. [ABSTRACT FROM PUBLISHER]

Face alignment has gained great popularity in computer vision due to its wide-spread applications. In this paper, we propose a novel learning architecture, i.e. , heterogenous output regression network (HORNet), for face alignment, which directly predicts facial landmarks from images. HORNet is based on kernel approximations and establishes a new compact multi-layer architecture. A nonlinear layer with cosine activations disentangles nonlinear relationships between representations of images and shapes of facial landmarks. A linear layer with identity activations explicitly encodes landmark correlations by low-rank learning via matrix elastic nets. HORNet is highly flexible and can work either with pre-built feature representations or with convolutional architectures for end-to-end learning. HORNet leverages the strengths of both kernel methods in modeling nonlinearities and of neural networks in structural prediction. This combination renders it effective and efficient for direct face alignment. Extensive experiments on five in-the-wild datasets show that HORNet delivers high performance and consistently exceeds state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Egocentric activity recognition has recently generated great popularity in computer vision due to its widespread applications in egocentric video analysis. However, it poses new challenges comparing to the conventional third-person activity recognition tasks, which are caused by significant body shaking, varied lengths, and poor recoding quality, etc. To handle these challenges, in this paper, we propose deep appearance and motion learning (DAML) for egocentric activity recognition, which leverages the great strength of deep learning networks in feature learning. In contrast to hand-crafted visual features or pre-trained convolutional neural network (CNN) features with limited generality to new egocentric videos, the proposed DAML is built on the deep autoencoder (DAE), and directly extracts appearance and motion feature, the main cue of activities, from egocentric videos. The DAML takes advantages of the great effectiveness and efficiency of the DAE in unsupervised feature learning, which provides a new representation learning framework of egocentric videos. The learned appearance and motion features by the DAML are seamlessly fused to accomplish a rich informative egocentric activity representation which can be readily fed into any supervised learning models for activity recognition. Experimental results on two challenging benchmark datasets show that the DAML achieves high performance on both short- and long-term egocentric activity recognition tasks, which is comparable to or even better than the state-of-the-art counterparts. [ABSTRACT FROM AUTHOR]

• We introduce physics-informed frameworks that analyse DCE-MRI data and output robust and reliable Extended Tofts-Kety parameters. • All frameworks showed to be substantially faster than conventional approaches. • The GRU network with attention layers outperforms the conventional non-linear least squares method at typical in vivo noise. • Recurrent networks show to generalize well over varying acquisition lengths, making them widely applicable. • In vivo, the GRU had better test-retest repeatability for all Extended Tofts-Kety parameters. [Display omitted] Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is an MRI technique for quantifying perfusion that can be used in clinical applications for classification of tumours and other types of diseases. Conventionally, the non-linear least squares (NLLS) methods is used for tracer-kinetic modelling of DCE data. However, despite promising results, NLLS suffers from long processing times (minutes-hours) and noisy parameter maps due to the non-convexity of the cost function. In this work, we investigated physics-informed deep neural networks for estimating physiological parameters from DCE-MRI signal-curves. Three voxel-wise temporal frameworks (FCN, LSTM, GRU) and two spatio-temporal frameworks (CNN, U-Net) were investigated. The accuracy and precision of parameter estimation by the temporal frameworks were evaluated in simulations. All networks showed higher precision than the NLLS. Specifically, the GRU showed to decrease the random error on v e by a factor of 4.8 with respect to the NLLS for noise (SD) of 1/20. The accuracy was better for the prediction of the v e parameter in all networks compared to the NLLS. The GRU and LSTM worked with arbitrary acquisition lengths. The GRU was selected for in vivo evaluation and compared to the spatio-temporal frameworks in 28 patients with pancreatic cancer. All neural network approaches showed less noisy parameter maps than the NLLS. The GRU had better test-retest repeatability than the NLLS for all three parameters and was able to detect one additional patient with significant changes in DCE parameters post chemo-radiotherapy. Although the U-Net and CNN had even better test-retest characteristics than the GRU, and were able to detect even more responders, they also showed potential systematic errors in the parameter maps. Therefore, we advise using our GRU framework for analysing DCE data. [ABSTRACT FROM AUTHOR]