Search

Action recognition has been extensively researched in computer vision due to its potential applications in a broad range of areas. The key to action recognition lies in modeling actions and measuring their similarity, which however poses great challenges. In this paper, we propose learning match kernels between actions on Grassmann manifold for action recognition. Specifically, we propose modeling actions as a linear subspace on the Grassmann manifold; the subspace is a set of convolutional neural network (CNN) feature vectors pooled temporally over frames in semantic video clips, which simultaneously captures local discriminant patterns and temporal dynamics of motion. To measure the similarity between actions, we propose Grassmann match kernels (GMK) based on canonical correlations of linear subspaces to directly match videos for action recognition; GMK is learned in a supervised way via kernel target alignment, which is endowed with a great discriminative ability to distinguish actions from different classes. The proposed approach leverages the strengths of CNNs for feature extraction and kernels for measuring similarity, which accomplishes a general learning framework of match kernels for action recognition. We have conducted extensive experiments on five challenging realistic data sets including Youtube, UCF50, UCF101, Penn action, and HMDB51. The proposed approach achieves high performance and substantially surpasses the state-of-the-art algorithms by large margins, which demonstrates the great effectiveness of proposed approach for action recognition. [ABSTRACT FROM AUTHOR]

Multi-target regression has recently regained great popularity due to its capability of simultaneously learning multiple relevant regression tasks and its wide applications in data mining, computer vision and medical image analysis, while great challenges arise from jointly handling inter-target correlations and input-output relationships. In this paper, we propose Multi-layer Multi-target Regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general framework via robust low-rank learning. Specifically, the MMR can explicitly encode inter-target correlations in a structure matrix by matrix elastic nets (MEN); the MMR can work in conjunction with the kernel trick to effectively disentangle highly complex nonlinear input-output relationships; the MMR can be efficiently solved by a new alternating optimization algorithm with guaranteed convergence. The MMR leverages the strength of kernel methods for nonlinear feature learning and the structural advantage of multi-layer learning architectures for inter-target correlation modeling. More importantly, it offers a new multi-layer learning paradigm for multi-target regression which is endowed with high generality, flexibility and expressive ability. Extensive experimental evaluation on 18 diverse real-world datasets demonstrates that our MMR can achieve consistently high performance and outperforms representative state-of-the-art algorithms, which shows its great effectiveness and generality for multivariate prediction. [ABSTRACT FROM PUBLISHER]

Cardiac four-chamber volume estimation serves as a fundamental and crucial role in clinical quantitative analysis of whole heart functions. It is a challenging task due to the huge complexity of the four chambers including great appearance variations, huge shape deformation and interference between chambers. Direct estimation has recently emerged as an effective and convenient tool for cardiac ventricular volume estimation. However, existing direct estimation methods were specifically developed for one single ventricle, i.e., left ventricle (LV), or bi-ventricles; they can not be directly used for four chamber volume estimation due to the great combinatorial variability and highly complex anatomical interdependency of the four chambers. In this paper, we propose a new, general framework for direct and simultaneous four chamber volume estimation. We have addressed two key issues, i.e., cardiac image representation and simultaneous four chamber volume estimation, which enables accurate and efficient four-chamber volume estimation. We generate compact and discriminative image representations by supervised descriptor learning (SDL) which can remove irrelevant information and extract discriminative features. We propose direct and simultaneous four-chamber volume estimation by the multioutput sparse latent regression (MSLR), which enables jointly modeling nonlinear input-output relationships and capturing four-chamber interdependence. The proposed method is highly generalized, independent of imaging modalities, which provides a general regression framework that can be extensively used for clinical data prediction to achieve automated diagnosis. Experiments on both MR and CT images show that our method achieves high performance with a correlation coefficient of up to 0.921 with ground truth obtained manually by human experts, which is clinically significant and enables more accurate, convenient and comprehensive assessment of cardiac functions. [ABSTRACT FROM AUTHOR]

Automating report generation for medical imaging promises to minimize labor and aid diagnosis in clinical practice. Deep learning algorithms have recently been shown to be capable of captioning natural photos. However, doing a similar thing for medical data, is difficult due to the variety in reports written by different radiologists with fluctuating levels of knowledge and experience. Current methods for automatic report generation tend to merely copy one of the training samples in the created report. To tackle this issue, we propose variational topic inference , a probabilistic approach for automatic chest X-ray report generation. Specifically, we introduce a probabilistic latent variable model where a latent variable defines a single topic. The topics are inferred in a conditional variational inference framework by aligning vision and language modalities in a latent space, with each topic governing the generation of one sentence in the report. We further adopt a visual attention module that enables the model to attend to different locations in the image while generating the descriptions. We conduct extensive experiments on two benchmarks, namely Indiana U. Chest X-rays and MIMIC-CXR. The results demonstrate that our proposed variational topic inference method can generate reports with novel sentence structure, rather than mere copies of reports used in training, while still achieving comparable performance to state-of-the-art methods in terms of standard language generation criteria. • Definition of the sentence generator net as a Transformer decoder and a fully Transformer-based definition of the VTI model. • A thorough explanation of the base VTI method, with additional derivation steps and algorithms. • Redesigned figures of the architecture for a better illustration of the building blocks of the model. • Additional ablation study and experiments for the new formulation of the model. • Discussion about the probabilistic formulation of the task and trade-off between different decoders for generating medical text. [ABSTRACT FROM AUTHOR]

Local methods based on spatio-temporal interest points (STIPs) have shown their effectiveness for human action recognition. The bag-of-words (BoW) model has been widely used and dominated in this field. Recently, a large number of techniques based on local features including improved variants of the BoW model, sparse coding (SC), Fisher kernels (FK), vector of locally aggregated descriptors (VLAD) as well as the naive Bayes nearest neighbor (NBNN) classifier have been proposed and developed for visual recognition. However, some of them are proposed in the image domain and have not yet been applied to the video domain and it is still unclear how effectively these techniques would perform on action recognition. In this paper, we provide a comprehensive study on these local methods for human action recognition. We implement these techniques and conduct comparison under unified experimental settings on three widely used benchmarks, i.e. , the KTH, UCF-YouTube and HMDB51 datasets. We discuss insightfully the findings from the experimental results and draw useful conclusions, which are expected to guide practical applications and future work for the action recognition community. [ABSTRACT FROM AUTHOR]

Direct estimation of cardiac ventricular volumes has become increasingly popular and important in cardiac function analysis due to its effectiveness and efficiency by avoiding an intermediate segmentation step. However, existing methods rely on either intensive user inputs or problematic assumptions. To realize the full capacities of direct estimation, this paper presents a general, fully learning-based framework for direct bi-ventricular volume estimation, which removes user inputs and unreliable assumptions. We formulate bi-ventricular volume estimation as a general regression framework which consists of two main full learning stages: unsupervised cardiac image representation learning by multi-scale deep networks and direct bi-ventricular volume estimation by random forests. By leveraging strengths of generative and discriminant learning, the proposed method produces high correlations of around 0.92 with ground truth by human experts for both the left and right ventricles using a leave-one-subject-out cross validation, and largely outperforms existing direct methods on a larger dataset of 100 subjects including both healthy and diseased cases with twice the number of subjects used in previous methods. More importantly, the proposed method can not only be practically used in clinical cardiac function analysis but also be easily extended to other organ volume estimation tasks. [ABSTRACT FROM AUTHOR]

Clinical routine often requires to analyze spinal images of multiple anatomic structures in multiple anatomic planes from multiple imaging modalities (M^3). Unfortunately, existing methods for segmenting spinal images are still limited to one specific structure, in one specific plane or from one specific modality (S^3). In this paper, we propose a novel approach, Regression Segmentation, that is for the first time able to segment M^3 spinal images in one single unified framework. This approach formulates the segmentation task innovatively as a boundary regression problem: modeling a highly nonlinear mapping function from substantially diverse M^3 images directly to desired object boundaries. Leveraging the advancement of sparse kernel machines, regression segmentation is fulfilled by a multi-dimensional support vector regressor (MSVR) which operates in an implicit, high dimensional feature space where M^3 diversity and specificity can be systematically categorized, extracted, and handled. The proposed regression segmentation approach was thoroughly tested on images from 113 clinical subjects including both disc and vertebral structures, in both sagittal and axial planes, and from both MRI and CT modalities. The overall result reaches a high dice similarity index (DSI) 0.912 and a low boundary distance (BD) 0.928 mm. With our unified and expendable framework, an efficient clinical tool for M^3 spinal image segmentation can be easily achieved, and will substantially benefit the diagnosis and treatment of spinal diseases. [ABSTRACT FROM AUTHOR]

We propose a novel method to model human actions by explicitly coding motion and structure features that are separately extracted from video sequences. Firstly, the motion template (one feature map) is applied to encode the motion information and image planes (five feature maps) are extracted from the volume of differences of frames to capture the structure information. The Gaussian pyramid and center-surround operations are performed on each of the six obtained feature maps, decomposing each feature map into a set of subband maps. Biologically inspired features are then extracted by successively applying Gabor filtering and max pooling on each subband map. To make a compact representation, discriminative locality alignment is employed to embed the high-dimensional features into a low-dimensional manifold space. In contrast to sparse representations based on detected interest points, which suffer from the loss of structure information, the proposed model takes into account the motion and structure information simultaneously and integrates them in a unified framework; it therefore provides an informative and compact representation of human actions. The proposed method is evaluated on the KTH, the multiview IXMAS, and the challenging UCF sports datasets and outperforms state-of-the-art techniques on action recognition [ABSTRACT FROM AUTHOR]

The representation of human actions in video sequences is one of the key steps in action classification and recognition, performances of which are greatly dependent on the distinctiveness and robustness of the descriptors used for representation. In this paper, a novel descriptor, named pyramid correlogram of oriented gradients (PCOG), is presented for feature representation. PCOG, combined with the motion history images, captures both shape and spatial layout of the motion and therefore gives more effective and powerful representation for human actions and can be used for the detection and recognition of a variety of actions. Experiments on challenging action data sets show that PCOG performs significantly better than the histogram of oriented gradients both as a global descriptor and as a local descriptor. [ABSTRACT FROM AUTHOR]

Face alignment has gained great popularity in computer vision due to its wide-spread applications. In this paper, we propose a novel learning architecture, i.e. , heterogenous output regression network (HORNet), for face alignment, which directly predicts facial landmarks from images. HORNet is based on kernel approximations and establishes a new compact multi-layer architecture. A nonlinear layer with cosine activations disentangles nonlinear relationships between representations of images and shapes of facial landmarks. A linear layer with identity activations explicitly encodes landmark correlations by low-rank learning via matrix elastic nets. HORNet is highly flexible and can work either with pre-built feature representations or with convolutional architectures for end-to-end learning. HORNet leverages the strengths of both kernel methods in modeling nonlinearities and of neural networks in structural prediction. This combination renders it effective and efficient for direct face alignment. Extensive experiments on five in-the-wild datasets show that HORNet delivers high performance and consistently exceeds state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Zero-shot Learning (ZSL) is a highly non-trivial task to generalize from seen to unseen classes. In this paper, we propose spherical zero-shot learning (SZSL) to address the major challenges in ZSL. By decoupling the similarity metric in the spherical embedding space into radius and angle, our SZSL can map classes to hyperspherical surfaces of different radiuses, which greatly increases its flexibility. Specifically, we introduce the spherical alignment on angles to spread classes as uniformly as possible to alleviate the hubness problem and simultaneously preserve the inter-class semantic structure to make the alignment more reasonable. We also introduce the spherical calibration with a minimum entropy based regularizer by adopting a larger radius for unseen classes than seen classes to reduce the prediction bias. Extensive experiments on five middle-scale benchmarks and large-scale ImageNet dataset demonstrate that the proposed approach consistently achieves superior performance for the traditional and generalized settings of ZSL. [ABSTRACT FROM AUTHOR]

Image deraining is an important yet challenging image processing task. Though deterministic image deraining methods are developed with encouraging performance, they are infeasible to learn flexible representations for probabilistic inference and diverse predictions. Besides, rain intensity varies both in spatial locations and across color channels, making this task more difficult. In this paper, we propose a Conditional Variational Image Deraining (CVID) network for better deraining performance, leveraging the exclusive generative ability of Conditional Variational Auto-Encoder (CVAE) on providing diverse predictions for the rainy image. To perform spatially adaptive deraining, we propose a spatial density estimation (SDE) module to estimate a rain density map for each image. Since rain density varies across different color channels, we also propose a channel-wise (CW) deraining scheme. Experiments on synthesized and real-world datasets show that the proposed CVID network achieves much better performance than previous deterministic methods on image deraining. Extensive ablation studies validate the effectiveness of the proposed SDE module and CW scheme in our CVID network. The code is available at https://github.com/Yingjun-Du/VID. [ABSTRACT FROM AUTHOR]

• Our new approach for continuous cross-modal hashing (CCMH) can handle arriving new classes (ANCs), which has not been explored before. • The proposed CT-CMH can be flexibly deployed in real-world applications by adopting a single-head setup. • The proposed CT-CMH consisting of HCD and AWIU can effectively address the problem of catastrophic forgetting and hash-code conflicts. • The experimental results and visualizations demonstrate the effectiveness of our approach for the CCMH task. Generally, multimodal data with new classes arrive continuously in the real world. While advanced cross-modal hashing (CMH) focuses primarily on batch-based data with previously observed classes (ASCs), it disregards the effect of newly arriving classes (ANCs) on hash-code conflicts. In addition, class-level continuous hashing scenarios do not suit themselves well with the generic CMH configuration. To solve the aforementioned issues, we propose a novel framework, called CT-CMH, for the new task of continuous cross-modal hashing. For dealing with ANCs, CMH models require the ability of continuous learning, i.e. they can preserve the knowledge of previously observed data and, more crucially, they can be adapted to unseen data with ANCs. Specifically, we introduce the adaptive weight importance updating (AWIU) mechanism to alleviate the catastrophic forgetting problem of CMH and a new hash-code divergence (HCD) method to eliminate hash-code conflicts between ASCs and ANCs. When CT-CMH is equipped with both AWIU and HCD, it can consistently achieve high retrieval performance. The experiment results and visualization analyses validate the effectiveness of our approach. To the best of our knowledge, we are the first to introduce and implement the task of CCMH for ANCs. [ABSTRACT FROM AUTHOR]

We present a summarization approach for egocentric video. Given hours of video, the proposed method produces a compact storyboard summary of the camera wearer's day. In contrast to traditional keyframe selection techniques, the resulting summary focuses on the most important video shots which reflect high stable salience, discrimination and representativeness. To accomplish this, we utilize egocentric salience cues, motion cues and a selection model to capture stable salience weight, discriminative weight and representative weight of a video shot respectively. We further combine these weights in a unified framework to predict the importance score of a shot, based on which, important shots are selected for the storyboard. Critically, the approach is neither camera-wearer-specific nor object-specific; that means the learned importance metric need not be trained for a given user or context, and it can predict the importance of shots that have never been seen previously. Experimental results on three video datasets across various genres demonstrate that our proposed approach clearly outperforms several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel subspace learning algorithm called Local Feature Discriminant Projection (LFDP) for supervised dimensionality reduction of local features. LFDP is able to efficiently seek a subspace to improve the discriminability of local features for classification. We make three novel contributions. First, the proposed LFDP is a general supervised subspace learning algorithm which provides an efficient way for dimensionality reduction of large-scale local feature descriptors. Second, we introduce the Differential Scatter Discriminant Criterion (DSDC) to the subspace learning of local feature descriptors which avoids the matrix singularity problem. Third, we propose a generalized orthogonalization method to impose on projections, leading to a more compact and less redundant subspace. Extensive experimental validation on three benchmark datasets including UIUC-Sports, Scene-15 and MIT Indoor demonstrates that the proposed LFDP outperforms other dimensionality reduction methods and achieves state-of-the-art performance for image classification. [ABSTRACT FROM AUTHOR]

The focus of few shot learning research has been on the development of meta-learning recently, where a meta-learner is trained on a variety of tasks in hopes of being generalizable to new tasks. Tasks in meta training and meta test are usually assumed to be from the same domain, which would not necessarily hold in real world scenarios. In this paper, we propose variational hyperparameter inference for few-shot learning across domains. Based on an especially successful algorithm named model agnostic meta learning, the proposed variational hyperparameter inference integrates meta learning and variational inference into the optimization of hyperparameters, which enables the meta-learner with adaptivity for generalization across domains. In particular, we choose to learn adaptive hyperparameters including the learning rate and weight decay to avoid the failure in the face of few labeled examples across domain. Moreover, we model hyperparameters as distributions instead of fixed values, which will further enhance the generalization ability by capturing the uncertainty. Extensive experiments are conducted on two benchmark datasets including few shot learning dataset within-domain and across-domain. The results demonstrate that our methods outperforms previous approaches consistently, and comprehensive ablation studies further validate its effectiveness on few shot learning both within domains and across domains. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation crowd counting (UDACC) has been studied with practical research utility by getting rid of the labeling burden on large-scale dense crowds in the target domain. Current methods generalize well within the specific domain gap by directly aligning domain distributions or translating synthetic data to realistic images. However, it is difficult to define domain gaps among complex real-world datasets, in which the images vary greatly in style, density level and/or content. To tackle this problem, in this paper, we propose a Cross-Domain Attention Network (CDANet), which can effectively generalize the model to the unlabeled domain on both unsupervised synthetic-to-realistic and realistic-to-realistic crowd counting. Specifically, we propose a Cross-Domain Attention Module (CDAM) to learn domain-related information between the source and target domain, which extracts relations in cross-domain attentive information, thus enhancing crowd-informative features. Moreover, to make our CDAM invariant to domain shifts, we introduce a consistency penalty to ensure that the attention maps are consistent before and after the domain shifting. Thus our CDANet can pay attention to the shared counting information across domains, while remaining its invariant ability during domain adaptation. Extensive experiments on several common benchmarks for UDACC demonstrate that our CDANet gets competitive results on both unsupervised synthetic-to-realistic and realistic-to-realistic UDACC tasks. [ABSTRACT FROM AUTHOR]

Recent progress in generative models has led to the drastic growth of research in image generation. Existing approaches show visually compelling results by learning multi-modal distributions, but they still lack realism, especially in certain scenarios like medical image synthesis. In this paper, we propose a novel Brain Generative Adversarial Network (BrainGAN) that explores GANs with multi-constraint and transferable property for cross-modal brain image synthesis. We formulate BrainGAN by introducing a unified framework with new constraints that can enhance modal matching, texture details and anatomical structure, simultaneously. We show how BrainGAN can learn meaningful tissue representations with rich variability of brain images. In addition to generating 3D volumes that are visually indistinguishable from real ones, we model adversarial discriminators and segmentors jointly, along with the proposed cost functions, which forces our networks to synthesize brain MRIs with realistic textures conditioned on anatomical structures. BrainGAN is evaluated on three public datasets, where it consistently outperforms the other state-of-the-art approaches by a large margin, advancing cross-modal synthesis of brain images both visually and practically. [ABSTRACT FROM AUTHOR]

Abnormal crowd behavior detection has recently attracted increasing attention due to its wide applications in computer vision research areas. However, it is still an extremely challenging task due to the great variability of abnormal behavior coupled with huge ambiguity and uncertainty of video contents. To tackle these challenges, we propose a new probabilistic framework named variational abnormal behavior detection (VABD), which can detect abnormal crowd behavior in video sequences. We make three major contributions: (1) We develop a new probabilistic latent variable model that combines the strengths of the U-Net and conditional variational auto-encoder, which also are the backbone of our model; (2) We propose a motion loss based on an optical flow network to impose the motion consistency of generated video frames and input video frames; (3) We embed a Wasserstein generative adversarial network at the end of the backbone network to enhance the framework performance. VABD can accurately discriminate abnormal video frames from video sequences. Experimental results on UCSD, CUHK Avenue, IITB-Corridor, and ShanghaiTech datasets show that VABD outperforms the state-of-the-art algorithms on abnormal crowd behavior detection. Without data augmentation, our VABD achieves 72.24% in terms of AUC on IITB-Corridor, which surpasses the state-of-the-art methods by nearly 5%. [ABSTRACT FROM AUTHOR]

Crowd counting that aims to estimate the crowd density has recently made significant progress but remains an unsolved problem due to several challenges. In this paper, we propose an Attentive Encoder-Decoder Network (AEDNet) to overcome the notorious scale-variation problem in crowd counting. Our major contributions can be summarized in three aspects. First, we design an Attentive Feature Refinement (AFR) block in the encoder to adaptively extract multi-scale features. AFR compares the spatial information in different scales through the attention mechanism and then adaptively assign importance weights to each point, which highlights the distinctive roles in multi-scale feature extraction. Second, we develop a Separable Non-local Fusion (SNF) block in the decoder with the self-attention mechanism to aggregate multi-scale features from different layers, which not only achieves the sufficient feature fusion by capturing long-range dependencies, but also vastly reduces the computation cost compared to the original non-local operation. Third, we propose a Regional MSE (R-MSE) loss to tackle the pixel-isolation problems in regular MSE loss. To demonstrate the effectiveness of the proposed AEDNet, we conduct extensive experiments on four widely-used crowd counting datasets, and our AEDNet consistently achieves the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Abstract: Finding correspondences between two related feature point sets is a basic task in computer vision and pattern recognition. In this paper, we present a novel method for point pattern matching via spectral graph analysis. In particular, we aim to render the spectral matching algorithm more robust for positional jitter and outlier. A local structural descriptor, namely the spectral context, is proposed to describe the attribute domain of point sets, which is fundamentally different from the previous methods. Furthermore, the approximate distance order is defined and employed as the metric for geometric consistency of neighboring points in this work. By combining these two novel ingredients, we formulate feature point set matching as an optimization problem with one-to-one constraints. The correspondences are then obtained by maximizing the given objective function via the technique of probabilistic relaxation. Comparative experiments conducted on both synthetic and real data demonstrate the effectiveness of the proposed method, especially in the presence of positional jitter and outliers. [Copyright &y& Elsevier]

Abstract: In this paper, we propose to integrate structural information with appearance features for human action recognition. In local representations based on detected spatio-temporal interest points (STIPs), the layout of STIPs carries important cues of motion structures in video sequences, and is assumed to contain complementary information to appearance features. We aim to incorporate structures into the description of STIPs by combing with appearance features for action representation. Based on the previous work of the 3D shape context, we present an optimised version of 3D shape context to encode the layout information of STIPs. By combining the proposed optimised 3D shape context with appearance descriptors, e.g., HOG3D and 3D gradients, we provide a more informative and discriminative description of STIPs for action classification. To validate the proposed descriptor, we have conducted extensive experiments on the KTH and the UCF YouTube datasets. The results prove that the optimised 3D shape context offers complementary information to appearance features, showing its effectiveness for action representation; moreover, the proposed descriptor yields comparable results with the state-of-the-art methods. [Copyright &y& Elsevier]

Few-shot learning has recently generated increasing popularity in machine learning, which addresses the fundamental yet challenging problem of learning to adapt to new tasks with the limited data. In this paper, we propose a new probabilistic framework that learns to fast adapt with external memory. We model the classifier parameters as distributions that are inferred from the support set and directly applied to the query set for prediction. The model is optimized by formulating as a variational inference problem. The probabilistic modeling enables better handling prediction uncertainty due to the limited data. We impose a discriminative constraint on the feature representations by exploring the class structure, which can improve the classification performance. We further introduce a memory unit to store task-specific information extracted from the support set and used for the query set to achieve explicit adaption to individual tasks. By episodic training, the model learns to acquire the capability of adapting to specific tasks, which guarantees its performance on new related tasks. We conduct extensive experiments on widely-used benchmarks for few-shot recognition. Our method achieves new state-of-the-art performance and largely surpassing previous methods by large margins. The ablation study further demonstrates the effectiveness of the proposed discriminative learning and memory unit. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new multi-layer learning architecture, the calibrated multivariate regression network (CMRN). Compared to previous multivariate models, the CMRN is able to simultaneously handle major challenges in multivariate regression including highly nonlinear input-output relationships, underlying inter-output correlations and calibration of multiple outputs within one single framework. The CMRN is comprised of a nonlinear module with cosine activations and a linear module with the low-rank expansion, which establishes a compact multivariate regression network. By seamlessly working with the $\ell _{2,1}$ loss, the CMRN automatically calibrates multiple outputs with distinct noise levels to achieve improved performance. Being succinctly formulated but theoretically well-founded, the CMRN offers a compact multi-layer learning architecture that can be efficiently trained to scale up with massive datasets. We conduct extensive experimental evaluation on two representative large multivariate regression tasks for both machine learning and computer vision. The proposed CMRN can produce high performance on all tasks, which is better or competitive to state-of-the-art models. Extensive ablation studies offer deep insights into the effectiveness of the proposed CMRN. [ABSTRACT FROM AUTHOR]

Representation learning is a fundamental but challenging problem, especially when the distribution of data is unknown. In this paper, we propose a new representation learning method, named Structure Transfer Machine (STM), which enables feature learning process to converge at the representation expectation in a probabilistic way. We theoretically show that such an expected value of the representation (mean) is achievable if the manifold structure can be transferred from the data space to the feature space. The resulting structure regularization term, named manifold loss, is incorporated into the loss function of the typical deep learning pipeline. The STM architecture is constructed to enforce the learned deep representation to satisfy the intrinsic manifold structure from the data, which results in robust features that suit various application scenarios, such as digit recognition, image classification and object tracking. Compared with state-of-the-art CNN architectures, we achieve better results on several commonly used public benchmarks. [ABSTRACT FROM AUTHOR]

Flying bird detection has recently attracted increasing attention in computer vision, which becomes an urgent task with the opening up of the low-altitude airspace. However, compared to conventional object detection tasks, it is much more challenging to trap flying birds in aerial videos due to small target sizes, complex backgrounds of great variations and disturbances of bird-like objects. In this paper, we propose a unified framework termed glance-and-stare detection (GSD) to trap flying birds in aerial videos. The GSD is inspired by the fact that human beings first glance at the whole image and then stare at the areas where the suspected object is most likely to appear until the confirmation is obtained. Specifically, we propose the zooming-in algorithm to generate region proposals for accurate localization of flying birds; to represent region proposal sequences of different lengths, we propose adaptive deep spatio-temporal features by leveraging the strength of 3D convolutional neural networks, based on which classification is conducted to achieve final detection. In contrast to conventional methods, the GSD enables localization and classification to be conducted jointly in an alternating iterative way, which mutually enhances each other to improve their performance. In order to validate the proposed GSD algorithm, we build flying bird data sets including images and videos, which provide new benchmarks for evaluation of flying bird detection systems. Experiments on the data sets demonstrate that the GSD can achieve high detection accuracy and largely outperform the state-of-the-art detection methods. [ABSTRACT FROM AUTHOR]

• We provide the fully probabilistic framework for few-shot semantic segmentation. • We introduce a variational attention mechanism to enable the model to capture the appearance variation of an object. • We formulate the optimization as a variational inference problem to jointly estimate posteriors over latent variables. This paper aims to address few-shot segmentation. While existing prototype-based methods have achieved considerable success, they suffer from uncertainty and ambiguity caused by limited labeled examples. In this work, we propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot segmentation. We define a global latent variable to represent the prototype of each object category, which we model as a probabilistic distribution. The probabilistic modeling of the prototype enhances the model's generalization ability by handling the inherent uncertainty caused by limited data and intra-class variations of objects. To further enhance the model, we introduce a local latent variable to represent the attention map of each query image, which enables the model to attend to foreground objects while suppressing the background. The optimization of the proposed model is formulated as a variational Bayesian inference problem, which is established by amortized inference networks. We conduct extensive experiments on four benchmarks, where our proposal obtains at least competitive and often better performance than state-of-the-art prototype-based methods. We also provide comprehensive analyses and ablation studies to gain insight into the effectiveness of our method for few-shot segmentation. [ABSTRACT FROM AUTHOR]

Dynamic scene classification has been extensively studied in computer vision due to its widespread applications. The key to dynamic scene classification lies in jointly characterizing spatial appearance and temporal dynamics to achieve informative representation, which remains an outstanding task in the literature. In this paper, we propose a unified framework to extract spatial and temporal features for dynamic scene representation. More specifically, we deploy two variants of deep convolutional neural networks to encode spatial appearance and short-term dynamics into short-term deep features (STDF). Based on STDF, we propose using the autoregressive moving average model to extract long-term frequency features (LTFF). By combining STDF and LTFF, we establish the long–short-term feature (LSTF) representations of dynamic scenes. The LSTF characterizes both spatial and temporal patterns of dynamic scenes for comprehensive and information representation that enables more accurate classification. Extensive experiments on three-dynamic scene classification benchmarks have shown that the proposed LSTF achieves high performance and substantially surpasses the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Egocentric activity recognition has recently generated great popularity in computer vision due to its widespread applications in egocentric video analysis. However, it poses new challenges comparing to the conventional third-person activity recognition tasks, which are caused by significant body shaking, varied lengths, and poor recoding quality, etc. To handle these challenges, in this paper, we propose deep appearance and motion learning (DAML) for egocentric activity recognition, which leverages the great strength of deep learning networks in feature learning. In contrast to hand-crafted visual features or pre-trained convolutional neural network (CNN) features with limited generality to new egocentric videos, the proposed DAML is built on the deep autoencoder (DAE), and directly extracts appearance and motion feature, the main cue of activities, from egocentric videos. The DAML takes advantages of the great effectiveness and efficiency of the DAE in unsupervised feature learning, which provides a new representation learning framework of egocentric videos. The learned appearance and motion features by the DAML are seamlessly fused to accomplish a rich informative egocentric activity representation which can be readily fed into any supervised learning models for activity recognition. Experimental results on two challenging benchmark datasets show that the DAML achieves high performance on both short- and long-term egocentric activity recognition tasks, which is comparable to or even better than the state-of-the-art counterparts. [ABSTRACT FROM AUTHOR]

• We introduce physics-informed frameworks that analyse DCE-MRI data and output robust and reliable Extended Tofts-Kety parameters. • All frameworks showed to be substantially faster than conventional approaches. • The GRU network with attention layers outperforms the conventional non-linear least squares method at typical in vivo noise. • Recurrent networks show to generalize well over varying acquisition lengths, making them widely applicable. • In vivo, the GRU had better test-retest repeatability for all Extended Tofts-Kety parameters. [Display omitted] Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is an MRI technique for quantifying perfusion that can be used in clinical applications for classification of tumours and other types of diseases. Conventionally, the non-linear least squares (NLLS) methods is used for tracer-kinetic modelling of DCE data. However, despite promising results, NLLS suffers from long processing times (minutes-hours) and noisy parameter maps due to the non-convexity of the cost function. In this work, we investigated physics-informed deep neural networks for estimating physiological parameters from DCE-MRI signal-curves. Three voxel-wise temporal frameworks (FCN, LSTM, GRU) and two spatio-temporal frameworks (CNN, U-Net) were investigated. The accuracy and precision of parameter estimation by the temporal frameworks were evaluated in simulations. All networks showed higher precision than the NLLS. Specifically, the GRU showed to decrease the random error on v e by a factor of 4.8 with respect to the NLLS for noise (SD) of 1/20. The accuracy was better for the prediction of the v e parameter in all networks compared to the NLLS. The GRU and LSTM worked with arbitrary acquisition lengths. The GRU was selected for in vivo evaluation and compared to the spatio-temporal frameworks in 28 patients with pancreatic cancer. All neural network approaches showed less noisy parameter maps than the NLLS. The GRU had better test-retest repeatability than the NLLS for all three parameters and was able to detect one additional patient with significant changes in DCE parameters post chemo-radiotherapy. Although the U-Net and CNN had even better test-retest characteristics than the GRU, and were able to detect even more responders, they also showed potential systematic errors in the parameter maps. Therefore, we advise using our GRU framework for analysing DCE data. [ABSTRACT FROM AUTHOR]

Abstract: Previous studies have consistently suggested that the ɛ4 allele of apolipoprotein E (APOE) gene is a major risk factor for Alzheimer''s disease (AD). However, whether the ɛ2 allele, a possible protective factor for AD, will express its protective effect in terms of cortical thickness in healthy elderly carriers is unclear. The goal of this study is to clarify the effects of APOE genotypes on cortical thickness in nondemented elderly subjects. We used 164 healthy, cognitively normal, elderly subjects, who were grouped into ɛ2 carriers, ɛ3 homozygotes, and ɛ4 carriers respectively. The APOE ɛ2 carriers had a significant thicker (corrected p <0.05) cortical thickness in the superior temporal cortex compared with the ɛ3 homozygotes. In addition to this area, the APOE ɛ2 carriers had a significantly thicker region in the dorsolateral prefrontal cortex (corrected p <0.05) than did the ɛ4 carriers. These findings suggest that the different alleles of the APOE gene have distinct neuroanatomic effects in elderly healthy subjects and may play specific roles in the development of AD. [Copyright &y& Elsevier]

Despite the fact that automatic target recognition (ATR) in Synthetic aperture radar (SAR) images has been extensively researched due to its practical use in both military and civil applications, it remains an unsolved problem. The major challenges of ATR in SAR stem from severe data scarcity and great variation of SAR images. Recent work started to adopt convolutional neural networks (CNNs), which, however, remain unable to handle the aforementioned challenges due to their high dependency on large quantities of data. In this paper, we propose a novel deep convolutional learning architecture, called Multi-Stream CNN (MS-CNN), for ATR in SAR by leveraging SAR images from multiple views. Specifically, we deploy a multi-input architecture that fuses information from multiple views of the same target in different aspects; therefore, the elaborated multi-view design of MS-CNN enables it to make full use of limited SAR image data to improve recognition performance. We design a Fourier feature fusion framework derived from kernel approximation based on random Fourier features which allows us to unravel the highly nonlinear relationship between images and classes. More importantly, MS-CNN is qualified with the desired characteristic of easy and quick manoeuvrability in real SAR ATR scenarios, because it only needs to acquire real-time GPS information from airborne SAR to calculate aspect differences used for constructing testing samples. The effectiveness and generalization ability of MS-CNN have been demonstrated by extensive experiments under both the Standard Operating Condition (SOC) and Extended Operating Condition (EOC) on the MSTAR dataset. Experimental results have shown that our proposed MS-CNN can achieve high recognition rates and outperform other state-of-the-art ATR methods. [ABSTRACT FROM AUTHOR]