Search

This article proposes the Mediterranean matrix multiplication, a new, simple and practical randomized algorithm that samples angles between the rows and columns of two matrices with sizes m, n, and p to approximate matrix multiplication in O(k(mn+np+mp)) steps, where k is a constant only related to the precision desired. The number of instructions carried out is mainly bounded by bitwise operators, amenable to a simplified processing architecture and compressed matrix weights. Results show that the method is superior in size and number of operations to the standard approximation with signed matrices. Equally important, this article demonstrates a first application to machine learning inference by showing that weights of fully connected layers can be compressed between 30 × and 100 × with little to no loss in inference accuracy. The requirements for pure floating-point operations are also down as our algorithm relies mainly on simpler bitwise operators.

The last half-decade has seen a surge in deep learning research on irregular domains and efforts to extend convolutional neural networks (CNNs) to work on irregularly structured data. The graph has emerged as a particularly useful geometrical object in deep learning, able to represent a variety of irregular domains well. Graphs can represent various complex systems, from molecular structure, to computer and social and traffic networks. Consequent on the extension of CNNs to graphs, a great amount of research has been published that improves the inferential power and computational efficiency of graph-based convolutional neural networks (GCNNs). The research is incipient, however, and our understanding is relatively rudimentary. The majority of GCNNs are designed to operate with certain properties. In this survey we review of the state of graph representation learning from the perspective of deep learning. We consider challenges in graph deep learning that have been neglected in the majority of work, largely because of the numerous theoretical difficulties they present. We identify four major challenges in graph deep learning: dynamic and evolving graphs, learning with edge signals and information, graph estimation, and the generalization of graph models. For each problem we discuss the theoretical and practical issues, survey the relevant research, while highlighting the limitations of the state of the art. Advances on these challenges would permit GCNNs to be applied to wider range of domains, in situations where graph models have previously been limited owing to the obstructions to applying a model owing to the domains’ natures.

An automatic system for heart arrhythmia classification can perform a substantial role in managing and treating cardiovascular diseases. In this paper, a deep learning-based multi-model system is proposed for the classification of electrocardiogram (ECG) signals. Two different deep learning bagging models are introduced to classify heartbeats into different arrhythmias types. The first model (CNN-LSTM) is based on a combination of a convolutional neural network (CNN) and long short-term memory (LSTM) network to capture local features and temporal dynamics in the ECG data. The second model (RRHOS-LSTM) integrates some classical features, i.e. RR intervals and higher-order statistics (HOS), with LSTM model to effectively highlight abnormality heartbeats classes. We create a bagging model from the CNN-LSTM and RRHOS-LSTM networks by training each model on a different sub-sampling dataset to handle the high imbalance distribution of arrhythmias classes in the ECG data. Each model is also trained using a weighted loss function to provide high weight for not sufficiently represented classes. These models are then combined using a meta-classifier to form a strong coherent model. The meta-classifier is a feedforward fully connected neural network that takes the different predictions of bagging models as an input and combines them into a final prediction. The result of the meta-classifier is then verified by another CNN-LSTM model to decrease the false positive of the overall system. The experimental results are acquired by evaluating the proposed method on ECG data from the MIT-BIH arrhythmia database. The proposed method achieves an overall accuracy of 95.81% in the “subject-oriented” patient independent evaluation scheme. The averages of F1 score and positive predictive value are higher than all other methods by more than 3% and 8% respectively. The experimental results show the superiority of the proposed method for ECG heartbeats classification compared to many state-of-the-art methods.

Precisely detecting solar active regions (AR) from multi-spectral images is a challenging task yet important in understanding solar activity and its influence on space weather. A main challenge comes from each modality capturing a different location of these 3D objects, as opposed to more traditional multi-spectral imaging scenarios where all image bands observe the same scene. We present a multi-task deep learning framework that exploits the dependencies between image bands to produce 3D AR detection where different image bands (and physical locations) each have their own set of results. Different feature fusion strategies are investigated in this work, where information from different image modalities is aggregated at different semantic levels throughout the network. This allows the network to benefit from the joint analysis while preserving the band-specific information. We compare our detection method against baseline approaches for solar image analysis (multi-channel coronal hole detection, SPOCA for ARs (Verbeeck et al. Astron Astrophys 561:16, 2013)) and a state-of-the-art deep learning method (Faster RCNN) and show enhanced performances in detecting ARs jointly from multiple bands. We also evaluate our proposed approach on synthetic data of similar spatial configurations obtained from annotated multi-modal magnetic resonance images.

Dementia is a chronic and degenerative condition affecting millions globally. The care of patients with dementia presents an ever-continuing challenge to healthcare systems in the 21st century. Medical and health sciences have generated unprecedented volumes of data related to health and wellbeing for patients with dementia due to advances in information technology, such as genetics, neuroimaging, cognitive assessment, free texts, routine electronic health records, etc. Making the best use of these diverse and strategic resources will lead to high-quality care of patients with dementia. As such, machine learning becomes a crucial factor in achieving this objective. The aim of this paper is to provide a state-of-the-art review of machine learning methods applied to health informatics for dementia care. We collate and review the existing scientific methodologies and identify the relevant issues and challenges when faced with big health data. Machine learning has demonstrated promising applications to neuroimaging data analysis for dementia care, while relatively less effort has been made to make use of integrated heterogeneous data via advanced machine learning approaches. We further indicate future potential and research directions in applying advanced machine learning, such as deep learning, to dementia informatics.

Networks derived from various disciplinary of sociality and nature are dynamic and incomplete, and temporal link prediction has wide applications in recommendation system and data mining system, etc. The current algorithms first obtain features by exploiting the topological or latent structure of networks, and then predict temporal links based on the obtained features. These algorithms are criticized by the separation of feature extraction and link prediction, which fails to fully characterize the dynamics of networks, resulting in undesirable performance. To overcome this problem, we propose a novel algorithm by joint multi-label learning and feature extraction (called MLjFE), where temporal link prediction and feature extraction are integrated into an overall objective function. The main advantage of MLjFE is that the features and parameter matrix for temporal link prediction are simultaneously learned during optimization procedure, which is more precise to capture dynamics of networks, improving the performance of algorithms. The experimental results on a number of artificial and real-world temporal networks demonstrate that the proposed algorithm significantly outperforms state-of-the-art methods, showing joint learning with feature extraction and temporal link prediction is promising.

Segmenting complex 3D geometry is a challenging task due to rich structural details and complex appearance variations of target object. Shape representation and foreground-background delineation are two of the core components of segmentation. Explicit shape models, such as mesh based representations, suffer from poor handling of topological changes. On the other hand, implicit shape models, such as level-set based representations, have limited capacity for interactive manipulation. Fully automatic segmentation for separating foreground objects from background generally utilizes non-interoperable machine learning methods, which heavily rely on the off-line training dataset and are limited to the discrimination power of the chosen model. To address these issues, we propose a novel semi-implicit representation method, namely Non-Uniform Implicit B-spline Surface (NU-IBS), which adaptively distributes parametrically blended patches according to geometrical complexity. Then, a two-stage cascade classifier is introduced to carry out efficient foreground and background delineation, where a simplistic Naive-Bayesian model is trained for fast background elimination, followed by a stronger pseudo-3D Convolutional Neural Network (CNN) multi-scale classifier to precisely identify the foreground objects. A localized interactive and adaptive segmentation scheme is incorporated to boost the delineation accuracy by utilizing the information iteratively gained from user intervention. The segmentation result is obtained via deforming an NU-IBS according to the probabilistic interpretation of delineated regions, which also imposes a homogeneity constrain for individual segments. The proposed method is evaluated on a 3D cardiovascular Computed Tomography Angiography (CTA) image dataset and Brain Tumor Image Segmentation Benchmark 2015 (BraTS2015) 3D Magnetic Resonance Imaging (MRI) dataset.

Data privacy has become an increasingly important issue in Machine Learning (ML), where many approaches have been developed to tackle this challenge, e.g. cryptography (Homomorphic Encryption (HE), Differential Privacy (DP), etc.) and collaborative training (Secure Multi-Party Computation (MPC), Distributed Learning and Federated Learning (FL)). These techniques have a particular focus on data encryption or secure local computation. They transfer the intermediate information to the third party to compute the final result. Gradient exchanging is commonly considered to be a secure way of training a robust model collaboratively in Deep Learning (DL). However, recent researches have demonstrated that sensitive information can be recovered from the shared gradient. Generative Adversarial Network (GAN), in particular, has shown to be effective in recovering such information. However, GAN based techniques require additional information, such as class labels which are generally unavailable for privacy-preserved learning. In this paper, we show that, in the FL system, image-based privacy data can be easily recovered in full from the shared gradient only via our proposed Generative Regression Neural Network (GRNN). We formulate the attack to be a regression problem and optimize two branches of the generative model by minimizing the distance between gradients. We evaluate our method on several image classification tasks. The results illustrate that our proposed GRNN outperforms state-of-the-art methods with better stability, stronger robustness, and higher accuracy. It also has no convergence requirement to the global FL model. Moreover, we demonstrate information leakage using face re-identification. Some defense strategies are also discussed in this work., The source code can be found at: https://github.com/Rand2AI/GRNN

International audience; Precisely detecting solar Active Regions (AR) from multi-spectral images is a challenging task yet important in understanding solar activity and its influence on space weather. A main challenge comes from each modality capturing a different location of these 3D objects, as opposed to more traditional multi-spectral imaging scenarios where all image bands observe the same scene. We present a multi-task deep learning framework that exploits the dependencies between image bands to produce 3D AR detection where different image bands (and physical locations) each have their own set of results. We compare our detection method against baseline approaches for solar image analysis (multi-channel coronal hole detection, SPOCA for ARs (Verbeeck et al., 2013)) and a state-of-the-art deep learning method (Faster RCNN) and show enhanced performances in detecting ARs jointly from multiple bands.

Managing and treating cardiovascular diseases can be substantially improved by automatic detection and classification of the heart arrhythmia. In this paper, we introduced a novel deep learning system for classifying the electrocardiogram (ECG) signals. The heartbeats are classified into different arrhythmia types using two proposed deep learning models. The first model is integrating the convolutional neural network (CNN) and long short-term memory (LSTM) network to extract useful features within the ECG signal. The second model combines several classical features with LSTM in order to effectively recognize abnormal classes. These deep learning models are trained using a bagging model then aggregated by a fusion classifier to form a robust unified model. The proposed system is evaluated on the MIT-BIH arrhythmia database and produces an overall accuracy of 95.81%, which significantly outperforms the state-of-the-art.

There has been growing interest recently in using machine learning techniques as an aid in clinical medicine. Machine learning offers a range of classification algorithms which can be applied to medical data to aid in making clinical predictions. Recent studies have demonstrated the high predictive accuracy of various classification algorithms applied to clinical data. Several studies have already been conducted in diagnosing or predicting chronic kidney disease at various stages using different sets of variables. In this study we are investigating the use of machine learning techniques with blood test data. Such a system could aid renal teams in making recommendations to primary care general practitioners to refer patients to secondary care where patients may benefit from earlier specialist assessment and medical intervention. We are able to achieve an overall accuracy of 88.48% using logistic regression, 87.12% using ANN and 85.29% using SVM. ANNs performed with the highest sensitivity at 89.74 % compared to 86.67 % for logistic regression and 85.51 % for SVM.

We present a comprehensive, detailed review of time-series data analysis, with emphasis on deep time-series clustering (DTSC), and a case study in the context of movement behavior clustering utilizing the deep clustering method. Specifically, we modified the DCAE architectures to suit time-series data at the time of our prior deep clustering work. Lately, several works have been carried out on deep clustering of time-series data. We also review these works and identify state-of-the-art, as well as present an outlook on this important field of DTSC from five important perspectives.

Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. This presents significant challenges and restricts many deep learning applications, making the focus on reducing the complexity of models while maintaining their powerful performance. In this paper, we present an overview of popular methods and review recent works on compressing and accelerating deep neural networks. We consider not only pruning methods but also quantization methods, and low-rank factorization methods. This review also intends to clarify these major concepts, and highlights their characteristics, advantages, and shortcomings.

Sepsis presents a significant challenge to healthcare providers during critical care scenarios such as within an intensive care unit. The prognosis of the onset of severe septic shock results in significant increases in mortality rate, length of stay and readmission rates. Continual advancements in health informatics data allows for applications within the machine learning field to predict sepsis onset in a timely manner, allowing for effective preventative intervention of severe septic shock. A novel deep learning application is proposed to provide effective prediction of sepsis onset by up to six hours prior, involving the use of novel concepts such as a boosted cascading training methodology and adjustable margin hinge loss function. The proposed methodology provides statistically significant improvements to that of current machine learning based modelling applications based off the Physionet Computing in Cardiology 2019 challenge. Results show test F1 scores of 0.420, a significant improvement of 0.281 as compared to the next best challenger results.

Precisely localising solar Active Regions (AR) from multi-spectral images is a challenging but important task in understanding solar activity and its influence on space weather. A main challenge comes from each modality capturing a different location of the 3D objects, as opposed to typical multi-spectral imaging scenarios where all image bands observe the same scene. Thus, we refer to this special multi-spectral scenario as multi-layer. We present a multi-task deep learning framework that exploits the dependencies between image bands to produce 3D AR localisation (segmentation and detection) where different image bands (and physical locations) have their own set of results. Furthermore, to address the difficulty of producing dense AR annotations for training supervised machine learning (ML) algorithms, we adapt a training strategy based on weak labels (i.e. bounding boxes) in a recursive manner. We compare our detection and segmentation stages against baseline approaches for solar image analysis (multi-channel coronal hole detection, SPOCA for ARs) and state-of-the-art deep learning methods (Faster RCNN, U-Net). Additionally, both detection and segmentation stages are quantitatively validated on artificially created data of similar spatial configurations made from annotated multi-modal magnetic resonance images. Our framework achieves an average of 0.72 IoU (segmentation) and 0.90 F1 score (detection) across all modalities, comparing to the best performing baseline methods with scores of 0.53 and 0.58, respectively, on the artificial dataset, and 0.84 F1 score in the AR detection task comparing to baseline of 0.82 F1 score. Our segmentation results are qualitatively validated by an expert on real ARs.

The achievement of neural networks in a variety of applications is accompanied by a dramatic increase in computational costs and memory requirements. In this paper, we propose an efficient method to simultaneously identify the critical neurons and prune the model during training without involving any pre-training or fine-tuning procedures. Unlike existing methods, which accomplish this task in a greedy fashion, we propose a majority voting technique to compare the activation values among neurons and assign a voting score to quantitatively evaluate their importance. This mechanism helps to effectively reduce model complexity by eliminating the less influential neurons and aims to determine a subset of the whole model that can represent the reference model with much fewer parameters within the training process. Experimental results show that majority voting efficiently compresses the network with no drop in model accuracy, pruning more than 79% of the original model parameters on CIFAR10 and more than 91% of the original parameters on MNIST. Moreover, we show that with our proposed method, sparse models can be further pruned into even smaller models by removing more than 60% of the parameters, whilst preserving the reference model accuracy.

Psychology studies and behavioural observation show that humans shift their attention from one location to another when viewing an image of a complex scene. This is due to the limited capacity of the human visual system in simultaneously processing multiple visual inputs. The sequential shifting of attention on objects in a non-task oriented viewing can be seen as a form of saliency ranking. Although there are methods proposed for predicting saliency rank, they are not able to model this human attention shift well, as they are primarily based on ranking saliency values from binary prediction. Following psychological studies, in this paper, we propose to predict the saliency rank by inferring human attention shift. Due to the lack of such data, we first construct a large-scale salient object ranking dataset. The saliency rank of objects is defined by the order that an observer attends to these objects based on attention shift. The final saliency rank is an average across the saliency ranks of multiple observers. We then propose a learning-based CNN to leverage both bottom-up and top-down attention mechanisms to predict the saliency rank. Experimental results show that the proposed network achieves state-of-the-art performances on salient object rank prediction. Code and dataset are available at https://github.com/SirisAvishek/Attention_Shift_Ranks

The achievement of convolutional neural networks (CNNs) in a variety of applications is accompanied by a dramatic increase in computational costs and memory requirements. In this paper, we propose a novel framework to measure the importance of individual hidden units by computing a measure of relevance to identify the most critical filters and prune them to compress and accelerate CNNs. Unlike existing methods, we introduce the use of the activation of feature maps to detect valuable information and the essential semantic parts to evaluate the importance of feature maps, inspired by novel neural network interpretability. A majority voting technique based on the degree of alignment between a semantic concept and individual hidden unit representations is proposed to quantitatively evaluate the importance of feature maps. We also propose a simple yet effective method to estimate new convolution kernels based on the remaining, crucial channels to accomplish effective CNN compression. Experimental results show the effectiveness of our filter selection criteria, which outperforms the state-of-the-art baselines. Furthermore, we evaluate our pruning method on CIFAR-10, CUB-200, and ImageNet (ILSVRC 2012) datasets. The experimental results show that the proposed method efficiently achieves a 50% FLOPs reduction on CIFAR-10, with only 0.86% accuracy drop on the VGG-16 model. Meanwhile, ResNet pruned on CIFAR-10 achieves a 30% reduction in FLOPs with only 0.12% and 0.02% drops in accuracy on ResNet-20 and ResNet-32 respectively. For ResNet-50 on ImageNet, our pruned model achieves a 50% reduction in FLOPs with only a top-5 accuracy drop of 0.27%, which significantly outperforms state-of-the-art methods.

In this paper, we present a novel image segmentation technique, based on hidden Markov model (HMM), which we then apply to simultaneously segment interior and exterior walls of fluorescent confocal images of lymphatic vessels. Our proposed method achieves this by tracking hidden states, which are used to indicate the locations of both the inner and outer wall borders throughout the sequence of images. We parameterize these vessel borders using radial basis functions (RBFs), thus enabling us to minimize the number of points we need to track as we progress through multiple layers and therefore reduce computational complexity. Information about each border is detected using patch-wise convolutional neural networks (CNN). We use the softmax function to infer the emission probability and use a proposed new training algorithm based on s-excess optimization to learn the transition probability. We also introduce a new optimization method to determine the optimum sequence of the hidden states. Thus, we transform the segmentation problem into one that minimizes an s-excess graph cut, where each hidden state is represented as a graph node and the weight of these nodes are defined by their emission probabilities. The transition probabilities are used to define relationships between neighboring nodes in the constructed graph. We compare our proposed method to the Viterbi and Baum-Welch algorithms. Both qualitative and quantitative analysis show superior performance of the proposed methods.

Deep convolutional auto-encoder (DCAE) allows to obtain useful features via its internal layer and provide an abstracted latent representation, which has been exploited for clustering analysis. DCAE allows a deep clustering method to extract similar patterns in lower-dimensional representation and find idealistic representative centers for distributed data. In this paper, we present a deep clustering model carried out in presence of varying degrees of supervision. We propose a new version of DCAE to include a supervision component. It introduces a mechanism to inject various levels of supervision into the learning process. This mechanism helps to effectively reconcile extracted latent representations and provided supervising knowledge in order to produce the best discriminative attributes. The key idea of our approach is distinguishing the discriminatory power of numerous structures, through varying degrees of supervision, when searching for a compact structure to form robust clusters. We evaluate our model on MNIST, USPS, MNIST fashion, SVHN datasets and show clustering accuracy on different supervisory levels.

3D human pose estimation is a very difficult task. In this paper we propose that this problem can be more easily solved by first finding the solutions to a set of easier sub-problems. These are to locally estimate pose conditioned on a fixed root node state, which defines the global position and orientation of the person. The global solution can then be found using information extracted during this procedure. This approach has two key benefits: The first is that each local solution can be found by modeling the articulated object as a kinematic chain, which has far less degrees of freedom than alternative models. The second is that by using this approach we can represent, or support, a much larger area of the posterior than is currently possible. This allows far more robust algorithms to be implemented since there is far less pressure to prune the search space to free up computational resources. We apply this approach to two problems: The first is single frame monocular 3D pose estimation, where we propose a method to directly extract 3D pose without first extracting any intermediate 2D representation or being dependent on strong spatial prior models. The second is multi-view 3D tracking where we show that using the above technique results in an approach that is far more robust than current approaches, without relying on strong temporal prior models. In both domains we demonstrate the strength and versatility of the proposed method. A new approach to searching global solution by first finding a set of local solutionsApplied to both monocular 3D pose detection and multiview 3D trackingAchieve good accuracy using weak likelihood model and no assumed motion modelSource code is publically available to ensure reproducible results.

The geometric potential force (GPF) used in segmentation of medical images is in general a robust method. However, calculation of the GPF is often time consuming and slow. In the present work, we propose several methods for improving the GPF calculation and evaluate their efficiency against the original method. Among different methods investigated, the procedure that combines Riesz transform and integration by part provides the fastest solution. Both static and dynamic images have been employed to demonstrate the efficacy of the proposed methods.

Automatic deformable 3D modeling is computationally expensive, especially when considering complex position, orientation and scale variations. We present a volume segmentation framework to utilize local and global regularizations in a data-driven approach. We introduce automated correspondence search to avoid manually labeling landmarks and improve scalability. We propose a novel marginal space learning technique, utilizing multi-resolution pooling to obtain local and contextual features without training numerous detectors or excessively dense patches. Unlike conventional convolutional neural network operators, graph-based operators allow spatially related features to be learned on the irregular domain of the multi-resolution space, and a graph-based convolutional neural network is proposed to learn representations for position and orientation classification. The graph-CNN classifiers are used within a marginal space learning framework to provide efficient and accurate shape pose parameter hypothesis prediction. During segmentation, a global constraint is initially non-iteratively applied, with local and geometric constraints applied iteratively for refinement. Comparison is provided against both classical deformable models and state-of-the-art techniques in the complex problem domain of segmenting aortic root structure from computerized tomography scans. The proposed method shows improvement in both pose parameter estimation and segmentation performance.

In this paper, we propose a two-step textural feature extraction method, which utilizes the feature learning ability of Convolutional Neural Networks (CNN) to extract a set of low level primitive filter kernels, and then generalizes the discriminative power by forming a histogram based descriptor. The proposed method is applied to a practical medical diagnosis problem of classifying different stages of Age-Related Macular Degeneration (AMD) using a dataset comprising long-wavelength Optical Coherence Tomography (OCT) images of the choroid. The experimental results show that the proposed method extracts more discriminative features than the features learnt through CNN only. It also suggests the feasibility of classifying different AMD stages using the textural information of the choroid region.

In this paper, we propose a clustering approach embedded in a deep convolutional auto-encoder (DCAE). In contrast to conventional clustering approaches, our method simultaneously learns feature representations and cluster assignments through DCAEs. DCAEs have been effective in image processing as it fully utilizes the properties of convolutional neural networks. Our method consists of clustering and reconstruction objective functions. All data points are assigned to their new corresponding cluster centers during the optimization, after that, clustering centers are iteratively updated to obtain a stable performance of clustering. The experimental results on the MNIST dataset show that the proposed method substantially outperforms deep clustering models in term of clustering quality.

Modeling the heart motion has important applications for diagnosis and intervention. We present a new method for modeling the deformation of the myocardium in the cardiac cycle. Our approach is based on manifold learning to build a representation of shape variation across time. We experiment with various manifold types to identify the best manifold method, and with real patient data extracted from cine MRIs. We obtain a representation, common to all subjects, that can discriminate cardiac cycle phases and heart function types.

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area., This paper provides a comprehensive review for researchers of computer vision techniques and methodologies in transcatheter intervention.

Face detection in the wild is a challenging vision problem due to large variations and unpredictable ambiguities commonly existed in real world images. Whilst introducing powerful but complex models is often computationally inefficient, using hand-crafted features is hence problematic. In this paper, we propose a nested CNN-cascade learning algorithm that adopts shallow neural network architectures that allow efficient and progressive elimination of negative hypothesis from easy to hard via self-learning discriminative representations from coarse to fine scales. The face detection problem is considered as solving three sub-problems: eliminating easy background with a simple but fast model, then localising the face region with a soft-cascade, followed by precise detection and localisation by verifying retained regions with a deeper and stronger model. The face detector is trained on the AFLW dataset following the standard evaluation procedure, and the method is tested on four other public datasets, i.e. FDDB, AFW, CMU-MIT and GENKI. Both quantitative and qualitative results on FDDB and AFW are reported, which show promising performances on detecting faces in unconstrained environment.

The field of Human Action Recognition has expanded greatly in previous years, exploring actions and interactions between individuals via the use of appearance and depth based pose information. There are numerous datasets that display action classes composed of behaviors that are well defined by their key poses, such as ‘kicking’ and ‘punching’. The CONVERSE dataset presents conversational interaction classes that show little explicit relation to the poses and gestures they exhibit. Such a complex and subtle set of interactions is a novel challenge to the Human Action Recognition community, and one that will push the cutting edge of the field in both machine learning and the understanding of human actions. CONVERSE contains recordings of two person interactions from 7 conversational scenarios, represented as sequences of human skeletal poses captured by the Kinect depth sensor. In this study we discuss a method providing ground truth labelling for the set, and the complexity that comes with defining such annotation. The CONVERSE dataset it made available online.

A novel fully automated graph cut method to segment IVUS and OCT images.Shape priors to impose global and local shape constraints.Adaptive cost function based on data driven artifact classification.Tracking integrated into graph segmentation to efficiently impose temporal consistency. We present a novel approach to segment coronary cross-sectional images acquired using catheterization imaging techniques, i.e. intra-vascular ultrasound (IVUS) and optical coherence tomography (OCT). The proposed approach combines cross-sectional segmentation with longitudinal tracking in order to tackle various forms of imaging artifacts and to achieve consistent segmentation. A node-weighted directed graph is constructed on two consecutive cross-sectional frames with embedded shape constraints within individual cross-sections or frames and between consecutive frames. The intra-frame constraints are derived from a set of training samples and are embedded in both graph construction and its cost function. The inter-frame constraints are imposed by tracking the borders of interest across multiple frames. The coronary images are transformed from Cartesian coordinates to polar coordinates. Graph partition can then be formulated as searching an optimal interface in the node-weighted directed graph without user initialization. It also allows efficient parametrization of the border using radial basis function (RBF) and thus reduces the tracking of a large number of border points to a very few RBF centers. Moreover, we carry out supervised column-wise tissue classification in order to automatically optimize the feature selection. Instead of empirically assigning weights to different feature detectors, we dynamically and automatically adapt those weighting depending on the tissue compositions in each individual column of pixels. The proposed approach is applied to IVUS and OCT images. Both qualitative and quantitative results show superior performance of the proposed method compared to a number of alternative segmentation techniques.

In this paper, we propose a multi-step textural feature extraction and classification method, which utilizes the feature learning ability of Convolutional Neural Networks (CNN) to extract a set of low level primitive filter kernels, extracts spatial information using clustering and Local Binary Patterns (LBP) and then generalizes the discriminative power by forming a histogram based descriptor. It integrates the concept of hierarchical texton mining and data driven kernel learning into a uniform framework. The proposed method is applied to a practical medical diagnosis problem of classifying different stages of Age-Related Macular Degeneration (AMD) using a dataset comprising long-wavelength Optical Coherence Tomography (OCT) images of the choroid. The results demonstrate the feasibility of our method for classifying different AMD stages using the textural information of the choroidal region.

There is an increasing interest in applying deep learning to 3D mesh segmentation. We observe that 1) existing feature-based techniques are often slow or sensitive to feature resizing, 2) there are minimal comparative studies and 3) techniques often suffer from reproducibility issue. This study contributes in two ways. First, we propose a novel convolutional neural network (CNN) for mesh segmentation. It uses 1D data, filters and a multi-branch architecture for separate training of multi-scale features. Together with a novel way of computing conformal factor (CF), our technique clearly out-performs existing work. Secondly, we publicly provide implementations of several deep learning techniques, namely, neural networks (NNs), autoencoders (AEs) and CNNs, whose architectures are at least two layers deep. The significance of this study is that it proposes a robust form of CF, offers a novel and accurate CNN technique, and a comprehensive study of several deep learning techniques for baseline comparison., Comment: 14 pages, 9 figures, 5 tables

In this paper, we present an efficient segmentation technique that extracts piecewise linear patterns from hand-drawn maps. The user is only required to place the starting and end points and the method is capable of extracting the route that connects the two, which closely colocates with the hand-drawn map. It provides an effective approach to interactively process and understand those historical maps. The proposed method employs supervised learning to evaluate at every pixel location the probability that such a lineage pattern exists, followed by shortest path segmentation to extract the border of interest.

Detecting faces in the wild is a challenging problem due to large visual variations introduced by uncontrolled facial expressions, head pose, illumination and so on. Employing strong classifier and designing more discriminative visual features are two main approaches to overcoming such difficulties. Notably, Deep Neural Network (DNN) based methods have been found to outperform most traditional detectors in a multitude of studies, employing deep network structures and complex training procedures. In this work, we propose a novel method that uses stacked denoising autoencoders (SdA) for feature extraction and random forests (RF) for object-background classification in a classical cascading framework. This architecture allows much simpler neural network structures, resulting in efficient training and detection. The proposed face detector was evaluated on two publicly available datasets and produced promising results.

International audience; We address the problem of object modeling from 3D and 3D+T data made up of images which contain different parts of an object of interest, are separated by large spaces, and are misaligned with respect to each other. These images have only a limited number of intersections, hence making their registration particularly challenging. Furthermore, such data may result from various medical imaging modalities and can therefore present very diverse spatial configurations. Previous methods perform registration and object modeling (segmentation and interpolation) sequentially. However, sequential registration is ill-suited for the case of images with few intersections. We propose a new methodology which, regardless of the spatial configuration of the data, performs the three stages of registration, segmentation, and shape interpolation from spaced and misaligned images simultaneously. We integrate these three processes in a level set framework, in order to benefit from their synergistic interactions. We also propose a new registration method that exploits segmentation information rather than pixel intensities, and that accounts for the global shape of the object of interest, for increased robustness and accuracy. The accuracy of registration is compared against traditional mutual information based methods, and the total modeling framework is assessed against traditional sequential processing and validated on artificial, CT, and MRI data.

Age-Related Macular Degeneration (AMD) is a progressive eye disease which damages the retina and causes visual impairment. Detecting those in the early stages at most risk of progression will allow more timely treatment and preserve sight. In this paper, we propose a machine learning based method to detect AMD and distinguish the different stages using choroidal images obtained from optical coherence tomography (OCT). We extract texture features using a Gabor filter bank and non-linear energy transformation. Then the histogram based feature descriptors are used to train the random forests, Support Vector Machine (SVM) and neural networks, which are tested on our choroid OCT image dataset with 21 participants. The experimental results show the feasibility of our method.

Conventionally, hand-crafted features are used to train machine learning algorithms, however choosing useful features is not a trivial task as they are very much data-dependent. Given raw image intensities as inputs, supervised neural networks (NNs) essentially learn useful features by adjusting the weights of its nodes using the back-propagation algorithm. In this paper we investigate the performance of NN architectures for the purpose of boundary detection, before integrating a chosen architecture in a data-driven deformable modelling framework for full segmentation. Boundary detection performed well, with boundary sensitivity of >88 % and specificity of >85 % for highly obscured and diffused lymphatic vessel walls. In addition, the vast majority of all boundary-classified pixels were in the immediate vicinity of the ground truth boundary. When integrated into a 3D deformable modelling framework it produced an area overlap with the ground truth of >98 %, and both point-to-mesh and Hausdorff distance errors were less than other approaches. To this end it has been shown that NNs are suitable for boundary detection in deformable modelling, where object boundaries are obscured, diffused and low in contrast.

Background Early prediction of the outcomes of dementia is important and challenging. This study aimed to identify influential predictors from primary care electronic health records that can robustly predict whether patients with dementia will be admitted to hospital or remain under GP care. Methods Health records of patients with dementia were collected from general practice (GP) and hospital data in Wales between 1980 and 2015. These records were linked at individual patient level via the Secure Anonymised Information Linkage databank. The GP records of each patient were selected 1 year before diagnosis up to hospital admission. An artificial intelligence technique, neural network with entropy regularisation (a multilayer feedforward neural network whose weights between input layer and the first hidden layer were regularised by an entropy metric into the fitness function during training process) was used to automatically identify the most influential predictors from initial GP read codes, sex, and age. 10-fold cross validation was used to assess the predictive performance of the identified signals. Findings 52·5 million individual records of 59 298 patients (20 674 men, 38 624 women) with dementia were used. 30 178 were admitted to hospital and 29 120 remained with GP care. More men were admitted to hospital than stayed with GP care (11 233 vs 9441), whereas more women stayed with GP care than were admitted (19 679 vs 18 945). From the 54 649 initial event codes, the ten most important signals identified for admission for dementia were two diagnostic events (nightmares, essential hypertension), five medication events (betahistine dihydrochloride, ibuprofen gel, simvastatin, influenza vaccine, calcium carbonate and colecalciferol chewable tablets), and three procedural events (third party encounter, social group 3—skilled, blood glucose raised). They performed significantly above chance to predict admission to hospital with sensitivity of 0·758 (95% CI 0·731–0·785), specificity 0·759 (0·71–0·808), precision 0·766 (0·735–0·797), and negative predictive value 0·751 (0·741–0·761). Linear regression with all raw features yielded values of 0·286 (0·26–0·313), 0·804 (0·792–0·816), 0·487 (0·463–0·511), and 0·633 (0·615–0·651), respectively, and with ten identified features yielded values of 0·684 (0·679–0·691), 0·712 (0·705–0·718), 0·747 (0·738–0·754), and 0·644 (0·64–0·651). Interpretation Outperforming traditional methods, the artificial intelligence technique provides an effective means of identifying influential clinical signals to predict hospital admission of patients with dementia significantly above chance. Funding Health Data Research UK Wales and Northern Ireland Site, National Centre for Population Health and Wellbeing Research (CA02), Major Project of National Social Science Foundation of China (16ZDA0092), Guangxi University Digital ASEAN Cloud Big Data Security and Mining Technology Innovation Team.

We present a review on the current state of publicly available datasets within the human action recognition community; highlighting the revival of pose based methods and recent progress of understanding person-person interaction modeling. We categorize datasets regarding several key properties for usage as a benchmark dataset; including the number of class labels, ground truths provided, and application domain they occupy. We also consider the level of abstraction of each dataset; grouping those that present actions, interactions and higher level semantic activities. The survey identifies key appearance and pose based datasets, noting a tendency for simplistic, emphasized, or scripted action classes that are often readily definable by a stable collection of sub-action gestures. There is a clear lack of datasets that provide closely related actions, those that are not implicitly identified via a series of poses and gestures, but rather a dynamic set of interactions. We therefore propose a novel dataset that represents complex conversational interactions between two individuals via 3D pose. 8 pairwise interactions describing 7 separate conversation based scenarios were collected using two Kinect depth sensors. The intention is to provide events that are constructed from numerous primitive actions, interactions and motions, over a period of time; providing a set of subtle action classes that are more representative of the real world, and a challenge to currently developed recognition methodologies. We believe this is among one of the first datasets devoted to conversational interaction classification using 3D pose features and the attributed papers show this task is indeed possible. The full dataset is made publicly available to the research community at www.csvision.swansea.ac.uk/converse., Presentation of pose-based conversational human interaction dataset, review of current appearance and depth based action recognition datasets, public dataset, 38 pages

A statistical shape model that accurately generalizes a family of 3D shapes requires establishing correspondences across the set of shapes. However in 3D anatomical meshes, finding a sufficient number of landmarks to accurately describe the shape can be a challenge, and often only a few points are easily identifiable due to the smooth nature of the object surface. Using a sparse set of landmarks, this paper finds a dense set of vertex correspondences across a set of 3D aortic root meshes. This is achieved by non-rigidly transforming a source mesh to a target mesh. Then, for every vertex on the target, a corresponding vertex on the deformed source is found, resulting in complete correspondence. A more accurate transformation results in better correspondence establishment, and our mesh registration experiments show an average Hausdorff distance of 3.65mm, and an average point-to-mesh distance of 0.41mm, i. e. within one voxel.

We present a machine learning based approach to automatically detect and segment cells in phase contrast images. The proposed method consists of a multi-stage classification scheme based on random forest (RF) classifier. Both low level and mid level image features are used to determine meaningful cell regions. Pixel-wise RF classification is first carried out to categorize pixels into 4 classes (dark cell, bright cell, halo artifact, and background) and generate a probability map for cell regions. K-means clustering is then applied on the probability map to group similar pixels into candidate cell regions. Finally, cell validation is performed by another RF to verify the candidate cell regions. The proposed method has been tested on U2-OS human osteosarcoma phase contrast images. The experimental results show better performance of the proposed method with precision 92.96% and recall 96.63% compared to a state-of-the-art segmentation technique.

We present a novel HMM based approach to simultaneous segmentation of vessel walls in Lymphatic confocal images. The vessel borders are parameterized using RBFs to minimize the number of tracking points. The proposed method tracks the hidden states that indicate border locations for both the inner and outer walls. The observation for both borders is obtained using edge-based features from steerable filters. Two separate Gaussian probability distributions for the vessel borders and background are used to infer the emission probability, and the transmission probability is learned using a Baum-Welch algorithm. We transform the segmentation problem into a minimization of an s-excess graph cost, with each node in the graph corresponding to a hidden state and the weight for each node being defined by its emission probability. We define the inter-relations between neighboring nodes based on the transmission probability. We present both qualitative and quantitative analysis in comparison to the popular Viterbi algorithm.

In this paper, we propose a unified approach to deformable model-based segmentation. The fundamental force field of the proposed method is based on computing the divergence of a gradient convolution field (GCF), which makes the full use of directional information of the image gradient vectors and their interactions across image domain. However, instead of directly using such a vector field for deformable segmentation as in the conventional approaches, we derive a more salient representation for contour evolution, and very importantly, we demonstrate that this representation of image force field not only leads to global minimum through convex relaxation but also can achieve the same result using the conventional gradient descent with an intrinsic regularization. Thus, the proposed method can handle arbitrary initializations. The proposed external force field for deformable segmentation has both edge-based properties in that the GCF is computed from image gradients, and the region-based attributes since its divergence can be treated as a region indication function. Moreover, nonlinear diffusion can be conveniently applied to GCF to improve its performance in dealing with noise interference. We also show the extension of GCF from 2D to 3D. In comparison to the state-of-the-art deformable segmentation techniques, the proposed method shows greater flexibility in model initialization and optimization realization, as well as better performance toward noise interference and appearance variation.

The use of pose estimation for human action recognition has seen a resurgence in previous years, due in part to the natural representation of the activity as a sequence of key poses and gestures. The use of sequence alignment techniques has aided the process of comparing between sequences of differing temporal rates, with aligned cluster analysis segmenting an observation into lower level action primitives. We suggest that the representation of a given action class via its lower level gestures can help to identify the higher-level action class label. We therefore present a method for the generation of key poses via the initial segmentation of an action class into gestures that are similar across numerous observations. We treat all training observations as a single observation in which there are repetitions of the same action class. By applying segmentation, we then identify common gestures across the class, which are used to generate the key poses we optimize via evolutionary programming. Global recognition rates of 97.4% are achieved using a subset of the MSR Action3D dataset. We then expand the method to recognize interaction events between two individuals using the SBU Kinect Interaction dataset, achieving recognition rates of 83.9% and over 96.4% when observing the first 6 classes.

3D shape co-segmentation is an important topic in computer graphics. The idea of co-analysis brings new insights into understanding a collection of shapes. Rather than analysing individual shapes, an entire set is looked at, giving much more information about the class of shape in the set. Existing co-segmentation techniques use both face and segment level descriptors in order to provide enough information to give an accurate co-segmentation result. In the literature, a lot of these descriptors are proposed but there is limited empirical studies to compare which would perform well. In this paper, we have two aims: (a) propose new useful face and segment level descriptors and (b) analyse the effectiveness of them. Our experiment indicates that smoothly varying descriptors (Average Euclidean Distance) that respects geometry would improve the segmentation results.

We present a novel method to segment the lymph vessel wall in confocal microscopy images using Optimal Surface Segmentation (OSS) and hidden Markov Models (HMM). OSS is used to preform a pre-segmentation on the images, to act as the initial state for the HMM. We utilize a steerable filter to determine edge based filters for both of these segmentations, and use these features to build Gaussian probability distributions for both the vessel walls and the background. From this we infer the emission probability for the HMM, and the transmission probability is learned using a Baum-Welch algorithm. We transform the segmentation problem into one of cost minimization, with each node in the graph corresponding to one state, and the weight for each node being defined using its emission probability. We define the inter-relations between neighboring nodes using the transmission probability. Having constructed the problem, it is solved using the Viterbi algorithm, allowing the vessel to be reconstructed. The optimal solution can be found in polynomial time. We present qualitative and quantitative analysis to show the performance of the proposed method.

Coronary artery segmentation plays a vital important role in coronary disease diagnosis and treatment. In this paper, we present a machine learning based interactive coronary artery segmentation method for 3D computed tomography angiography images. We first apply vessel diffusion to reduce noise interference and enhance the tubular structures in the images. A few user strokes are required to specify region of interest and background. Various image features for detecting the coronary arteries are then extracted in a multi-scale fashion, and are fed into a random forests classifier, which assigns each voxel with probability values of being coronary artery and background. The final segmentation is carried through an MRF based optimization using primal dual algorithm. A connectivity component analysis is carried out as post processing to remove isolated, small regions to produce the segmented coronary arterial vessels. The proposed method requires limited user interference and achieves robust segmentation results.