Search

Domain adaptation aims to exploit useful information from the source domain where annotated training data are easier to obtain to address a learning problem in the target domain where only limited or even no annotated data are available. In classification problems, domain adaptation has been studied under the assumption all classes are available in the target domain regardless of the annotations. However, a common situation where only a subset of classes in the target domain are available has not attracted much attention. In this paper, we formulate this particular domain adaptation problem within a generalized zero-shot learning framework by treating the labelled source-domain samples as semantic representations for zero-shot learning. For this novel problem, neither conventional domain adaptation approaches nor zero-shot learning algorithms directly apply. To solve this problem, we present a novel Coupled Conditional Variational Autoencoder (CCVAE) which can generate synthetic target-domain image features for unseen classes from real images in the source domain. Extensive experiments have been conducted on three domain adaptation datasets including a bespoke X-ray security checkpoint dataset to simulate a real-world application in aviation security. The results demonstrate the effectiveness of our proposed approach both against established benchmarks and in terms of real-world applicability. [ABSTRACT FROM AUTHOR]

Deep Neural Networks have received considerable attention in recent years. As the complexity of network architecture increases in relation to the task complexity, it becomes harder to manually craft an optimal neural network architecture and train it to convergence. As such, Neural Architecture Search (NAS) is becoming far more prevalent within computer vision research, especially when the construction of efficient, smaller network architectures is becoming an increasingly important area of research, for which NAS is well suited. However, despite their promise, contemporary and end-to-end NAS pipeline require vast computational training resources. In this paper, we present a comprehensive overview of contemporary NAS approaches with respect to image classification, object detection, and image segmentation. We adopt consistent terminology to overcome contradictions common within existing NAS literature. Furthermore, we identify and compare current performance limitations in addition to highlighting directions for future NAS research. [ABSTRACT FROM AUTHOR]

Despite significant research focus on 3D scene capture systems, numerous unresolved challenges remain in relation to achieving full coverage scene depth estimation which is the key part of any modern 3D sensing system. This has created an area of research where the goal is to complete the missing 3D information post capture via a secondary depth filling process. In many downstream applications, an incomplete depth scene is of limited value, requiring many special cases for subsequent utilization, and thus techniques are required to “fill the holes” that exist in terms of both missing depth and color scene information. An analogous problem exists within the scope of scene filling post object removal in the same context. Although considerable research has resulted in notable progress in the synthetic expansion or reconstruction of missing color scene information in both statistical (texture synthesis) and structural (image completion) forms, work on the plausible completion of missing scene depth is contrastingly limited. This survey aims to provide a state of the art overview within this growing field of depth synthesis work whilst noting related solutions in the space of traditional texture synthesis and color image completion for hole filling. To these ends, we concentrate on the plausible completion of both underlying depth structure and relief texture to provide both greater understanding and future development in the area. Our analyses are in part supported by illustrative experimental examples of the comparative use of a subset of representative approaches over common depth completion examples. [ABSTRACT FROM AUTHOR]

We present a novel technique for the 3D segmentation of unknown objects from cluttered dual-energy Computed Tomography (CT) data obtained in the baggage security-screening domain. Initial materials-based coarse segmentations, generated using the Dual-Energy Index (DEI), are refined by partitioning at automatically detected regions. Partitioning is guided by a novel random forest based quality metric, trained to recognise high-quality, single-object segments. A second novel segmentation quality measure is presented for quantifying the quality of full segmentations based on the random forest metric of the constituent parts and the error in the number of objects segmented. In a comparative evaluation between the proposed approach and three state-of-the-art volumetric segmentation techniques designed for single-energy CT data (two region-growing [1,2] and one graph-based [3] ) our method is shown to outperform both region-growing methods in terms of segmentation quality and speed. Although the graph-based approach generates more accurate partitions, it is characterised by high processing times and is significantly outperformed by the proposed method in this regard. The observations made in this study indicate that the proposed segmentation technique is well-suited to the baggage security-screening domain, where the demand for computational efficiency is paramount to maximise throughput. [ABSTRACT FROM AUTHOR]

Abstract: We present an experimental comparison of 3D feature descriptors with application to threat detection in Computed Tomography (CT) airport baggage imagery. The detectors range in complexity from a basic local density descriptor, through local region histograms and three-dimensional (3D) extensions to both to the RIFT descriptor and the seminal SIFT feature descriptor. We show that, in the complex CT imagery domain containing a high degree of noise and imaging artefacts, a specific instance object recognition system using simpler descriptors appears to outperform a more complex RIFT/SIFT solution. Recognition rates in excess of 95% are demonstrated with minimal false-positive rates for a set of exemplar 3D objects. [Copyright &y& Elsevier]

Abstract: Typical stereo and laser scan based 3D acquisition approaches are essentially limited to 2.5D capture. The resulting 3D completion problem, to derive missing information in 2.5D scenes from limited contextual information, has received increasing attention in literature. Here we propose a hierarchical extension to our recent non-parametric approach for the 3D completion of surface relief detail to allow the resolution of inconsistencies arising in the global structure of an area completed with this technique. We test our approach over a range of surface types and contrast the presence of global discontinuities in the resulting completion with those of the earlier approach. [Copyright &y& Elsevier]

• Extending locality preserving projection (LPP) to multi-domain scenarios. • Proposing a cross-domain structure preserving projection (CDSPP) algorithm for heterogeneous domain adaptation (HDA). • A progressive pseudo-labelling strategy is proposed for semi-supervised HDA. • A new benchmark for HDA with significantly more classes than the existing ones is presented. • Extensive experiments are conducted to validate the effectiveness of our proposed method for both supervised and semi-supervised HDA. Heterogeneous Domain Adaptation (HDA) addresses the transfer learning problems where data from the source and target domains are of different modalities (e.g., texts and images) or feature dimensions (e.g., features extracted with different methods). It is useful for multi-modal data analysis. Traditional domain adaptation algorithms assume that the representations of source and target samples reside in the same feature space, hence are likely to fail in solving the heterogeneous domain adaptation problem. Contemporary state-of-the-art HDA approaches are usually composed of complex optimization objectives for favourable performance and are therefore computationally expensive and less generalizable. To address these issues, we propose a novel Cross-Domain Structure Preserving Projection (CDSPP) algorithm for HDA. As an extension of the classic LPP to heterogeneous domains, CDSPP aims to learn domain-specific projections to map sample features from source and target domains into a common subspace such that the class consistency is preserved and data distributions are sufficiently aligned. CDSPP is simple and has deterministic solutions by solving a generalized eigenvalue problem. It is naturally suitable for supervised HDA but has also been extended for semi-supervised HDA where the unlabelled target domain samples are available. Extensive experiments have been conducted on commonly used benchmark datasets (i.e. Office-Caltech, Multilingual Reuters Collection, NUS-WIDE-ImageNet) for HDA as well as the Office-Home dataset firstly introduced for HDA by ourselves due to its significantly larger number of classes than the existing ones (65 vs 10, 6 and 8). The experimental results of both supervised and semi-supervised HDA demonstrate the superior performance of our proposed method against contemporary state-of-the-art methods. [ABSTRACT FROM AUTHOR]

We address the Unsupervised Domain Adaptation (UDA) problem in image classification from a new perspective. In contrast to most existing works which either align the data distributions or learn domain-invariant features, we directly learn a unified classifier for both the source and target domains in the high-dimensional homogeneous feature space without explicit domain alignment. To this end, we employ the effective Selective Pseudo-Labelling (SPL) technique to take advantage of the unlabelled samples in the target domain. Surprisingly, data distribution discrepancy across the source and target domains can be well handled by a computationally simple classifier (e.g., a shallow Multi-Layer Perceptron) trained in the original feature space. Besides, we propose a novel generative model norm-AE to generate synthetic features for the target domain as a data augmentation strategy to enhance the classifier training. Experimental results on several benchmark datasets demonstrate the pseudo-labelling strategy itself can lead to comparable performance to many state-of-the-art methods whilst the use of norm-AE for feature augmentation can further improve the performance in most cases. As a result, our proposed methods (i.e. naive-SPL and norm-AE-SPL) can achieve comparable performance with state-of-the-art methods with the average accuracy of 93.4% and 90.4% on Office-Caltech and ImageCLEF-DA datasets, and achieve competitive performance on Digits, Office31 and Office-Home datasets with the average accuracy of 97.2%, 87.6% and 68.6% respectively. [ABSTRACT FROM AUTHOR]

This paper presents a novel noise robust edge detector based upon the automatic anisotropic Gaussian kernels (ANGKs), which also addresses the current problem that the seminal Canny edge detector may miss some obvious crossing edge details. Firstly, automatic ANGKs are designed according to the noise suppression, edge resolution and localization precision, which also conciliate the conflict between them. Secondly, reasons why cross-edge points are missing from Canny detector results using isotropic Gaussian kernel are analyzed. Thirdly, the automatic ANGKs are used to smooth image and a revised edge extraction method is used to extract edges. Finally, the aggregate test receiver-operating-characteristic (ROC) curves and Pratt's Figure of Merit (FOM) are used to evaluate the proposed detector against state-of-the-art edge detectors. The experiment results show that the proposed algorithm can obtain better performance for noise-free and noisy images. [ABSTRACT FROM AUTHOR]

We investigate the performance of a Bag of (Visual) Words (BoW) object classification model as an approach for automated threat object detection within 3D Computed Tomography (CT) imagery from a baggage security context. This poses a novel and unique challenge for rigid object classification within complex and cluttered volumetric imagery. Within this context it extends the BoW model to 3D transmission imagery (X-ray CT) from its conventional application in 2D reflectance (photographic) imagery. We explore combinations of four 3D feature descriptors (Density Histogram (DH), Density Gradient Histogram (DGH), Scale Invariant Feature Transform (SIFT) and Rotation Invariant Feature Transform (RIFT)), three codebook assignment methodologies (hard, kernel and uncertainty) and seven codebook sizes. Optimal performance is achieved using the DH and DGH descriptors in conjunction with an uncertainty assignment methodology. Successful detection rates in excess of 97% for handguns and 89% for bottles and false-positive rates of approximately 2–3% are achieved. We demonstrate that the underlying imaging modality and the irrelevance of illumination and scale invariance within the transmission imagery context considered here result in the favourable performance of simpler density histogram descriptors (DH, DGH) over 3D extensions of the well-established SIFT and RIFT feature descriptor approaches. [ABSTRACT FROM AUTHOR]

Rivers are among the world's most threatened ecosystems. Enabled by the rapid development of drone technology, hyperspatial resolution (<10 cm) images of fluvial environments are now a common data source used to better understand these sensitive habitats. However, the task of image classification remains challenging for this type of imagery and the application of traditional classification algorithms such as maximum likelihood, still in common use among the river remote sensing community, yields unsatisfactory results. We explore the possibility that a classifier of river imagery based on deep learning methods can provide a significant improvement in our ability to classify fluvial scenes. We assemble a dataset composed of RGB images from 11 rivers in Canada, Italy, Japan, the United Kingdom, and Costa Rica. The images were labelled into 5 land-cover classes: water, dry exposed sediment, green vegetation, senescent vegetation and roads. In total, >5 billion pixels were labelled and partitioned for the tasks of training (1 billion pixels) and validation (4 billion pixels). We develop a novel supervised learning workflow based on the NASNet convolutional neural network (CNN) called 'CNN-Supervised Classification' (CSC). First, we compare the classification performance of maximum likelihood, a multilayer perceptron, a random forest, and CSC. Results show median F1 scores (a commonly used quality metric in machine learning) of 71%, 78%, 72% and 95%, respectively. Second, we train our classifier using data for 5 of 11 rivers. We then predict the validation data for all 11 rivers. For the 5 rivers that were used in model training, median F1 scores reach 98%. For the 6 rivers not used in model training, median F1 scores are 90%. We reach two conclusions. First, in the traditional workflow where images are classified one at a time, CSC delivers an unprecedented mix of labour savings and classification F1 scores above 95%. Second, deep learning can predict land-cover classifications (F1 = 90%) for rivers not used in training. This demonstrates the potential to train a generalised open-source deep learning model for airborne river surveys suitable for most rivers 'out of the box'. Research efforts should now focus on further development of a new generation of deep learning classification tools that will encode human image interpretation abilities and allow for fully automated, potentially real-time, interpretation of riverine landscape images. Unlabelled Image • Deep Learning can classify RGB river imagery to 90%–99% F1. • This result exceeds the state-of-the-art in fluvial scene classification. • Deep Learning models can encode river features that transfer to new rivers. • Hyper- and multispectral data are not required. • We provide open source GIS integration via PyQGIS. [ABSTRACT FROM AUTHOR]

Highlights • Depth completion can be performed by learning the context and contents of the scene. • Depth holes can be predicted based on the scene and capture device characteristics. • Using absolute deviations loss in frequency domain (DCT) improves reconstruction. • Adversarial training (Wasserstein metric) can improve training and mode selection. • Using domain transfer, models trained on synthetic data are used in the real world. Graphical abstract Abstract In this work, the issue of depth filling is addressed using a self-supervised feature learning model that predicts missing depth pixel values based on the context and structure of the scene. A fully-convolutional generative model is conditioned on the available depth information and full RGB colour information from the scene and trained in an adversarial fashion to complete scene depth. Since ground truth depth is not readily available, synthetic data is instead used with a separate model developed to predict where holes would appear in a sensed (non-synthetic) depth image based on the contents of the RGB image. The resulting synthetic data with realistic holes is utilized in training the depth filling model which makes joint use of a reconstruction loss which employs the Discrete Cosine Transform for more realistic outputs, an adversarial loss which measures the distribution distances via the Wasserstein metric and a bottleneck feature loss that aids in better contextual feature execration. Additionally, the model is adversarially adapted to perform well on naturally-obtained data with no available ground truth. Qualitative and quantitative evaluations demonstrate the efficacy of the approach compared to contemporary depth filling techniques. The strength of the feature learning capabilities of the resulting deep network model is also demonstrated by performing the task of monocular depth estimation using our pre-trained depth hole filling model as the initialization for subsequent transfer learning. [ABSTRACT FROM AUTHOR]