Search

Person re-identification is a crucial task of identifying pedestrians of interest across multiple surveillance camera views. For person re-identification, a pedestrian is usually represented with features extracted from a rectangular image region that inevitably contains the scene background, which incurs ambiguity to distinguish different pedestrians and degrades the accuracy. Thus, we propose an end-to-end foreground-aware network to discriminate against the foreground from the background by learning a soft mask for person re-identification. In our method, in addition to the pedestrian ID as supervision for the foreground, we introduce the camera ID of each pedestrian image for background modeling. The foreground branch and the background branch are optimized collaboratively. By presenting a target attention loss, the pedestrian features extracted from the foreground branch become more insensitive to backgrounds, which greatly reduces the negative impact of changing backgrounds on pedestrian matching across different camera views. Notably, in contrast to existing methods, our approach does not require an additional dataset to train a human landmark detector or a segmentation model for locating the background regions. The experimental results conducted on three challenging datasets, i.e., Market-1501, DukeMTMC-reID, and MSMT17, demonstrate the effectiveness of our approach.

Extreme learning machine algorithm proposed in recent years has been widely used in many fields due to its fast training speed and good generalization performance. Unlike the traditional neural network, the ELM algorithm greatly improves the training speed by randomly generating the relevant parameters of the input layer and the hidden layer. However, due to the randomly generated parameters, some generated "bad" parameters may be introduced to bring negative effect on the final generalization ability. To overcome such drawback, this paper combines the artificial immune system (AIS) with ELM, namely, AIS-ELM. With the help of AIS's global search and good convergence, the randomly generated parameters of ELM are optimized effectively and efficiently to achieve a better generalization performance. To evaluate the performance of AIS-ELM, this paper compares it with relevant algorithms on several benchmark datasets. The experimental results reveal that our proposed algorithm can always achieve superior performance.

This article presents one-bit supervision, a novel setting of learning with fewer labels, for image classification. Instead of the training model using the accurate label of each sample, our setting requires the model to interact with the system by predicting the class label of each sample and learn from the answer whether the guess is correct, which provides one bit (yes or no) of information. An intriguing property of the setting is that the burden of annotation largely is alleviated in comparison to offering the accurate label. There are two keys to one-bit supervision: (i) improving the guess accuracy and (ii) making good use of the incorrect guesses. To achieve these goals, we propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm. Theoretical analysis shows that one-bit annotation is more efficient than full-bit annotation in most cases and gives the conditions of combining our approach with active learning. Inspired by this, we further integrate the one-bit supervision framework into the self-supervised learning algorithm, which yields an even more efficient training schedule. Different from training from scratch, when self-supervised learning is used for initialization, both hard example mining and class balance are verified to be effective in boosting the learning performance. However, these two frameworks still need full-bit labels in the initial stage. To cast off this burden, we utilize unsupervised domain adaptation to train the initial model and conduct pure one-bit annotations on the target dataset. In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision. [ABSTRACT FROM AUTHOR]

Differentiable architecture search (DARTS) transforms architectural optimization into a super network optimization by stacking two cells (2 c.). However, repeatedly stacking two cells is a suboptimal operation since cells in different depths should be various. Besides, we find that the performance is slightly improved by increasing the number of searched cells (e.g., from 2 c. to 5 c.), but it leads to uneven resource allocation. This letter proposes a regularized DARTS (RDARTS) to adjust the architectural differences and balance degrees of freedom and resource allocation. Specifically, we use separate architectural parameters for two reduction cells and three normal cells, and then propose an Reg distance to calculate the difference between cells. We design a new validation loss which is the weighting of cross-entropy and Reg loss and introduce an adaptive adjustment method. Results show that RDARTS achieves the top-1 accuracy of 97.64% and 75.8% on CIFAR and ImageNet. [ABSTRACT FROM AUTHOR]

Establishing a systematic molecular information analysis strategy for cell culture models is of great significance for drug development and tissue engineering technologies. Here, we fabricated single silver nanowires with high surface-enhanced Raman scattering activity to extract SERS spectra in situ from two-dimensional (2D) and three-dimensional (3D) cell culture models. The silver nanowires were super long, flexible and thin enough to penetrate through multiple cells. A single silver nanowire was used in combination with a four-dimensional microcontroller as a cell endoscope for spectrally analyzing the components in cell culture models. Then, we adopted a machine learning algorithm to analyze the obtained spectra. Our results show that the abundance of proteins differs significantly between the 2D and 3D models, and that nucleic acid-rich and protein-rich regions can be distinguished with satisfactory accuracy. [ABSTRACT FROM AUTHOR]

Cell division cycle-associated (CDCA) gene family members are essential cell proliferation regulators and play critical roles in various cancers. However, the function of the CDCA family genes in gliomas remains unclear. This study aims to elucidate the role of CDCA family members in gliomas using in vitro and in vivo experiments and bioinformatic analyses. We included eight glioma cohorts in this study. An unsupervised clustering algorithm was used to identify novel CDCA gene family clusters. Then, we utilized multi-omics data to elucidate the prognostic disparities, biological functionalities, genomic alterations, and immune microenvironment among glioma patients. Subsequently, the scRNA-seq analysis and spatial transcriptomic sequencing analysis were carried out to explore the expression distribution of CDCA2 in glioma samples. In vivo and in vitro experiments were used to investigate the effects of CDCA2 on the viability, migration, and invasion of glioma cells. Finally, based on ten machine-learning algorithms, we constructed an artificial intelligence-driven CDCA gene family signature called the machine learning-based CDCA gene family score (MLCS). Our results suggested that patients with the higher expression levels of CDCA family genes had a worse prognosis, more activated RAS signaling pathways, and more activated immunosuppressive microenvironments. CDCA2 knockdown inhibited the proliferation, migration, and invasion of glioma cells. In addition, the MLCS had robust and favorable prognostic predictive ability and could predict the response to immunotherapy and chemotherapy drug sensitivity. • An integration analysis of CDCA gene family in gliomas by multiple omics data. • Novel CDCA family clusters with distinct signaling, genomic features, TIME, and prognosis. • Experiments showed CDCA2 acted as an oncogene in gliomas by RAS signaling. • An artificial intelligence-driven CDCA family signature was developed. • The MLCS was a stable and robust biomarker for prognosis and immunotherapy. [ABSTRACT FROM AUTHOR]

Porosity produced in laser metal deposition hampers its application due to the absence of an effective prediction method. Measured thermal images of the melt pool provide a unique opportunity for porosity analytics. Furthermore, a physical model may provide complementary rich data that cannot be measured otherwise. How to leverage both types of data to predict porosity is very challenging. This paper presents a physics-driven deep learning model to predict porosity by integrating both measured and predicted data of the melt pool. The model fidelity is validated with the predicted pore occurrence and size with enhanced interpretability of Ti–6Al–4V thin-wall structures. [ABSTRACT FROM AUTHOR]

Diffusion process has advanced object retrieval greatly as it can capture the underlying manifold structure. Recent studies have experimentally demonstrated that tensor product diffusion can better reveal the intrinsic relationship between objects than other variants. However, the principle remains unclear, i.e., what kind of manifold structure is captured. In this paper, we propose a new affinity learning algorithm called Regularized Diffusion Process (RDP). By deeply exploring the properties of RDP, our first yet basic contribution is providing a manifold-based explanation for tensor product diffusion. A novel criterion measuring the smoothness of the manifold is defined, which simultaneously regularizes four vertices in the affinity graph. Inspired by this observation, we further contribute two variants towards two specific goals. While ARDP can learn similarities across heterogeneous domains, HRDP performs affinity learning on tensor product hypergraph, considering the relationships between objects are generally more complex than pairwise. Consequently, RDP, ARDP and HRDP constitute a generic tool for object retrieval in most commonly-used settings, no matter the input relationships between objects are derived from the same domain or not, and in pairwise formulation or not. Comprehensive experiments on 10 retrieval benchmarks, especially on large scale data, validate the effectiveness and generalization of our work. [ABSTRACT FROM AUTHOR]

The huge variance of human pose and the misalign-ment of detected human images significantly increase the difficulty of pedestrian image matching in person Re-Identification (Re-ID). Moreover, the massive visual data being produced by surveillance video cameras requires highly efficient person Re-ID systems. Targeting to solve the first problem, this work proposes a robust and discriminative pedestrian image descriptor, namely, the Global–Local-Alignment Descriptor (GLAD). For the second problem, this work treats person Re-ID as image retrieval and proposes an efficient indexing and retrieval framework. GLAD explicitly leverages the local and global cues in the human body to generate a discriminative and robust representation. It consists of part extraction and descriptor learning modules, where several part regions are first detected and then deep neural networks are designed for representation learning on both the local and global regions. A hierarchical indexing and retrieval framework is designed to perform offline relevance mining to eliminate the huge person ID redundancy in the gallery set, and accelerate the online Re-ID procedure. Extensive experimental results on widely used public benchmark datasets show GLAD achieves competitive accuracy compared to the state-of-the-art methods. On a large-scale person, with the Re-ID dataset containing more than 520 K images, our retrieval framework significantly accelerates the online Re-ID procedure while also improving Re-ID accuracy. Therefore, this work has the potential to work better on person Re-ID tasks in real scenarios. [ABSTRACT FROM AUTHOR]

This paper aims at learning discriminative part detectors with only image-level labels. To this end, we need to develop effective technologies for both pattern mining and detection learning. Different from previous methods, which train part detectors in one step, we divide the detector learning process into two stages and formulate it as a weak to strong learning framework. In particular, we first learn exemplar detectors from the unaligned patterns and perform a detector-based spectral clustering to produce weak detectors that are only responsible for a few discriminative patterns. In this way, the weak detectors are able to offer right initial patterns for strong detector learning. Second, we learn strong detectors with patterns discovered from the weak detectors, which we formulate as a confidence-loss sparse multiple instance learning (cls-MIL) task. The cls-MIL considers the diversity of positive samples while avoiding drifting away from the well localized ones by assigning a confidence value to each positive sample. The responses of the learned detectors produce an effective mid-level image representation for both image classification and object localization. Experiments conducted on benchmark data sets well demonstrate the superiority of our method over existing approaches. [ABSTRACT FROM AUTHOR]

As a post-processing procedure, the diffusion process has demonstrated its ability of substantially improving the performance of various visual retrieval systems. Whereas, great efforts are also devoted to similarity (or metric) fusion, seeing that only one individual type of similarity cannot fully reveal the intrinsic relationship between objects. This stimulates a great research interest of considering similarity fusion in the framework of the diffusion process (i.e., fusion with diffusion) for robust retrieval. In this paper, we first revisit representative methods about fusion with diffusion and provide new insights which are ignored by previous researchers. Then, observing that existing algorithms are susceptible to noisy similarities, the proposed regularized ensemble diffusion (RED) is bundled with an automatic weight learning paradigm, so that the negative impacts of noisy similarities are suppressed. Though formulated as a convex optimization problem, one advantage of RED is that it converts back into the iteration-based solver with the same computational complexity as the conventional diffusion process. At last, we integrate several recently-proposed similarities with the proposed framework. The experimental results suggest that we can achieve new state-of-the-art performances on various retrieval tasks, including 3D shape retrieval on the ModelNet data set, and image retrieval on the Holidays and Ukbench data sets. [ABSTRACT FROM AUTHOR]

Emotion recognition is challenging due to the emotional gap between emotions and audio–visual features. Motivated by the powerful feature learning ability of deep neural networks, this paper proposes to bridge the emotional gap by using a hybrid deep model, which first produces audio–visual segment features with Convolutional Neural Networks (CNNs) and 3D-CNN, then fuses audio–visual segment features in a Deep Belief Networks (DBNs). The proposed method is trained in two stages. First, CNN and 3D-CNN models pre-trained on corresponding large-scale image and video classification tasks are fine-tuned on emotion recognition tasks to learn audio and visual segment features, respectively. Second, the outputs of CNN and 3D-CNN models are combined into a fusion network built with a DBN model. The fusion network is trained to jointly learn a discriminative audio–visual segment feature representation. After average-pooling segment features learned by DBN to form a fixed-length global video feature, a linear Support Vector Machine is used for video emotion classification. Experimental results on three public audio–visual emotional databases, including the acted RML database, the acted eNTERFACE05 database, and the spontaneous BAUM-1s database, demonstrate the promising performance of the proposed method. To the best of our knowledge, this is an early work fusing audio and visual cues with CNN, 3D-CNN, and DBN for audio–visual emotion recognition. [ABSTRACT FROM AUTHOR]

We propose Multi-Task Learning with Low Rank Attribute Embedding (MTL-LORAE) to address the problem of person re-identification on multi-cameras. Re-identifications on different cameras are considered as related tasks, which allows the shared information among different tasks to be explored to improve the re-identification accuracy. The MTL-LORAE framework integrates low-level features with mid-level attributes as the descriptions for persons. To improve the accuracy of such description, we introduce the low-rank attribute embedding, which maps original binary attributes into a continuous space utilizing the correlative relationship between each pair of attributes. In this way, inaccurate attributes are rectified and missing attributes are recovered. The resulting objective function is constructed with an attribute embedding error and a quadratic loss concerning class labels. It is solved by an alternating optimization strategy. The proposed MTL-LORAE is tested on four datasets and is validated to outperform the existing methods with significant margins. [ABSTRACT FROM PUBLISHER]

• We enlarge search space by introducing the Cartesian product of operators. • We propose a factorized search to search separately in two subspaces. • We optimize architectural parameters sequentially, avoiding undesirable competition. • We perform diagnostic experiments and explanations, demonstrating our superiority. • Our approach achieves SOTA performance on CIFAR10, ImageNet, and COCO. Neural Architecture Search (NAS) replaces manually designed networks with automatically searched networks and has become a hot topic in machine learning and computer vision. One key factor of NAS, scaling up the search space by adding operators, could help bring about more possibilities for effective architectures. However, existing works require high search costs and are prone to make confused selections caused by competition among similar operators. This paper presents a scalable NAS that utilizes a factorized method, which improves existing art in the following aspects. (1) Our work explores a broader search space. We construct an ample search space through the cartesian product of activation operators and regular operators. (2) Our work experiences a limited computation burden even though searching in a more extensive search space. We factorize a search space into two subspaces and adopt different architectural parameters to control corresponding subspaces. (3) Our work avoids competition among similarly combined operators (e.g. , ReLU & sep-conv-3x3 and ReLU6 & sep-conv-3x3 and PReLU & sep-conv-3x3). That is because our architectural parameters are optimized sequentially. Experimental results show that our approach achieves SOTA on CIFAR10 (2. 38 % test error) and ImageNet (23. 8 % test error). Furthermore, the excellent performance of 35. 8 % (AP) and 55. 7 % ( AP 50 ) on COCO further proves the superiority of our factorized method. [ABSTRACT FROM AUTHOR]

Statistical-learning-based object detection is an important topic in computer vision. It learns visual representation from annotated exemplars to identify semantic defined objects in images. High-performance object detection is usually carried out in feature space and effective feature representation can improve the performance significantly. Feature representation is the encoding process which maps raw image pixels inside local regions into discriminant feature space. The motivation of this paper is to present a review on feature representation in recent object detection methods. Visual features applied in object detection are categorized according to the differences in computation and visual properties. The most valued features are introduced and discussed in detail. Representative extensions are introduced briefly for comparison. Descriptive power, robustness, compactness as well as computational efficiency are viewed as important properties. According to these properties, discussions are presented on the advantages and drawbacks of features. Besides, generic techniques such as dimension reduction and combination are introduced. Through this review, we would like to draw the feature sketch and provide new insights for feature utilization, in order to tackle future challenges of object detection. [ABSTRACT FROM AUTHOR]

Feature detection is a fundamental and important problem in computer vision and image processing. It is a low-level processing step which serves as the essential part for computer vision based applications. The goal of this paper is to present a survey of recent progress and advances in visual feature detection. Firstly we describe the relations among edges, corners and blobs from the psychological view. Secondly we classify the algorithms in detecting edges, corners and blobs into different categories and provide detailed descriptions for representative recent algorithms in each category. Considering that machine learning becomes more involved in visual feature detection, we put more emphasis on machine learning based feature detection methods. Thirdly, evaluation standards and databases are also introduced. Through this survey we would like to present the recent progress in visual feature detection and identify future trends as well as challenges. [ABSTRACT FROM AUTHOR]

Deep model-based semantic segmentation has received ever increasing research focus in recent years. However, due to the complex model architectures, existing works are still unable to achieve high accuracy in real-time applications. In this paper, we propose a novel Sequential Prediction Network (termed SPNet) to seek a better trade-off between accuracy and efficiency. SPNet is also an end-to-end encoder-decoder architecture, which introduces a sequential prediction method to spread the contextual information from the low-level layers to the high-level layers. Besides, the proposed method is equipped with a stream Spatial Semantic and Edge Loss (termed SEL) and an adversarial network at multiple resolutions, which greatly improves the segmentation accuracy with a negligible increase in computation cost. To further utilize the extra unlabeled data, we present a knowledge distillation scheme to distill the structured knowledge from cumbersome to compact networks. Without using any pre-trained model, our method achieves state-of-the-art performance among exiting real-time segmentation models on several challenging datasets. Impressively, on the Cityscapes test dataset, it obtains 75.8 % mIoU at a speed of 61.2 FPS. [ABSTRACT FROM AUTHOR]

Traditional Bag-of-Words (BoWs) model is commonly generated with many steps, including local feature extraction, codebook generation and feature quantization, etc. Those steps are relatively independent with each other and are hard to be jointly optimized. Moreover, the dependency on hand-crafted local feature makes BoWs model not effective in conveying high-level semantics. These issues largely hinder the performance of BoWs model in large-scale image applications. To conquer these issues, we propose an End-to-End BoWs (E2BoWs) model based on Deep Convolutional Neural Network (DCNN). Our model takes an image as input, then identifies and separates semantic objects in it, and finally outputs visual words with high semantic discriminative power. Specifically, our model firstly generates Semantic Feature Maps (SFMs) corresponding to different object categories through convolutional layers, then introduces Bag-of-Words Layers (BoWL) to generate visual words from each individual feature map. We also introduce a novel learning algorithm to reinforce the sparsity of the generated E2BoWs model, which further ensures the time and memory efficiency. We evaluate the proposed E2BoWs model on several image search datasets including MNIST, SVHN, CIFAR-10, CIFAR-100, MIRFLICKR-25K and NUS-WIDE. Experimental results show that our method achieves promising accuracy and efficiency compared with recent deep learning based retrieval works. [ABSTRACT FROM AUTHOR]

Due to the ability of capturing the geometry structure of data manifold, context-sensitive similarity has demonstrated impressive performances in the retrieval task. The key idea of context-sensitive similarity is that the similarity between two data points can be more reliably estimated with the local context of other points in the affinity graph. Therefore, neighborhood selection is a crucial factor for those algorithms, which affects the performance dramatically. In this paper, we propose a new algorithm called Smooth Neighborhood (SN) that mines the neighborhood structure to satisfy the manifold assumption. By doing so, nearby points on the underlying manifold are guaranteed to yield similar neighbors as much as possible. Moreover, SN is adjusted to tackle multiple affinity graphs by imposing a weight learning paradigm, and this is the primary difference compared with related works which are only applicable with one affinity graph. Finally, we integrate SN with Sparse Contextual Activation (SCA), a representative context-sensitive similarity proposed recently. Extensive experimental results and comparisons manifest that with the neighborhood structure generated by SN, the proposed framework can yield state-of-the-art performances on shape retrieval, image retrieval and 3D model retrieval. [ABSTRACT FROM AUTHOR]

• Multi-view Adaptive Semi-supervised Feature Selection framework is proposed. • The self-paced learning is applied to semi-supervised feature selection. • Multi-view learning is utilized to enhance the feature selection performance. • An iterative algorithm is proposed with convergence and complexity analysis. • Experiments show MASFS can obtain good semi-supervised feature selection performance. In recent decades, semi-supervised feature selection (SSFS) has gained extensive research in the field of machine learning and computer vision. Most SSFS algorithms are based on graph-based semi-supervised learning (GSSL), and their performance depend heavily on the quality of the Laplacian weight graph. However, the Laplacian weight graph can't be changed once it is constructed, which greatly restricts the performance of SSFS. To address this defect, in this paper we propose a novel Multi-view Adaptive Semi-supervised Feature Selection (MASFS) algorithm, which introduces the self-paced learning (SPL) into SSFS to make the Laplacian weight graph adaptively change according to the current predicted information. Meanwhile, the MASFS algorithm utilizes the multi-view learning to effectively explore the complementary and related information contained in different views to enhance SSFS performance. We propose a valid iterative algorithm for optimizing the objective function, followed by the convergence analysis and the complexity analysis. To illustrate the effectiveness of the MASFS algorithm, some experiments are carried out on NUS-WIDE dataset and MSRA-MM2.0 dataset and the experimental results indicate that MASFS has better performance than other SSFS algorithms based on GSSL. [ABSTRACT FROM AUTHOR]