Search

Mirror‐image proteins (D‐proteins) are useful in biomedical research for purposes such as mirror‐image screening for D‐peptide drug discovery, but the chemical synthesis of many D‐proteins is often low yielding due to the poor solubility or aggregation of their constituent peptide segments. Here, we report a Lys‐C protease‐cleavable solubilizing tag and its use to synthesize difficult‐to‐obtain D‐proteins. Our tag is easily installed onto multiple amino acids such as DLys, DSer, DThr, and/or the N‐terminal amino acid of hydrophobic D‐peptides, is impervious to various reaction conditions, such as peptide synthesis, ligation, desulfurization, and transition metal‐mediated deprotection, and yet can be completely removed by Lys‐C protease under denaturing conditions to give the desired D‐protein. The efficacy and practicality of the new method were exemplified in the synthesis of two challenging D‐proteins: D‐enantiomers of programmed cell death protein 1 IgV domain and SARS‐CoV‐2 envelope protein, in high yield. This work demonstrates that the enzymatic cleavage of solubilizing tags under denaturing conditions is feasible, thus paving the way for the production of more D‐proteins. [ABSTRACT FROM AUTHOR]

Mirror‐image proteins (D‐proteins) are useful in biomedical research for purposes such as mirror‐image screening for D‐peptide drug discovery, but the chemical synthesis of many D‐proteins is often low yielding due to the poor solubility or aggregation of their constituent peptide segments. Here, we report a Lys‐C protease‐cleavable solubilizing tag and its use to synthesize difficult‐to‐obtain D‐proteins. Our tag is easily installed onto multiple amino acids such as DLys, DSer, DThr, and/or the N‐terminal amino acid of hydrophobic D‐peptides, is impervious to various reaction conditions, such as peptide synthesis, ligation, desulfurization, and transition metal‐mediated deprotection, and yet can be completely removed by Lys‐C protease under denaturing conditions to give the desired D‐protein. The efficacy and practicality of the new method were exemplified in the synthesis of two challenging D‐proteins: D‐enantiomers of programmed cell death protein 1 IgV domain and SARS‐CoV‐2 envelope protein, in high yield. This work demonstrates that the enzymatic cleavage of solubilizing tags under denaturing conditions is feasible, thus paving the way for the production of more D‐proteins. [ABSTRACT FROM AUTHOR]

Membrane‐associated D‐proteins are an important class of synthetic molecules needed for D‐peptide drug discovery, but their chemical synthesis using canonical ligation methods such as native chemical ligation is often hampered by the poor solubility of their constituent peptide segments. Here, we describe a Backbone‐Installed Split Intein‐Assisted Ligation (BISIAL) method for the synthesis of these proteins, wherein the native L‐forms of the N‐ and C‐intein fragments of the unique consensus‐fast (Cfa) (i.e. L–CfaN and L–CfaC) are separately installed onto the two D‐peptide segments to be ligated via a removable backbone modification. The ligation proceeds smoothly at micromolar (μM) concentrations under strongly chaotropic conditions (8.0 M urea), and the subsequent removal of the backbone modification groups affords the desired D‐proteins without leaving any "ligation scar" on the products. The effectiveness and practicality of the BISIAL method are exemplified by the synthesis of the D‐enantiomers of the extracellular domains of T cell immunoglobulin and ITIM domain (TIGIT) and tropomyosin receptor kinase C (TrkC). The BISIAL method further expands the chemical protein synthesis ligation toolkit and provides practical access to challenging D‐protein targets. [ABSTRACT FROM AUTHOR]

Membrane‐associated D‐proteins are an important class of synthetic molecules needed for D‐peptide drug discovery, but their chemical synthesis using canonical ligation methods such as native chemical ligation is often hampered by the poor solubility of their constituent peptide segments. Here, we describe a Backbone‐Installed Split Intein‐Assisted Ligation (BISIAL) method for the synthesis of these proteins, wherein the native L‐forms of the N‐ and C‐intein fragments of the unique consensus‐fast (Cfa) (i.e. L–CfaN and L–CfaC) are separately installed onto the two D‐peptide segments to be ligated via a removable backbone modification. The ligation proceeds smoothly at micromolar (μM) concentrations under strongly chaotropic conditions (8.0 M urea), and the subsequent removal of the backbone modification groups affords the desired D‐proteins without leaving any "ligation scar" on the products. The effectiveness and practicality of the BISIAL method are exemplified by the synthesis of the D‐enantiomers of the extracellular domains of T cell immunoglobulin and ITIM domain (TIGIT) and tropomyosin receptor kinase C (TrkC). The BISIAL method further expands the chemical protein synthesis ligation toolkit and provides practical access to challenging D‐protein targets. [ABSTRACT FROM AUTHOR]

Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&E-stained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67-stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis. [ABSTRACT FROM AUTHOR]

Representation learning is a fundamental but challenging problem, especially when the distribution of data is unknown. In this paper, we propose a new representation learning method, named Structure Transfer Machine (STM), which enables feature learning process to converge at the representation expectation in a probabilistic way. We theoretically show that such an expected value of the representation (mean) is achievable if the manifold structure can be transferred from the data space to the feature space. The resulting structure regularization term, named manifold loss, is incorporated into the loss function of the typical deep learning pipeline. The STM architecture is constructed to enforce the learned deep representation to satisfy the intrinsic manifold structure from the data, which results in robust features that suit various application scenarios, such as digit recognition, image classification and object tracking. Compared with state-of-the-art CNN architectures, we achieve better results on several commonly used public benchmarks. [ABSTRACT FROM AUTHOR]

Cerebrovascular segmentation from time-of-flight magnetic resonance angiography (TOF-MRA) data is of great importance in blood supply structure analysis, diagnosis, and treatment of cerebrovascular pathologies. However, complete and accurate segmentation is still a challenge due to the complex image context and vascular morphology. The existing model-driven methods are often difficult to obtain prominent accuracy and robustness. Deep-learning based technology has achieved unimaginable success, but always faces the problem of insufficient labeled data. In this paper, a novel strategy is proposed to automate cerebrovascular segmentation, which integrates model- and data-driven methods. Firstly, the TOF-MRA data are sparsely labeled by three radiologists. Secondly, a semi-supervised mixture probability model is proposed to fit the cerebrovascular intensity distribution precisely, which starts from the sparse annotations and generates massive labeled points. Thirdly, mislabeled points are corrected by a Clean-Mechanism model, to acquire a well-labeled point-set of good quality. Finally, we construct and train a dilated dense convolution network (DD-CNN) by the resultant labeled point-set. The proposed method is validated on 109 clinical TOR-MRA data from a public dataset. Compared with the other state-of-the-art segmentation methods, our method segments cerebrovascular structure with better completeness and sensibility, especially for slender vascularity. The experimental results show that our method reaches an average dice score of 93.20%, which also indicates that the DD-CNN is very competent for cerebrovascular segmentation from TOF-MRA volume. [ABSTRACT FROM AUTHOR]

During the total chemical synthesis of the water‐soluble globular Haemophilus Influenzae DNA ligase (Hin‐Lig), we observed the surprising phenomenon of a soluble peptide segment that failed to undergo native chemical ligation. Based on dynamic light scattering and transmission electron microscopy experiments, we determined that the peptide formed soluble colloidal particles in a homogeneous solution containing 6 m guanidine hydrochloride. Conventional peptide performance‐improving strategies, such as installation of a terminal/side‐chain Arg tag or O‐acyl isopeptide, failed to enable the reaction, presumably because of their inability to disrupt the formation of soluble colloidal particles. However, a removable backbone modification strategy recently developed for the synthesis of membrane proteins did disrupt the formation of the colloids, and the desired ligation of this soluble but unreactive system was eventually accomplished. This work demonstrates that an appropriate solution dispersion state, in addition to good peptide solubility, is a prerequisite for successful peptide ligation. [ABSTRACT FROM AUTHOR]

During the total chemical synthesis of the water‐soluble globular Haemophilus Influenzae DNA ligase (Hin‐Lig), we observed the surprising phenomenon of a soluble peptide segment that failed to undergo native chemical ligation. Based on dynamic light scattering and transmission electron microscopy experiments, we determined that the peptide formed soluble colloidal particles in a homogeneous solution containing 6 m guanidine hydrochloride. Conventional peptide performance‐improving strategies, such as installation of a terminal/side‐chain Arg tag or O‐acyl isopeptide, failed to enable the reaction, presumably because of their inability to disrupt the formation of soluble colloidal particles. However, a removable backbone modification strategy recently developed for the synthesis of membrane proteins did disrupt the formation of the colloids, and the desired ligation of this soluble but unreactive system was eventually accomplished. This work demonstrates that an appropriate solution dispersion state, in addition to good peptide solubility, is a prerequisite for successful peptide ligation. [ABSTRACT FROM AUTHOR]

Abstract Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing. Most recent deep learning based methods have demonstrated superior performance over the previous hand-crafted methods. In this paper, we propose an end-to-end one-two-one (OTO) network, to combine different deep models, i.e., summation and difference models, to solve the CAR problem. Particularly, the difference model motivated by the Laplacian pyramid is designed to obtain the high frequency information, while the summation model aggregates the low frequency information. We provide an in-depth investigation into our OTO architecture based on the Taylor expansion, which shows that these two kinds of information can be fused in a nonlinear scheme to gain more capacity of handling complicated image compression artifacts, especially the blocking effect in compression. Extensive experiments are conducted to demonstrate the superior performance of the OTO networks, as compared to the state-of-the-arts on remote sensing datasets and other benchmark datasets. The source code will be available here: https://github.com/bczhangbczhang/. [ABSTRACT FROM AUTHOR]

In this paper, we leverage the manifold structure of visual data in order to improve performance in general optimization problems subject to linear constraints. As the main theoretical result, we show that manifold constraints can be transferred from the data to the optimized variables if these are linearly correlated. We also show that the resulting optimization problem can be solved with an efficient alternating direction method of multipliers that can consistently integrate the manifold constraints during the optimization process. This leads to a simplification of the approach, which instead of directly optimizing on the manifold, we can iteratively recast the problem as the projection over the manifold via an embedding method. The proposed method is extremely versatile since it can be applied to different problems including Kernel Ridge Regression (KRR) and sparse coding which have numerous applications in machine learning and computer vision. In particular, we apply the methods to different problems such as tracking, object recognition and categorization showing a consistent increase of performance with respect to the state of the art. [ABSTRACT FROM AUTHOR]

Correlation filters are special classifiers designed for shift-invariant object recognition, which are robust to pattern distortions. The recent literature shows that combining a set of sub-filters trained based on a single or a small group of images obtains the best performance. The idea is equivalent to estimating variable distribution based on the data sampling (bagging), which can be interpreted as finding solutions (variable distribution approximation) directly from sampled data space. However, this methodology fails to account for the variations existed in the data. In this paper, we introduce an intermediate step—solution sampling—after the data sampling step to form a subspace, in which an optimal solution can be estimated. More specifically, we propose a new method, named latent constrained correlation filters (LCCF), by mapping the correlation filters to a given latent subspace, and develop a new learning framework in the latent subspace that embeds distribution-related constraints into the original problem. To solve the optimization problem, we introduce a subspace-based alternating direction method of multipliers, which is proven to converge at the saddle point. Our approach is successfully applied to three different tasks, including eye localization, car detection, and object tracking. Extensive experiments demonstrate that LCCF outperforms the state-of-the-art methods.1 1

The source code will be publicly available. https://github.com/bczhangbczhang/

Human action recognition is an important yet challenging task. This paper presents a low-cost descriptor called 3D histograms of texture (3DHoTs) to extract discriminant features from a sequence of depth maps. 3DHoTs are derived from projecting depth frames onto three orthogonal Cartesian planes, i.e., the frontal, side, and top planes, and thus compactly characterize the salient information of a specific action, on which texture features are calculated to represent the action. Besides this fast feature descriptor, a new multi-class boosting classifier (MBC) is also proposed to efficiently exploit different kinds of features in a unified framework for action classification. Compared with the existing boosting frameworks, we add a new multi-class constraint into the objective function, which helps to maintain a better margin distribution by maximizing the mean of margin, whereas still minimizing the variance of margin. Experiments on the MSRAction3D, MSRGesture3D, MSRActivity3D, and UTD-MHAD data sets demonstrate that the proposed system combining 3DHoTs and MBC is superior to the state of the art. [ABSTRACT FROM PUBLISHER]

In this paper, we present a new strategy for modeling the motion of local patches for single-object tracking that can be seamlessly applied to most part-based trackers in the literature. The proposed adaptive local movement modeling method is able to model the local movement distribution of the image patches defining the object to track and the reliability of each image patch. Given the output of a base tracking algorithm, a Gaussian mixture model (GMM) is first used to model the distribution of the movement of local patches relative to the center of gravity of the tracked object. Then, the GMM is combined with the chosen base tracker in a boosting framework, which gives an efficient integrated scheme for the tracking task. This provides a robust procedure to detect outliers in the local motion of the patches. The algorithm is highly configurable with the possibility to change the number of local patches used for tracking and to adapt to the variations of the tracked object. The extensive tracking results on standard data sets show that equipping state-of-the-art trackers with our technique remarkably improves their performance. [ABSTRACT FROM PUBLISHER]

The kernelized correlation filter (KCF) is one of the state-of-the-art object trackers. However, it does not reasonably model the distribution of correlation response during tracking process, which might cause the drifting problem, especially when targets undergo significant appearance changes due to occlusion, camera shaking, and/or deformation. In this paper, we propose an output constraint transfer (OCT) method that by modeling the distribution of correlation response in a Bayesian optimization framework is able to mitigate the drifting problem. OCT builds upon the reasonable assumption that the correlation response to the target image follows a Gaussian distribution, which we exploit to select training samples and reduce model uncertainty. OCT is rooted in a new theory which transfers data distribution to a constraint of the optimized variable, leading to an efficient framework to calculate correlation filters. Extensive experiments on a commonly used tracking benchmark show that the proposed method significantly improves KCF, and achieves better performance than other state-of-the-art trackers. To encourage further developments, the source code is made available. [ABSTRACT FROM AUTHOR]

This paper presents a new deep learning architecture for robust object representation, aiming at efficiently combining the proposed synchronized multi-stage feature (SMF) and a boosting-like algorithm. The SMF structure can capture a variety of characteristics from the inputting object based on the fusion of the handcraft features and deep learned features. With the proposed boosting-like algorithm, we can obtain more convergence stability on training multi-layer network by using the boosted samples. We show the generalization of our object representation architecture by applying it to undertake various tasks, i.e. pedestrian detection and action recognition. Our approach achieves 15.89% and 3.85% reduction in the average miss rate compared with ACF and JointDeep on the largest Caltech dataset, and acquires competitive results on the MSRAction3D dataset. [ABSTRACT FROM AUTHOR]

This paper proves the uncertainty bound for the multiple Gaussian functions, termed multiple Gaussians Uncertainty (MGU), which significantly generalizes the uncertainty principle for the single Gaussian function. First, as a theoretical contribution, we prove that the momentum (velocity) and position for the sum of multiple Gaussians wave function are theoretically bounded. Second, as for a practical application, we show that the bound can be well exploited for object tracking to detect anomalies of local movement in an online learning framework. By integrating MGU with a given object tracker, we demonstrate that uncertainty principle can provide remarkable robustness in tracking. Extensive experiments are done to show that the proposed MGU can significantly help base trackers overcome the object drifting and reach state-of-the-art results. [ABSTRACT FROM AUTHOR]

Sparse representation classifier (SRC) is the state-of-the-art method, and the theory of SRC has interesting links to compressed sensing. This paper proposes a new method named Sparse Regression Analysis (SRA) for object representation and recognition. In SRA, the L1-norm minimization method is combined with regression process to represent the input signal. We show that the discriminative ability of SRC and SRA derives from the fact that the subset which most compactly expresses the input signal is activated in the regression analysis, and SRA is a more direct and powerful way to use compressed sensing for the recognition tasks. To achieve a further improvement, kernelized SRA (KSRA) is developed to make a nonlinear extension of SRA. The experiments are extensively conducted on both palmprint and face recognition, which show that the proposed methods achieve a much better performance than sparse representation classifier, the linear regression classification method, principal component analysis, kernel discriminant analysis, and linear discriminant analysis. [ABSTRACT FROM AUTHOR]

This paper proposes a novel Compressed Sensing Ensemble Classifier (CSEC) for human detection. The proposed CSEC employs the compressed sensing technique to get a more sparse model with a more reasonable selection of base classifiers. The major contributions of this paper are: (1) a new principled framework for ensemble classifier design based on compressed sensing; (2) a new concept of considering both the simplicity of ensemble classifier and irrelevance of base classifiers towards optimal classifier design; and (3) a quadratic function for CSEC optimization which includes a new optimizable positive semi-definite relevance matrix to simultaneously select appropriate base classifiers with minimized relevance. Experimental results on INRIA and SDL databases show that the performance of CSEC is better than two most popular classifiers SVM and AdaBoost, as well as a most recent method CLML. [ABSTRACT FROM AUTHOR]

A simple, specific and highly sensitive assay using liquid chromatography–tandem mass spectrometry with electrospray ionization was developed for the determination of olprinone in human plasma. Following the addition of codeine as an internal standard and simple liquid–liquid extraction with ethyl acetate–isopropanol (95:5, v/v), the analytes were separated on a Zorbax SB-C18 column (4.6 × 150 mm i.d., 5 µm) and eluted with a mobile phase consisting of methanol–10 mmol/L ammonium acetate containing 1% formic acid (50:50, v/v) at a flow rate of 1.0 mL/min. Detection was achieved with electrospray positive ionization mass spectrometry using multiple reaction monitoring mode. Linear calibration curves were obtained in the concentration range of 0.1–100 ng/mL (r = 0.9995), with a lower limit of quantification of 0.1 ng/mL. The intra-day and inter-day precision values, as relative deviation, were below 6.06% and the accuracy, as relative error, was below 9.92% at all quality control concentrations. The method was applicable to clinical pharmacokinetic study of an olprinone hydrochloride hydrate injection in healthy Chinese volunteers. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a new learning algorithm, named as the Cooperative and Geometric Learning Algorithm (CGLA), to solve problems of maneuverability, collision avoidance and information sharing in path planning for Unmanned Aerial Vehicles (UAVs). The contributions of CGLA are three folds: (1) CGLA is designed for path planning based on cooperation of multiple UAVs. Technically, CGLA exploits a new defined individual cost matrix, which leads to an efficient path planning algorithm for multiple UAVs. (2) The convergence of the proposed algorithm for calculating the cost matrix is proven theoretically, and the optimal path in terms of path length and risk measure from a starting point to a target point can be calculated in polynomial time. (3) In CGLA, the proposed individual weight matrix can be efficiently calculated and adaptively updated based on the geometric distance and risk information shared among UAVs. Finally, risk evaluation is introduced first time in this paper for UAV navigation and extensive computer simulation results validate the effectiveness and feasibility of CGLA for safe navigation of multiple UAVs. [Copyright &y& Elsevier]

Abstract: This paper proposes a new Weighted Margin Sparse Embedded (WMSE) classifier for brake cylinder detection, which is a big challenge in Trouble of Freight car Detection System (TFDS) of China. The major contributions of this paper are in three folds. (1) The proposed method is a combination of the Sparse Embedded (SE) and the Weighted Margin LearningS (WML) models, which are iteratively performed toward optimal classifier ensemble. The final classifier in cascades takes advantages of VC-dimension minimization and weighted margin learning, which provides a new investigation into the literature of classifier design. (2) Convergence of the WMSE classifier is theoretically proven, which is a desirable characteristic for object detection due to existence of large-scale training datasets in real applications. (3)To evaluate the performance of the proposed method, we establish and distribute the challenging BeiHang Brake Cylinder (BH-BC) Database containing over 2000 annotated brake cylinder images with various appearances and almost indistinguishable backgrounds. Comparative experimental results on the BH-BC database show that our approach can get a much higher detection performance than the state-of-the-art classifiers (Support Vector Machine and Adaboost). [Copyright &y& Elsevier]

We present a flexible, general framework for few-shot learning where both inter-class differences and intra-class relationships are fully considered to improve recognition performance significantly. We introduce complex-valued convolutional neural networks (CNNs) to describe the subtle difference among inter-class samples and Dependable Learning to capture the intra-class relationship. Conventional CNNs use only real-valued CNNs and fail to extract more detailed information. Complex-valued CNNs, on the other hand, can provide amplitude and phase information to enhance the feature representation ability based on the proposed complex metric module (CMM). Building upon the recent episodic training mechanism, CMMs can improve the representation capacity by extracting robust complex-valued features to facilitate the modeling of subtle relationships among few-shot samples. Furthermore, we use Dependable Learning as a new learning paradigm, to promote a robust model against perturbation based on a new bilinear optimization to enhance the feature extraction capacity for very few available intra-class samples. Experiments on two benchmark datasets show that the proposed methods significantly improve the performance over other approaches and achieve state-of-the-art results. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes a high-order Texture Pattern Flow (TPF) for complex background modeling and motion detection. The pattern flow is proposed to encode the binary pattern changes among the neighborhoods in the space–time domain. To model the distribution of the TPF pattern flow, the TPF integral histograms are used to extract the discriminative features to represent the input video. The Gaussian Mixture Model (GMM) is exploited to calculate an adaptive threshold in propagation way for the histogram similarity measure to decide which part/pixel is background or moving object. Experimental results on the public databases testify the effectiveness of the proposed method in comparison to LBP and GMM based background modeling methods. [Copyright &y& Elsevier]

This paper proposes a novel kernel similarity modeling of texture pattern flow (KSM-TPF) for background modeling and motion detection in complex and dynamic environments. The texture pattern flow encodes the binary pattern changes in both spatial and temporal neighborhoods. The integral histogram of texture pattern flow is employed to extract the discriminative features from the input videos. Different from existing uniform threshold based motion detection approaches which are only effective for simple background, the kernel similarity modeling is proposed to produce an adaptive threshold for complex background. The adaptive threshold is computed from the mean and variance of an extended Gaussian mixture model. The proposed KSM-TPF approach incorporates machine learning method with feature extraction method in a homogenous way. Experimental results on the publicly available video sequences demonstrate that the proposed approach provides an effective and efficient way for background modeling and motion detection. [ABSTRACT FROM PUBLISHER]

Abstract: This paper proposes a new Local Kernel Feature Analysis (LKFA) method for object recognition. LKFA captures the nonlinear local relationship in an image via kernel functions. Different from traditional kernel methods for object recognition, the proposed method does not need to reserve the training samples. LKFA is designed to extract the eigenvalue features from the Hermite matrix of a local feature representation, which we have theoretically proven its robustness to noise and perturbations. Experiment results on palmprint and face recognitions demonstrated the effectiveness of the proposed LKFA that significantly improved the performance of the local feature based object recognition method. [Copyright &y& Elsevier]

Abstract: This paper introduces the establishment of PolyU near-infrared face database (PolyU-NIRFD) and presents a new coding scheme for face recognition. The PolyU-NIRFD contains images from 350 subjects, each contributing about 100 samples with variations of pose, expression, focus, scale, time, etc. In total, 35,000 samples were collected in the database. The PolyU-NIRFD provides a platform for researchers to develop and evaluate various near-infrared face recognition techniques under large scale, controlled and uncontrolled conditions. A new coding scheme, namely directional binary code (DBC), is then proposed for near-infrared face recognition. Finally, we provide three protocols to evaluate and compare the proposed DBC method with baseline face recognition methods, including Gabor based Eigenface, Fisherface and LBP (local binary pattern) on the PolyU-NIRFD database. In addition, we also conduct experiments on the visible light band FERET database to further validate the proposed DBC scheme. [Copyright &y& Elsevier]

This paper proposes a novel high-order local pattern descriptor, local derivative pattern (LDP), for face recognition. LDP is a general framework to encode directional pattern features based on local derivative variations. The nth-order LDP is proposed to encode the (n -- 1)th-order local derivative direction variations, which can capture more detailed information than the first-order local pattern used in local binary pattern (LBP). Different from LBP encoding the relationship between the central point and its neighbors, the LDP templates extract high-order local information by encoding various distinctive spatial relationships contained in a given local region. Both gray-level images and Gabor feature images are used to evaluate the comparative performances of LDP and LBP. Extensive experimental results on FERET, CAS-PEAL, CMU-PIE, Extended Yale B, and FRGC databases show that the high-order LDP consistently performs much better than LBP for both face identification and face verification under various conditions. [ABSTRACT FROM AUTHOR]

Abstract: In order to study the effect of styrene–butadiene–rubber/montmorillonite (SBR/MMT) modification on the characteristics and properties of asphalt, modified asphalts were prepared by incorporating MMT into SBR and mixing this into asphalt. The X-ray diffraction (XRD) results show that SBR/MMT composites may form an intercalated structure and SBR/MMT modified asphalts may form an exfoliated structure. The morphologies of the samples have been characterized by fluorescent microscopy. The results indicate that SBR/MMT modified asphalts formed an ideal fine network structure. The addition of SBR/MMT to asphalt increased both the softening point and viscosity and decreased the penetration of the modified asphalts at high temperatures. High-temperature storage stability tests indicated that the modified asphalts are very stable in some SRB/MMT, SBR/MMT content range. Especially, the modified asphalts exhibited higher complex modulus (G ∗) and lower damping factor (tan δ). It implies that SBR/MMT displays improved viscoelastic properties, resulting in enhancing its resistance to rutting at high temperature. [Copyright &y& Elsevier]

Self-supervised monocular depth estimation methods have become the focus of research since ground truth data are not required. Current single-image-based works only leverage appearance-based features, thus achieving a limited performance. Deep learning based multiview stereo works facilitate the research on multi-frame depth estimation methods. Some multi-frame methods build cost volumes and take multiple frames as inputs at the time of test to fully utilize geometric cues between adjacent frames. Nevertheless, low-textured regions, which are dominant in indoor scenes, tend to cause unreliable depth hypotheses in the cost volume. Few self-supervised multi-frame methods have been used to conduct research on the issue of low-texture areas in indoor scenes. To handle this issue, we propose SIM-MultiDepth, a self-supervised indoor monocular multi-frame depth estimation framework. A self-supervised single-frame depth estimation network is introduced to learn the relative poses and supervise the multi-frame depth learning. A texture-aware depth consistency loss is designed considering the calculation of the patch-based photometric loss. Only the areas where multi-frame depth prediction is considered unreliable in low-texture regions are supervised by the single-frame network. This approach helps improve the depth estimation accuracy. The experimental results on the NYU Depth V2 dataset validate the effectiveness of SIM-MultiDepth. The zero-shot generalization studies on the 7-Scenes and Campus Indoor datasets aid in the analysis of the application characteristics of SIM-MultiDepth. [ABSTRACT FROM AUTHOR]

Crowd counting that aims to estimate the crowd density has recently made significant progress but remains an unsolved problem due to several challenges. In this paper, we propose an Attentive Encoder-Decoder Network (AEDNet) to overcome the notorious scale-variation problem in crowd counting. Our major contributions can be summarized in three aspects. First, we design an Attentive Feature Refinement (AFR) block in the encoder to adaptively extract multi-scale features. AFR compares the spatial information in different scales through the attention mechanism and then adaptively assign importance weights to each point, which highlights the distinctive roles in multi-scale feature extraction. Second, we develop a Separable Non-local Fusion (SNF) block in the decoder with the self-attention mechanism to aggregate multi-scale features from different layers, which not only achieves the sufficient feature fusion by capturing long-range dependencies, but also vastly reduces the computation cost compared to the original non-local operation. Third, we propose a Regional MSE (R-MSE) loss to tackle the pixel-isolation problems in regular MSE loss. To demonstrate the effectiveness of the proposed AEDNet, we conduct extensive experiments on four widely-used crowd counting datasets, and our AEDNet consistently achieves the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) have dominated as the best performers on almost all computer vision tasks over the past several years. However, it remains a major challenge to deploy these powerful DCNNs in resource-limited environments, such as embedded devices and smartphones. To this end, 1-bit CNNs have emerged as a feasible solution as they are much more resource-efficient. Unfortunately, they often suffer from a significant performance drop compared to their full-precision counterparts. In this paper, we propose a novel Bayesian Optimized compact 1-bit CNNs (BONNs) model, which has the advantage of Bayesian learning, to improve the performance of 1-bit CNNs significantly. BONNs incorporate the prior distributions of full-precision kernels, features, and filters into a Bayesian framework to construct 1-bit CNNs in a comprehensive end-to-end manner. The proposed Bayesian learning algorithms are well-founded and used to optimize the network simultaneously in different kernels, features, and filters, which largely improves the compactness and capacity of 1-bit CNNs. We further introduce a new Bayesian learning-based pruning method for 1-bit CNNs, which significantly increases the model efficiency with very competitive performance. This enables our method to be used in a variety of practical scenarios. Extensive experiments on the ImageNet, CIFAR, and LFW datasets show that BONNs achieve the best in classification performance compared to a variety of state-of-the-art 1-bit CNN models. In particular, BONN achieves a strong generalization performance on the object detection task. [ABSTRACT FROM AUTHOR]

Traditional neural architecture search (NAS) has a significant impact in computer vision by automatically designing network architectures for various tasks. In this paper, binarized neural architecture search (BNAS), with a search space of binarized convolutions, is introduced to produce extremely compressed models to reduce huge computational cost on embedded devices for edge computing. The BNAS calculation is more challenging than NAS due to the learning inefficiency caused by optimization requirements and the huge architecture space, and the performance loss when handling the wild data in various computing applications. To address these issues, we introduce operation space reduction and channel sampling into BNAS to significantly reduce the cost of searching. This is accomplished through a performance-based strategy that is robust to wild data, which is further used to abandon less potential operations. Furthermore, we introduce the upper confidence bound to solve 1-bit BNAS. Two optimization methods for binarized neural networks are used to validate the effectiveness of our BNAS. Extensive experiments demonstrate that the proposed BNAS achieves a comparable performance to NAS on both CIFAR and ImageNet databases. An accuracy of 96.53% vs. 97.22% is achieved on the CIFAR-10 dataset, but with a significantly compressed model, and a 40% faster search than the state-of-the-art PC-DARTS. On the wild face recognition task, our binarized models achieve a performance similar to their corresponding full-precision models. [ABSTRACT FROM AUTHOR]

Neural architecture search (NAS) proves to be among the effective approaches for many tasks by generating an application-adaptive neural architecture, which is still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binary weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework, while searching the 1-bit CNNs is more challenging due to the more complicated processes involved. In this paper, we introduce Discrepant Child–Parent Neural Architecture Search (DCP–NAS) to efficiently search 1-bit CNNs, based on a new framework of searching the 1-bit model (Child) under the supervision of a real-valued model (Parent). Particularly, we first utilize a Parent model to calculate a tangent direction, based on which the tangent propagation method is introduced to search the optimized 1-bit Child. We further observe a coupling relationship between the weights and architecture parameters existing in such differentiable frameworks. To address the issue, we propose a decoupled optimization method to search an optimized architecture. Extensive experiments demonstrate that our DCP–NAS achieves much better results than prior arts on both CIFAR-10 and ImageNet datasets. In particular, the backbones achieved by our DCP–NAS achieve strong generalization performance on person re-identification and object detection. [ABSTRACT FROM AUTHOR]

Video segmentation is a task of partitioning pixels that exhibit homogeneous appearance and motion into coherent spatial–temporal groups, which is still challenging for aerial applications. In this paper, a principled combination of superpixel labeling priors and Markov random field (S-MRF) is proposed for aerial video segmentation. The proposed approach has several contributions: 1) we develop a metadata-based global projection model with coordinate transformation to estimate motion information between frames; 2) the superpixel labeling priors from previous frames are incorporated into the segmentation of the current frame, leading to a highly efficient probabilistic label propagation algorithm; and 3) we perform an MRF optimization on the initial segments with propagated labeling priors to improve the temporal coherency. In addition, a new video dataset is collected and will be made publicly available to evaluate the performance of aerial video segmentation algorithms. The experimental results show that the proposed approach outperforms the state-of-the-art video segmentation methods. [ABSTRACT FROM AUTHOR]

Object tracking is challenged by the varying appearances of targets and the real‐time requirement. Siamese regression trackers, being one of the most popular tracking paradigms, excel in efficiency but suffer at adaptability to cope with appearance variations. To improve their adaptability, the authors propose a new adaptive Siamese (ASiam) tracker, which integrates a novel adversarial template generation module and a motion‐based failure recovery module. The template generation module exploits the temporal coherence and evolution of target appearance variations encoded in preceding tracklets and then generates an adaptive target template online which approximates the varying target in the coming frame. This generation module is optimised via adversarial learning to achieve accurate appearance prediction and sharp template quality. The generated template, together with a search region, are fed into a Siamese tracking backbone to compute an appearance response map via dense similarity computation in a sliding‐window way. At frames where the Siamese tracking fails, the failure recovery module is invoked to perform deep frame differencing motion detection to provide a motion response map. By fusing different response maps, the drifted tracker can be re‐calibrated. Extensive experiments on the OTB2013, OTB2015, and VOT2016 datasets prove the accuracy and efficiency of the proposed tracker. [ABSTRACT FROM AUTHOR]

Neural Architecture Search (NAS) is a highly challenging task that requires consideration of search space, search efficiency, and adversarial robustness of the network. In this paper, to accelerate the training speed, we reformulate NAS as a multi-armed bandit problem and present Anti-Bandit NAS (ABanditNAS) method, which exploits Upper Confidence Bounds (UCB) to abandon arms for search efficiency and Lower Confidence Bounds (LCB) for fair competition between arms. Based on the presented ABanditNAS, the adversarially robust optimization and architecture search can be solved in a unified framework. Specifically, our proposed framework defends against adversarial attacks based on a comprehensive search of denoising blocks, weight-free operations, Gabor filters, and convolutions. The theoretical analysis on the rationality of the two confidence bounds in ABanditNAS are provided and extensive experiments on three benchmarks are conducted. The results demonstrate that the presented ABanditNAS achieves competitive accuracy at a reduced search cost compared to prior methods. [ABSTRACT FROM AUTHOR]

Abstract Developing flexible and deformable supercapacitors electrodes based on porous materials is of high interest in energy related fields. We propose a novel approach called "soaking-squeezing-calcining" to fabricate a unique carbon nanotubes melamine sponge electrode, and demonstrate its application as a highly compressible supercapacitor electrode with high performance. Our supercapacitor electrode consists of three-dimensional melamine sponge as flexible skeleton and conductive skeleton, the carbon nanotubes filled in the sponge skeleton by physical adsorption and calcination process to enhance the overall conductivity and mechanical properties. After adding carbon nanotubes, the volume specific capacity of the supercapacitors increases by 3 times and the mass specific capacity is basically unchanged. Furthermore, it is found that its capacity retention rate could still reach 91% after the device is compressed by 50% for 1000 cycles of charging and discharging and 88% in the original uncompressed state for 5000 cycles of charging and discharging, respectively, which deeply proves that the carbon nanotubes reinforced sponge electrode has excellent electrochemical stability. Most importantly, the mechanical strength of the electrode increases by 5 times compared with that without carbon nanotubes. The results indicate that this electrode has the potential to fabricate deformable, robust supercapacitors with stable performance and many other energy devices. Highlights • Volume specific capacity increases by 3 times compared with no carbon nanotubes. • Capacity retention rate could reach 91% after the device was compressed by 50%. • Capacity retention rate could reach 88% for 5000 cycles of charging and discharging. • Mechanical strength raises 5 times after adding carbon nanotubes. [ABSTRACT FROM AUTHOR]

In steerable filters, a filter of arbitrary orientation can be generated by a linear combination of a set of “basis filters.” Steerable properties dominate the design of the traditional filters, e.g., Gabor filters and endow features the capability of handling spatial transformations. However, such properties have not yet been well explored in the deep convolutional neural networks (DCNNs). In this paper, we develop a new deep model, namely, Gabor convolutional networks (GCNs or Gabor CNNs), with Gabor filters incorporated into DCNNs such that the robustness of learned features against the orientation and scale changes can be reinforced. By manipulating the basic element of DCNNs, i.e., the convolution operator, based on Gabor filters, GCNs can be easily implemented and are readily compatible with any popular deep learning architecture. We carry out extensive experiments to demonstrate the promising performance of our GCNs framework, and the results show its superiority in recognizing objects, especially when the scale and rotation changes take place frequently. Moreover, the proposed GCNs have much fewer network parameters to be learned and can effectively reduce the training complexity of the network, leading to a more compact deep learning model while still maintaining a high feature representation capacity. The source code can be found at https://github.com/bczhangbczhang. [ABSTRACT FROM AUTHOR]

Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints. The architectures searched by our method achieve remarkable top-1 accuracies, 97.56 and 77.2 on CIFAR-10 and ImageNet (mobile settings), respectively, with the fastest search process, i.e., only 1.8 GPU hours on a Tesla V100. Codes for searching and network generation are available at: https://openi.pcl.ac.cn/PCL_AutoML/XNAS. [ABSTRACT FROM AUTHOR]

Gesture recognition becomes a popular analytics tool for extracting the characteristics of user movement and enables numerous practical applications in the biometrics field. Despite recent advances in this technique, complex user interaction and the limited amount of data pose serious challenges to existing methods. In this paper, we present a novel approach for hand gesture recognition based on user interaction on mobile devices. We have developed two deep models by integrating Bidirectional Long-Short Term Memory (BiLSTM) network and Bidirectional Gated Recurrent Unit (BiGRU) with Fisher criterion, termed as F-BiLSTM and F-BiGRU respectively. These two Fisher discriminative models can classify user’s gesture effectively by analyzing the corresponding acceleration and angular velocity data of hand motion. In addition, we build a large Mobile Gesture Database (MGD) containing 5547 sequences of 12 gestures. With extensive experiments, we demonstrate the superior performance of the proposed method compared to the state-of-the-art BiLSTM and BiGRU on MGD database and two other benchmark databases ( i.e. , BUAA mobile gesture and SmartWatch gesture). The source code and MGD database will be made publicly available at https://github.com/bczhangbczhang/Fisher-Discriminant-LSTM. [ABSTRACT FROM AUTHOR]

Deep learning has recently attracted much attention due to its excellent performance in processing audio, image, and video data. However, few studies are devoted to the field of automatic modulation classification (AMC). It is one of the most well-known research topics in communication signal recognition and remains challenging for traditional methods due to complex disturbance from other sources. This paper proposes a heterogeneous deep model fusion (HDMF) method to solve the problem in a unified framework. The contributions include the following: (1) a convolutional neural network (CNN) and long short-term memory (LSTM) are combined by two different ways without prior knowledge involved; (2) a large database, including eleven types of single-carrier modulation signals with various noises as well as a fading channel, is collected with various signal-to-noise ratios (SNRs) based on a real geographical environment; and (3) experimental results demonstrate that HDMF is very capable of coping with the AMC problem, and achieves much better performance when compared with the independent network. [ABSTRACT FROM AUTHOR]

aerial scene recognition. A novel superpixel-based feature is proposed in this study to characterize aerial image scenes. Then, based on the proposed feature, a scene recognition method of the Bag-of-Words model for aerial imaging is designed. The proposed superpixel-based feature that utilizes landform information establishes top-task superpixel extraction of landforms to bottom-task expression of feature vectors. This characterization technique comprises the following steps: simple linear iterative clustering based superpixel segmentation, adaptive filter bank construction, Lie group-based feature quantification, and visual saliency model-based feature weighting. Experiments of image scene recognition are carried out using real image data captured by an unmanned aerial vehicle (UAV). The recognition accuracy of the proposed superpixel-based feature is 95.1%, which is higher than those of scene recognition algorithms based on other local features. [ABSTRACT FROM AUTHOR]

LBP based feature descriptors have gotten encouraging performance in image recognition. As the improved versions of LBP, Local Ternary Pattern (LTP) and Center-symmetric LBP (CS-LBP) have been successfully applied to image recognition and matching. Both use a threshold to address the noise. However, it is difficult to manually set a suitable threshold in LTP and CS-LBP. Here, we propose an adaptive local feature descriptor for face recognition. First, inspired by Weber's Law, we introduce an adaptive Local Ternary Pattern (ALTP) feature descriptor based on an automatic strategy selecting the threshold for LTP; Second, based on ALTP, we further propose a center-symmetric adaptive local ternary pattern (CS-ALTP) feature description method for face recognition. CS-ALTP improves CS-LBP from two aspects: (1) An automatic threshold is proposed based on Weber's law; (2) double channel patterns are exploited to extract more discriminative information. The experiments on ORL and FERET face databases show that ALTP and CS-ALTP have good and robust recognition performance. [ABSTRACT FROM AUTHOR]

Gait recognition can be used in person identification and re-identification by itself or in conjunction with other biometrics. Although gait has both spatial and temporal attributes, and it has been observed that decoupling spatial feature and temporal feature can better exploit the gait feature on the fine-grained level. However, the spatial-temporal correlations of gait video signals are also lost in the decoupling process. Direct 3D convolution approaches can retain such correlations, but they also introduce unnecessary interferences. Instead of common 3D convolution solutions, this paper proposes an integration of decoupling process into a 3D convolution framework for cross-view gait recognition. In particular, a novel block consisting of a Parallel-insight Convolution layer integrated with a Spatial-Temporal Dual-Attention (STDA) unit is proposed as the basic block for global spatial-temporal information extraction. Under the guidance of the STDA unit, this block can well integrate spatial-temporal information extracted by two decoupled models and at the same time retain the spatial-temporal correlations. In addition, a Multi-Scale Salient Feature Extractor is proposed to further exploit the fine-grained features through context awareness extension of part-based features and adaptively aggregating the spatial features. Extensive experiments on three popular gait datasets, namely CASIA-B, OULP and OUMVLP, demonstrate that the proposed method outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we find that the conventional convolution operation becomes the bottleneck for extremely efficient binary neural networks (BNNs). To address this issue, we open up a new direction by introducing a reshaped point-wise convolution (RPC) to replace the conventional one to build BNNs. Specifically, we conduct a point-wise convolution after rearranging the spatial information into depth, with which at least $2.25\times $ computation reduction can be achieved. Such an efficient RPC allows us to explore more powerful representational capacity of BNNs under a given computation complexity budget. Moreover, we propose to use a balanced activation (BA) to adjust the distribution of the scaled activations after binarization, which enables significant performance improvement of BNNs. After integrating RPC and BA, the proposed network, dubbed as RB-Net, strikes a good trade-off between accuracy and efficiency, achieving superior performance with lower computational cost against the state-of-the-art BNN methods. Specifically, our RB-Net achieves 66.8% Top-1 accuracy with ResNet-18 backbone on ImageNet, exceeding the state-of-the-art Real-to-Binary Net (65.4%) by 1.4% while achieving more than $3\times $ reduction (52M vs. 165M) in computational complexity. [ABSTRACT FROM AUTHOR]