Search

There remains an open question about the usefulness and the interpretation of machine learning (ML) approaches for discrimination of spatial patterns of brain images between samples or activation states. In the last few decades, these approaches have limited their operation to feature extraction and linear classification tasks for between-group inference. In this context, statistical inference is assessed by randomly permuting image labels or by the use of random effect models that consider between-subject variability. These multivariate ML-based statistical pipelines, whilst potentially more effective for detecting activations than hypotheses-driven methods, have lost their mathematical elegance, ease of interpretation, and spatial localization of the ubiquitous General linear Model (GLM). Recently, the estimation of the conventional GLM parameters has been demonstrated to be connected to an univariate classification task when the design matrix in the GLM is expressed as a binary indicator matrix. In this paper we explore the complete connection between the univariate GLM and ML-based regressions. To this purpose we derive a refined statistical test with the GLM based on the parameters obtained by a linear Support Vector Regression (SVR) in the inverse problem (SVR-iGLM). Subsequently, random field theory (RFT) is employed for assessing statistical significance following a conventional GLM benchmark. Experimental results demonstrate how parameter estimations derived from each model (mainly GLM and SVR) result in different experimental design estimates that are significantly related to the predefined functional task. Moreover, using real data from a multisite initiative the proposed ML-based inference demonstrates statistical power and the control of false positives, outperforming the regular GLM. [ABSTRACT FROM AUTHOR]

• The proposed method introduces objectness constraint by using multiple output heads. • The target position consistency is enhanced by introducing positional encoding. • The multi-head soft aggregation based instance attention is designed. • Our method surpasses previous top performers and maintains a competitive speed. Objectness is a notion of how a generic object looks like. Recently, matching-based methods have yielded impressive video object segmentation results. However, these methods suffer from mis-segmentation of distractors that are distant from the targets due to the lack of objectness constraint. The segmentation mask has a very low objectness when both the correct object and the distant distractor are considered as a target. Position information is also ignored in matching-based methods. In this paper, a video object segmentation method based on target objectness constraint and multi-head soft aggregation is proposed. The target objectness constraint is introduced by forcing the network to learn to predict object center and pixel-to-object boundary distances. The problem of mis-segmentation of objects or regions with similar appearance at a distance is alleviated. Besides, the target position consistency constraint is strengthened by introducing the target position information in the feature matching of adjacent frames. Furthermore, an instance attention mechanism is implemented using multi-head soft aggregation to re-calibrate channel importance. Multi-head soft aggregation enables better utilization of target feature information than global average pooling. Comprehensive experiments demonstrate that the proposed method obtains a competitive performance, surpassing many other existing methods. [ABSTRACT FROM AUTHOR]

Time-varying Lyapunov and Stein matrix equations (TVLSMEs) are very ubiquitous in engineering applications. To solve TVLSMEs, many zeroing neural networks (ZNNs) activated by nonlinear activation functions have been proposed. In the conventional ZNN schemes, the designed convergence parameters (DCPs) before the nonlinear activation functions are extraordinarily important because DCPs determine their convergent rates basically. However, the DCPs are usually stated to be constant, which is not practical because the DCPs are usually time-varying in realistic hardware situations, especially the external noises injected. Hence, many varying-parameter ZNNs (VP-ZNNs) with time-varying DCPs have been proposed previously. Compared with finite-parameter ZNNs, these traditional VP-ZNNs are proved to have better convergence, however their downside is that DCPs usually increases over time, and becomes even infinite eventually. Evidently, infinity large DCPs would be lack of robustness are unacceptable in practice, especially the external noises injected. Additionally, even if VP-ZNNs converge over time, the growth of DCPs will result in a huge waste of computing resources. Inspired by that, a new hyperbolic tangent varying-parameter ZNNs (HTVP-ZNNs) with time-varying DCPs are proposed in this paper. Considering the noisy environment, we also develop the robust HTVP-ZNNs (HTVPR-ZNNs). Both of them are able to solve the time-variant Lyapunov and Stein matrix equations in prescribed-time. Theoretically the convergent prescribed-time of the HTVPR-ZNN and the upper time threshold of the DCPs are analyzed mathematically. And numerical experiments and trajectory tracking tasks of the mobile robot substantiate the outstanding convergence of the HTVPR-ZNN schemes. [ABSTRACT FROM AUTHOR]

Ensemble learning has gained success in machine learning with major advantages over other learning methods. Bagging is a prominent ensemble learning method that creates subgroups of data, known as bags, that are trained by individual machine learning methods such as decision trees. Random forest is a prominent example of bagging with additional features in the learning process. Evolutionary algorithms have been prominent for optimisation problems and also been used for machine learning. Evolutionary algorithms are gradient-free methods that work with a population of candidate solutions that maintain diversity for creating new solutions. In conventional bagged ensemble learning, the bags are created once and the content, in terms of the training examples, are fixed over the learning process. In our paper, we propose evolutionary bagged ensemble learning, where we utilise evolutionary algorithms to evolve the content of the bags in order to iteratively enhance the ensemble by providing diversity in the bags. The results show that our evolutionary ensemble bagging method outperforms conventional ensemble methods (bagging and random forests) for several benchmark datasets under certain constraints. We find that evolutionary bagging can inherently sustain a diverse set of bags without reduction in performance accuracy. [ABSTRACT FROM AUTHOR]

Temporal action localization, which aims to locate temporal regions where actions take place and recognize their corresponding classes in untrimmed real-world videos, is a challenging task. As a critical cue to video understanding, exploiting the video context has become an important strategy to boost the localization performance. However, previous methods mainly focus on exploring semantic context which captures the feature similarity among frames or proposals. The temporal position context which is also vital for temporal action localization is less explored. In this paper, we propose a position-sensitive context modeling approach to fuse both semantic and position context for more precise action localization. Specifically, we first propose a position encoding method tailored for temporal action localization on both frame-level and proposal-level, which ensures that the generated position representations can model the distance and chronological relationships among frames or proposals. Then we conduct attention-based context aggregation to produce discriminative features and help with precise boundary detection and proposal evaluation. Our method achieves state-of-the-art performance on two widely used datasets, THUMOS-14 and ActivityNet-1.3, demonstrating the effectiveness and generalizability of our method. [ABSTRACT FROM AUTHOR]

Compressive imaging aims to recover a latent image from under-sampled measurements, suffering from a serious ill-posed inverse problem. Recently, deep neural networks have been applied to this problem with superior results, owing to the learned advanced image priors. These approaches, however, require training separate models for different imaging modalities and sampling ratios, leading to overfitting to specific settings. In this paper, a dynamic proximal unrolling network (dubbed DPUNet) was proposed, which can handle a variety of measurement matrices via one single model without retraining. Specifically, DPUNet can exploit both the embedded observation model via gradient descent and imposed image priors by learned dynamic proximal operators, achieving joint reconstruction. A key component of DPUNet is a dynamic proximal mapping module, whose parameters can be dynamically adjusted at the inference stage and make it adapt to different imaging settings. Moreover, in order to eliminate the image blocking artifacts, an enhanced version DPUNet+ is developed, which integrates a dynamic deblocking module and reconstructs jointly with DPUNet to further improve the performance. Experimental results demonstrate that the proposed method can effectively handle multiple compressive imaging modalities under varying sampling ratios and noise levels via only one trained model, and outperform the state-of-the-art approaches. Our code is available at https://github.com/Yixiao-Yang/DPUNet-PyTorch. [ABSTRACT FROM AUTHOR]

• A novel anomalous subgraph detection method fuses the multiple experts' interactions. • Maximizing the non-parametric scan statistic over the connected subgraphs. • Detecting the subgraph as an event by the data-driven and user-driven procedures. • Guaranteeing to get the globally optimal solution without disconnecting the subgraph. • Achieving better performance than representative baselines on three datasets. Event detection on networks is an important research task in data mining. Most previous approaches usually detect a particular type of event that satisfies the predefined rules in the model, i.e., data-driven methods. However, few methods consider the diverse interests of multiple experts during the detection process to discover unexplored events. In this paper, we regard interactive event detection on attributed networks as Multi-users Interaction Anomalous Subgraph Detection (MIASD), where events are represented as connected subgraphs on networks. The core of our method is automatically maximizing a non-parametric scan statistic over connected subgraphs to identify the most anomalous subgraph as the event and integrating the interactions from multiple experts simultaneously, i.e., under both data-driven and user-driven paradigm. In our approach, we first define individual and shared environments of experts on attributed networks to model the interests of experts, including the interactive operations and Recommended Interaction Domain. Afterwards, we propose an efficient anomalous subgraph detection algorithm in which each expert gives feedback separately on the vertices in the Recommended Interaction Domain; then, the non-parametric scan statistic over the connected subgraph is approximately maximized by integrating various feedback from experts into the iterative subgraph expansion process. In this way, our method retrieves the most anomalous subgraph on the network as the final event, which contains the potential unexplored information. We have conducted extensive experiments on three real-world datasets and proved that our algorithm could achieve better performance compared with several competitive baselines. Furthermore, the case study shows that our method could detect global abnormal events effectively. [ABSTRACT FROM AUTHOR]

Over the past few years, the computer vision domain has evolved and made a revolutionary transition from human-engineered features to automated features to address challenging tasks. Computer vision is an ever-evolving domain with its roots deeply correlated with neuroscience; any new findings that trigger a more intuitive understanding and working of the human visual system generally impact the design strategy of computer vision algorithms. The convolutional neural network is one such algorithm that is currently the de facto standard for most computer vision tasks such as image classification, object detection, image segmentation, etc. As convolutional neural networks are associated with inherent constraints such as the requirement for an immense amount of labeled data and an inefficient data routing policy, capsule networks could be a viable alternative. Upheld by the backpropagation and the dynamic routing algorithm, the capsule network has set the new paradigm for developing reliable computer vision algorithms. Despite the phenomenal theoretical backing from neuroscience and the groundbreaking performance on benchmark datasets, the lack of information concerning the conception and working of capsule networks becomes the major impediment to adopting them for computer vision algorithms. This paper presents a concise overview of capsule network-based classification architectures, routing algorithms, performance analysis, limitations, and future scope, helping the research community to adopt capsule networks at the forefront of modern computer vision research. [ABSTRACT FROM AUTHOR]

In the real world, large-scale datasets for visual recognition typically exhibit a long-tailed distribution, where only a few classes contain adequate samples but the others have (much) fewer samples. With the advancement of data-hungry deep models for visual recognition, the low-tail power-law data distribution that biases the model training has attracted significant attention. When training with the long-tailed data, the majority classes dominate the training procedure, resulting in poor performance in instance-scarce classes. To tackle this problem, numerous strategies, such as re-sampling, cost-sensitive loss, meta-learning and transfer learning, have been proposed. This paper systematically reviews contemporary approaches for the long-tailed visual recognition task and categorizes these methods based on the stage applied as training, fine-tuning, and inference. Furthermore, we categorize training stage methods into data augmentation, re-sampling strategy, cost-sensitive loss, as well as multiple experts and transfer learning. Next, comprehensive comparisons are made in the balanced test set performance of long-tailed benchmarks and method robustness in diverse test distributions using metrics including top-1 accuracy, per-class accuracy, multi-class ROC AUC and Expected Calibration Error (ECE). At last, we outline the challenges in this field and future research trends. Our reviews and intriguing findings can be a tutorial for researchers working in the field of open-world deep learning. [ABSTRACT FROM AUTHOR]

Most text classification methods achieve great success based on the large-scale annotated data and the pre-trained language models. However, the labeled data is insufficient in practice, and the pre-trained language models are difficult to deploy due to their high computing resources and slow inference speed. In this paper, we propose cross-domain knowledge distillation, where the teacher and student tasks belong to different domains. It not only acquires knowledge from multiple teachers but also accelerates inference and reduces model size. Specifically, we train the pre-trained language models on factual knowledge obtained by aligned Wikipedia text to Wikidata triplets and fine-tune it as the teacher model. Then we use the heterogeneous multi-teacher knowledge distillation to transfer knowledge from the multiple teacher models to the student model. Multi-teacher knowledge vote can distill knowledge related to the target domain. Moreover, we also introduce the teacher assistant to help distill large pre-trained language models. Finally, we reduce the difference between the source domain and target domain by multi-source domain adaptation to solve the domain shift problem. Experiments on the multiple public datasets demonstrate that our method can achieve competitive performance while having fewer parameters and less inference time. [ABSTRACT FROM AUTHOR]

It has been widely noticed the performance of algorithms for high-resolution face recognition (HRFR) degrades significantly for low-resolution face recognition (LRFR). In this paper, we discover the main source of this performance degradation comes from the human-defined inductive bias of CNN, which constrains the model to absorb effective information and leads to overfitting. To overcome the shortcoming, for the first time, we adopt a transformer-based DNN algorithm named DeiT to accomplish LRFR tasks. On the other hand, we further borrow the form of knowledge distillation. Traditional knowledge distillation network for LRFR sets the student model to be simpler than or the same as the teacher model while using the teacher model off-the-shelf, leading to the model capacity gap. Instead, we fuse an online network into the original parameter-fixed teacher model to learn how to transfer knowledge. The final "knowledge" is the sum fusion of the outputs of both the teacher model and the student model. Experiments show that even without training on LR face images, the performance of our model can be comparable to recent baselines delicately designed for LRFR tasks on certain LR face datasets. After the entire training process, in either real-world LR datasets, artificially down-sampled datasets, or generated LR face datasets, our method performs favorably against the state-of-the-art (SOTA) algorithms. [ABSTRACT FROM AUTHOR]

In this paper, a class of quaternion-valued BAM neural networks (QVBAMNNs) fractional-order model with impulses and proportional delays is proposed in discrete-time case. The QVBAMNNs fractional-order model is investigated directly rather than through real decomposition method or the plural one. By employing homeomorphic mapping theorem, Lyapunov stability theory and inequality technology, several criteria for the discrete-time QVBAMNNs fractional-order model are derived to guarantee the existence, uniqueness and global exponential stability (GES) of equilibrium point. The novelty of these results comes from the generality with the fractional-order systems and the integer-order ones. Finally, two examples are given to demonstrate the effectiveness and availability of the derived criteria. [ABSTRACT FROM AUTHOR]

In recent years, low-resource Machine Reading Comprehension (MRC) attracts increasing attention. Due to the difficulty in data collecting, current low-resource MRC approaches often suffer from poor generalizing capability: the model only learns limited task-aware and domain-aware knowledge from a small-scale training dataset. Previous works generally address such deficiency by learning the required knowledge from out-of-domain MRC datasets and in-domain self-supervised datasets. However, such approaches also introduce domain noise and task noise. This paper proposes a Domain Adaptive Multi-Task Transformer (DAMT2) to tackle these noises. For task noise, DAMT2 utilizes a well-designed Multi-Task Transformer (MT2) as the backbone to model the high-level features separately from different tasks. For domain noise, two kinds of domain adaptation approaches are incorporated into MT2 to learn domain-invariant representations. The experimental results show that our method outperforms several baselines on multiple datasets, and especially achieves a new SOTA on the RRC dataset. Moreover, using only 40%-60% training data, our work achieves comparable performance with the classic BERT model. [ABSTRACT FROM AUTHOR]

In recent years, there has been an increasing amount of works on improving the salient object detection performance based on the edge information. However, previous studies applied a parameter-sharing encoder to extract edge features and region features, leading to incomplete regions and blurry edges. In this paper, we focus on the independence and complementarity between salient edges and regions, and propose a novel salient object detection network with two-stream encoder and interactive decoder(TSEID). Specifically, instead of a parameter-sharing encoder, we construct a two-stream encoder consisting of the region extraction branch and the edge extraction branch to balance the feature domain differences between the regions and edges. Meanwhile, we develop a super-pixel segmentation-based method to extract the salient edges, so that the generated distinctive edges will ensure the integrality of edge features. The region features are obtained via a feature pyramid network and hierarchically enhanced with Channel Adaptation (CA) and Spatial Adaptation (SA). Further, the extracted region features and edge features are fed into an interactive decoder to explore their complementarity. Accordingly, a location guidance module (LGM) and an edge compensation module (ECM) are designed to refine object positions, enrich object boundaries, and generate saliency features. Extensive experiments on five challenging benchmarks well demonstrate that our approach outperforms existing deep learning-based methods, including the latest edge-guided ones. [ABSTRACT FROM AUTHOR]

Community Question Answering (CQA) provides platforms for users with different backgrounds to share information and knowledge. With the increasing popularity of CQA, more and more question-answer (Q-A) pairs, with numerous duplicates, have accumulated. Therefore, many researchers focus on detecting duplicate questions in CQA. However, most existing techniques utilize only questions to solve the duplicate question detection task, while paired answers which may also contain necessary information are not considered. In this paper, we propose a BERT-encoded Hierarchical Question-Answer Cross-Attention Network for Duplicate Question Detection (Bert-QAnet) for detecting duplicate questions. Our model applies BERT to encode text and extract text features. Further, we use cross-attention to integrate word-level features both in question and answer. Also, inner attention is used to capture the interaction between question and answer. Hence, our model Bert-QAnet makes full use of semantic information in paired answers at both word-level and sentence-level. We evaluate our model on two datasets: the Yahoo! Answers dataset and the Stack Overflow dataset. To meet the special requirements of this study, both datasets are extended by paired answers. Experimental results demonstrate that our proposed model achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Recently, the attention mechanism became the key issue for image super-resolution (SR) because it has the ability to extract different features from the image according to the used attention type. Despite the great success of attention-based methods, the conflict among features of different attention types can affect the SR performance. In this paper, we propose an efficient single image SR model called balanced spatial feature distillation and pyramid attention (BSPAN). The idea of BSPAN is based on the trade-off among the extracted features of different attention types. Also, we propose a balanced spatial feature distillation block (BSFDB) as the backbone of BSPAN so that the network can effectively advantage from the different attention features. Two different attention types, namely spatial attention residual feature distillation (SARFD) and classical attention (CA) are considered in the BSFDB to achieve a leveling between them based on the contents of the low-resolution feature map using the balancing attention block. The BSFDB block is designed to improve the SR performance and has lightweight parameters and low computation complexity. Moreover, to further improve the SR performance, the pyramid attention is introduced in the middle of the BSPAN network for extracting long-range features at a variety of locations and scales. Evaluation based on five benchmark datasets, we concluded that the balancing between features of a variety of attention types can effectively improve the SR performance. So, the proposed BSPAN model achieves significant enhancements in comparison with the state-of-the-art and superior visual quality and reconstruction accuracy. [ABSTRACT FROM AUTHOR]

Fractional diffusion equations with conformable derivative have become an important research topic in Newtonian mechanics, quantum mechanics, arbitrary time scale problems, diffusion transport, neutron dynamics and other fields. In this paper, we propose to study time fractional diffusion equations with conformable derivative by using physics-informed neural networks (PINNs) for the first time. By solving the supervised learning task, we design a new spatio-temporal function approximator with high data efficiency. L-BFGS algorithm is used to optimize our loss function, and back propagation algorithm is used to update our parameters to give the numerical solutions. Since the conformable derivative satisfies the Leibniz rule and the chain rule, we can directly derive the network through automatic differentiation without any additional numerical discretization or truncation. For the forward problem, we can take IC/BCs as the data, and use PINNs to solve the corresponding partial differential equation. Three numerical examples are carried out to demonstrate the effectiveness of our methods. In particular, when the order of the conformable fractional derivative α tends to 1, a class of weighted PINNs is introduced to overcome the accuracy degradation caused by the singularity of solutions. For the inverse problem, we use the obtained data to train the neural network and specify the estimation of the parameter λ in the equation. Numerical results show that the proposed method is easy to implement and can accurately identify parameters, even when the training data is corrupted by 1% uncorrelated noise. [ABSTRACT FROM AUTHOR]

In the era of big data, People's lives are filled with all kinds of information. Scientific and technological information is utilized for scholars to understand the current technology trends, and to think about the source of information for future development prospects. More and more scholars are no longer satisfied with single-modal retrieval methods. However, to get more intelligent cross-media retrieval results we should give higher requirements to the search engine. And how to span the semantic gap between different modalities is a key issue that needs to be solved. In response to the above problems, this paper proposes a Multi-feature Fusion based Cross-Media Retrieval (MFCMR) method. Our method is capable of integrating multiple features to promote semantic understanding, and adopting adversarial learning to further improve the accuracy of public subspace representation. Then we use similarity in the same space to sort the retrieval results. We conduct a lot of experiments on real datasets, and the results show that our method obtains better cross-media retrieval performance than other methods. [ABSTRACT FROM AUTHOR]

This paper deals with the state estimate problem for a class of discrete-time complex networks. The target plant involves Markovian jumping parameters and mode-dependent distributed time-delays. It is assumed that the measurements for the estimate purpose are just from a portion of network nodes, and corrupted by a certain stochastic noise. A unified framework is developed to cope with the state estimation issue based on the outputs from the fraction of nodes for the discrete-time complex networks with distributed time delay and measurement noise. By employing the Lyapunov stability theory and some new techniques, the sufficient conditions are established to ensure the state estimation error is exponentially ultimately mean-square bounded. As the special case when the measurement is noise-free, the resulting criteria are to guarantee that the dynamics of state estimation error is exponentially stable. In addition, the estimator gain matrices are given in the explicit formula. Finally, numerical simulations are presented to illustrate the proposed approach. [ABSTRACT FROM AUTHOR]

Learning node representations in graphs is a widespread problem in node classification and link prediction. Current research has focused on static heterogeneous, homogeneous networks, and dynamic homogeneous networks. However, many existing graphs, such as citation and social networks, are heterogeneous. Therefore, it is still a great challenge to obtain heterogeneous information in a dynamic heterogeneous graph to assist node representation learning. In this paper, we propose a Dynamic HEterogeneous Network (DynHEN) for user-item bipartite networks. It is a discrete dynamic graph neural network model that can be used directly for node representation learning by utilizing dynamic heterogeneous graphs. Specifically, DynHEN takes a bipartite graph at each time step as input, gets the corresponding embedding by capturing the deep heterogeneous information of the nodes while fusing the temporal information. To illustrate the effectiveness of heterogeneous information for graph representation learning, we compare with the current SOTA methods in the two type experiments of link prediction and node classification, and achieve outstanding results. [ABSTRACT FROM AUTHOR]

• We study the problem of distributed image-to-image translation. • A novel framework for distributed image-to-image translation. • A unified framework for distributed image-to-image translation and autoencoding. Data-driven paradigms using machine learning are becoming ubiquitous in image processing and communications. In particular, image-to-image (I2I) translation is a generic and widely used approach to image processing problems, such as image synthesis, style transfer, and image restoration. At the same time, neural image compression has emerged as a data-driven alternative to traditional coding approaches in visual communications. In this paper, we study the combination of these two paradigms into a joint I2I compression and translation framework, focusing on multi-domain image synthesis. We first propose distributed I2I translation by integrating quantization and entropy coding into an I2I translation framework (i.e. I2Icodec). In practice, the image compression functionality (i.e. autoencoding) is also desirable, requiring to deploy alongside I2Icodec a regular image codec. Thus, we further propose a unified framework that allows both translation and autoencoding capabilities in a single codec. Adaptive residual blocks conditioned on the translation/compression mode provide flexible adaptation to the desired functionality. The experiments show promising results in both I2I translation and image compression using a single model. [ABSTRACT FROM AUTHOR]

Attributed network representation learning aims at learning node embeddings by integrating network structure and attribute information. It is a challenge to fully capture the microscopic structure and the attribute semantics simultaneously, where the microscopic structure includes the one-step, two-step and multi-step relations, indicating the first-order, second-order and high-order proximity of nodes, respectively. In this paper, we propose a deep attributed network representation learning via attribute enhanced neighborhood (DANRL-ANE) model to improve the robustness and effectiveness of node representations. The DANRL-ANE model adopts the idea of the autoencoder, and expands the decoder component to three branches to capture different order proximity. We linearly combine the adjacency matrix with the attribute similarity matrix as the input of DANRL-ANE, where the attribute similarity matrix is calculated by the cosine similarity between the attributes based on the social homophily. Moreover, the sigmoid cross-entropy loss function is extended to capture the neighborhood character, so that the first-order proximity could be well-preserved. We compare our model with the state-of-the-art models, especially, the latest graph autoencoders (GAEs) method, ARGA, and demonstrate the contribution of each module on real-world datasets and two network analysis tasks, i.e. , link prediction and node classification. The DANRL-ANE model performs well on various networks, even on sparse networks or networks with isolated nodes when the attribute information is sufficient. [ABSTRACT FROM AUTHOR]

Inspired by image-to-curve transformation and multi-exposure fusion, in this paper, we have developed a new method to treat the low light image enhancement tasks as an extended problem with multiple virtual exposures by a non-linear intensity mapping function. Considering that existing image-to-curve methods have difficulty in obtaining the desired detail and brightness recovery in any one iteration without relying on any ground truth, we propose a virtual multi-exposure fusion strategy to merge the outputs from these different iterations. Specifically, a simple CNN is trained to learn a pixel-wise intensity mapping function and accordingly adjust a given image multiple times. Then the results of all iterations are retained together with the original input image for fusion via a WGIF-based Multi-scale pyramid to obtain a final enhanced output. We present experimental results to demonstrate the effectiveness of the new technique and its state-of-the-art performances. [ABSTRACT FROM AUTHOR]

This paper is concerned with the set-membership filtering issue for a class of artificial neural networks subject to mixed attacks. The encoding–decoding communication mechanism is adopted in the processing of data sharing between neurons. During the information exchanges among neurons, both injection and DoS attacks are concurrently considered to reflect the practical operating conditions of the investigated neural networks. The purpose of the addressed problem is to present an algorithm to estimate the neurons' states in the presence of mixed attacks, while guaranteeing the estimation errors at each neuron are confined within certain prescribed ellipsoidal region. Sufficient conditions are derived, in terms of convex optimization approach, to ensure the existence of desired filter, and the explicit filtering parameters are obtained via solving the provided set of matrix inequalities. Finally, a numerical simulation example is proposed to show the validity of the obtained theoretical results. [ABSTRACT FROM AUTHOR]

The Iterative Closest Point (ICP) algorithm is one of the most important methods for rigid registration between point sets. However, its performance begins to degenerate with the point data are overly contaminated by noise, outliers, and missing data. In this paper, we propose an adaptive weighted robust ICP method (AW-RICP). A sparse weight vector can be automatically learned by adaptive neighbors assigning process. The sparseness of the weight vector can be achieved by fitting the number of selected samples. The adaptive weight vector leads to the robustness of AW-RICP by eliminating the negative effect caused by point cloud pairs with largest registration errors. Further, the retained point pairs are assigned suitable weights to suppress the noises and outliers. The new error metric can effectively improve the robustness of ICP method. Additionally, we propose an efficient iterative solution to optimize our problem. Experiments on both synthetic and real data sets show that the proposed method can achieve superior performance with other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Obtaining the optimal cluster number and generating reliable clustering results in nonlinear manifolds are necessary but challenging tasks. Most existing clustering algorithms have considerable limitations in dealing with local and nonlinear data patterns, while graph-based clustering has shown impressive performance in identifying clusters in such data patterns. In this paper, we propose a robust clustering method with noise cutting based on directed k-nearest neighbor graph (CDKNN) to identify the desired cluster number automatically and produce reliable clustering results simultaneously on nonlinear, non-overlapping but locally tight-connected data patterns. This method draws support from the k-nearest neighbor graph to represent the complex nonlinear datasets and applies parameter adaptive process to make the proposed clustering method better adapt to specific data patterns. The proposed method is robust to the noises of arbitrary shape datasets because it uses a directed K-nearest neighbor to cut out sparse nodes. We use simulation and UCI real-world datasets to prove the validity of the innovatory method by comparing it to k-means, DBSCAN, OPTICS, AP, SC, and CutPC algorithms in terms of clustering ACC, ARI, NMI, and FMI. The experimental results confirm that the proposed method outperforms the alternative nonlinear clustering methods. [ABSTRACT FROM AUTHOR]

Online updating is an important statistical method for the analysis of big data arriving in streams due to its ability to break the storage barrier and the computational barrier under certain circumstances. The quantile regression, as a widely used regression model in many fields, faces challenges in model fitting and variable selection with big data arriving in streams. Chen et al. (2019, Annals of Statistics) has proposed a quantile regression method for streaming data, but a strong additional condition is required. In this paper, renewable optimized objective functions for regression parameter estimation and variable selection in a quantile regression are proposed. The proposed methods are illustrated using current data and the summary statistics of historical data. Theoretically, the proposed statistics are shown to have the same asymptotic distributions as the standard version computed on an entire data stream with the data batches pooled into one data set, without additional condition. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods. [ABSTRACT FROM AUTHOR]

To improve the classification performance of existing adaptive boosting (AdaBoost) based algorithms effectively, an improved AdaBoost algorithm based on misclassified samples oriented feature selection and weighted non-negative matrix factorization (WNMF) is proposed in this paper. Firstly, in order to consider the effects of sample weights, a misclassified samples oriented feature selection (called MOFS) is proposed to select the most discriminative features which occur in the samples with high weights. Secondly, the explicit features and the part-based features of the training samples are both considered, and the WNMF algorithm is introduced and combined with MOFS to reduce the dimension of the training sample set. Finally, the concept of misclassification degree is introduced and a fine grained sample weight updating method is proposed to distinguish the samples with different misclassification degrees. Numerical experiments show that the proposed MOFS method achieves higher accuracy compared to traditional feature selection methods, and the proposed MOFS and WNMF based AdaBoost method obtains significant improvement on classification accuracy when comparing with typical existing AdaBoost based algorithms using different classifiers. [ABSTRACT FROM AUTHOR]

In this paper, the measured input–output data sequences with pinning gain are proposed via the topology of multi-agent systems (MAS), where the requirement of reinforcement learning algorithm on internal state is avoided by pinning gain and the measured data of leader and neighbors. Besides, a data-based tracking state is given, which can be applied to MAS with different control matrices. According to the sequences and tracking state, a distributed control policy and corresponding reinforcement learning algorithm are proposed for the output synchronization. The proposed algorithm overcomes the shortcoming that previous algorithms can not be applied to MAS with different control matrices in the absence of model information and full-state vector. Finally, the effectiveness of proposed algorithm is verified by simulation examples. [ABSTRACT FROM AUTHOR]

For NP-hard combinatorial optimization problems, it is usually challenging to find high-quality solutions in polynomial time. Designing either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods have provided new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input, masks, and decoder context vectors. The proposed framework aims to improve the models in literacy in terms of the neural network model and the training algorithm. The solution quality of TSP and the CVRP up to 100 nodes are significantly improved via our framework. Compared with the best results of the state-of-the-art methods, the average optimality gap is reduced from 4.53% to 3.67% for TSP with 100 nodes and from 7.34% to 6.68% for CVRP with 100 nodes when using the greedy decoding strategy. Besides, the proposed framework can be used to solve a multi-depot CVRP case without any structural modification. Furthermore, our framework uses about 1/3 ∼ 3/4 training samples compared with other existing learning methods while achieving better results. The results performed on randomly generated instances, and the benchmark instances from TSPLIB and CVRPLIB confirm that our framework has a linear running time on the problem size (number of nodes) during training and testing phases and has a good generalization performance from random instance training to real-world instance testing. [ABSTRACT FROM AUTHOR]

Lately, Autonomous vehicles (AV) have been gaining traction globally owing to their huge social, economic and environmental benefits. However, the rising safety apprehensions for vulnerable road users (VRU) alongside have become a stumbling block to the large-scale implementation of AVs. Deciphering intention in advance considering all social norms and context surrounding the VRU is a highly challenging task due to the huge amount of variability in their motion, actions and end goals. Based on this pensiveness, this paper extensively surveys the variety of techniques applied to anticipate pedestrian intention and classifies them from multiple perspectives. Some newly introduced datasets with added complexities of human behaviour on road have also been outlined. It also provides a comparative analysis of the performance of pedestrian intention prediction approaches on several benchmark datasets as per the various assessment parameters available. In addition to this, several potential challenges and their possible solutions paving way for future research initiatives have also been thoroughly analysed in this endeavour. [ABSTRACT FROM AUTHOR]

Video clip retrieval and captioning tasks play an essential role in multimodal research and are the fundamental research problem for multimodal understanding and generation. The CLIP (Contrastive Language-Image Pre-training) model has demonstrated the power of visual concepts learning from web collected image-text datasets. In this paper, we propose a CLIP4Clip model to transfer the knowledge of the image-text pretrained CLIP model to video-text tasks in an end-to-end manner. Furthermore, we conduct several empirical studies including 1) Whether image feature is enough for video-text retrieval and captioning? 2) How a post-pretraining on a large-scale video-text dataset based on the CLIP affect the performance? 3) What is the practical mechanism to model temporal dependency between video frames? And 4) The Hyper-parameters sensitivity of the model. Extensive experimental results present that the CLIP4Clip model transferred from the CLIP can achieve SOTA results on various video-text datasets, including MSR-VTT, MSVD, LSMDC, and DiDeMo for multimodal understanding and generation tasks. [ABSTRACT FROM AUTHOR]

In the field of saliency detection, salient instance segmentation is a novel challenging task that has received widespread attention. Due to the limited scale of the available dataset and the high cost of mask annotations, a substantial quantity of supervision sources is urgently required to train a high-performing salient instance model. To this end, we aim to train a novel salient instance segmentation model by weak supervisions that make full use of the existing salient object detection dataset. In this paper, we present a cyclic global context salient instance segmentation network (CGCNet) supervised by the combination of salient regions and bounding boxes from ready-made salient object detection datasets. To locate salient instances more accurately, a global feature refining layer is designed to expand the size of the features from the region of interest (ROI) to the global field in a scene. Moreover, a labeling updating scheme is embedded in the proposed framework to iteratively update the weak labels. Extensive experimental results demonstrate that our CGCNet trained by weak labels is competitive with the existing fully-supervised state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper explores the collective behavior of bipartite flocking for a set of autonomous mobile agents based on Cucker-Smale update rules on directed cooperation-competition networks. In our model, cooperative or competitive degree between neighbouring agents is set by a nonlinear function of the communication distance between agents, which is called weight function, and further the influence of communication delays on the system stability and the final distance between agents is analyzed. With the help of related conclusions about sub-stochastic matrices, a sufficient condition relying on the topology structure, weight functions and initial states is established to guarantee the realization of bipartite flocking behavior. In the end the effectiveness of dynamic results is verified through a numerical example. [ABSTRACT FROM AUTHOR]

Automated skin lesion segmentation is an essential yet challenging task for computer-aided skin disease diagnosis. One major challenge for learning-based segmentation method is the limited manually annotated dermoscopy images. Many semi-supervised methods are proposed to exploit unlabeled data by self-training with pseudo labels. However, the plain pseudo labels are less accurate and the pixel-wise features of unlabeled data are always not well formulated due to the large variations among different lesions. Aiming at producing a good segmentation embedding space in a semi-supervised manner, in this paper, we propose a novel dynamic prototypical feature representation learning framework to address these problems. Specifically, we propose a novel denoised pseudo label generation method, which effectively filters out the unreliable components in plaint pseudo labels and provides the guidance for the subsequent feature representation learning. Then, we propose a memory relation learning method to enhance the intermediate feature representation globally. Additionally, we propose a prototype-based confidence-aware contrastive learning method to learn a better local feature structure in semi-supervised training, strengthening intra-class compactness and inter-class separability. Extensive experiments on two skin lesion segmentation datasets demonstrate that our method outperforms other popular semi-supervised segmentation methods. [ABSTRACT FROM AUTHOR]

In this paper, a distributed nonsmooth nonconvex optimization (DNNO) problem with affine inequality and nonsmooth convex inequality constraints is studied. A continuous-time distributed neurodynamic algorithm is proposed to solve this problem. Under the assumed conditions, for any initial state, the solution of distributed neurodynamic algorithm is bounded and globally exists, and will converge to the critical point set of distributed problems in a finite time. Compared with other DNNO algorithms, distributed neurodynamic algorithm has a lower dimension and does not need to satisfy the assumption that the feasible region is bounded. Finally, a series of numerical examples are given to verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The electroencephalogram (EEG) signal is commonly applied in the brain-computer interface (BCI) system of the motor imagery paradigm because it is noninvasive and has a high time resolution. This paper proposes a motor imagery classification method based on convolutional neural networks and Riemannian geometry to overcome the problem of noise and extreme values impacting motor imagery classification performance. The time-domain properties of EEG signals are extracted using multiscale temporal convolutions, whereas the spatial aspects of EEG signals are extracted using multiple convolutional kernels learned by spatial convolution. The extracted features are mapped to a Riemannian manifold space, and bilinear mapping and logarithmic operations are performed on the features to solve the problem of noise and extreme values. The effectiveness of the proposed method is validated using four types of motor imagery in the BCI competition IV dataset 2a to evaluate the classification ability. The experimental results show that the proposed approach has obvious advantages in the classification performance of motor imagery. [ABSTRACT FROM AUTHOR]

Researchers have recently proposed plenty of heterogeneous graph neural networks (HGNNs) due to the ubiquity of heterogeneous graphs in both academic and industrial areas. Instead of pursuing a more powerful HGNN model, in this paper, we are interested in devising a versatile plug-and-play module, which accounts for distilling relational knowledge from pre-trained HGNNs. To the best of our knowledge, we are the first to propose a HI gh-order RE lational (HIRE) knowledge distillation framework on heterogeneous graphs, which can significantly boost the prediction performance regardless of model architectures of HGNNs. Concretely, our HIRE framework initially performs first-order node-level knowledge distillation, which encodes the semantics of the teacher HGNN with its prediction logits. Meanwhile, the second-order relation-level knowledge distillation imitates the relational correlation between node embeddings of different types generated by the teacher HGNN. Extensive experiments on various popular HGNNs models and three real-world heterogeneous graphs demonstrate that our method obtains consistent and considerable performance enhancement, proving its effectiveness and generalization ability. [ABSTRACT FROM AUTHOR]

To handle the occlusion issue, this paper proposes a simple but effective pedestrian detector who designs a novel bi-center prediction mechanism (namely Bi-Center) based on the anchor-free network. Specifically, we modify the training loss or ground truth of the two designed center prediction branches in the pedestrian detection head. In the first branch, the negative samples around centers of occluded pedestrians, which are mainly covered by the background region, are down-weighted in a novel weighted Focal Loss according to the occlusion levels. In the second branch, we further integrate the occlusion ratio into the Gaussian mask of center ground truth to improve the predicted probability of occluded pedestrian instances. We optimize the designed detection head of the network in an end-to-end fashion, and utilize the complementary outputs of the designed two center prediction branches to boost the detection performance. The proposed bi-center prediction mechanism forces the network to pay more attention to the occluded pedestrian instances. Experimental results on the challenging CityPersons, Caltech, and CrowdHuman benchmarks sufficiently validate the efficacy of our Bi-Center detector for occlusion handling in pedestrian detection. [ABSTRACT FROM AUTHOR]

This paper deals with global exponential stabilization (GES) and exponential lag synchronization (ELS) for impulsive complex-valued inertial neural networks (ICVINNs) with proportional delays, without employing the standard order-reduction transformation and the general separation approach of the real and imaginary part of the ICVINNs. Several algebraic criteria involving multi-parameters for examining the GES and ELS of the ICVINNs are built, via establishing an appropriate Lyapunov functional, and utilizing inequality techniques, which can be achieved by the novel design of the adaptive controller. Numerical simulations are shown to sustain the availability and efficiency of the proposed criteria. [ABSTRACT FROM AUTHOR]

The success of convolutional neural networks (CNNs) has made low-latency inference services on Graphic Processing Units (GPUs) a hot research topic. However, GPUs are hardware processors with high power consumption. To have the least energy consumption while meeting latency Service-Level-Objective (SLO), batching strategy and dynamic voltage frequency scaling (DVFS) are two important solutions. However, existing studies do not coordinate them and regard CNN as a black box, which makes inference services less energy-efficient. In this paper, we propose EALI, an energy-aware layer-level adaptive scheduling framework that is comprised of a power prediction model, a layer combination strategy, and an energy-aware layer-level scheduler. The power prediction model uses classic machine learning techniques to predict fine-grained layer-level power consumption. The layer combination strategy combines multiple layers into optimization units to lower scheduling overhead and complexity. The energy-aware layer-level scheduler adaptively coordinates batching strategy and layer-level DVFS according to workloads to minimize the energy consumption while meeting SLO. Our experimental results on NVIDIA Tesla M40 and V100 GPUs show that, compared to the state-of-the-art approaches, EALI decreases energy consumption by up to 36.24% while meeting SLO. [ABSTRACT FROM AUTHOR]

Chaotic time series prediction, a challenging research topic in dynamic system modeling, has drawn great attention from researchers around the world. In recent years extensive researches have been done on developing chaotic time series prediction methods, and various models have been proposed. Among them, recurrent fuzzy neural networks (RFNNs) have shown significant potential in this area. Most of the proposed RFNNs learn a single function, but when dealing with chaotic time series, different outputs may be generated for a specific input based on the system's state. So, a network is required that can learn multiple functions simultaneously. Based on this concept, a novel multi-functional recurrent fuzzy neural network (MFRFNN) is proposed in this paper. MFRFNN consists of two fuzzy neural networks with Takagi-Sugeno-Kang fuzzy rules, one is used to produce the output, and the other to determine the system's state. There is a feedback loop between these two networks, which makes MFRFNN capable of learning and memorizing historical information of past observations. Employing the states allows the proposed network to learn multiple functions simultaneously. Moreover, a new learning algorithm, which employs the particle swarm optimization algorithm, is developed to train the networks' weights. The effectiveness of MFRFNN is validated using the Lorenz and Rossler chaotic time series and four real-world datasets, including Box–Jenkins gas furnace, wind speed prediction, Google stock price prediction, and air quality index prediction. Based on the root mean square error, the proposed method shows a decrease of 35.12 % , 13.95 % , and 49.62 % from the second best methods in the Lorenz time series, Box–Jenkins gas furnace, and wind speed prediction dataset, respectively. [ABSTRACT FROM AUTHOR]

Retinal optical coherence tomography (OCT) images provide fundamental information regarding the health of the posterior eye (e.g., the retina and choroid). Thus, the development of automatic image analysis methods (e.g., segmentation) is fundamental to provide clinicians and researchers with quantitative data that facilitates decision making. In recent years, various machine learning (ML) methods have been developed to perform these automated image analyses, improving performance over traditional methods, increasing repeatability, and reducing the use of time-consuming manual analysis. Deep learning (DL), a sub-field of ML, represents a new approach that can improve image processing outcomes. Results published to date demonstrate that DL architectures generally achieve superior performance to previously proposed methods based on traditional image analysis or early ML techniques. Thus, DL methods for OCT image analysis have provided an important advance in the area of retinal layer segmentation in images from healthy eyes as well as those with pathologies. This paper provides a comprehensive narrative literature review of current DL layer segmentation methods applied to OCT images of the posterior segment of the eye. The manuscript provides an overview on the state-of-the-art in deep learning as well as highlighting some important areas for future developments to extend the analysis methods in this field. [ABSTRACT FROM AUTHOR]

Hashing technology can support large-scale recommendation very effectively due to its advantages of low storage cost and high recommendation efficiency. However, existing hashing-based recommendation methods often suffer from item cold-start problem. This is because they simply consider the user-item interaction and the single content information of the items, but the full interaction history is not always available and the single auxiliary information may be missing. To solve this issue, in this paper we propose a Binary Multi-modal Matrix Factorization (BMMF) method. First, we propose an efficient consensus multi-modal mapping to transform the heterogeneous multi-modal features to the unified factors by exploiting the complementarity of multiple modalities. Then, binary matrix factorization is simultaneously performed on the multi-modal features of the items and past user preferences to learn the compact binary codes of the users/items in a common Hamming space. In addition, inspired by the observation that similar instances often have similar binary codes within a short Hamming distance, we formulate a semantic structure regularization term to preserve the similarities of the items during the binary embedding process. Finally, we develop an effective Discrete Coordinate Descent (DCD) approach to tackle the formulated discrete hash optimization problem directly. Experiments on three publicly available real-world datasets demonstrate the superiority of the proposed method against the state-of-the-art methods. Our source codes and testing datasets are available at https://github.com/pcm1217/BMMF. [ABSTRACT FROM AUTHOR]

In this paper, a new data-driven reinforcement learning method based on Monte Carlo simulation is developed to solve the optimal control problem of unmanned aerial vehicle (UAV) systems. Based on the data which are generated by Monte Carlo simulation, neural network (NN) is used to construct the dynamics of the UAV system with unknown disturbances, where the mathematical model of the UAV system is unnecessary. An effective iterative framework of action and critic is constructed to obtain the optimal control law. The convergence property is developed to guarantee that the iterative performance cost function converges to a finite neighborhood of the optimal performance cost function. Finally, numerical results are given to illustrate the effectiveness of the developed method. [ABSTRACT FROM AUTHOR]

Knowledge graph alignment aims to identify entity pairs having the same meaning between different knowledge graphs, which is essential to the automated construction of a coherent knowledge base. With the development of knowledge representation learning, researchers have proposed several useful embedding-based alignment methods. In practice, some entities are easily aligned with high confidence while others are not. However, there is a lack of effective methods which boost the alignment of implicitly aligned entities based on explicitly aligned ones. This paper presents AEP, which combines several effective schemes for accurate knowledge graph alignment. First, we employ a word-embedding model to encode the semantic information contained in entities' surface names. We propose an attention-based graph convolutional network model to incorporate the structural information via supervised embedding propagation. Besides, we develop a multi-view bootstrapping strategy to address the over-fitting problem caused by insufficient training sets. Next, an embedding-propagation-based alignment scheme is proposed to improve the alignment accuracy by propagating explicitly aligned entities' embeddings to implicitly aligned ones in an unsupervised manner. Finally, we conduct extensive experiments to validate the superiority of AEP and evaluate the effects of proposed schemes. Experimental results show that AEP outperforms state-of-the-art methods, and the proposed schemes improve the alignment accuracy significantly. [ABSTRACT FROM AUTHOR]

Previous work like Category Structure Knowledge Distillation proposes to construct category-wise relations for knowledge distillation by introducing intra-category and inter-category relations based on category centers. However, category centers may be unreliable when feature representations of wrongly classified samples are used to form category centers. Besides, inter-category relations based on category centers are coarse-grained. In this paper, we propose a Marginal Sample Knowledge Distillation (MSKD) to construct reliable category centers and fine-grained inter-category relations by introducing label filtering and marginal samples. Label filtering removes feature representations of wrongly classified samples from the calculation of category centers to create unbiased and reliable category centers. Marginal samples are defined as the correctly classified samples close to category boundaries. Marginal samples contain information of other categories and form fine-grained category boundaries for knowledge distillation. Extensive experiments on different datasets and teacher-student architecture settings show that our method has excellent performances when compared with closely related methods. [ABSTRACT FROM AUTHOR]

Distance metric learning plays a significant role in improving the generalization of algorithms related to distance metrics. In this paper, we first propose a generalized Mahalanobis metric learning framework (called GLML) to make use of the prior knowledge from the samples including global and local information, the main goal of which is to enlarge the distance of dissimilar sample pairs and shrink the distance of similar sample pairs simultaneously. Specifically, the metric learning problem is guided by a discriminative regularization through incorporating the pair-wise and class-wise information. Moreover, an effective alternating iterative algorithm is developed to optimize the proposed GLML, and the convergence of the algorithm is demonstrated theoretically. Then a boosting algorithm of GLML (BGLML) is proposed, where the low-rank basis learning is jointly optimized with the metric to better uncover the data structure and mitigate the computational burden. Following that, numerical experiments are carried out on binary classification and multi-classification problems. With different assessment criteria, experiment results from different scale datasets confirm that the proposed methods either improve generalization or have comparable results compared with the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In this paper, synchronization problem of coupled neural networks (CNNs) with both the distributed delay coupling (DDC) and the nonlinear coupling is studied. Firstly, a novel array of delayed inequalities with mixed delays is introduced. Specifically, mixed delays involve the distributed delay and discrete impulsive delays. Stemmed from such newly proposed inequalities, some sufficient conditions ensuring the exponential synchronization of CNNs are obtained by utilizing technique of linear matrix inequalities (LMIs) and Kronecker product. Furthermore, the delay-dependent distributed impulsive controller is devised, where discrete delays can be nonidentical at different impulsive instants. In addition, we pay special attention on synchronization issue of large-scale CNNs and derive some low-dimensional sufficient conditions by employing the technique of the matrix decomposition. Finally, two illustrative numerical examples are provided to demonstrate the effectiveness of theoretical results. [ABSTRACT FROM AUTHOR]

Product quantization (PQ) is a fundamental technique for approximate nearest neighbor (ANN) search in many applications such as information retrieval, computer vision and pattern recognition. In the existing PQ-based methods for approximate nearest neighbor search, the reachable best search accuracy is achieved by using fixed codebooks. The search performance is limited by the quality of the hard codebooks. Unlike the existing methods, in this paper, we present a novel codebook-softened product quantization (CSPQ) method to achieve more quantization levels by softening codebooks. We firstly analyze how well the database vectors match the trained codebooks by examining quantization error for each database vector, and select the bad-matching database vectors. Then, we give the trained codebooks b -bit freedom to soften codebooks. Finally, by minimizing quantization errors, the bad-matching vectors are encoded by softened codebooks and the labels of best-matching codebooks are recorded. Experimental results on SIFT1M, GIST1M and SIFT10M show that, despite its simplicity, our proposed method achieves higher accuracy compared with PQ and it can be combined with other non-exhaustive frameworks to achieve fast search. [ABSTRACT FROM AUTHOR]