Search

With the abundance of data, machine learning applications engaged increased attention in the last decade. An attractive approach to robustify the statistical analysis is to preprocess the data through clustering. This paper develops a low-complexity Riemannian optimization framework for solving optimization problems on the set of positive semidefinite stochastic matrices. The low-complexity feature of the proposed algorithms stems from the factorization of the optimization variable $\mathbf {X}=\mathbf {Y}\mathbf {Y}^{\mathrm{T}}$ X = Y Y T and deriving conditions on the number of columns of $\mathbf {Y}$ Y under which the factorization yields a satisfactory solution. The paper further investigates the embedded and quotient geometries of the resulting Riemannian manifolds. In particular, the paper explicitly derives the tangent space, Riemannian gradients and Hessians, and a retraction operator allowing the design of efficient first and second-order optimization methods for the graph-based clustering applications of interest. The numerical results reveal that the resulting algorithms present a clear complexity advantage as compared with state-of-the-art euclidean and Riemannian approaches for graph clustering applications. [ABSTRACT FROM AUTHOR]

In this paper, we propose to extend the standard Convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be “low-rank” through a Canonical Polyadic decomposition. We show that this additional constraint increases the robustness of the CDL with respect to noise and improve the interpretability of the final results. In addition, we discuss in detail the advantages of this representation and introduce two algorithms, based on ADMM or FISTA, that efficiently solve this problem. We show that by exploiting the low rank property of activations, they achieve lower complexity than the main CDL algorithms. Finally, we evaluate our approach on a wide range of experiments, highlighting the modularity and the advantages of this tensorial low-rank formulation. [ABSTRACT FROM AUTHOR]

Stochastic unit commitment is an efficient method for grid operation in the presence of significant uncertainties. An example is an operation during a predicted hurricane with uncertain line out-ages. However, the solution quality comes at the cost of substantial computational burden, which makes its adoption challenging. This paper evaluates some possible ways that machine learning can be used to reduce this computational burden. First, a set of feasibility studies is conducted. Results suggest that using machine learning as an assistant to the stochastic unit commitment solver is more advantageous than using it as a standalone solver. In particular, the machine learning model is trained to facilitate solving the problem by determining the unnecessary constraints that can be removed from the original problem without affecting the final accuracy. The variables that can be used as input features/predictors or outputs for the machine learning model are determined through feasibility studies. Then, an algorithm to train and utilize a machine learning model is proposed. The method is tested on a 500-bus synthetic South Carolina system. Various test cases show an average reduction in solution time by more than 90% by using the trained machine learning model to assist the stochastic unit commitment solver. [ABSTRACT FROM AUTHOR]

$k$ k -means method using Lloyd heuristic is a traditional clustering method which has played a key role in multiple downstream tasks of machine learning because of its simplicity. However, Lloyd heuristic always finds a bad local minimum, i.e., the bad local minimum makes objective function value not small enough, which limits the performance of $k$ k -means. In this paper, we use coordinate descent (CD) method to solve the problem. First, we show that the $k$ k -means minimization problem can be reformulated as a trace maximization problem, then a simple and efficient coordinate descent scheme is proposed to solve the maximization problem. Two interesting findings through theory are that Lloyd cannot decrease the objective function value of $k$ k -means produced by our CD further, and our proposed method CD to solve $k$ k -means problem can avoid produce empty clusters. In addition, according to the computational complexity analysis, it is verified CD has the same time complexity with original $k$ k -means method. Extensive experiments including statistical hypothesis testing, on several real-world datasets with varying number of clusters, varying number of samples and varying number of dimensions show that CD performs better compared to Lloyd, i.e., lower objective value, better local minimum and fewer iterations. And CD is more robust to initialization than Lloyd whether the initialization strategy is random or initialization of $k$ k -means++. [ABSTRACT FROM AUTHOR]

Estimating set similarity and detecting highly similar sets are fundamental problems in areas such as databases and machine learning. MinHash is a well-known technique for approximating Jaccard similarity of sets and has been successfully used for many applications. Its two compressed versions, $b$ b -bit MinHash and Odd Sketch, can significantly reduce the memory usage of the MinHash, especially for estimating high similarities (i.e., similarities around 1). Although MinHash can be applied to static sets as well as streaming sets, of which elements are given in a streaming fashion, unfortunately, $b$ b -bit MinHash and Odd Sketch fail to deal with streaming data. To solve this problem, we previously designed a memory-efficient sketch method, MaxLogHash, to accurately estimate Jaccard similarities in streaming sets. Compared with MinHash, our method uses smaller sized registers (each register consists of less than 7 bits) to build a compact sketch for each set. In this paper, we further develop a faster method, MaxLogOPH++. Compared with MaxLogHash, MaxLogOPH++ reduces the time complexity for updating each coming element from $O(k)$ O (k) to $O(1)$ O (1) with a small additional memory. We conduct experiments on a variety of datasets, and experimental results demonstrate the efficiency and effectiveness of our methods. [ABSTRACT FROM AUTHOR]

With growth in the number of smart devices and advancements in their hardware, in recent years, data-driven machine learning techniques have drawn significant attention. However, due to privacy and communication issues, it is not possible to collect this data at a centralized location. Federated learning is a machine learning setting where the centralized location trains a learning model over remote devices. Federated learning algorithms cannot be employed in the real world scenarios unless they consider unreliable and resource-constrained nature of the wireless medium. In this paper, we propose a federated learning algorithm that is suitable for cellular wireless networks. We prove its convergence, and provide a sub-optimal scheduling policy that improves the convergence rate. We also study the effect of local computation steps and communication steps on the convergence of the proposed algorithm. We prove, in practice, federated learning algorithms may solve a different problem than the one that they have been employed for if the unreliability of wireless channels is neglected. Finally, through numerous experiments on real and synthetic datasets, we demonstrate the convergence of our proposed algorithm. [ABSTRACT FROM AUTHOR]

Evolutionary algorithms (EAs) have become the standards and paradigms for solving inverse problems. However, their two inherited operations, namely, the crossover and mutation operations, are complicated and difficult, both in theory and in numerical implementations. In this regard, increasing efforts have been devoted to EAs which are based on probabilistic models (EAPMs) to overcome the shortcomings of available EAs. The population-based incremental learning (PBIL) is an EAPM; moreover, it can bridge the gap between machine learning and the EAs, hence enjoying several merits compared with other EAs. However, lukewarm efforts have been devoted to PBILs, especially the real coded PBILs, in the study of inverse problems in electromagnetics. In this regard, a novel real coded PBIL is being proposed in this paper. In the proposed real coded PBIL, a probability matrix is proposed to randomly generate a population, and the updating formulas for this probability matrix using the so far searched best solution and the best solution of the current population are introduced to strike a balance between convergence performance and solution quality. The proposed real coded PBIL algorithm is numerically experimented on several case studies and promising results are reported in this paper. [ABSTRACT FROM PUBLISHER]

In multiple instance learning problems, patterns are often given as bags and each bag consists of some instances. Most of existing research in the area focuses on multiple instance classification and multiple instance regression, while very limited work has been conducted for multiple instance clustering (MIC). This paper formulates a novel framework, maximum margin multiple instance clustering (M^3IC), for MIC. However, it is impractical to directly solve the optimization problem of M^3IC. Therefore, M^3IC is relaxed in this paper to enable an efficient optimization solution with a combination of the constrained concave-convex procedure and the cutting plane method. Furthermore, this paper presents some important properties of the proposed method and discusses the relationship between the proposed method and some other related ones. An extensive set of empirical results are shown to demonstrate the advantages of the proposed method against existing research for both effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Sparse unmixing has become a popular tool for hyperspectral imagery interpretation. It refers to finding the optimal subset of a spectral library to reconstruct the image data and further estimate the proportions of different materials. Recently, multi-objective based sparse unmixing methods have presented promising performance because of their advantages in addressing combinatorial problems. A spectral and multi-objective based sparse unmixing (SMoSU) algorithm was proposed in our previous work, which solves the decision-making problem well. However, it does not show outstanding advantages in strong noise cases. To solve the problem, in this paper, SMoSU is improved based on the estimation of distribution algorithms (EDAs). The machine learning based EDAs have been a reliable approach in solving multi-objective problems. However, most of them are for special problems and relatively weak in theoretical foundations. Thus, it is unreliable to extend it directly to sparse unmixing. Here, we improve EDA on the basis of classification and propose a classification-based model for individual generating under the framework of SMoSU (CM-MoSU). In CM-MoSU, the whole population is divided to be positive and negative. Then, the macroinformation of positive individuals is used to guide the generation of new individuals. Therefore, the optimization task could pay more attention to the feasible space with high quality. Moreover, some theoretical analyses are presented to prove the reliability of CM-MoSU. In experiments, several state-of-the-art sparse unmixing algorithms are compared. Both synthetic and real-world experiments demonstrate the effectiveness of CM-MoSU. [ABSTRACT FROM AUTHOR]

Texts in natural scenes carry critical semantic clues for understanding images. When capturing natural scene images, especially by handheld cameras, a common artifact, i.e., blur, frequently happens. To improve the visual quality of such images, deblurring techniques are desired, which also play an important role in character recognition and image understanding. In this paper, we study the problem of recovering the clear scene text by exploiting the text field characteristics. A series of text-specific multiscale dictionaries (TMD) and a natural scene dictionary is learned for separately modeling the priors on the text and nontext fields. The TMD-based text field reconstruction helps to deal with the different scales of strings in a blurry image effectively. Furthermore, an adaptive version of nonuniform deblurring method is proposed to efficiently solve the real-world spatially varying problem. Dictionary learning allows more flexible modeling with respect to the text field property, and the combination with the nonuniform method is more appropriate in real situations where blur kernel sizes are depth dependent. Experimental results show that the proposed method achieves the deblurring results with better visual quality than the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Both cost-sensitive classification and online learning have been extensively studied in data mining and machine learning communities, respectively. However, very limited study addresses an important intersecting problem, that is, “Cost-Sensitive Online Classification”. In this paper, we formally study this problem, and propose a new framework for Cost-Sensitive Online Classification by directly optimizing cost-sensitive measures using online gradient descent techniques. Specifically, we propose two novel cost-sensitive online classification algorithms, which are designed to directly optimize two well-known cost-sensitive measures: (i) maximization of weighted sum of sensitivity and specificity, and (ii) minimization of weighted misclassification cost. We analyze the theoretical bounds of the cost-sensitive measures made by the proposed algorithms, and extensively examine their empirical performance on a variety of cost-sensitive online classification tasks. Finally, we demonstrate the application of the proposed technique for solving several online anomaly detection tasks, showing that the proposed technique could be a highly efficient and effective tool to tackle cost-sensitive online classification tasks in various application domains. [ABSTRACT FROM AUTHOR]

In this paper, we propose a Hierarchical Sampling-based Multi-Instance ensemble LEarning (HSMILE) method. Due to the unique multi-instance learning nature, a positive bag contains at least one positive instance whereas samples (instance and sample are interchangeable terms in this paper) in a negative bag are all negative, simply applying bootstrap sampling to individual bags may severely damage a positive bag because a sampled positive bag may not contain any positive sample at all. To solve the problem, we propose to calculate probable positive sample distributions in each positive bag and use the distributions to preserve at least one positive instance in a sampled bag. The hierarchical sampling involves inter- and intrabag sampling to adequately perturb bootstrap sample sets for multi-instance ensemble learning. Theoretical analysis and experiments confirm that HSMILE outperforms existing multi-instance ensemble learning methods. [ABSTRACT FROM PUBLISHER]

Bottom-up saliency detection aims to detect salient areas within natural images usually without learning from labeled images. Typically, the saliency map of an image is inferred by only using the information within this image (referred to as the “current image”). While efficient, such single-image-based methods may fail to obtain reliable results, because the information within a single image may be insufficient for defining saliency. In this paper, we investigate how saliency detection can benefit from the nearest neighbor structure in the image space. First, we show that existing methods can be improved by extending them to include the visual neighborhood information. This verifies the significance of the neighbors. Next, a solution of multitask sparsity pursuit is proposed to integrate the current image and its neighbors to collaboratively detect saliency. The integration is done by first representing each image as a feature matrix, and then seeking the consistently sparse elements from the joint decompositions of multiple matrices into pairs of low-rank and sparse matrices. The computational procedure is formulated as a constrained nuclear norm and \ell2,1-norm minimization problem, which is convex and can be solved efficiently with the augmented Lagrange multiplier method. Besides the nearest neighbor structure in the visual feature space, the proposed model can also be generalized to handle multiple visual features. Extensive experiments have clearly validated its superiority over other state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Assessing the similarity between cases is a key aspect of the retrieval phase in case-based reasoning (CBR). In most CBR work, similarity is assessed based on feature value descriptions of cases using similarity metrics, which use these feature values. In fact, it might be said that this notion of a feature value representation is a defining part of the CBR worldview—it underpins the idea of a problem space with cases located relative to each other in this space. Recently, a variety of similarity mechanisms have emerged that are not founded on this feature space idea. Some of these new similarity mechanisms have emerged in CBR research and some have arisen in other areas of data analysis. In fact, research on kernel-based learning is a rich source of novel similarity representations because of the emphasis on encoding domain knowledge in the kernel function. In this paper, we present a taxonomy that organizes these new similarity mechanisms and more established similarity mechanisms in a coherent framework. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs) have been extensively used. However, it is known that SVMs face difficulty in solving large complex problems due to the intensive computation involved in their training algorithms, which are at least quadratic with respect to the number of training examples. This paper proposes a new, simple, and efficient network architecture which consists of several SVMs each trained on a small subregion of the whole data sampling space and the same number of simple neural quantizer modules which inhibit the outputs of all the remote SVMs and only allow a single local SVM to fire (produce actual output) at any time. In principle, this region-computing based modular network method can significantly reduce the learning time of SYM algorithms without sacrificing much generalization performance. The experiments on a few real large complex bench- mark problems demonstrate that our method can be significantly faster than single SVMs without losing much generalization performance. [ABSTRACT FROM AUTHOR]

This paper introduces a framework of experienced-based decision making as an extension of case-based decision making, a recently proposed alternative to expected utility theory. In experienced-based decision making, an agent faced with a new decision problem acts on the basis of experience gathered from previous problems in the past, either through predicting the utility of potential actions or through establishing a direct relationship between decision problems and appropriate actions. In the paper, a realization of the latter approach in the form of "satisficing" decision trees is proposed. [ABSTRACT FROM AUTHOR]

Machine learning provides an effective approach to execute big data analysis. As a branch of machine learning, classification has been widely adopted in data processing. However, the sensitivity of data raises the concern of data privacy. How to balance data utility and data privacy is a challenging issue. Privacy-preserving data classification, which supports flexible and privacy-friendly access to datasets and data classification, enables users' data to be collected in an authenticated manner. However, the priva-cy-preserving data classification approach has a limitation in that the correctness of data classification cannot be guaranteed. As a consequence, it is possible for a malicious classifier to manipulate the classification result. To solve these problems, in this article, we propose a transparent and accountable privacy-preserving data classification framework, which involves a tracer to assert the behavior of the classifier and maintains the utility and privacy of data. Specifically, we take advantage of cryptography techniques to balance data privacy and data utility, and use blockchain to achieve transparency and accountability for the behavior of the classifier. To illustrate the practicability of this framework, we implement concrete cryptographic algorithms and develop a prototype system to evaluate and test its performance. [ABSTRACT FROM AUTHOR]

Recently, artificial intelligence (AI) has seen great success in solving problems from various domains. It is envisioned that AI and Machine Learning (ML) will be an important technology for the successful development of future networks, especially in the dynamic service provisioning, adaptive traffic control, and security issues. Compared to meticulously manually designed strategies, AI&ML provides a generalized model and uniform learning method without prespecified processes for various network scenarios. Until now, limited research efforts have been done in Al-ernpowered future communications networks. [ABSTRACT FROM AUTHOR]

Process discovery is the learning task that entails the construction of process models from event logs of information systems. Typically, these event logs are large data sets that contain the process executions by registering what activity has taken place at a certain moment in time. By far the most arduous challenge for process discovery algorithms consists of tackling the problem of accurate and comprehensible knowledge discovery from highly flexible environments. Event logs from such flexible systems often contain a large variety of process executions which makes the application of process mining most interesting. However, simply applying existing process discovery techniques will often yield highly incomprehensible process models because of their inaccuracy and complexity. With respect to resolving this problem, trace clustering is one very interesting approach since it allows to split up an existing event log so as to facilitate the knowledge discovery process. In this paper, we propose a novel trace clustering technique that significantly differs from previous approaches. Above all, it starts from the observation that currently available techniques suffer from a large divergence between the clustering bias and the evaluation bias. By employing an active learning inspired approach, this bias divergence is solved. In an assessment using four complex, real-life event logs, it is shown that our technique significantly outperforms currently available trace clustering techniques. [ABSTRACT FROM PUBLISHER]

In this paper, we use shape grammars (SGs) for facade parsing, which amounts to segmenting 2D building facades into balconies, walls, windows, and doors in an architecturally meaningful manner. The main thrust of our work is the introduction of reinforcement learning (RL) techniques to deal with the computational complexity of the problem. RL provides us with techniques such as Q-learning and state aggregation which we exploit to efficiently solve facade parsing. We initially phrase the 1D parsing problem in terms of a Markov Decision Process, paving the way for the application of RL-based tools. We then develop novel techniques for the 2D shape parsing problem that take into account the specificities of the facade parsing problem. Specifically, we use state aggregation to enforce the symmetry of facade floors and demonstrate how to use RL to exploit bottom-up, image-based guidance during optimization. We provide systematic results on the Paris building dataset and obtain state-of-the-art results in a fraction of the time required by previous methods. We validate our method under diverse imaging conditions and make our software and results available online. [ABSTRACT FROM AUTHOR]

Level set methods are a popular way to solve the image segmentation problem. The solution contour is found by solving an optimization problem where a cost functional is minimized. Gradient descent methods are often used to solve this optimization problem since they are very easy to implement and applicable to general nonconvex functionals. They are, however, sensitive to local minima and often display slow convergence. Traditionally, cost functionals have been modified to avoid these problems. In this paper, we instead propose using two modified gradient descent methods, one using a momentum term and one based on resilient propagation. These methods are commonly used in the machine learning community. In a series of 2-D/3-D-experiments using real and synthetic data with ground truth, the modifications are shown to reduce the sensitivity for local optima and to increase the convergence rate. The parameter sensitivity is also investigated. The proposed methods are very simple modifications of the basic method, and are directly compatible with any type of level set implementation. Downloadable reference code with examples is available online. [ABSTRACT FROM PUBLISHER]

Pedestrian dead reckoning (PDR) is a popular indoor localization method due to its independence of additional infrastructures and the wide availability of smart devices. Step length estimation is a key component of PDR, which has an important influence on the performance of PDR. Existing step length estimation models suffer from various limitations such as requiring knowledge of user’s height, lack of consideration of varying phone carrying ways, and dependence on spatial constraints. To solve these problems, we propose a deep learning-based step length estimation model, which can adapt to different phone carrying ways and does not require individual stature information and spatial constraints. Experimental results show that the proposed method outperforms existing popular step length estimation methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose two cooperative ensemble learning algorithms, i.e., NegBagg and NegBoost, for designing neural network (NN) ensembles. The proposed algorithms incrementally train different individual NNs in an ensemble using the negative correlation learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different NNs in the ensemble. The idea behind using negative correlation learning in conjunction with the bagging/boosting algorithm is to facilitate interaction and cooperation among NNs during their training. Both NegBagg and NegBoost use a constructive approach to automatically determine the number of hidden neurons for NNs. NegBoost also uses the constructive approach to automatically determine the number of NNs for the ensemble. The two algorithms have been tested on a number of benchmark problems in machine learning and NNs, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, satellite, soybean, and waveform problems. The experimental results show that NegBagg and NegBoost require a small number of training epochs to produce compact NN ensembles with good generalization. [ABSTRACT FROM AUTHOR]

Even though evolutionary algorithm (EA) has now become the standard and paradigm for solving multi-objective design problems, the complexity of its genetic operation is, however, limiting its popularity in engineering applications. Hitherto, there has been insufficient research that addresses the inadequacy of EAs in extracting the characteristic landscape features of an objective function. However, increasing attentions have now been devoted to EAs based on probabilistic models (EAPMs) in computational intelligence studies. A real coded scalar population-based incremental learning algorithm, an EAPM, is proposed for multi-objective optimizations of electromagnetic devices. Major improvements include the design of a generating mechanism for new intermediate solutions, the selection of elite solutions to update the probability matrix, matrix updating formulations, and refinement mechanism for intervals to precisely generate intermediate solution. Also, a methodology to consider quantitatively both the number of improved objectives and the amount of improvements in a specified objective of multi-objective design problems is introduced in fitness assignments. Numerical results on a high frequency and a low inverse problem are reported to showcase the merits of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Online social media have aroused much research interest in recent years. In contrast to previous work that focused on the detection of emerging topics, this article undertakes the prediction of active users in online social events, which is so far rarely explored. This prediction task is formulated as a binary classification problem that built on real-world tweet streams, taking Ferguson event and New York Chockhold event as examples. Then, a comprehensive user feature system is designed to characterize the events’ online participants, which includes not only basic statistical characteristics and image-pixel-level features, but also some emotional features and personality features. Next, the Weighted Random Forest (Weighted-RF) classifier is adopted to solve the classification problem. Based on the user feature system and the classifier, the experience of a previous event can be archived and applied to the prediction of later similar events. Experimental results show that the Weighted-RF trained by samples of Ferguson event can effectively predict active users in NYC event, with an AUC value around 0.8392. Besides, the image-content based personality model provides a new tool for depicting user portraits, which further contributes to the quantitative analysis of online social events. [ABSTRACT FROM AUTHOR]

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm. [ABSTRACT FROM AUTHOR]

The encoder–decoder framework is prevalent in existing remote-sensing image captioning (RSIC) models. The appearance of attention mechanisms brings significant results. However, current attention-based caption models only build up the relationships between the local features without introducing the global visual feature and removing redundant feature components. It will cause caption models to generate descriptive sentences that are weakly related to the scene of images. To solve the problems, this article proposed a global visual feature-guided attention (GVFGA) mechanism. First, GVFGA introduces the global visual feature and fuses them with local visual features to build up their relationships between them. Second, an attention gate utilizing the global visual feature is proposed in GVFGA to filter out redundant feature components in the fused image features and provide more salient image features. In addition, to relieve the hidden state’s burden, a linguistic state (LS) is proposed to specifically provide textual features, making the hidden state only guiding visual–textual attention process. What’s more, to further refine the fusion of visual features and textual features, a LS-Guided Attention (LSGA) mechanism is proposed. It can also filter out the irrelevant information in the fused visual–textual feature with the help of an attention gate. The experimental results show that this proposed image captioning model can achieve better results on three RSIC datasets, UCM-Captions, Sydney-Captions, and RSICD datasets. [ABSTRACT FROM AUTHOR]

Given the prior information of the target, hyperspectral target detection focuses on exploiting spectral differences to separate objects of interest from the background, which can be treated as information retrieval (IR) task in machine learning (ML). Most traditional detection methods work in the original feature space and rely heavily on specific assumptions, which cannot guarantee effective extraction of features for the target and background in hyperspectral images (HSIs). Mass estimation (ME) is a base modeling mechanism that has been proven to effectively solve problems in IR and is not restricted by specific assumptions. In this article, we propose a novel target detection method through ensemble-based IR with ME (EIRME). By directly deriving the ordering from a sample set to rank data points, ME provides a simple and straightforward ranking measure to ensure that points similar to the given target are far away from dissimilar points. For the estimation of mass distribution, the proposed method utilizes a tree-structured mapping to generate a feature space, in which the separability of the target and background is further improved. In particular, to break through the technical difficulty that the direct migration of IR methods with mass measure cannot specifically meet the high-precision requirements of target detection in HSIs, we develop a specialized measurement, topological mass, which innovatively combines the mass measure with tree topology to quantify the spectral difference for detection output. Moreover, the IR with ME based on parallel measurements through ensemble trees provides a robust solution with better generalization capacity and higher precision for hyperspectral target detection, facilitating practical applications. Experimental results on benchmark HSI datasets prove that the specialized measurement that we developed successfully overcomes the drawbacks of the direct migration of IR methods with ME and exhibits unique advantages. In addition, comparisons with the most classic and advanced detection algorithms demonstrate the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Visual Question Answering (VQA) is a task to answer natural language questions tied to the content of visual images. Most recent VQA approaches usually apply attention mechanism to focus on the relevant visual objects and/or consider the relations between objects via off-the-shelf methods in visual relation reasoning. However, they still suffer from several drawbacks. First, they mostly model the simple relations between objects, which results in many complicated questions cannot be answered correctly, because of failing to provide sufficient knowledge. Second, they seldom leverage the harmony cooperation of visual appearance feature and relation feature. To solve these problems, we propose a novel end-to-end VQA model, termed Multi-modal Relation Attention Network (MRA-Net). The proposed model explores both textual and visual relations to improve performance and interpretability. In specific, we devise 1) a self-guided word relation attention scheme, which explore the latent semantic relations between words; 2) two question-adaptive visual relation attention modules that can extract not only the fine-grained and precise binary relations between objects but also the more sophisticated trinary relations. Both kinds of question-related visual relations provide more and deeper visual semantics, thereby improving the visual reasoning ability of question answering. Furthermore, the proposed model also combines appearance feature with relation feature to reconcile the two types of features effectively. Extensive experiments on five large benchmark datasets, VQA-1.0, VQA-2.0, COCO-QA, VQA-CP v2, and TDIUC, demonstrate that our proposed model outperforms state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The synthesis of a metasurface exhibiting a specific set of desired scattering properties is a time-consuming and resource-demanding process, which conventionally relies on many cycles of full-wave simulations. It requires an experienced designer to choose the number of metallic layers, the scatterer shapes and dimensions, and the type and the thickness of the separating substrates. In this article, we propose a generative machine learning (ML)-based approach to solve this one-to-many mapping and automate the inverse design of dual- and triple-layer metasurfaces. Using this approach, it is possible to solve optimization problems with single or more constraints by synthesizing thin structures composed of potentially brand-new scatterer designs, in cases where the interlayer coupling between the layers is nonnegligible and synthesis by traditional methods becomes cumbersome. Various examples to provide specific magnitude and phase responses of $x$ - and $y$ -polarized scattering coefficients across a frequency range as well as bounded responses for different metasurface applications are presented to verify the practicality of the proposed method. [ABSTRACT FROM AUTHOR]

In recent decades, deep learning (DL) has become a rapidly growing research direction, redefining the state-of-the-art performances in a wide range of techniques, such as object detection and speech recognition. In the aircraft design, dynamics, and control field, many works hinge on the information-rich data-driven approach, which includes the fusion-based prognostic and health management, the airliner's flight safety monitoring, intelligent sensing, and flight control systems development. While DL provides great potentials to solve these data-driven problems, a systematic review and discussion as to how the DL has been/can be used for these problems are still missing in relation to the rapidly developing and widely used DL techniques. In this article, we aim to address this urgent issue to provide a timely overview of the state-of-the-art for applying DL to the aircraft design, dynamics, and control field. In particular, we briefly introduce five representative DL methods, i.e., deep neural network, deep autoencoder, deep belief network, convolutional neural network, and recurrent neural network. Mathematical definitions for each method are presented, and illustrative applications are also discussed. We then review the existing DL-based works that have appeared in the aircraft design, dynamics, and control field. The review efforts are divided into two major groups, i.e., the own-ship aircraft modeling, wherein the works have been/can be implemented online for the aircraft design/dynamics/control, and other airplanes research works, wherein DL-based schemes provide offline monitoring of the aircraft operation. We then summarize the data sources and DL architectures. Referring to the experiences of DL research works/techniques development in other related fields, future opportunities, challenges, and potential solutions for implementing DL in the aircraft design, dynamics, and control field are also discussed. [ABSTRACT FROM AUTHOR]

This article proposes Bayesian optimization augmented factoring self-scheduling (BO FSS), a new parallel loop scheduling strategy. BO FSS is an automatic tuning variant of the factoring self-scheduling (FSS) algorithm and is based on Bayesian optimization (BO), a black-box optimization algorithm. Its core idea is to automatically tune the internal parameter of FSS by solving an optimization problem using BO. The tuning procedure only requires online execution time measurement of the target loop. In order to apply BO, we model the execution time using two Gaussian process (GP) probabilistic machine learning models. Notably, we propose a locality-aware GP model, which assumes that the temporal locality effect resembles an exponentially decreasing function. By accurately modeling the temporal locality effect, our locality-aware GP model accelerates the convergence of BO. We implemented BO FSS on the GCC implementation of the OpenMP standard and evaluated its performance against other scheduling algorithms. Also, to quantify our method’s performance variation on different workloads, or workload-robustness in our terms, we measure the minimax regret. According to the minimax regret, BO FSS shows more consistent performance than other algorithms. Within the considered workloads, BO FSS improves the execution time of FSS by as much as 22% and 5% on average. [ABSTRACT FROM AUTHOR]

Approximate Nearest Neighbor Search (ANNS) is a fundamental problem in many areas of machine learning and data mining. During the past decade, numerous hashing algorithms are proposed to solve this problem. Every proposed algorithm claims to outperform Locality Sensitive Hashing (LSH), which is the most popular hashing method. However, the evaluation of these hashing article was not thorough enough, and the claim should be re-examined. If implemented correctly, almost all the hashing methods will have their performance improved as the code length increases. However, many existing hashing article only report the performance with the code length shorter than 128. In this article, we carefully revisit the problem of search-with-a-hash-index and analyze the pros and cons of two popular hash index search procedures. Then we proposed a simple but effective novel hash index search approach and made a thorough comparison of eleven popular hashing algorithms. Surprisingly, the random-projection-based Locality Sensitive Hashing ranked the first, which is in contradiction to the claims in all the other 10 hashing article. Despite the extreme simplicity of random-projection-based LSH, our results show that the capability of this algorithm has been far underestimated. For the sake of reproducibility, all the codes used in the article are released on GitHub, which can be used as a testing platform for a fair comparison between various hashing algorithms. [ABSTRACT FROM AUTHOR]

Cognitive Radio (CR) technology was first introduced to solve the problem of radio spectrum under-utilization. A cognitive radio network consists of smart radio devices that have the ability to sense radio environment variables and take actions accordingly. To realize their full potential and to become fully cognitive, the CR nodes need to be equipped with learning and reasoning capabilities. Machine learning has been one of the enabling vehicles for intelligent CR networks. Inspired by the cognition cycle of a CR node, over the past years there has been an ever growing interest in using machine learning techniques to enhance the performance of CR networks. In this article, an overview of the various learning techniques currently used in the literature of CR networks is given. We focus on feature classification and clustering algorithms, and their application in cooperative CR networks. We outline the steps to establishing a learning-based cooperative secondary network, highlighting factors that impact detection performance. Additionally, current state-of-the-art learning-based applications in Cognitive Internet of Things (CIoT) are presented. Finally, the key challenges and future directions of intelligent cognitive networks are discussed. [ABSTRACT FROM AUTHOR]

The trade-off between feature representation power and spatial localization accuracy is crucial for the dense classification/semantic segmentation of remote sensing images (RSIs). High-level features extracted from the late layers of a neural network are rich in semantic information, yet have blurred spatial details; low-level features extracted from the early layers of a network contain more pixel-level information but are isolated and noisy. It is therefore difficult to bridge the gap between high- and low-level features due to their difference in terms of physical information content and spatial distribution. In this article, we contribute to solve this problem by enhancing the feature representation in two ways. On the one hand, a patch attention module (PAM) is proposed to enhance the embedding of context information based on a patchwise calculation of local attention. On the other hand, an attention embedding module (AEM) is proposed to enrich the semantic information of low-level features by embedding local focus from high-level features. Both proposed modules are lightweight and can be applied to process the extracted features of convolutional neural networks (CNNs). Experiments show that, by integrating the proposed modules into a baseline fully convolutional network (FCN), the resulting local attention network (LANet) greatly improves the performance over the baseline and outperforms other attention-based methods on two RSI data sets. [ABSTRACT FROM AUTHOR]

Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully applied to various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions. [ABSTRACT FROM AUTHOR]

We propose a framework for localization and classification of masses in breast ultrasound images. We have experimentally found that training convolutional neural network-based mass detectors with large, weakly annotated datasets presents a non-trivial problem, while overfitting may occur with those trained with small, strongly annotated datasets. To overcome these problems, we use a weakly annotated dataset together with a smaller strongly annotated dataset in a hybrid manner. We propose a systematic weakly and semi-supervised training scenario with appropriate training loss selection. Experimental results show that the proposed method can successfully localize and classify masses with less annotation effort. The results trained with only 10 strongly annotated images along with weakly annotated images were comparable to results trained from 800 strongly annotated images, with the 95% confidence interval (CI) of difference −3%–5%, in terms of the correct localization (CorLoc) measure, which is the ratio of images with intersection over union with ground truth higher than 0.5. With the same number of strongly annotated images, additional weakly annotated images can be incorporated to give a 4.5% point increase in CorLoc, from 80% to 84.50% (with 95% CIs 76%–83.75% and 81%–88%). The effects of different algorithmic details and varied amount of data are presented through ablative analysis. [ABSTRACT FROM AUTHOR]

By leveraging advances in deep learning, challenging pattern recognition problems have been solved in computer vision, speech recognition, natural language processing, and more. Mobile computing has also adopted these powerful modeling approaches, delivering astonishing success in the field’s core application domains, including the ongoing transformation of human activity recognition technology through machine learning. [ABSTRACT FROM AUTHOR]

How can the advantages of deep learning be brought to the emerging world of embedded IoT devices? The authors discuss several core challenges in embedded and mobile deep learning, as well as recent solutions demonstrating the feasibility of building IoT applications that are powered by effective, efficient, and reliable deep learning models. [ABSTRACT FROM AUTHOR]

Compressive sensing of signals drawn from a Gaussian mixture model (GMM) admits closed-form minimum mean squared error reconstruction from incomplete linear measurements. An accurate GMM signal model is usually not available a priori, because it is difficult to obtain training signals that match the statistics of the signals being sensed. We propose to solve that problem by learning the signal model in situ, based directly on the compressive measurements of the signals, without resorting to other signals to train a model. A key feature of our method is that the signals being sensed are treated as random variables and are integrated out in the likelihood. We derive a maximum marginal likelihood estimator (MMLE) that maximizes the likelihood of the GMM of the underlying signals given only their linear compressive measurements. We extend the MMLE to a GMM with dominantly low-rank covariance matrices, to gain computational speedup. We report extensive experimental results on image inpainting, compressive sensing of high-speed video, and compressive hyperspectral imaging (the latter two based on real compressive cameras). The results demonstrate that the proposed methods outperform state-of-the-art methods by significant margins. [ABSTRACT FROM AUTHOR]

A Bayesian dynamic model based on multitask learning (MTL) is developed for radar automatic target recognition (RATR) using high-resolution range profile (HRRP). The aspect-dependent HRRP sequence is modeled using a truncated stick-breaking hidden Markov model (TSB-HMM) with time-evolving transition probabilities, in which the spatial structure across range cells is described by the hidden Markov structure and the temporal dependence between HRRP samples is described by the time evolution of the transition probabilities. This framework imposes the belief that temporally proximate HRRPs are more likely to be drawn from similar HMMs, while also allowing for possible distant repetition or “innovation”. In addition, as formulated the stick-breaking prior and MTL mechanism are employed to infer the number of hidden states in an HMM and learn the target-dependent states collectively for all targets. The form of the proposed hierarchical model allows efficient variational Bayesian (VB) inference, of interest for large-scale problems. To validate the formulation, example results are presented for an illustrative synthesized dataset and our main application—RATR, for which we consider the measured HRRP data. For the latter, we also make comparisons to the model with the independent state-transition statistics and some other existing statistical models for radar HRRP data. [ABSTRACT FROM PUBLISHER]