Search

Adversarial attacks have been extensively studied in recent years since they can identify the vulnerability of deep learning models before deployed. In this paper, we consider the black-box adversarial setting, where the adversary needs to craft adversarial examples without access to the gradients of a target model. Previous methods attempted to approximate the true gradient either by using the transfer gradient of a surrogate white-box model or based on the feedback of model queries. However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information. To address these problems and improve black-box attacks, we propose two prior-guided random gradient-free (PRGF) algorithms based on biased sampling and gradient averaging, respectively. Our methods can take the advantage of a transfer-based prior given by the gradient of a surrogate model and the query information simultaneously. Through theoretical analyses, the transfer-based prior is appropriately integrated with model queries by an optimal coefficient in each method. Extensive experiments demonstrate that, in comparison with the alternative state-of-the-arts, both of our methods require much fewer queries to attack black-box models with higher success rates. [ABSTRACT FROM AUTHOR]

Nonlinear optimization problems with dynamical parameters are widely arising in many practical scientific and engineering applications, and various computational models are presented for solving them under the hypothesis of short-time invariance. To eliminate the large lagging error in the solution of the inherently dynamic nonlinear optimization problem, the only way is to estimate the future unknown information by using the present and previous data during the solving process, which is termed the future dynamic nonlinear optimization (FDNO) problem. In this paper, to suppress noises and improve the accuracy in solving FDNO problems, a novel noise-tolerant neural (NTN) algorithm based on zeroing neural dynamics is proposed and investigated. In addition, for reducing algorithm complexity, the quasi-Newton Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is employed to eliminate the intensively computational burden for matrix inversion, termed NTN-BFGS algorithm. Moreover, theoretical analyses are conducted, which show that the proposed algorithms are able to globally converge to a tiny error bound with or without the pollution of noises. Finally, numerical experiments are conducted to validate the superiority of the proposed NTN and NTN-BFGS algorithms for the online solution of FDNO problems. [ABSTRACT FROM AUTHOR]

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

In sensor networks (SNs), how to allocate the resources so as to optimize data gathering and network utility is an important and challenging task. This paper studies the distributed optimization problem in SNs. A distributed hybrid-driven algorithm based on the coordinate descent method is presented for the optimization purpose. The proposed optimization algorithm differs from the existing ones since the hybrid driven scheme allows more choices of actuation time, resulting a tradeoff between communications and computation performance. Applying the proposed algorithm, each sensor node is driven in a hybrid event time manner, which removes the requirement of strict time synchronization. The convergence and optimality of the proposed algorithm are analyzed, and then verified by simulation examples. The developed results also show the tradeoff between communications and computation performance. [ABSTRACT FROM AUTHOR]

Estimating the probability of failures or accidents with aerospace systems is often necessary when new concepts or designs are introduced, as it is being done for autonomous aircraft. If the design is safe, as it is supposed to be, accident cases are hard to find. Such analysis needs some variance reduction technique and several algorithms exist for that; however, specific model features may cause difficulties in practice, such as the case of system models where independent agents have to autonomously accomplish missions within finite time, and likely with the presence of human agents. For handling these scenarios, this paper presents a novel estimation approach, based on the combination of the well-established variation reduction technique of interacting particles system with the long-standing optimization algorithm denominated DIviding RECTangles. When combined, these two techniques yield statistically significant results for extremely low probabilities. In addition, this novel approach allows the identification of intermediate events and simplifies the evaluation of sensitivity of the estimated probabilities to certain system parameters. [ABSTRACT FROM AUTHOR]

In quantitative magnetic resonance ${T}_{\textsf {1}}$ mapping, the variable flip angle (VFA) steady state spoiled gradient recalled echo (SPGR) imaging technique is popular as it provides a series of high resolution ${T}_{\textsf {1}}$ weighted images in a clinically feasible time. Fast, linear methods that estimate ${T}_{\textsf {1}}$ maps from these weighted images have been proposed, such as DESPOT1 and iterative re-weighted linear least squares. More accurate, non-linear least squares (NLLS) estimators are in play, but these are generally much slower and require careful initialization. In this paper, we present NOVIFAST, a novel NLLS-based algorithm specifically tailored to VFA SPGR ${T}_{\textsf {1}}$ mapping. By exploiting the particular structure of the SPGR model, a computationally efficient, yet accurate and precise ${T}_{\textsf {1}}$ map estimator is derived. Simulation and in vivo human brain experiments demonstrate a twenty-fold speed gain of NOVIFAST compared with conventional gradient-based NLLS estimators while maintaining a high precision and accuracy. Moreover, NOVIFAST is eight times faster than the efficient implementations of the variable projection (VARPRO) method. Furthermore, NOVIFAST is shown to be robust against initialization. [ABSTRACT FROM AUTHOR]

It is a promising solution for real-time multitarget tracking to implement particle cardinality balanced multitarget multi-Bernoulli (CBMeMBer) filters in hardware platforms. However, this solution is difficult to materialize since there is a contradiction between the time-varying number of Bernoulli intensity components in CBMeMBer filters and the limited number of particles in hardware platforms. Moreover, real-time hardware implementation requires a resampling procedure that is suitable for parallel processing, while the existing parallel resampling algorithms oversimplify this procedure, resulting in estimation performance degradation. In this paper, we propose an optimization algorithm of particle allocation to overcome the above-mentioned contradiction, and a parallel resampling algorithm to improve the estimation performance. Numerical experiments demonstrate the effectiveness of the proposed algorithms in multitarget tracking. [ABSTRACT FROM AUTHOR]

Disparity estimation is an essential task taking part in many light-field applications. Due to the complexity of algorithms and high dimensional property of light-field data, performing this task involves a significant computational effort and results in very long processing time on CPU. Graphics processing units (GPUs), which is capable of massively parallel processing, is a promising solution to cover the computation requirement and speed up the task. In this paper, we develop a GPU-accelerated approach for light-field disparity estimation using a variational computation framework (GVLD). Our algorithm combines the intrinsic sub-pixel precision of variational formulation and the effectiveness of weighted median filtering to produce a highly accurate solution. The proposed algorithm is fully parallelized and optimized for the implementation using the OpenCL framework. An intensive evaluation including a quantitative comparison to related works and a detailed analysis of the proposed approach’s performance is presented. Experimental results demonstrate our superior performance compared to state-of-the-art approaches. The proposed approach is 10+ times faster than other approaches running on a similar GPU platform and provides the most accurate solution among optimization-based approaches. Compared to the implementation running on CPU, our GPU-accelerated method achieves up to $365\times $ speed up. [ABSTRACT FROM AUTHOR]

The unceasing demand for continuous situational awareness calls for innovative and large-scale signal processing algorithms, complemented by collaborative and adaptive sensing platforms to accomplish the objectives of layered sensing and control. Towards this goal, the present paper develops a spline-based approach to field estimation, which relies on a basis expansion model of the field of interest. The model entails known bases, weighted by generic functions estimated from the field's noisy samples. A novel field estimator is developed based on a regularized variational least-squares (LS) criterion that yields finite-dimensional (function) estimates spanned by thin-plate splines. Robustness considerations motivate well the adoption of an overcomplete set of (possibly overlapping) basis functions, while a sparsifying regularizer augmenting the LS cost endows the estimator with the ability to select a few of these bases that “better” explain the data. This parsimonious field representation becomes possible, because the sparsity-aware spline-based method of this paper induces a group-Lasso estimator for the coefficients of the thin-plate spline expansions per basis. A distributed algorithm is also developed to obtain the group-Lasso estimator using a network of wireless sensors, or, using multiple processors to balance the load of a single computational unit. The novel spline-based approach is motivated by a spectrum cartography application, in which a set of sensing cognitive radios collaborate to estimate the distribution of RF power in space and frequency. Computer simulations and tests on real data corroborate that the estimated power spectrum density atlas yields the desired RF state awareness, since the maps reveal spatial locations where idle frequency bands can be reused for transmission, even when fading and shadowing effects are pronounced. [ABSTRACT FROM PUBLISHER]

In this paper, we analyze and compare the properties of different well-known and also new nonparametric discrete Fourier transform (DFT)-based methods for resonant frequency and logarithmic decrement estimation in application to mechanical spectroscopy. We derive a new DFT interpolation algorithm for a signal analyzed with Rife–Vincent class-I windows and also propose new formulas that extend Bertocco and Yoshida methods. We study errors of the resonant frequency and logarithmic decrement estimation in realistic conditions that include measurement noise and a zero-point drift. We also investigate the systematic errors of the estimation methods of interest. A nonlinear least squares time-domain parametric signal fitting is used to determine the boundaries of statistical efficiency in all tests. [ABSTRACT FROM AUTHOR]

Blind image deblurring is a challenging problem in computer vision, which aims to restore both the blur kernel and the latent sharp image from only a blurry observation. Inspired by the prevalent self-example prior in image super-resolution, in this paper, we observe that a coarse enough image down-sampled from a blurry observation is approximately a low-resolution version of the latent sharp image. We prove this phenomenon theoretically and define the coarse enough image as a latent structure prior of the unknown sharp image. Starting from this prior, we propose to restore sharp images from the coarsest scale to the finest scale on a blurry image pyramid and progressively update the prior image using the newly restored sharp image. These coarse-to-fine priors are referred to as multi-scale latent structures (MSLSs). Leveraging the MSLS prior, our algorithm comprises two phases: 1) we first preliminarily restore sharp images in the coarse scales and 2) we then apply a refinement process in the finest scale to obtain the final deblurred image. In each scale, to achieve lower computational complexity, we alternately perform a sharp image reconstruction with fast local self-example matching, an accelerated kernel estimation with error compensation, and a fast non-blind image deblurring, instead of computing any computationally expensive non-convex priors. We further extend the proposed algorithm to solve more challenging non-uniform blind image deblurring problem. The extensive experiments demonstrate that our algorithm achieves the competitive results against the state-of-the-art methods with much faster running speed. [ABSTRACT FROM AUTHOR]

Reconstructing a full-resolution stereoscopic video from an asymmetric stereoscopic video is a challenging task. The existing approaches require depth information, which imposes an additional challenge in data acquisition. In this paper, we propose a novel scheme that is capable of obtaining super-resolution and depth estimation simultaneously from an asymmetric stereoscopic video. The proposed scheme models the video super-resolution and stereo matching with a unified energy function. Then, we apply an alternating optimization method to minimize this energy function, which can be implemented with a two-step algorithm. In the first step we calculate the initial depth map by using a region-based cooperative optimization technique while considering the temporal consistency in video. In the second step we resolve the super-resolution problem under the guidance of the depth information. It is effective because each step benefits from the additional improvement over the previous step. We iteratively update the two steps until stable depth and super-resolution results are obtained. We have conducted a series of experiments on public stereoscopic video sequences to evaluate the performance of the proposed method. Both objective indexes and subjective visual comparisons verify that the proposed scheme can achieve satisfactory super-resolution results and high-quality depth map simultaneously. In particular, the subjective evaluation experiments on a 3-D monitor show that this scheme outperforms others and achieves the best visual sharpness. [ABSTRACT FROM AUTHOR]

We propose Graph-Cut RANSAC, GC-RANSAC in short, a new robust geometric model estimation method where the local optimization step is formulated as energy minimization with binary labeling, applying the graph-cut algorithm to select inliers. The minimized energy reflects the assumption that geometric data often form spatially coherent structures – it includes both a unary component representing point-to-model residuals and a binary term promoting spatially coherent inlier-outlier labelling of neighboring points. The proposed local optimization step is conceptually simple, easy to implement, efficient with a globally optimal inlier selection given the model parameters. Graph-Cut RANSAC, equipped with “the bells and whistles” of USAC and MAGSAC++, was tested on a range of problems using a number of publicly available datasets for homography, 6D object pose, fundamental and essential matrix estimation. It is more geometrically accurate than state-of-the-art robust estimators, fails less often and runs faster or with speed similar to less accurate alternatives. The source code is available at https://github.com/danini/graph-cut-ransac. [ABSTRACT FROM AUTHOR]

Many statistical estimation procedures lead to nonconvex optimization problems. Algorithms to solve these problems are often guaranteed to output a stationary point of the optimization problem. Oracle inequalities are an important theoretical instrument to assess the statistical performance of an estimator. Oracle results have focused on the theoretical properties of the uncomputable (global) minimum or maximum. In this paper, a general framework used for convex optimization problems to derive oracle inequalities for stationary points is extended. A main new ingredient of these oracle inequalities is that they are sharp: they show closeness to the best approximation within the model plus a remainder term. We apply this framework to different estimation problems. [ABSTRACT FROM AUTHOR]

This paper presents a variational-based approach for fusing hyperspectral and multispectral images. The fusion problem is formulated as an inverse problem whose solution is the target image assumed to live in a lower dimensional subspace. A sparse regularization term is carefully designed, relying on a decomposition of the scene on a set of dictionaries. The dictionary atoms and the supports of the corresponding active coding coefficients are learned from the observed images. Then, conditionally on these dictionaries and supports, the fusion problem is solved via alternating optimization with respect to the target image (using the alternating direction method of multipliers) and the coding coefficients. Simulation results demonstrate the efficiency of the proposed algorithm when compared with state-of-the-art fusion methods. [ABSTRACT FROM PUBLISHER]

A fundamental problem in multimodal signal processing is to quantify relations between two different signals with respect to a certain phenomenon. In this paper, we address this problem from a kernel-based perspective and propose a measure that is based on affinity kernels constructed separately in each modality. This measure is motivated from both a kernel density estimation point of view of predicting the signal in one modality based on the other, as well as from a statistical model, which implies that high values of the proposed measure are expected when signals highly correspond to each other. Considering an online setting, we propose an efficient algorithm for the sequential update of the proposed measure, and demonstrate its application to eye-fixation prediction in audio-visual recordings. The goal is to predict locations within a video recording at which people gaze when watching the video. As studies in psychology imply, people tend to gaze at the location of the audio source, so that their prediction becomes equivalent to locating the audio source within the video. Therefore, we propose to predict eye-fixations as regions within the video with the highest correspondence to the audio signal, thereby demonstrating the improved performance of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we present new algorithms and analysis for the linear inverse sensor placement and scheduling problems over multiple time instances with power and communications constraints. The proposed algorithms, which deal directly with minimizing the mean squared error (MSE), are based on the convex relaxation approach to address the binary optimization scheduling problems that are formulated in sensor network scenarios. We propose to balance the energy and communications demands of operating a network of sensors over time while we still guarantee a minimum level of estimation accuracy. We measure this accuracy by the MSE for which we provide average case and lower bounds analyses that hold in general, irrespective of the scheduling algorithm used. We show experimentally how the proposed algorithms perform against state-of-the-art methods previously described in the literature. [ABSTRACT FROM AUTHOR]

This paper presents a technique for noise and vibration reduction in low-speed sensorless control algorithm of surface mounted permanent magnet synchronous machines (SM-PMSM). The signal processing involved in high-frequency-based sensorless control algorithms has a significant effect on the vibration and noise accompanying such algorithms. Therefore, a wavelet filtering approach is proposed. This wavelet filter is capable of extracting the position information even at low values of injection voltage amplitude where conventional filters fail to extract the information. This leads to a reduction in the machine's noise and vibration. Furthermore, an optimization using hardware-in-the-loop method based on genetic algorithms was implemented in real time to obtain the optimum value of the injected voltages amplitude and frequency causing the least possible vibration with acceptable position estimation accuracy. Consequently, a significant reduction in vibration and noise was achieved. The proposed techniques were implemented on a 2-hp SM-PMSM motor both experimentally and through simulations. Comparisons between numerical and test results confirm the validity of the proposed techniques. [ABSTRACT FROM PUBLISHER]

This paper proposes a decentralized state-estimation approach that relies on an elaborated instance of the Lagrangian relaxation decomposition technique. The proposed algorithm does not require a central coordinator but just to moderate interchanges of information among neighboring regions, and exploits the structure of the problem to achieve a fast and accurate convergence. Additionally, a decentralized bad measurement identification procedure is developed, which is efficient and robust in terms of identifying bad measurements within regions and in border tie-lines. The accuracy and efficiency of the proposed procedures are assessed by a large number of simulations, which allows drawing statistically sound conclusions. [ABSTRACT FROM PUBLISHER]

Recent studies have shown high interest in statistical methods dedicated to the prediction of the maximum confidence levels in simulations and measurements for risk assessment in electromagnetic compatibility. In particular, it has been shown that one of the main issues remains the access to a number of samples allowing the assessment of the risks in regards to test setup random variables. In this paper, it is argued that a real-time bootstrapping module enables to optimize the number of experiments while estimating the maximum confidence levels of the accessible samples. An efficient technique is presented to substitute the full experimental requirements to a simple and low time-consuming mathematical algorithm. The benefits expected of this method are illustrated throughout experimental shielding effectiveness power measurements. Based upon an optimized statistical approach, the first statistical moments (e.g., mean and standard deviation) of power received inside a cabinet are provided jointly with an accurate assessment of power's quantiles, leading to a better evaluation of extreme values. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new video inpainting method which applies to both static or free-moving camera videos. The method can be used for object removal, error concealment, and background reconstruction applications. To limit the computational time, a frame is inpainted by considering a small number of neighboring pictures which are grouped into a group of pictures (GoP). More specifically, to inpaint a frame, the method starts by aligning all the frames of the GoP. This is achieved by a region-based homography computation method which allows us to strengthen the spatial consistency of aligned frames. Then, from the stack of aligned frames, an energy function based on both spatial and temporal coherency terms is globally minimized. This energy function is efficient enough to provide high quality results even when the number of pictures in the GoP is rather small, e.g. 20 neighboring frames. This drastically reduces the algorithm complexity and makes the approach well suited for near real-time video editing applications as well as for loss concealment applications. Experiments with several challenging video sequences show that the proposed method provides visually pleasing results for object removal, error concealment, and background reconstruction context. [ABSTRACT FROM PUBLISHER]

We present a novel iterative feedback approach for the simultaneous estimation of depth and all-in-focus (AIF) videos from a defocused video pair by joint spatiotemporal optimization. Depth and AIF videos benefit each other in the iterative optimization. First, for the recovery of AIF video, the sparse prior of natural video is incorporated to ensure a high-quality defocus blur removal even under inaccurate depth estimation. Second, in depth estimation step, we feed back the spatial and temporal constraints from the high-quality AIF video and adopt a numerical solution, which is robust to the inaccuracy of AIF recovery to further boost the performance of depth from the defocus algorithm. Benefitting from the incorporation of AIF video priors and the temporal consistency constraint, the proposed framework can effectively reconstruct the depth of the textureless region and is insensitive to camera parameter changes. Our approach provides better temporal consistency and higher depth accuracy than the conventional method that applies postsmoothing to the sequential frame estimation. We not only demonstrate the feasibility of our approach via real experimentation but also provide visual and quantitative evaluation on synthetic data. [ABSTRACT FROM AUTHOR]

The Levenberg–Marquardt (LM) algorithm is a nonlinear parameter learning algorithm that converges accurately and quickly. This paper demonstrates for the first time to our knowledge, a real-time implementation of the LM algorithm on field programmable gate arrays (FPGAs). It was used to train neural networks to solve the eXclusive Or function (XOR), and for 3D-to-2D camera calibration parameter estimation. A Xilinx Virtex-5 ML506 was used to implement the LMA as a hardware-in-the-loop system. The XOR function was approximated in only 13 iterations from zero initial conditions, usually the same function is approximated in thousands of iterations using the error backpropagation algorithm. Also, this type of training not only reduced the number of iterations but also achieved a speed up in excess of 3 \times 10^6 when compared to the software implementation. A real-time camera calibration and parameter estimation was performed successfully on FPGAs. Compared to the software implementation the FPGA implementation led to an increase in the mean squared error and standard deviation by only 17.94% and 8.04% respectively. The FPGA increased the calibration speed by a factor of 1.41 \times 10^6. There are a wide range of systems problems solved via nonlinear parameter optimization, this study demonstrated that a hardware solution for systems such as automated optical inspection systems or systems dealing with projective geometry estimation and motion compensation systems in robotic vision systems is possible in real time. [ABSTRACT FROM PUBLISHER]

Autonomous agents deployed or moving on land for the purpose of carrying out coordinated tasks need to have good knowledge of their absolute or relative position. For large formations, it is often impractical to equip each agent with an absolute sensor such as GPS, whereas relative range sensors measuring interagent distances are cheap and commonly available. In this setting, this paper considers the problem of autonomous distributed estimation of the position of each agent in a networked formation using noisy measurements of interagent distances. The underlying geometrical problem has been studied quite extensively in various fields, ranging from molecular biology to robotics, and it is known to lead to a hard nonconvex optimization problem. Centralized algorithms do exist that work reasonably well in finding local or global minimizers for this problem (e.g., semidefinite programming relaxations). Here, we explore a fully decentralized approach for localization from range measurements, and we propose a computational scheme based on a distributed gradient algorithm with Barzilai–Borwein stepsizes. The advantage of this distributed approach is that each agent may autonomously compute its position estimate, exchanging information only with its neighbors, without need of communicating with a central station and without needing complete knowledge of the network structure. [ABSTRACT FROM PUBLISHER]

A new method for robust estimation, MAGSAC++, is proposed. It introduces a new model quality (scoring) function that does not make inlier-outlier decisions, and a novel marginalization procedure formulated as an M-estimation with a novel class of M-estimators (a robust kernel) solved by an iteratively re-weighted least squares procedure. Instead of the inlier-outlier threshold, it requires only its loose upper bound which can be chosen from a significantly wider range. Also, we propose a new termination criterion and a technique for selecting a set of inliers in a data-driven manner as a post-processing step after the robust estimation finishes. On a number of publicly available real-world datasets for homography, fundamental matrix fitting and relative pose, MAGSAC++ produces results superior to the state-of-the-art robust methods. It is more geometrically accurate, fails fewer times, and it is often faster. It is shown that MAGSAC++ is significantly less sensitive to the setting of the threshold upper bound than the other state-of-the-art algorithms to the inlier-outlier threshold. Therefore, it is easier to be applied to unseen problems and scenes without acquiring information by hand about the setting of the inlier-outlier threshold. The source code and examples both in C++ and Python are available at https://github.com/danini/magsac. [ABSTRACT FROM AUTHOR]

Nonlinear dimensionality reduction methods have demonstrated top-notch performance in many pattern recognition and image classification tasks. Despite their popularity, they suffer from highly expensive time and memory requirements, which render them inapplicable to large-scale datasets. To leverage such cases we propose a new method called “Path-Based Isomap”. Similar to Isomap, we exploit geodesic paths to find the low-dimensional embedding. However, instead of preserving pairwise geodesic distances, the low-dimensional embedding is computed via a path-mapping algorithm. Due to the much fewer number of paths compared to number of data points, a significant improvement in time and memory complexity with a comparable performance is achieved. The method demonstrates state-of-the-art performance on well-known synthetic and real-world datasets, as well as in the presence of noise. [ABSTRACT FROM PUBLISHER]

While the convergence rate of any optimization algorithm can be increased by selecting a larger stepsize parameter, this also magnifies the parameter estimation variance. This research introduces an analytical methodology for comparing the estimation variance when optimizing Soft-Max models using two different loss functions (Cross Entropy and Mean Square Error). This method can be used to adjust the stepsize parameters of algorithms with different loss functions so that they attain equal estimation variances after converging; this provides a fair starting point for a comparison of the convergence rates of the algorithms. The polyphonic pitch detection problem is used as a case study, and the performance of the models trained by the cross entropy loss function and the mean square error loss function are compared with each other and with a k-nearest neighbor classifier. [ABSTRACT FROM PUBLISHER]

Geo-localization services are an important functionality in cellular networks. Besides, the use of Ultra Dense Networks and small cells, in current and future cellular networks, greatly increases the complexity of centralized localization approaches. Consequently, we propose a Self-Synchronization Positioning Estimation (SSPE) algorithm that estimates the transmitter position in a distributed fashion. The proposed SSPE algorithm reaches consensus for the posterior distribution of the transmitter position rather than on the final estimates. Such consensus ensures that the proposed SSPE algorithm converges to the centralized Direct Positioning Estimation (DPE) approach, which has the best performance of all localization approaches. We show that the proposed algorithm is related to the Iterative Positioning Estimation (IPE) algorithm, since both exploit the self-synchronization mechanism. As a result, the improvements and extensions for IPE, previously studied in other works, can be directly applied to the proposed SSPE algorithm. In addition, the proposed algorithm is able to localize the transmitter even when it is not time synchronized with the network as it is usually the case. The performance of the algorithms is numerically assessed through Monte-Carlo simulations by the mean distance error and mean range offset error. Finally, we not only show that our approach gets close to the DPE performance after a few iterations, but also that it converges for different logical network configurations. [ABSTRACT FROM AUTHOR]

Wireless communication in vehicular networks is highly vulnerable to electromagnetic interference, which seriously undermines the communication effectiveness. Thus, interference estimation and elimination are essential in vehicular communications. To solve the multitasking problem of interference estimation and cancelation within the transformation domain of vehicular communications where they are divided into the consistency and inconsistency based on the different conditions of spectrum sensing, the present study proposes an iterative Bayesian sparse learning approach for simultaneous interference recovery and elimination which applies multiple Bayesian learning for measured signals, and the structured sparse learning method for simultaneous interference mitigation with signal recovery. Furthermore, the updating optimization based on Bayesian estimation framework is performed to facilitate the learning process and improve the performance of interference estimation and signal reconstruction. Simulation results show that the proposed algorithms achieved more rapid convergence and more accurate recovery (31.84% and 27.88% higher in spectrum sensing with consistency and inconsistency, respectively) than conventional algorithms. During all iterations, the proposed algorithms, because of updating optimization, were more robust than conventional ones, but the computational complexity remained largely the same. To explore the correlation between the parameters and the interference recovery performance, we performed experiments, which revealed linear, logarithmic, and exponential regression correlations for particular forms of interference. We also compared the demodulation performance of the proposed algorithm with existing methods, and it turned out the proposed algorithm also obtained higher gains on average in the bit error rate. [ABSTRACT FROM AUTHOR]

A nonlinear operator approach to estimation in discrete-time systems is described. It involves inferential estimation of a signal which enters a communications channel involving both nonlinearities and transport delays. The measurements are assumed to be corrupted by a colored noise signal which is correlated with the signal to be estimated. The system model may also include a communications channel involving either static or dynamic nonlinearities. The signal channel is represented in a very general nonlinear operator form. The algorithm is relatively simple to derive and to implement. [ABSTRACT FROM AUTHOR]

We study the problem of learning mixtures of low-rank models, i.e. reconstructing multiple low-rank matrices from unlabelled linear measurements of each. This problem enriches two widely studied settings — low-rank matrix sensing and mixed linear regression — by bringing latent variables (i.e. unknown labels) and structural priors (i.e. low-rank structures) into consideration. To cope with the non-convexity issues arising from unlabelled heterogeneous data and low-complexity structure, we develop a three-stage meta-algorithm that is guaranteed to recover the unknown matrices with near-optimal sample and computational complexities under Gaussian designs. In addition, the proposed algorithm is provably stable against random noise. We complement the theoretical studies with empirical evidence that confirms the efficacy of our algorithm. [ABSTRACT FROM AUTHOR]

The plane-wave destruction (PWD) method has been a widely used local slope estimation method in the seismic community. It is based on the discretization of the plane-wave partial differential equation (PDE) and the linearization of the PDE with respect to the local slope. Solving the linearized inverse problem for the slope perturbation is equivalent to solving a smoothness constrained optimization based on a shaping regularization method. The smoothness constraint in the shaping regularization is important in that it not only controls the stability and smoothness of the solution, i.e., slope perturbation, but also affects the accuracy and resolution of the solution. The traditional PWD algorithm is not easy to compromise between the smoothness and resolution of the estimated local slope because it uses a stationary triangle smoothing operator as the shaping operator. Here, we propose to improve the robustness of the PWD algorithm by introducing a nonstationary triangle smoothing operator into the shaping regularization framework in order to adaptively constrain the solution according to the local signal reliability. The smoothing is weak in areas with a higher probability of signals and is strong in areas with a higher likelihood of noise. The smoothing radius can be adaptively estimated based on an optimization model, which is solved by a line-search method. The proposed new slope estimation method is referred to as a nonstationary method compared with the traditional stationary one. The effectiveness and benefits of the new slope estimation method are validated via several synthetic and field data examples. [ABSTRACT FROM AUTHOR]

Matrix factorization (MF) has earned great success on recommender systems. However, the common-used regression-based MF not only is sensitive to outliers but also unable to guarantee that the predicted values are in line with the user preference orders, which is the basis of common measures of recommender systems, e.g., nDCG. To overcome the aforementioned drawbacks, we propose a framework for personalized ranking of Poisson factorization that utilizes learning-to-rank based posteriori instead of the classical regression-based ones. Owing to the combination, the proposed framework not only preserves user preference but also performs well on a sparse matrix. Since the posteriori that combines learning to rank and Poisson factorization does not follow the conjugate prior relationship, we estimate variational parameters approximately and propose two optimization approaches based on variational inference. As long as the used learning-to-rank model has the 1st and 2nd order partial derivatives, by exploiting our framework, the proposed optimizing algorithm can maximize the posteriori whichever the used learning-to-rank model is. In the experiment, we show that the proposed framework outperforms the state-of-the-art methods and achieves promising results on consuming log and rating datasets for multiple recommendation tasks. [ABSTRACT FROM AUTHOR]

In this article, we consider the problem of planar graph-based simultaneous localization and mapping (SLAM) that involves both poses of the autonomous agent and positions of observed landmarks. We present complex (CPL)-SLAM, an efficient and certifiably correct algorithm to solve planar graph-based SLAM using the complex number representation. We formulate and simplify planar graph-based SLAM as the maximum likelihood estimation on the product of unit complex numbers, and relax this nonconvex quadratic complex optimization problem to convex complex semidefinite programming (SDP). Furthermore, we simplify the corresponding complex SDP to Riemannian staircase optimization (RSO) on the complex oblique manifold that can be solved with the Riemannian trust region method. In addition, we prove that the SDP relaxation and RSO simplification are tight as long as the noise magnitude is below a certain threshold. The efficacy of this work is validated through applications of CPL-SLAM and comparisons with existing state-of-the-art methods on planar graph-based SLAM, which indicates that our proposed algorithm is capable of solving planar graph-based SLAM certifiably, and is more efficient in numerical computation and more robust to measurement noise than existing state-of-the-art methods. The C++ code for CPL-SLAM is available at https://github.com/MurpheyLab/CPL-SLAM. [ABSTRACT FROM AUTHOR]

Hyperspectral super-resolution (HSR) is a problem that aims to estimate an image of high spectral and spatial resolutions from a pair of coregistered multispectral (MS) and hyperspectral (HS) images, which have coarser spectral and spatial resolutions, respectively. In this article, we pursue a low-rank matrix estimation approach for HSR. We assume that the spectral–spatial matrices associated with the whole image and the local areas of the image have low-rank structures. The local low-rank assumption, in particular, has the aim of providing a more flexible model for accounting for local variation effects due to endmember variability. We formulate the HSR problem as a global–local rank-regularized least-squares problem. By leveraging on the recent advances in nonconvex large-scale optimization, namely the smooth Schatten- $p$ approximation and the accelerated majorization–minimization method, we develop an efficient algorithm for the global–local low-rank problem. Numerical experiments on synthetic, semi-real, and real data show that the proposed algorithm outperforms a number of benchmark algorithms in terms of recovery performance. [ABSTRACT FROM AUTHOR]

Advanced collaborative filtering methods based on explicit feedback assume that unknown ratings are missing not at random. The state-of-the-art algorithm hypothesizes that unknown items are weakly rated and sets an explicit prior to unknown ratings. However, the prior assuming unknown ratings be close to zero may be questionable and it is challenging to set appropriate prior ratings for unknown items. In this article, to avert the use of prior ratings, we propose a ranking-based prior by hypothesizing that each user's unknown ratings are close to each other. This prior essentially acts as a regularizer to penalize the discrepancy of predicted ratings between any two unknown items. With the ranking-based prior, we design a generic collaborative filtering framework for explicit feedback and develop an efficient optimization algorithm for parameter learning. We finally evaluate the proposed algorithms on four real-world rating datasets. The results show that the proposed algorithms consistently outperform the state-of-the-art baselines and that the ranking-based prior leads to superior recommendation accuracy. [ABSTRACT FROM AUTHOR]

Clutter background suppression and velocity estimation for moving targets are two critical problems in synthetic aperture radar–ground moving target indication (SAR-GMTI). A robust principal component analysis (RPCA) method is used to separate the sparse matrix of moving targets from the low-rank matrix of static backgrounds by using amplitude information in the image domain. However, the nonsparsity of the moving target echoes limits the performance of the RPCA in SAR-GMTI, and the velocity of the moving target cannot be estimated since the phase information is destroyed by the soft-thresholding operator in the RPCA process. To solve these problems, a novel moving target detection method based on RPCA (NRPCA) for SAR systems is proposed in this article. An atomic norm-based optimization program is first constructed to transform the data sparsity requirement into a moving target sparsity requirement. Although this optimization program is NP-hard, it is transformed to semidefinite programming by relaxation. Furthermore, accurate velocity estimation is performed using dual function theory and the alternating direction method of multipliers (ADMM) algorithm while the selection of the sparsity order k is avoided. Simulations and analyses based on experimental data illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

We propose an algorithm to efficiently compute approximate solutions of the piecewise affine Mumford-Shah model. The algorithm is based on a novel reformulation of the underlying optimization problem in terms of Taylor jets. A splitting approach leads to linewise segmented jet estimation problems for which we propose an exact and efficient solver. The proposed method has the combined advantages of prior algorithms: it directly yields a partition, it does not need an initialization procedure, and it is highly parallelizable. The experiments show that the algorithm has lower computation times and that the solutions often have lower functional values than the state-of-the-art. [ABSTRACT FROM AUTHOR]

Disparity acquisition is beneficial to various 3D multimedia applications, i.e., free viewpoint TV (FTV), 3DTV, and 3D scene model reconstruction. In this paper, we propose an efficient disparity map estimation method with occlusion handling taken into account. In order to detect occlusion, we obtain left and right initial disparity maps via optimization based on modified constant-space belief propagation, a low complexity algorithm. The initial disparity maps provide clues for occlusion detection. From the clues, an energy function is defined and optimized by energy minimization framework. We define two occlusion types and apply suitable occlusion handling processes, accordingly. The proposed occlusion handling method extends disparity values of visible pixels to occluded pixels. Experimental results show that our method provides near-ground truth occlusion and a superior disparity map compared to other stateof- the-art methods with occlusion handling. Our contribution provides solid advantages regarding disparity map estimation, which is useful for various 3D multimedia applications. [ABSTRACT FROM AUTHOR]

Trajectory estimation of moving targets is examined; in particular, quasi-linear trajectories are considered. Background subtraction methods, exploiting low-rank backgrounds, and sparse features of interest are extended to incorporate linear constraints. The line constraint is enforced via a rotation that yields an additional low rank condition. The proposed method is applied to single object tracking in video, wherein the trajectory can be parameterized as a line. The optimization is solved via the augmented Lagrange multiplier method. An average performance improvement of 4 dB is observed over previous background subtraction methods for estimating the position and velocity of the target. Furthermore, about a 6.2 dB gain is seen over previous target tracking methods that do not exploit the linear nature of the trajectory. The Cramér–Rao bound (CRB) for background subtraction with a linear constraint is derived and numerical results show that the proposed method achieves near optimal performance via comparison to the CRB. An aggregated error is shown to converge to zero and a boundedness analysis is conducted which suggests that the iterative algorithm is convergent as confirmed by simulations. Finally, the proposed technique is applied to real video data and is shown to be effective in estimating quasi-linear trajectories. [ABSTRACT FROM AUTHOR]

Solving Generalized LASSO (GL) problems is challenging, particularly when analyzing many features with a complex interacting structure. Recent developments have found effective ways to identify inactive features so that they can be removed or aggregated to reduce the problem size before applying optimization solvers for learning. However, existing methods are mostly devoted to special cases of GL problems with special structures for feature interactions, such as chains or trees. Developing screening rules, particularly, safe screening rules to remove or aggregate features with general interaction structures, calls for a very different screening approach for GL problems. To tackle this challenge, we formulate the GL screening problem as a bound estimation problem in a large linear inequality system when solving them in the dual space. We propose a novel bound propagation algorithm for efficient safe screening for general GL problems, which can be further enhanced by developing novel transformation methods that can effectively decouple interactions among features. The proposed propagation and transformation methods are applicable with dynamic screening that can easily initiate the screening process while existing screening methods require the knowledge of the solution under a desirable regularization parameter. Experiments on both synthetic and real-world data demonstrate the effectiveness of the proposed screening method. [ABSTRACT FROM AUTHOR]

Images acquired in low-light conditions with handheld cameras are often blurry, so steady poses and long exposure time are required to alleviate this problem. Although significant advances have been made in image deblurring, state-of-the-art approaches often fail on low-light images, as a sufficient number of salient features cannot be extracted for blur kernel estimation. On the other hand, light streaks are common phenomena in low-light images that have not been extensively explored in existing approaches. In this work, we propose an algorithm that utilizes light streaks to facilitate deblurring low-light images. The light streaks, which commonly exist in the low-light blurry images, contain rich information regarding camera motion and blur kernels. A method is developed in this work to detect light streaks for kernel estimation. We introduce a non-linear blur model that explicitly takes light streaks and corresponding light sources into account, and pose them as constraints for estimating the blur kernel in an optimization framework. For practical applications, the proposed algorithm is extended to handle images undergoing non-uniform blur. Experimental results show that the proposed algorithm performs favorably against the state-of-the-art methods on deblurring real-world low-light images. [ABSTRACT FROM AUTHOR]

We consider multiagent stochastic optimization problems over reproducing kernel Hilbert spaces. In this setting, a network of interconnected agents aims to learn decision functions, i.e., nonlinear statistical models, that are optimal in terms of a global convex functional that aggregates data across the network, with only access to locally and sequentially observed samples. We propose solving this problem by allowing each agent to learn a local regression function while enforcing consensus constraints. We use a penalized variant of functional stochastic gradient descent operating simultaneously with low-dimensional subspace projections. These subspaces are constructed greedily by applying orthogonal matching pursuit to the sequence of kernel dictionaries and weights. By tuning the projection-induced bias, we propose an algorithm that allows each individual agent to learn, based on its locally observed data stream and message passing with its neighbors only, a regression function that is close to the globally optimal regression function. That is, we establish that with constant step-size selections agents’ functions converge to a neighborhood of the globally optimal one while satisfying the consensus constraints as the penalty parameter is increased. Moreover, the complexity of the learned regression functions is guaranteed to remain finite. On both multiclass kernel logistic regression and multiclass kernel support vector classification with data generated from class-dependent Gaussian mixture models, we observe an stable function estimation and the state-of-the-art performance for distributed online multiclass classification. Experiments on the Brodatz textures further substantiate the empirical validity of this approach. [ABSTRACT FROM AUTHOR]

A semiparametric, nonlinear regression model in the presence of latent variables is introduced. These latent variables can correspond to unmodeled phenomena or unmeasured agents in a complex networked system. This new formulation allows joint estimation of certain nonlinearities in the system, the direct interactions between measured variables and the effects of unmodeled elements on the observed system. The particular form of the model adopted is justified, and learning is posed as a regularized empirical risk minimization. This leads to classes of structured convex optimization problems with a “sparse plus low-rank” flavor. Relations between the proposed model and several common model paradigms, such as those of robust principal component analysis and vector autoregression (VAR), are established. Particularly in the VAR setting, the low-rank contributions can come from broad trends exhibited in the time series. Details of the algorithm for learning the model are presented. Experiments demonstrate the performance of the model and the estimation algorithm on simulated and real data. [ABSTRACT FROM AUTHOR]