Search

Asynchronous training based on the parameter server architecture is widely used for scaling up the DNN training over large datasets and DNN models. Communication has been identified as the major bottleneck when deploying the DNN training over the large-scale distributed deep learning systems. Recent studies try to reduce the communication traffic through gradient sparsification and quantization approaches. We identify three limitations in previous studies. First, the fundamental guideline for gradient sparsification of their work is the magnitude of the gradient. However, the gradients’ magnitude represents the current optimization direction while it cannot indicate the significance of the parameters, which potentially results in delayed updating for the significant parameters. Second, their gradient quantization methods based on the entire model often lead to error accumulation for gradients aggregation since the gradients from different layers of the DNN model follow different distributions. Third, previous quantization approaches are CPU intensive, which generates strong overhead for the server. We propose MIPD, an adaptive and layer-wise gradient sparsification framework that compresses the gradients based on model interpretability and probability distribution of gradients. MIPD compresses the gradients according to the corresponding significance of its parameters, which is defined by model interpretability. An Exponential Smoothing method is also proposed to compensate for the dropped gradients on the server to reduce the gradients error. MIPD proposes to update half of the parameters for each training step to reduce the CPU overhead of the server. It encodes the gradients based on their probability distribution, thereby minimizing the approximated errors. Extensive experimental results generated on the GPU cluster indicate that the proposed framework effectively improves the training performance of DNNs by up to 36.2%, which ensures high accuracy as compared to state-of-art solutions. Accordingly, the CPU and network usage of the server dropped by up to 42.0% and 32.7% respectively. [ABSTRACT FROM AUTHOR]

Low-frequency (LF) signal content in seismic data as well as a realistic initial model are key ingredients for robust and efficient full-waveform inversions (FWIs). However, acquiring LF data is challenging in practice for active seismic surveys. Data-driven solutions show promise to extrapolate LF data given a high-frequency counterpart. While being established for synthetic acoustic examples, the application of bandwidth extrapolation to field datasets remains nontrivial. Rather than aiming to reach superior accuracy in bandwidth extrapolation, we propose to jointly reconstruct LF data and a smooth background subsurface model within a multitask deep learning framework. We automatically balance data, model, and trace-wise correlation loss terms in the objective functional and show that this approach improves the extrapolation capability of the network. We also design a pipeline for generating synthetic data suitable for field data applications. Finally, we apply the same trained network to synthetic and real marine streamer datasets and run an elastic FWI from the extrapolated dataset. [ABSTRACT FROM AUTHOR]

The reliability and precision of dynamic database are vital for the optimal operating and global control of integrated energy systems. One of the effective ways to obtain the accurate states is state estimations.A novel robust dynamic state estimation methodology for integrated natural gas and electric power systems is proposed based on the Kalman filter. To take full advantage of measurement redundancies and predictions for enhancing the estimating accuracy, the dynamic state estimation model coupling gas and power systems by gas turbine units are established. The exponential smoothing technique and gas physical model are integrated into the Kalman filter. Additionally, the time-varying scalar matrix is proposed to conquer bad data in the Kalman filter algorithm. The proposed method is applied to integrated gas and power systems formed by GasLib-40 and IEEE 39-bus system with five gas turbine units. The simulating results show that the method can obtain the accurate dynamic states under three different measurement error conditions, and the filtering performance is better than separate estimation methods. Additionally, the proposed method is robust when the measurements experience bad data. [ABSTRACT FROM AUTHOR]

Defense Meteorological Satellite Program Operational Linescan System (DMSP-OLS) and Suomi National Polar-orbiting Partnership Visible Infrared Imaging Radiometer Suite (SNPP-VIIRS) data are valuable records of nighttime lights (NTLs) in analyzing socioeconomic development. However, inconsistencies between these data have severely restricted long time-series analyses. Published time-series NTL data sets are not widely available or accurate because the DMSP-OLS calibration is inadequate and some missing data in the SNPP-VIIRS data are seldom considered for patching. To address these issues, we calibrated DMSP-OLS data (1992–2013) by using a quadratic model based on a “pseudo-invariant pixel” method. Thereafter, an exponential smoothing model was used to predict and patch missing data in the monthly SNPP-VIIRS data (2013–2019). Outliers and noise were also removed from the annual data. In addition, a sigmoid model was employed to generate improved simulated DMSP-OLS (SDMSP-OLS) data (2013–2019), which were appended with the calibrated DMSP-OLS data (1992–2013) to develop improved DMSP-OLS-like data (1992–2019) in China. Finally, we qualitatively and quantitatively compared these data with published NTL data to examine data availability. Results showed that choosing invariant pixels to calibrate DMSP-OLS data can minimize discontinuity. The correlation between the SNPP-VIIRS data synthesized by the patched monthly SNPP-VIIRS data and the official annual SNPP-VIIRS data in 2015 ($R^{2} =0.931$) and 2016 ($R^{2} =0.930$) was higher than those of the two existing correction methods with $R^{2}$ values below 0.90. Spatial patterns of pixels in the improved SDMSP-OLS data in 2013 were more similar with the DMSP-OLS data than those in the published data. Strong correlations likewise existed between the total (average) pixel values of the improved SDMSP-OLS data (2013–2019) and the DMSP-OLS data in 2012. We also found that the improved DMSP-OLS-like data held strong linear correlations with different statistics, the average $R^{2}$ values of which were 0.931 and 0.654 at the national and provincial levels, respectively. Meanwhile, the average regression $R^{2}$ values between the two published data sets and statistics were 0.858/0.506 and 0.911/0.611, respectively. Our study has proven that the improved DMSP-OLS-like data (1992–2019) have immense potential to effectively evaluate socioeconomic development and anthropic activities. [ABSTRACT FROM AUTHOR]

Rendering is far from solved. Even today, the rendered results still look artificial and overly perfect. To make rendering more realistic, we need details. However, rendering a complex surface with lots of details is far from easy. Traditionally, the surface microstructure is approximated using a smooth statistical distribution, but this ignores all the details on the surface, completely eliminating the "glinty" visual effects that are easily observable in the real world. While modeling the actual surface microstructure is possible, the resulting rendering problem is prohibitively expensive using Monte Carlo point sampling. We consider the highly complicated distribution of normals on a surface patch seen through a single pixel, and evaluate this actual distribution efficiently with closed-form solutions, in both geometric and wave optics. Results show complicated, temporally varying glints from materials such as bumpy plastics, brushed and scratched metals, metallic paint and ocean waves—bringing the interesting and important details to Computer Graphics for the first time. [ABSTRACT FROM AUTHOR]

An image dehazing technique can restore hazy images to clear ones. Under hazy conditions, it can be a preceding stage of advanced-driver assistance systems (ADAS) to keep driving images clear. ADAS pursues a larger image resolution with a higher frame rate to keep its reliability for driving safety. This trend drives the image dehazing to face a severe challenge of considerable throughput. In this study, a hardware-efficient image dehazing engine with low computational burdens is proposed. It comprises two core techniques: Depth-Refinement Transmission-Rate Estimation (DTE) and Distribute Airlight Estimation (DAE). DTE provides an accurate depth map to estimate transmission rate, where the depth of white-objects is carefully calibrated. Base on the depth information, DAE predicts the airlight with the average of distant illumination sources to compensate nonuniform haze concentration. DTE and DAE can reduce the number of line buffers by 33% at least, while keeping comparable image dehazing quality. This study is implemented using TSMC CMOS $0.18~\mu \text{m}$ technology. To construct a smooth data scheduling, it is organized into a six-stage pipelinging strategy with an interleaving access technique. The logic gate count is 14.4K, and the power consumption is 15.2mW@250MHz. With a two-level parallelism, its throughput can support 3, $840\times 2.160$ @60fps. The experiment results show this study presents the superior combined performance of energy efficiency, area efficiency and line buffer reduction. [ABSTRACT FROM AUTHOR]

Data-driven methods have shown its great latent capacity in the field of industrial process monitoring. However, the existing methods usually achieve good results under the assumption that the offline learning data and the online monitoring data are drawn from the same distribution. Unfortunately, in the industrial system, the assumption is often violated due to the harsh operating environment. Especially, with the increasing complexity and scale of industrial production, the supervisory control and data acquisition (SCADA) data of the industrial production process often collected from different machines, seasons, or operating modes. In addition, due to the cost of manual data labeling and real-time requirement of process monitoring, the offline learning data, which was used to build the model, often have abundant source-domain data and insufficient target-domain data. Consequently, these methods have bad performance on the online monitoring data collected from the target domain. In order to make full use of the knowledge from the abundant source-domain data, a transfer dictionary learning method is proposed to address the cross-domain problem in this article. The proposed method can learn an initial dictionary from the abundant source-domain data, and then, the final dictionary is updated by incorporating the feature of insufficient target-domain data in a smooth subspace interpolation way. The effectiveness of the proposed method is evaluated through a numerical simulation case, a continuous stirred tank heater (CSTH) case, and a wind turbine system case, from which we can see the proposed method has a better performance compared with some state-of-the-art methods. [ABSTRACT FROM AUTHOR]

For the purpose of improving economic benefits, manufacturers pay more attention on key-performance-indicator prediction, since it plays a central role in process monitoring, quality control, and operational optimization. However, the existing methods only develop prediction models under Euclidean distance, which is not flexible enough for different problems. In addition, few methods are able to extract useful features through supervised learning. In this article, a novel metric-learning-based algorithm is proposed, named as metric-learning-based least-squares support vector machine (MLSSVM). The MLSSVM learns a task-dependent metric through optimizing the metric matrix of the Mahalanobis distance employed in the kernel function of the least-squares support vector machine. Locality preserving projection is employed to preserve the local structure of data. Besides, by enforcing the low-rank property of the metric matrix, the MLSSVM is able to explore the low-dimensional representation of features in a supervised and effective way. An accelerated proximal method is employed to solve the constructed nonsmooth constrained optimization problem. The effectiveness of the MLSSVM is illustrated through the tests on several problems, including a practical one from the glass production process. [ABSTRACT FROM AUTHOR]

Early change detection in dynamic running status has become an important problem in the online monitoring of industrial machineries. Existing methods, such as using mean and root mean square (rms), often produce significant false alarms in the detected results. To address this problem, this article proposes a new stability indicator based on the exponential smoothing model (ESM) and the structural graph similarity measure, in order to characterize the dynamic machine running status. The proposed stability indicator benefits from two stages, i.e., ESM and graph, and thus is advantageous in terms of computational efficiency and robustness to noise. By means of the extracted stability indicator, a common null hypothesis testing is employed to make the change decision, which does not require a prior collection of anomaly data samples. The proposed method is validated on both simulated and real scenarios. Experimental results demonstrate the outperforming performance of the method over the state-of-the-art competitors, suggesting its great potential in real applications. [ABSTRACT FROM AUTHOR]

Heat diffusion has been widely used in brain imaging for surface fairing, mesh regularization and cortical data smoothing. Motivated by diffusion wavelets and convolutional neural networks on graphs, we present a new fast and accurate numerical scheme to solve heat diffusion on surface meshes. This is achieved by approximating the heat kernel convolution using high degree orthogonal polynomials in the spectral domain. We also derive the closed-form expression of the spectral decomposition of the Laplace-Beltrami operator and use it to solve heat diffusion on a manifold for the first time. The proposed fast polynomial approximation scheme avoids solving for the eigenfunctions of the Laplace-Beltrami operator, which is computationally costly for large mesh size, and the numerical instability associated with the finite element method based diffusion solvers. The proposed method is applied in localizing the male and female differences in cortical sulcal and gyral graph patterns obtained from MRI in an innovative way. The MATLAB code is available at http://www.stat.wisc.edu/~mchung/chebyshev. [ABSTRACT FROM AUTHOR]

Alzheimer’s disease (AD) is a serious neurodegenerative condition that affects millions of individuals across the world. As the average age of individuals in the United States and the world increases, the prevalence of AD will continue to grow. To address this public health problem, the research community has developed computational approaches to sift through various aspects of clinical data and uncover their insights, among which one of the most challenging problem is to determine the biological mechanisms that cause AD to develop. To study this problem, in this paper we present a novel Joint Multi-Modal Longitudinal Regression and Classification method and show how it can be used to identify the cognitive status of the participants in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort and the underlying biological mechanisms. By intelligently combining clinical data of various modalities (i.e., genetic information and brain scans) using a variety of regularizations that can identify AD-relevant biomarkers, we perform the regression and classification tasks simultaneously. Because the proposed objective is a non-smooth optimization problem that is difficult to solve in general, we derive an efficient iterative algorithm and rigorously prove its convergence. To validate our new method in predicting the cognitive scores of patients and their clinical diagnosis, we conduct comprehensive experiments on the ADNI cohort. Our promising results demonstrate the benefits and flexibility of the proposed method. We anticipate that our new method is of interest to clinical communities beyond AD research and have open-sourced the code of our method online. 1 The code package for the proposed Joint Multi-Modal Longitudinal Regression and Classification model have been made publicly available online at https://github.com/minds-mines/jmmlrc. [ABSTRACT FROM AUTHOR]

In this paper we propose a novel turbo equalizer based on the expectation propagation (EP) algorithm. Optimal equalization is computationally unfeasible when high-order modulations and/or large memory channels are used. In these scenarios, low-cost and suboptimal equalizers, such as those based on the linear minimum mean square error (LMMSE), are commonly used. The LMMSE-based equalizer can be efficiently implemented with a Kalman smoother (KS), i.e., a forward and backward Kalman filtering whose predictions are merged in a posterior smoothing step. Recently, it was shown that applying EP at the forward and backward stages of a KS equalizer could significantly improve its performance. In this paper, we investigate applying EP at the smoothing level instead. Also, we propose some further modifications to better exploit the information coming from the channel decoder in turbo equalization schemes. Overall, we remarkably reduce the computational complexity while highly improving the performance in terms of bit error rate. [ABSTRACT FROM AUTHOR]

Non-parametric density estimation methods are more flexible than parametric methods, due to the fact that they do not assume any specific shape or structure for the data. Most non-parametric methods, like Kernel estimation, require tuning of parameters to achieve good data smoothing, a non-trivial task, even in low dimensions. In higher dimensions, sparsity of data in local neighborhoods becomes a challenge even for non-parametric methods. In this paper, we use the copula transform and two efficient non-parametric methods to develop a new method for improved non-parametric density estimation in multivariate domain. After separation of marginal and joint densities using copula transform, a diffusion-based kernel estimator is employed to estimate the marginals. Next, Bayesian sequential partitioning (BSP) is used in the joint density estimation. [ABSTRACT FROM AUTHOR]

This article describes a phasor data prediction and recovery algorithm that can be used to optimally extrapolate invalid or missing phasor data measurements. The basic output of the algorithm is the prediction of the phasor data when bad data quality is detected from a phasor measurement unit (PMU) at a given sampling rate. The approach is based on optimal one-sample prediction, where the sampling time between sample predictors is tuned based on the spectral content of phasor data. Optimality of prediction using such optimal filters is addressed in this article by iterative (recursive) search of the parameters of a nonlinear discrete-time integrated filter that minimizes the norm of the prediction error. The extrapolated data are then estimated by interpolation or reconstruction of the signal from the resulting optimally spaced predictors. Finally, a Kalman-filtering-based recovery method is discussed for smoothing the data for transitioning to actual valid measurements for use in real-time control applications. The approach is illustrated on phasor data obtained from a micro-PMU system developed by the Power Standards Lab collected at 60 Hz. [ABSTRACT FROM AUTHOR]

Objective: We present a physiologically motivated eye movement analysis framework for model-based separation, detection, and classification (MBSDC) of eye movements. By estimating kinematic and neural controller signals for saccades, smooth pursuit, and fixational eye movements in a mechanistic model of the oculomotor system we are able to separate and analyze these eye movements independently. Methods: We extended an established oculomotor model for horizontal eye movements by neural controller signals and by a blink artifact model. To estimate kinematic (position, velocity, acceleration, forces) and neural controller signals from eye position data, we employ Kalman smoothing and sparse input estimation techniques. The estimated signals are used for detecting saccade start and end points, and for classifying the recording into saccades, smooth pursuit, fixations, post-saccadic oscillations, and blinks. Results: On simulated data, the reconstruction error of the velocity profiles is about half the error value obtained by the commonly employed approach of filtering and numerical differentiation. In experiments with smooth pursuit data from human subjects, we observe an accurate signal separation. In addition, in neural recordings from non-human primates, the estimated neural controller signals match the real recordings strikingly well. Significance: The MBSDC framework enables the analysis of multi-type eye movement recordings and provides a physiologically motivated approach to study motor commands and might aid the discovery of new digital biomarkers. Conclusion: The proposed framework provides a model-based approach for a wide variety of eye movement analysis tasks. [ABSTRACT FROM AUTHOR]

Frequency-resolved optical gating (FROG) is a popular technique for complete characterization of ultrashort laser pulses. The acquired data in FROG, called FROG trace, is the Fourier magnitude of the product of the unknown pulse with a time-shifted version of itself, for several different shifts. To estimate the pulse from the FROG trace, we propose an algorithm that minimizes a smoothed non-convex least-squares objective function. The method consists of two steps. First, we approximate the pulse by an iterative spectral algorithm. Then, the attained initialization is refined based upon a sequence of block stochastic gradient iterations. The algorithm is theoretically simple, numerically scalable, and easy-to-implement. Empirically, our approach outperforms the state-of-the-art when the FROG trace is incomplete, that is, when only few shifts are recorded. Simulations also suggest that the proposed algorithm exhibits similar computational cost compared to a state-of-the-art technique for both complete and incomplete data. In addition, we prove that in the vicinity of the true solution, the algorithm converges to a critical point. A Matlab implementation is publicly available at https://github.com/samuelpinilla/FROG. [ABSTRACT FROM AUTHOR]

Divergence measures have been used for a long time for different purposes in information theory and statistics. In particular, density-based minimum divergence estimation is a popular tool in the statistical literature. Given the sampled data and a parametric model, we estimate the model parameter by choosing the member of the model family that is closest to the data distribution in terms of the given divergence. In the absolutely continuous set up, when the distributions from the model family and the unknown data generating distribution are assumed to have densities, the application of kernel based non-parametric smoothing is sometimes unavoidable to get an estimate of the true data density. The use of kernels (or other non-parametric smoothing techniques) makes the estimation process considerably more complex, as now one has to impose necessary conditions not just on the model but also on the kernel and its bandwidth. In higher dimensions the efficiency of the kernel density estimator (KDE) often becomes too low for the minimum divergence procedure to be practically useful. It can, therefore, lead to a significant advantage to have a divergence which allows minimum divergence estimation bypassing the use of non-parametric smoothing. For the same reason, characterizing the class of such divergences would be a notable achievement. In this work, we provide a characterization of the class of divergences that bypasses the use of non-parametric smoothing in the construction of divergences, providing a solution to this very important problem. [ABSTRACT FROM AUTHOR]

Reliability classification of gear safety has long been a challenging issue in transmission industry because of complicated calculations and great classification errors of coupled parameters with insufficient data. This paper proposes a model based on generative adversarial network (GAN) as pretreatment to improve the accuracy of reliability classification. First, we present bounded-GAN to generate gear data within required boundaries without massive computations. In bounded-GAN, three bounded layers are designed to bound generated data in terms of different data characteristics; smooth targets are developed to enhance the ability of generating high-quality instances by the generator; Adam optimizer is used to train both generator and discriminator to avoid nonconvergence. Second, to overcome unlabeling defect of bounded-GAN, a mean–covariance labeling scheme is introduced to label the data according to the nearest classes of gear reliability within specific ranges. Finally, original and qualified data are combined to train classifiers. Simulations on gear data from industry show that our proposed model outperforms other methods on operational metrics. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new approach to signal smoothing when the data are generated (or represented) by an autoregressive moving average with exogenous inputs (ARMAX) model. In this approach, the original ARMAX recurrence relation is directly employed and combined with a constrained least squares optimization framework to filter out both system and measurement noise components and estimate the desired signal in form of block-wise matrix formulation. The approach is also driven from a forward backward filtering, which is accomplished through linear time invariant system. While the impulse response of the proposed filter can be found using deconvolution operator, a closed-form expression is presented for its impulse response without resorting to any transform methods. Two examples of its applications for variable-Q filter design and spectral density estimation are given, which demonstrate the present approach is more effective and robust than the existing variable-Q filter designs in the literature and it can be used to improve the spectral density estimation. [ABSTRACT FROM AUTHOR]

This paper proposes a novel data-driven adaptive robust optimization (ARO) framework for the unit commitment (UC) problem integrating wind power into smart grids. By leveraging a Dirichlet process mixture model, a data-driven uncertainty set for wind power forecast errors is constructed as a union of several basic uncertainty sets. Therefore, the proposed uncertainty set can flexibly capture a compact region of uncertainty in a nonparametric fashion. Based on this uncertainty set and wind power forecasts, a data-driven adaptive robust UC problem is then formulated as a four-level optimization problem. A decomposition-based algorithm is further developed. Compared to conventional robust UC models, the proposed approach does not presume single mode, symmetry, or independence in uncertainty. Moreover, it not only substantially withstands wind power forecast errors, but also significantly mitigates the conservatism issue by reducing operational costs. We also compare the proposed approach with the state-of-the-art data-driven ARO method based on principal component analysis and kernel smoothing to assess its performance. The effectiveness of the proposed approach is demonstrated with the six-bus and IEEE 118-bus systems. Computational results show that the proposed approach scales gracefully with problem size and generates solutions that are more cost effective than the existing data-driven ARO method. [ABSTRACT FROM AUTHOR]

We describe a computationally efficient heuristic algorithm based on a renormalization-group procedure which aims at solving the problem of finding a minimal surface given its boundary (curve) in any hypercubic lattice of dimension $D>2$. We use this algorithm to correct errors occurring in a four-dimensional variant of the toric code, having open as opposed to periodic boundaries. For a phenomenological error model which includes measurement errors we use a five-dimensional version of our algorithm, achieving a threshold of 4.35 ± 0.1%. For this error model, this is the highest known threshold of any topological code. Without measurement errors, a four-dimensional version of our algorithm can be used and we find a threshold of 7.3 ± 0.1%. For the gate-based depolarizing error model, we find a threshold of 0.31 ± 0.01% which is below the threshold found for the two-dimensional toric code. [ABSTRACT FROM AUTHOR]

Predictive models can be used on high-dimensional brain images to decode cognitive states or diagnosis/prognosis of a clinical condition/evolution. Spatial regularization through structured sparsity offers new perspectives in this context and reduces the risk of overfitting the model while providing interpretable neuroimaging signatures by forcing the solution to adhere to domain-specific constraints. Total variation (TV) is a promising candidate for structured penalization: it enforces spatial smoothness of the solution while segmenting predictive regions from the background. We consider the problem of minimizing the sum of a smooth convex loss, a non-smooth convex penalty (whose proximal operator is known) and a wide range of possible complex, non-smooth convex structured penalties such as TV or overlapping group Lasso. Existing solvers are either limited in the functions they can minimize or in their practical capacity to scale to high-dimensional imaging data. Nesterov’s smoothing technique can be used to minimize a large number of non-smooth convex structured penalties. However, reasonable precision requires a small smoothing parameter, which slows down the convergence speed to unacceptable levels. To benefit from the versatility of Nesterov’s smoothing technique, we propose a first order continuation algorithm, CONESTA, which automatically generates a sequence of decreasing smoothing parameters. The generated sequence maintains the optimal convergence speed toward any globally desired precision. Our main contributions are: gap to probe the current distance to the global optimum in order to adapt the smoothing parameter and the To propose an expression of the duality convergence speed. This expression is applicable to many penalties and can be used with other solvers than CONESTA. We also propose an expression for the particular smoothing parameter that minimizes the number of iterations required to reach a given precision. Furthermore, we provide a convergence proof and its rate, which is an improvement over classical proximal gradient smoothing methods. We demonstrate on both simulated and high-dimensional structural neuroimaging data that CONESTA significantly outperforms many state-of-the-art solvers in regard to convergence speed and precision. [ABSTRACT FROM AUTHOR]

Phase retrieval is an inverse problem which consists in recovering an unknown signal from a set of absolute squared projections. Recently, gradient descent algorithms have been developed to solve this problem. However, their optimization cost functions are non-convex and non-smooth. To address the non-smoothness of the cost function, some of these methods use truncation thresholds to calculate a truncated step gradient direction. But, the truncation requires designing parameters to obtain a desired performance in the phase recovery, which drastically modifies the search direction update, increasing the sampling complexity. Therefore, this paper develops the Phase Retrieval Smoothing Conjugate Gradient method (PR-SCG) which uses a smoothing function to retrieve the signal. PR-SCG is based on the smooth-ing projected gradient method which is useful for non-convex optimization problems. PR-SCG uses a nonlinear conjugate gradient of the smoothing function as the search direction to accelerate the convergence. Furthermore, the incremental Stochastic Smoothing Phase Retrieval algorithm (SSPR) is developed. SSPR involves a single equation per iteration which results in a simple, scalable, and fast approach useful when the size of the signal is large. Also, it is shown that SSPR converges linearly to the true signal, up to a global unimodular constant. Additionally, the proposed methods do not require truncation parameters. Simulation results are provided to validate the efficiency of PR-SCG and SSPR compared to existing phase retrieval algorithms. It is shown that PR-SCG and SSPR are able to reduce the number of measurements and iterations to recover the phase, compared with recently developed algorithms. [ABSTRACT FROM AUTHOR]

Structure-preserving image decomposition separates a given image into structure and texture by smoothing the image, simultaneously preserving or enhancing image edges. The well-studied problem of image decomposition is applied to various areas, such as image smoothing, detail enhancement, non-photorealistic rendering, image artistic rendering, and high-dynamic-range compression. In this paper, we propose a fast algorithm for structure-preserving image decomposition that adopts total variation (TV) minimization to the moving least squares (MLS) method with non-local weights, called structure adaptive TV (SATV) minimization. MLS with non-local weights provides high accuracy approximation that is robust to noise, and allows a fast convergence with TV regularization term. As a result, our proposed SATV preserves the dominant structure while flattening fine-scale details. The experimental results show that the SATV minimization algorithm provides faster and more robust image decomposition than the well-known previous approaches. We demonstrate the usefulness of our algorithm by presenting successful applications in image smoothing and detail enhancement. [ABSTRACT FROM AUTHOR]

We present approximate algorithms for performing smoothing in a class of high-dimensional state-space models via sequential Monte Carlo methods (particle filters). In high dimensions, a prohibitively large number of Monte Carlo samples (particles), growing exponentially in the dimension of the state space, are usually required to obtain a useful smoother. Employing blocking approximations, we exploit the spatial ergodicity properties of the model to circumvent this curse of dimensionality. We thus obtain approximate smoothers that can be computed recursively in time and parallel in space. First, we show that the bias of our blocked smoother is bounded uniformly in the time horizon and in the model dimension. We then approximate the blocked smoother with particles and derive the asymptotic variance of idealized versions of our blocked particle smoother to show that variance is no longer adversely effected by the dimension of the model. Finally, we employ our method to successfully perform maximum-likelihood estimation via stochastic gradient-ascent and stochastic expectation–maximization algorithms in a 100-dimensional state-space model. [ABSTRACT FROM PUBLISHER]

Many patch-based image denoising algorithms can be formulated as applying a smoothing filter to the noisy image. Expressed as matrices, the smoothing filters must be row normalized, so that each row sums to unity. Surprisingly, if we apply a column normalization before the row normalization, the performance of the smoothing filter can often be significantly improved. Prior works showed that such performance gain is related to the Sinkhorn–Knopp balancing algorithm, an iterative procedure that symmetrizes a row-stochastic matrix to a doubly stochastic matrix. However, a complete understanding of the performance gain phenomenon is still lacking. In this paper, we study the performance gain phenomenon from a statistical learning perspective. We show that Sinkhorn–Knopp is equivalent to an expectation-maximization (EM) algorithm of learning a Gaussian mixture model of the image patches. By establishing the correspondence between the steps of Sinkhorn–Knopp and the EM algorithm, we provide a geometrical interpretation of the symmetrization process. This observation allows us to develop a new denoising algorithm called Gaussian mixture model symmetric smoothing filter (GSF). GSF is an extension of the Sinkhorn–Knopp and is a generalization of the original smoothing filters. Despite its simple formulation, GSF outperforms many existing smoothing filters and has a similar performance compared with several state-of-the-art denoising algorithms. [ABSTRACT FROM PUBLISHER]

In this paper, we consider the filtering and smoothing recursions in nonparametric finite state space hidden Markov models (HMMs) when the parameters of the model are unknown and replaced by estimators. We provide an explicit and time uniform control of the filtering and smoothing errors in total variation norm as a function of the parameter estimation errors. We prove that the risk for the filtering and smoothing errors may be uniformly upper bounded by the \mathrm L^1 -risk of the estimators. It has been proved very recently that statistical inference for finite state space nonparametric HMMs is possible. We study how the recent spectral methods developed in the parametric setting may be extended to the nonparametric framework and we give explicit upper bounds for the \mathrm L^2 -risk of the nonparametric spectral estimators. In the case where the observation space is compact, this provides explicit rates for the filtering and smoothing errors in total variation norm. The performance of the spectral method is assessed with simulated data for both the estimation of the (nonparametric) conditional distribution of the observations and the estimation of the marginal smoothing distributions. [ABSTRACT FROM AUTHOR]

Spatial smoothing is a widely used preprocessing scheme to improve the performance of high-resolution parameter estimation algorithms in case of coherent signals or if only a small number of snapshots is available. In this paper, we present a first-order performance analysis of the spatially smoothed versions of $R$-D Standard ESPRIT and $R$-D Unitary ESPRIT for sources with arbitrary signal constellations as well as $R$-D NC Standard ESPRIT and $R$-D NC Unitary ESPRIT for strictly second-order (SO) non-circular (NC) sources. The derived expressions are asymptotic in the effective signal-to-noise ratio (SNR), i.e., the approximations become exact for either high SNRs or a large sample size. Moreover, no assumptions on the noise statistics are required apart from a zero mean and finite SO moments. We show that both $R$-D NC ESPRIT-type algorithms with spatial smoothing perform asymptotically identical in the high effective SNR regime. Generally, the performance of spatial smoothing based algorithms depends on the number of subarrays, which is a design parameter that needs to be chosen beforehand. In order to gain more insights into the optimal choice of the number of subarrays, we simplify the derived analytical $R$-D mean square error (MSE) expressions for the special case of a single source. The obtained MSE expression explicitly depends on the number of subarrays in each dimension, which allows us to analytically find the optimal number of subarrays for spatial smoothing. Based on this result, we additionally derive the maximum asymptotic gain from spatial smoothing and compute the asymptotic efficiency for the single source case in closed-form. All the analytical results are verified by simulations. [ABSTRACT FROM PUBLISHER]

We propose minimum entropy rate simplification (MERS), an information-theoretic, parameterization-independent framework for simplifying generative models of stochastic processes. Applications include improving model quality for sampling tasks by concentrating the probability mass on the most characteristic and accurately described behaviors while de-emphasizing the tails, and obtaining clean models from corrupted data (nonparametric denoising). This is the opposite of the smoothing step commonly applied to classification models. Drawing on rate-distortion theory, MERS seeks the minimum entropy-rate process under a constraint on the dissimilarity between the original and simplified processes. We particularly investigate the Kullback-Leibler divergence rate as a dissimilarity measure, where, compatible with our assumption that the starting model is disturbed or inaccurate, the simplification rather than the starting model is used for the reference distribution of the divergence. This leads to analytic solutions for stationary and ergodic Gaussian processes and Markov chains. The same formulas are also valid for maximum-entropy smoothing under the same divergence constraint. In experiments, MERS successfully simplifies and denoises models from audio, text, speech, and meteorology. [ABSTRACT FROM AUTHOR]

Online nuclear counting represents a challenge due to the stochastic nature of radioactivity. The counting data have to be filtered in order to provide a precise and accurate estimation of the count rate, while ensuring a response time compatible with the application in view. An innovative filter is presented in this paper to address this issue. The filter is nonlinear and based on a Centered Significance Test (CST) providing a local maximum likelihood estimation of the signal. This nonlinear approach allows enables to smooth the counting signal while maintaining a fast response when brutal change in activity occurs. The filter is then improved by the implementation of a Brown’s double Exponential Smoothing (BES). The filter has been validated and compared to other state-of-the-art smoothing filters. The CST* filter shows a significant improvement compared to all tested smoothing filters. [ABSTRACT FROM PUBLISHER]

Electricity demand forecasts are needed for decisions regarding generation dispatch for lead times as short as just a few minutes. Imbalance between generation and demand causes deviation of the system frequency from its target, which in Great Britain is 50 Hz. This, in turn, causes a change in demand, due largely to motor loads. For Great Britain, the change is estimated to be 2.5% of demand per 1 Hz of frequency deviation from its target. This can be used to calculate the demand that would have occurred if frequency had been at 50 Hz. Modeling and forecasting the resulting frequency-corrected demand provides a better basis for dispatching generation. This paper evaluates methods for forecasting frequency-corrected demand up to 10 min ahead. We introduce an exponential smoothing model that, like the system operator's proposed Kalman filter approach, jointly models frequency and demand. We also evaluate a set of univariate methods applied directly to the series of frequency-corrected demand. These methods have not previously been considered for lead times less than 10 min. In our empirical analysis, the best results were produced by a seasonal exponential smoothing method applied directly to the series of frequency-corrected demand. [ABSTRACT FROM AUTHOR]

In this paper, we consider a dual-mode process for the electrophotographic laser printer—a low-frequency halftoning for smooth regions and a high-frequency halftoning for detail regions. These regions are described by an object map that is extracted from the page description language version of the document. This manner of switching screens depending on the local content provides a stable halftone without artifacts in smooth areas and preserves the detail rendering in detail or texture areas. However, when switching between halftones with two different frequencies, jaggies may occur along the boundaries between areas halftoned with low- and high-frequency screens. To reduce the jaggies, our screens obey a harmonic relationship. In addition, we implement a blending process based on a transition region. We propose a nonlinear blending process in which at each pixel, we choose the maximum of the two weighted halftones, where the weights vary according to the position in the transition region. Moreover, we describe an online tone-mapping for the boundary blending process, based on an offline calibration procedure that effectively assures the desired tone values within the transition region. [ABSTRACT FROM PUBLISHER]

Good initial values are crucial to obtain solutions of nonconvex optimization problems. When estimating the transfer function of physical systems from measured noisy data, obtaining good initial parameter estimates is therefore a primordial step. In this paper, it is shown that smoothing the measured frequency response function of a linear time-invariant system enhances the construction of initial estimates significantly, resulting in the optimization schemes to converge to a better optimum. This is achieved with minimal user interaction. Two smoothing techniques, the time-truncated local polynomial method and the regularized finite impulse response, are compared with the existing generalized total least squares and the bootstrapped total least squares initial estimates. The improvement attributable to smoothing is demonstrated by a simulation and by measurements of an electrical filter. The results ultimately show that the parametric models obtained using the proposed starting values are much more likely to give a good description of the measured system and hence lead to more useful models. [ABSTRACT FROM PUBLISHER]

In the field of high spatial resolution (HSR) remote sensing imagery classification, object-oriented classification and conditional random field (CRF) approaches are widely used due to their ability to incorporate the spatial contextual information. However, the selection of the optimal segmentation scale in object- oriented classification is not an easy task, and some pairwise CRF models always show an oversmooth performance. In this paper, a detail-preserving smoothing classifier based on conditional random fields (DPSCRF) for HSR imagery is proposed to apply the object-oriented strategy in the CRF classification framework, thus integrating the merits of both approaches to consider the spatial contextual information and preserve the detail information in the classification. The DPSCRF model defines suitable potential functions based on the CRF model for HSR image classification, which comprise the spatial smoothing and local class label cost terms. Both terms favor spatial smoothing in a local neighborhood to consider the spatial information. In addition, the local class label cost also considers the different label information of neighboring pixels at each iterative step in the classification to preserve the detail information. In order to deal with the spectral variability of HSR imagery, a segmentation prior is used by the object-oriented processing strategy. This models the probability of each pixel based on the segmentation regions obtained by the connected-component labeling algorithm. The experimental results with three HSR images demonstrate that the proposed classification algorithm shows a competitive performance in both the quantitative and the qualitative evaluation when compared to the other state-of-the-art classification algorithms. [ABSTRACT FROM AUTHOR]

This paper presents a cost-sensitive ordinal hyperplanes ranking algorithm for human age estimation based on face images. The proposed approach exploits relative-order information among the age labels for rank prediction. In our approach, the age rank is obtained by aggregating a series of binary classification results, where cost sensitivities among the labels are introduced to improve the aggregating performance. In addition, we give a theoretical analysis on designing the cost of individual binary classifier so that the misranking cost can be bounded by the total misclassification costs. An efficient descriptor, scattering transform, which scatters the Gabor coefficients and pooled with Gaussian smoothing in multiple layers, is evaluated for facial feature extraction. We show that this descriptor is a generalization of conventional bioinspired features and is more effective for face-based age inference. Experimental results demonstrate that our method outperforms the state-of-the-art age estimation approaches. [ABSTRACT FROM PUBLISHER]

This paper presents a nonparametric time-domain estimation method for linear slowly time-varying systems. The proposed method uses a 2-D impulse response function. That is modeled by a generalized B-spline smoothing technique. This means that double smoothing is applied: once over the different realizations (referring to the system behavior) and once over the global time (referring to the system memory). With respect to the system dynamics using the proposed method, it is possible to reduce the model order and to decrease the effect of the disturbing noise. [ABSTRACT FROM PUBLISHER]

Optimal placement of sensors or landmarks for the localization of an autonomous guided vehicle (AGV) based on range measurements is considered. The optimization relies on the Fisher information matrix (FIM). A closed-form expression of the FIM determinant is obtained for 2-D and 3-D spaces, considering heterogeneous sensors and distance-dependent ranging errors. Analytical bounds on the FIM determinant are derived, and several optimal landmark placement expressions for specific scenarios using two groups of landmarks are given. A minimax formulation for the optimal landmark placement is proposed and iteratively solved using exponential smoothing. There are several key features integrated into the proposed approach. These include 1) averaging of the cost function over elementary regions of the AGV search space to include the effect of the whole search space, 2) using a projected gradient search to stay within the boundaries of the landmark search space, and 3) searching on a convex space for placing the landmarks. Convergence issues of the proposed algorithm are discussed. Numerical results demonstrate that the proposed optimal landmark placement enables accurate AGV localization over significantly large volume or area of the search space compared with the case when landmarks are randomly placed. [ABSTRACT FROM AUTHOR]

The analysis and interpretation of heart rate variability (HRV) during exercise is challenging not only because of the nonstationary nature of exercise, the time-varying mean heart rate, and the fact that respiratory frequency exceeds 0.4 Hz, but there are also other factors, such as the component centered at the pedaling frequency observed in maximal cycling tests, which may confuse the interpretation of HRV analysis. The objectives of this study are to test the hypothesis that a component centered at the running stride frequency (SF) appears in the HRV of subjects during maximal treadmill exercise testing, and to study its influence in the interpretation of the low-frequency (LF) and high-frequency (HF) components of HRV during exercise. The HRV of 23 subjects during maximal treadmill exercise testing is analyzed. The instantaneous power of different HRV components is computed from the smoothed pseudo-Wigner–Ville distribution of the modulating signal assumed to carry information from the autonomic nervous system, which is estimated based on the time-varying integral pulse frequency modulation model. Besides the LF and HF components, the appearance is revealed of a component centered at the running SF as well as its aliases. The power associated with the SF component and its aliases represents 22 \pm 7% (median \pm  median absolute deviation) of the total HRV power in all the subjects. Normalized LF power decreases as the exercise intensity increases, while normalized HF power increases. The power associated with the SF does not change significantly with exercise intensity. Consideration of the running SF component and its aliases is very important in HRV analysis since stride frequency aliases may overlap with LF and HF components. [ABSTRACT FROM PUBLISHER]

Although promising for studying the microstructure of in vivo tissues, the performance and the potentiality of diffusion tensor magnetic resonance imaging are hampered by the presence of high-level noise in diffusion weighted (DW) images. This paper proposes a novel smoothing approach, called the nonstationarity adaptive filtering, which estimates the intensity of a pixel by averaging intensities in its adaptive homogeneous neighborhood. The latter is determined according to five constraints and spatiodirectional nonstationarity measure maps. The proposed approach is compared with an anisotropic diffusion method used in DW image smoothing. Experimental results on both synthetic and real human DW images show that the proposed method achieves a better compromise between the smoothness of homogeneous regions and the preservation of desirable features such as boundaries, even for highly noisy data, thus leading to homogeneously consistent tensor fields and consequently more coherent fibers. [ABSTRACT FROM PUBLISHER]

Accurate tracking of facial tissue in thermal infrared imaging is challenging because it is affected not only by positional but also physiological (functional) changes. This paper presents a particle filter tracker driven by a probabilistic template function with both spatial and temporal smoothing components, which is capable of adapting to abrupt positional and physiological changes. The method was tested on tracking facial regions of subjects under varying physiological and environmental conditions in 25 thermal clips. It demonstrated robustness and accuracy, outperforming other strategies. This new method promises improved performance in a number of biomedical applications that involve physiological measurements on the face, such as unobtrusive sleep and stress studies. [ABSTRACT FROM PUBLISHER]

We consider the smoothing problem for a general state space system using sequential Monte Carlo (SMC) methods. The marginal smoother is assumed to be available in the form of weighted random particles from the SMC output. New algorithms are developed to extract the smoothed marginal maximum a posteriori (MAP) estimate of the state from the existing marginal particle smoother. Our method does not need any kernel fitting to obtain the posterior density from the particle smoother. The proposed estimator is then successfully applied to find the unknown initial state of a dynamical system and to address the issue of parameter estimation problem in state space models. [ABSTRACT FROM PUBLISHER]

This paper addresses the problem of estimating simultaneously a local polynomial signal and an approximately piecewise constant signal from a noisy additive mixture. The approach developed in this paper synthesizes the total variation filter and least-square polynomial signal smoothing into a unified problem formulation. The method is based on formulating an \ell_1-norm regularized inverse problem. A computationally efficient algorithm, based on variable splitting and the alternating direction method of multipliers (ADMM), is presented. Algorithms are derived for both unconstrained and constrained formulations. The method is illustrated on experimental data involving the detection of nano-particles with applications to real-time virus detection using a whispering-gallery mode detector. [ABSTRACT FROM PUBLISHER]

This paper provides an overview of the intra coding techniques in the High Efficiency Video Coding (HEVC) standard being developed by the Joint Collaborative Team on Video Coding (JCT-VC). The intra coding framework of HEVC follows that of traditional hybrid codecs and is built on spatial sample prediction followed by transform coding and postprocessing steps. Novel features contributing to the increased compression efficiency include a quadtree-based variable block size coding structure, block-size agnostic angular and planar prediction, adaptive pre- and postfiltering, and prediction direction-based transform coefficient scanning. This paper discusses the design principles applied during the development of the new intra coding methods and analyzes the compression performance of the individual tools. Computational complexity of the introduced intra prediction algorithms is analyzed both by deriving operational cycle counts and benchmarking an optimized implementation. Using objective metrics, the bitrate reduction provided by the HEVC intra coding over the H.264/advanced video coding reference is reported to be 22% on average and up to 36%. Significant subjective picture quality improvements are also reported when comparing the resulting pictures at fixed bitrate. [ABSTRACT FROM PUBLISHER]

The problem of out-of-sequence measurement (OOSM) processing when the filtering technique used at the tracker is the particle filter (PF) is considered. First, an exact Bayesian algorithm for updating with OOSMs is derived. Then, the PF implementation of the exact Bayesian algorithm, called A-PF, is developed. Since A-PF is rooted in exact Bayesian inference, if the number of particles is sufficiently large, A-PF is the one (and the only one) that is able to achieve the optimal performance obtained from the in-sequence processing. This is confirmed by the simulation results. Also, it is shown that the performance of A-PF is always superior to previous (heuristic) PF-based algorithms with the same number of particles. [ABSTRACT FROM AUTHOR]