Search

We consider the reconstruction problem of video snapshot compressive imaging (SCI), which captures high-speed videos using a low-speed 2D sensor (detector). The underlying principle of SCI is to modulate sequential high-speed frames with different masks and then these encoded frames are integrated into a snapshot on the sensor and thus the sensor can be of low-speed. On one hand, video SCI enjoys the advantages of low-bandwidth, low-power and low-cost. On the other hand, applying SCI to large-scale problems (HD or UHD videos) in our daily life is still challenging and one of the bottlenecks lies in the reconstruction algorithm. Existing algorithms are either too slow (iterative optimization algorithms) or not flexible to the encoding process (deep learning based end-to-end networks). In this paper, we develop fast and flexible algorithms for SCI based on the plug-and-play (PnP) framework. In addition to the PnP-ADMM method, we further propose the PnP-GAP (generalized alternating projection) algorithm with a lower computational workload. We first employ the image deep denoising priors to show that PnP can recover a UHD color video with 30 frames from a snapshot measurement. Since videos have strong temporal correlation, by employing the video deep denoising priors, we achieve a significant improvement in the results. Furthermore, we extend the proposed PnP algorithms to the color SCI system using mosaic sensors, where each pixel only captures the red, green or blue channels. A joint reconstruction and demosaicing paradigm is developed for flexible and high quality reconstruction of color video SCI systems. Extensive results on both simulation and real datasets verify the superiority of our proposed algorithm. [ABSTRACT FROM AUTHOR]

Compressive covariance estimation has arisen as a class of techniques whose aim is to obtain second-order statistics of stochastic processes from compressive measurements. Recently, these methods have been used in various image processing and communications applications, including denoising, spectrum sensing, and compression. Notice that estimating the covariance matrix from compressive samples leads to ill-posed minimizations with severe performance loss at high compression rates. In this regard, a regularization term is typically aggregated to the cost function to consider prior information about a particular property of the covariance matrix. Hence, this paper proposes an algorithm based on the projected gradient method to recover low-rank or Toeplitz approximations of the covariance matrix from compressive measurements. The proposed algorithm divides the compressive measurements into data subsets projected onto different subspaces and accurately estimates the covariance matrix by solving a single optimization problem assuming that each data subset contains an approximation of the signal statistics. Furthermore, gradient filtering is included at every iteration of the proposed algorithm to minimize the estimation error. The error induced by the proposed splitting approach is analytically derived along with the convergence guarantees of the proposed method. The proposed algorithm estimates the covariance matrix of hyperspectral images from synthetic and real compressive samples. Extensive simulations show that the proposed algorithm can effectively recover the covariance matrix of hyperspectral images from compressive measurements with high compression ratios ($8-15\%$ approx) in noisy scenarios. Moreover, simulations and theoretical results show that the filtering step reduces the recovery error up to twice the number of eigenvectors. Finally, an optical implementation is proposed, and real measurements are used to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

Wavelet transform using Haar filter is a fast process for decomposing an image into low and high frequency sub-bands, which is an important step in multi-focus image fusion. This transformation alone is usually not sufficient to produce satisfying fused images, and therefore additional tools like focused region decision map and feature extraction are needed to enhance the fusion quality. In this paper, we propose a multi-focus image fusion algorithm that is suitable for efficient hardware implementation. The algorithm speed is significantly improved by utilizing Haar wavelet and computationally simple fusion rules. To further improve the circuit power and delay, we limit the logic operations to additions, subtractions, and multiplications only. Experiments on different benchmarks demonstrate that our proposed algorithm can reduce the fusion time by up to 99.97% compared with the state-of-the-art competing methods without compromising the fusion quality. In addition, the hardware implementation of the proposed algorithm reduces the circuit area and delay by 79.5% and 92.5%, respectively, when compared to the most competitive algorithm in this paper while maintaining similar fusion quality. [ABSTRACT FROM AUTHOR]

Color demosaicing is an essential step in the camera image processing pipeline, especially on hardware platforms for real-time video applications. Though many demosaicing algorithms have been proposed in the past two decades, there remains a substantial gap between industrial needs and academic research. On one hand, industry requires a high perceptual quality, artifact-free, and low-cost demosaicing algorithm that is noniterative and small window based ready to be implemented on a hardware platform with a limited line buffer. On the other hand, academia is targeting high PSNR/SSIM, with the computation cost and line buffer receiving second priority, and often a frame buffer and iterative operation are used. The cost, large line buffer and iteration requirement make most existing demosaicing algorithms inapplicable on hardware platforms. In this paper, we introduce a novel low-cost demosaicing algorithm to narrow the gap. We keep the operation window size and computation cost as the first priority and achieve both high PSNR/SSIM and visual perceptual quality compared to previous state-of-the-art methods on standard test datasets. We fully investigate the a priori knowledge of natural scene raw images and find several key cues that are beneficial to demosaicing. Our demosaicing algorithm is a smart fusion of these useful cues. Furthermore, we solve typical demosaicing issues that occur in many traditional methods, including false color artifacts, T-section closing, and zippering artifacts. The proposed method has no learning stage, no iteration operation, a small line buffer and a limited number of parameters, so it can easily be applied to hardware platforms. [ABSTRACT FROM AUTHOR]

Articulation angle sensing is an essential component of manoeuvrability and stability control systems for articulated heavy goods vehicles, particularly long combination vehicles. Existing solutions to this sensing task are limited by reliance on trailer modifications or information or by measurement accuracy, or both, restricting commercial adoption. In this paper we present a purely tractor-based sensor concept comprising a rear-facing camera and the parallel tracking and mapping (PTAM) image processing algorithm. The system requires no prior knowledge of or modifications to the trailer, is compatible with planar and non-planar trailer shapes, and with multiply-articulated vehicle combinations. The system is validated in full-scale vehicle tests on both a tractor semi-trailer combination and a truck and full-trailer combination, demonstrating robust performance in a number of conditions, including trailers with non-planar geometry and with minimal visual features. Average RMS measurement errors of 1.19, 1.03 and 1.53 degrees were demonstrated for the semi-trailer and full-trailer (drawbar and semi-trailer) respectively. This compares favourably with the state-of-the-art in the published literature. A number of improvements are proposed for future development based on the observations in this research. [ABSTRACT FROM AUTHOR]

A fast wind turbine abnormal data cleaning algorithm via image processing for wind turbine power generation performance measurement and evaluation is proposed in this paper. The proposed method includes two stages, data cleaning and data classification. At the data cleaning stage, pixels of normal data are extracted via image processing based on pixel spatial distribution characteristics of abnormal and normal data in wind power curve (WPC) images. At the data classification stage, wind power data points are classified as normal and abnormal based on the existence of corresponding pixels in the processed WPC image. To accelerate the proposed method, the cleaning operation is executed parallelly using graphics processing units (GPUs) via compute unified device architecture (CUDA). The effectiveness of the proposed method is validated based on real data sets collected from 37 wind turbines of two commercial farms and three types of GPUs are employed to implement the proposed algorithm. The computational results prove the proposed approach has achieved better performance in cleaning abnormal wind power data while the execution time is tremendously reduced. Therefore, the proposed method is available and practical for real wind turbine power generation performance evaluation and monitoring tasks. [ABSTRACT FROM AUTHOR]

Semi-global matching (SGM) is favored for high accuracy real-time stereo matching design as it achieves a good trade-off between disparity image quality and computational complexity. Nevertheless, most of previous SGM designs so far are restricted to the real-time processing of small image resolution and disparity range, or achieve high throughput by simplifying the original algorithm at the penalty of significant disparity image quality degradation. We analyze that the major challenge to efficient SGM design is its memory architecture, including both on-chip memory cost and off-chip memory bandwidth. We address the memory architecture challenge by algorithm and architecture co-design. Based on two observed features of SGM algorithm, i.e. incompleteness and inaccuracy, this paper proposes several efficient techniques to reduce on-chip memory cost and compress off-chip memory bandwidth respectively. Moreover, we also design high throughput and pipelined architecture to implement the proposed techniques. The disparity image quality and hardware efficiency of the proposed SGM design are evaluated on both KITTI2015 and Middlebury V3 stereo datasets. Evaluation results demonstrate that, the throughput of the proposed circuit designs can easily achieve 1080P@30fps at the disparity range of 128, and can reduce the on-chip memory cost and off-chip memory bandwidth by up to $4\times $ and $2\times $ respectively while achieving better or the same disparity image quality, compared with the best reference design techniques. [ABSTRACT FROM AUTHOR]

This paper presents low-cost compressors for video display systems, which compress video data to reduce the size of the frame memory in a display panel in a television or a smartphone. The hardware solution exploits the discrete wavelet transform (DWT) and 1-D set partitioning in hierarchical trees (SPIHT) to support the raster-scan processing order in a display system and meet the fixed target compression ratio (CR). However, the compression efficiency and hardware cost often increase in proportion to the block size and thus, designing a compressor with low cost but high fidelity is challenging. This paper proposes a generic partitioned SPIHT algorithm, which achieves a low-cost design for mobile devices by allowing various sizes of partitioned sub-blocks. By taking advantage of DWT decomposition properties, the proposed algorithm partitions a coding block into sub-blocks, each of which can be processed independently. The independent coding operations among multiple sub-blocks allow the size of the hardware buffer to be decreased, by storing the temporary results only for an individual sub-block. However, the proposed sub-block coding slightly decreases the compression efficiency when each sub-block is compressed with the same target CR. To reduce the drop-off in the compression efficiency, the boundary-pixel handling in DWT is exploited and the proposed partitioned SPIHT method adjusts the target CR for each sub-block depending on the potential quality loss. When compared with the previous 1-D design, experimental results show that the hardware gate count and internal memory are reduced by 59.34% and 75%, respectively. Furthermore, the proposed boundary handling and bit-allocation schemes mitigate the PSNR degradation due to the sub-block coding by 1.11 dB. [ABSTRACT FROM PUBLISHER]

Dictionary learning has emerged as a powerful tool for a range of image processing applications and a proper dictionary always plays a key issue to the final achievable performance. In this paper, a class-oriented discriminative dictionary learning (CODDL) method is presented for image classification applications. It takes a comprehensive consideration of multiple optimization objectives, emphasizing class discrimination of both dictionary atoms and representation coefficients. The atoms of the learned dictionary should be grouped into class level sub-dictionaries. Meanwhile, the sparse representation coefficients of an input sample should be concentrated on the sub-dictionary of the class it belongs to. Then, based on the learned class-oriented discriminative dictionary, the structured representation coefficients can thus be used for image classification with a simple and efficient classification scheme. The superior performance of the proposed algorithm is demonstrated through extensive experiments. [ABSTRACT FROM AUTHOR]

Capon’s minimum-variance method (MVM) and amplitude and phase estimation (APES) spectral estimation algorithms can be applied to synthetic aperture radar (SAR) processing to improve the resolution and suppress sidelobe levels. In this paper, we use Capon-/APES-based SAR reprocessing algorithms to increase the persistent scatterer (PS) density in PS interferometry (PSI). We propose a PS candidate (PSC) selection algorithm applicable to the superresolution reprocessed images and the corresponding processing chain. The performance of the proposed algorithm is evaluated by a number of simulations and a stack of TerraSAR-X data. The results show that the Capon algorithm outperforms others in PSC selection. We present a full PSI time-series analysis on the PSCs extracted from the Capon-reprocessed stacks. The results show that the PS density is increased between 50% and 60%, while their interferometric quality is maintained. [ABSTRACT FROM AUTHOR]

We deal with the classical problem of testing two simple statistical hypotheses but, as a new element, it is assumed that the data vector is observed after an unknown permutation of its entries. What is the fundamental limit for the detection performance in this case? How much information for detection is contained in the entry values and how much in their positions? In the first part of this paper, we answer these questions. In the second part, we focus on practical algorithms. A low-complexity detector solves the detection problem without attempting to estimate the permutation. A modified version of the auction algorithm is then considered, and two greedy algorithms with affordable worst case complexity are presented. The detection operational characteristics of these detectors are investigated by computer experiments. The problem we address is referred to as unlabeled detection and is motivated by large sensor network applications, but applications are also foreseen in different fields, including image processing, social sensing, genome research, and molecular communication. [ABSTRACT FROM AUTHOR]

Hyperspectral unmixing has attracted much attention in recent years. Single sparse unmixing assumes that a pixel in a hyperspectral image consists of a relatively small number of spectral signatures from large, ever-growing, and available spectral libraries. Joint-sparsity (or row-sparsity) model typically enforces all pixels in a neighborhood to share the same set of spectral signatures. The two sparse models are widely used in the literature. In this paper, we propose a joint-sparsity-blocks model for abundance estimation problem. Namely, the abundance matrix of size $m\times n$ is partitioned to have one row block and $s$ column blocks and each column block itself is joint-sparse. It generalizes both the single (i.e., $s=n$) and the joint (i.e., $s=1$) sparsities. Moreover, concatenating the proposed joint-sparsity-blocks structure and low rankness assumption on the abundance coefficients, we develop a new algorithm called joint-sparse-blocks and low-rank unmixing. In particular, for the joint-sparse-blocks regression problem, we develop a two-level reweighting strategy to enhance the sparsity along the rows within each block. Simulated and real-data experiments demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

With the rapid increase of remote sensing images in temporal, spectral, and spatial resolutions, it is urgent to develop effective techniques for joint interpretation of spatial-temporal images. Multitype change detection (CD) is a significant research topic in multitemporal remote sensing image analysis, and its core is to effectively measure the difference degree and represent the difference among the multitemporal images. In this paper, we propose a novel difference representation learning (DRL) network and present an unsupervised learning framework for multitype CD task. Deep neural networks work well in representation learning but rely too much on labeled data, while clustering is a widely used classification technique free from supervision. However, the distribution of real remote sensing data is often not very friendly for clustering. To better highlight the changes and distinguish different types of changes, we combine difference measurement, DRL, and unsupervised clustering into a unified model, which can be driven to learn Gaussian-distributed and discriminative difference representations for nonchange and different types of changes. Furthermore, the proposed model is extended into an iterative framework to imitate the bottom-up aggregative clustering procedure, in which similar change types are gradually merged into the same classes. At the same time, the training samples are updated and reused to ensure that it converges to a stable solution. The experimental studies on four pairs of multispectral data sets demonstrate the effectiveness and superiority of the proposed model on multitype CD. [ABSTRACT FROM AUTHOR]

Despeckling is a fundamental preprocessing step for applications using polarimetric synthetic aperture radar data in most cases. In this paper, a guided filter with nonlinear weight kernels and adaptive filtering windows is presented for PolSAR image despeckling, in which the guidance image is constructed by a weighted average using the statistical traits of the speckled image. The output result is then estimated by another weighted average, with the aid of the fully polarimetric information from both the guidance image and the speckled image. In the experimental part, the filtering results obtained with both simulated and real PolSAR images reveal the positive performance of the proposed method in both reducing speckle and retaining details, when compared with some of the state-of-the-art algorithms. Furthermore, the relatively low computational complexity is another strength of the proposed method. [ABSTRACT FROM AUTHOR]

Fast inspection of a defect is a challenging task in mass production of curved surfaces, and photometric stereo (PS) utilizing multiple images from a single camera under a number of different illumination directions is a promising technique for this task due to its high sensitivity to surface normal perturbations. This paper adapts conventional PS and extends the technique to the inspection of non-Lambertian surfaces with high accuracy and efficiency. A data-driven PS is presented by establishing the Gaussian process (GP) model to represent the nonlinear reflectance behavior of various materials based on measured reflectance data sets. With the trained GP model, the surface normal can be estimated in two steps: prediction of bidirectional reflectance distribution function values under different light directions and the subsequent least-squares estimation of a surface normal. Comparison tests with other algorithms on the Mitsubishi Electric Research Laboratories data set and real workpieces with non-Lambertian materials show the superior accuracy and efficiency of the proposed method in surface normal estimation. After the surface normal of the workpiece is recovered, the defects can be detected by filtering out the perturbation of the surface normal. Experiments on steel and glossy polyester workpieces validate the efficacy of the proposed approach in detecting defects on curved non-Lambertian surfaces. [ABSTRACT FROM AUTHOR]

Band selection is an effective approach to mitigate the “Hughes phenomenon” of hyperspectral image (HSI) classification. Recently, sparse representation (SR) theory has been successfully introduced to HSI band selection, and many SR-based methods have been developed and shown great potential and superiority. However, due to the inherent limitations of the SR scheme, i.e., individually representing each band with only a few other bands from the same subspace, the SR-based methods cannot effectively capture the global structures of the data, which limit the band selection performance. In this paper, to overcome this obstacle, the novel Laplacian-regularized low-rank subspace clustering (LLRSC) algorithm is proposed for HSI band selection. On the one hand, the low-rank subspace clustering model is introduced to capture the global structure information for the learned representation coefficient matrix and deal with the HSI band selection task in the clustering framework. On the other hand, considering the high correlation between adjacent bands, 1-D Laplacian regularization is utilized to incorporate the neighboring band information and further reduce the representation bias. Lastly, an eigenvalue analysis algorithm based on band mutation information is utilized to estimate the appropriate size of the band subset. The experimental results indicate that the proposed LLRSC algorithm outperforms the other state-of-the-art methods and achieves a very competitive band selection performance for HSIs. [ABSTRACT FROM AUTHOR]

Multitemporal airborne and satellite imagery data with frequent repeat coverage provide great capability for change detection (CD). When comparing two images taken at different times of day or in different seasons for CD, the variation of topographic shades and shadows caused by the change of sunlight angle can be so significant that it overwhelms the real object and environmental changes, making automatic detection unreliable. An effective CD algorithm, therefore, has to be robust to the illumination variation. In this paper, the robustness of phase correlation (PC) to shadow effects is proven via mathematical analysis, and then, an illumination-invariant change detection (IICD) metric is proposed based on pixel-wise dense PC matching. In the proposed IICD method, a graph-based visual saliency map is introduced for the initial CD followed by an active contour-based segmentation to precisely quantize the change region. Compared to the state-of-the-art CD algorithms, experiments using daily images of a landscape model and Landsat satellite images demonstrate that only the proposed method can effectively detect and precisely segment appearance changes under daily and seasonal sunlight changes. [ABSTRACT FROM AUTHOR]

The advanced Himawari imager (AHI) aboard the Himawari-8 geostationary satellite provides high-frequency observations with broad coverage, multiple spectral channels, and high spatial resolution. In this paper, AHI data were used to develop an algorithm for joint retrieval of aerosol optical depth (AOD) over land and land surface bidirectional reflectance. Instead of performing surface reflectance estimation before calculating AOD, the AOD and surface bidirectional reflectance were retrieved simultaneously using an optimal estimation method. The algorithm uses an atmospheric radiative transfer model coupled with a surface bidirectional reflectance factor (BRF) model. Based on the assumption that the surface bidirectional reflective properties are invariant during a short time period (i.e., a day), multiple temporal AHI observations were combined to calculate the AOD and surface BRF. The algorithm was tested over East Asia for year 2016, and the AOD retrieval results were validated against the aerosol robotic network (AERONET) sites observation and compared with the Moderate Resolution Imaging Spectroradiometer Collection 6.0 AOD product. The validation of the retrieved AOD with AERONET measurements using 14 713 colocation points in 2016 over East Asia shows a high correlation coefficient: $R = 0.88$ , root-mean-square error = 0.17, and approximately 69.9% AOD retrieval results within the expected error of $\pm 0.2\cdot {\mathrm {AOD}}_{\mathrm {AERONET}}\pm 0.05$. A brief comparison between our retrieval and AOD product provided by Japan Meteorological Agency is also presented. The comparison and validation demonstrates that the algorithm has the ability to estimate AOD with considerable accuracy over land. [ABSTRACT FROM AUTHOR]

This paper presents a processing framework to separate moving targets from the clutter, under multichannel synthetic aperture radar (SAR) scenarios, and addresses the moving target imaging and velocity estimation problems for ground moving target indication (GMTI) applications. A practical implementation is introduced to break the SAR/GMTI problem into two processing stages, and the sparsity of the moving targets in the observed scene is exploited throughout the stages. The two-stage process extracts the moving targets from the monitored region via a sparsity-based iterative decomposition algorithm and subsequently estimates the complete velocity vectors of moving targets by enforcing sparsity constraints. The model is sufficiently versatile to incorporate digital elevation map information, which further improves the moving target relocation accuracy. The effectiveness of the presented framework is demonstrated using the Air Force Research Laboratory Gotcha GMTI challenge data. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of leveraging existing fully labeled categories to improve the weakly supervised detection (WSD) of new object categories, which we refer to as mixed supervised detection (MSD). Different from previous MSD methods that directly transfer the pre-trained object detectors from existing categories to new categories, we propose a more reasonable and robust objectness transfer approach for MSD. In our framework, we first learn domain-invariant objectness knowledge from the existing fully labeled categories. The knowledge is modeled based on invariant features that are robust to the distribution discrepancy between the existing categories and new categories; therefore the resulting knowledge would generalize well to new categories and could assist detection models to reject distractors (e.g., object parts) in weakly labeled images of new categories. Under the guidance of learned objectness knowledge, we utilize multiple instance learning (MIL) to model the concepts of both objects and distractors and to further improve the ability of rejecting distractors in weakly labeled images. Our robust objectness transfer approach outperforms the existing MSD methods, and achieves state-of-the-art results on the challenging ILSVRC2013 detection dataset and the PASCAL VOC datasets. [ABSTRACT FROM AUTHOR]

Light-field stereo matching problems are commonly modeled by Markov Random Fields (MRFs) for statistical inference of depth maps. Nevertheless, most previous approaches did not adapt to image statistics but instead adopted fixed model parameters. They explored explicit vision cues, such as depth consistency and occlusion, to provide local adaptability and enhance depth quality. However, such additional assumptions could end up confining their applicability, e.g. algorithms designed for dense view sampling are not suitable for sparse one. In this paper, we get back to MRF fundamentals and develop an empirical Bayesian framework—Robust Pseudo Random Field—to explore intrinsic statistical cues for broad applicability. Based on pseudo-likelihoods with hidden soft-decision priors, we apply soft expectation-maximization (EM) for good model fitting and perform hard EM for robust depth estimation. We introduce novel pixel difference models to enable such adaptability and robustness simultaneously. Accordingly, we devise a stereo matching algorithm to employ this framework on dense, sparse, and even denoised light fields. It can be applied to both true-color and grey-scale pixels. Experimental results show that it estimates scene-dependent parameters robustly and converges quickly. In terms of depth accuracy and computation speed, it also outperforms state-of-the-art algorithms constantly. [ABSTRACT FROM AUTHOR]

A path planning and control approach of a non-holonomic three-wheeled mobile robot (WMR) for online navigation in road following and roundabout environments is presented in this paper. We proposed a complete navigation algorithm that enables the WMR to autonomously navigate on the road with various scenarios. With such an algorithm, the robot is able to localize itself within the road environment and find a collision free-path starting from a pre-defined start position to a goal point using a novel approach called laser simulator (LS). The path planning and roundabout detection are determined based on LS and sensor fusion of a laser range finder, camera, and odometry measurements. The sensor fusion algorithm is used to remove noises and uncertainties from sensors’ data and provide optimum measurements for path planning. A robot motion control scheme is used for the purpose of controlling the kinematic parameters of WMR using a resolved acceleration control coupled with an active force control for rejecting the disturbances. Experimental results show the capability of the proposed algorithms to robustly drive the robot on the road following and roundabout environments. [ABSTRACT FROM AUTHOR]

Immersive media, such as free view point video and 360° video, are expected to be dominant as broadcasting services. The light field (LF) imaging is being considered as a next generation imaging technology offering the possibility to provide new services, including six degree-of-freedom video. The drawback of this technology is in the size of the generated content thus requiring novel compression systems and the design of ad-hoc methodologies for evaluating the perceived quality. In this paper, the relation between depth map quality and overall quality of LF image is studied. Next, a reduced reference quality assessment metric for LF images is presented. To predict the quality of distorted LF images, the measure of distortion in the depth map is exploited. To test and validate the proposed framework, a subjective experiment has been performed, and a LF image quality dataset has been created. The dataset is also used for evaluating the performance of state-of-the-art quality metrics, when applied to LF images. The achieved results evidence that the estimated quality score by the proposed framework has a significant correlation with subjective quality rating. Consequently, reference data can be delivered to the clients thus allowing the local estimation of the perceived quality of service. [ABSTRACT FROM AUTHOR]

The emerging of depth-camera technology is paving the way for a variety of new applications and it is believed that plane detection is one of them. In fact, planes are common in man-made living structures, thus their accurate detection can benefit many visual-based applications. The use of depth information allows detecting planes characterized by complex pattern and texture, where the texture-based plane detection algorithms usually fail. In this paper, we propose a robust depth-driven plane detection (DPD) algorithm which consists of two parts: the growing-based plane detection and a two-stage refinement. The proposed approach starts from the seed patch with the highest planarity and uses the estimated equation of the growing plane and a dynamic threshold function to steer the growing process. Aided with this mechanism, each seed patch can grow to its maximum extent, and then the next seed patch starts to grow. This process is iteratively repeated so as to detect all the planes. Moreover, the refinement is proposed to tackle two common problems suffered by growing-based approaches, the over-growing problem, and the under-growing problem. Validated by extensive experiments, the proposed DPD algorithm is able to accurately detect planes and robust to various testing conditions. In terms of applications, it can be used as the pre-processing step for a variety of applications, such as, planar object recognition, super-resolution of the time-of-flight depth images with intrinsically low resolution. [ABSTRACT FROM AUTHOR]

Frame rate up-conversion (FRUC) is a technology for generating a higher frame rate video from one with a lower frame rate. The multi-view plus depth map (MVD) system has been recently considered a promising solution for three-dimensional video systems. Accordingly, an FRUC efficient solution for the MVD system is greatly needed. In this paper, we thus propose an FRUC approach that considers the MVD system characteristics. It divides a frame into multiple layers using a depth map, and it reconstructs occluded regions using neighboring views and depth map information. With reconstructed information, it accurately performs motion estimation when a region occlusion occurs. Experimental results demonstrated that the proposed method significantly improved the quality of interpolated frames, and it can effectively complement other conventional FRUC approaches. [ABSTRACT FROM AUTHOR]

Multifractal analysis has become a powerful signal processing tool that characterizes signals or images via the fluctuations of their pointwise regularity, quantified theoretically by the so-called multifractal spectrum. The practical estimation of the multifractal spectrum fundamentally relies on exploiting the scale dependence of statistical properties of appropriate multiscale quantities, such as wavelet leaders, that can be robustly computed from discrete data. Despite successes of multifractal analysis in various real-world applications, current estimation procedures remain essentially limited to providing concave upper-bound estimates, while there is a priori no reason for the multifractal spectrum to be a concave function. This paper addresses this severe practical limitation and proposes a novel formalism for multifractal analysis that enables nonconcave multifractal spectra to be estimated in a stable way. The key contributions reside in the development and theoretical study of a generalized multifractal formalism to assess the multiscale statistics of wavelet leaders, and in devising a practical algorithm that permits this formalism to be applied to real-world data, allowing for the estimation of nonconcave multifractal spectra. Numerical experiments are conducted on several synthetic multifractal processes as well as on a real-world remote-sensing image and demonstrate the benefits of the proposed multifractal formalism over the state of the art. [ABSTRACT FROM AUTHOR]

Mismatch removal is a key step in many computer vision problems that involve point matching. The existing methods for checking geometric consistency mainly focus on similarity or affine transformations. In this paper, we propose a novel mismatch removal method that can cope with the projective transformation between two corresponding point sets. Our approach is based on the augmented homogeneous coordinates matrix constructed from the coordinates of anchor matches, whose degeneracy can indicate the correctness of anchor matches. The set of anchor matches is initially all the matches and is iteratively updated by calculating the difference between the estimated matched points, which can be easily computed in a closed form, and the actually matched points and removing those with large differences. Experimental results on synthetic 2D point matching data sets and real image matching data sets verify that our method achieves the highest $F$ -score among all the methods under similarity, affine, and projective transformations with noises and outliers. Our method can also achieve faster speed than all other iterative methods. Those non-iterative methods with slight advantage in speed are not competitive in accuracy when compared with ours. We also show that the set of anchor matches is stable through the iteration and the computation time grows very slowly with respect to the number of matched points. When applied to mismatch removal in partial-duplicate image search, our method achieves the best retrieval precision, and its computing time is also highly competitive. [ABSTRACT FROM AUTHOR]

The success of minimally invasive interventions and the remarkable technological and medical progress have made endoscopic image enhancement a very active research field. Due to the intrinsic endoscopic domain characteristics and the surgical exercise, stereo endoscopic images may suffer from different degradations which affect its quality. Therefore, in order to provide the surgeons with a better visual feedback and improve the outcomes of possible subsequent processing steps, namely, a 3-D organ reconstruction/registration, it would be interesting to improve the stereo endoscopic image quality. To this end, we propose, in this paper, two joint enhancement methods which operate in the wavelet transform domain. More precisely, by resorting to a joint wavelet decomposition, the wavelet subbands of the right and left views are simultaneously processed to exploit the binocular vision properties. While the first proposed technique combines only the approximation subbands of both views, the second method combines all the wavelet subbands yielding an inter-view processing fully adapted to the local features of the stereo endoscopic images. Experimental results, carried out on various stereo endoscopic datasets, have demonstrated the efficiency of the proposed enhancement methods in terms of perceived visual image quality. [ABSTRACT FROM AUTHOR]

In quantitative magnetic resonance ${T}_{\textsf {1}}$ mapping, the variable flip angle (VFA) steady state spoiled gradient recalled echo (SPGR) imaging technique is popular as it provides a series of high resolution ${T}_{\textsf {1}}$ weighted images in a clinically feasible time. Fast, linear methods that estimate ${T}_{\textsf {1}}$ maps from these weighted images have been proposed, such as DESPOT1 and iterative re-weighted linear least squares. More accurate, non-linear least squares (NLLS) estimators are in play, but these are generally much slower and require careful initialization. In this paper, we present NOVIFAST, a novel NLLS-based algorithm specifically tailored to VFA SPGR ${T}_{\textsf {1}}$ mapping. By exploiting the particular structure of the SPGR model, a computationally efficient, yet accurate and precise ${T}_{\textsf {1}}$ map estimator is derived. Simulation and in vivo human brain experiments demonstrate a twenty-fold speed gain of NOVIFAST compared with conventional gradient-based NLLS estimators while maintaining a high precision and accuracy. Moreover, NOVIFAST is eight times faster than the efficient implementations of the variable projection (VARPRO) method. Furthermore, NOVIFAST is shown to be robust against initialization. [ABSTRACT FROM AUTHOR]

The automatic reconstruction of single neurons from microscopic images is essential to enable large-scale data-driven investigations in neuron morphology research. However, few previous methods were able to generate satisfactory results automatically from 3-D microscopic images without human intervention. In this paper, we developed a new algorithm for automatic 3-D neuron reconstruction. The main idea of the proposed algorithm is to iteratively track backward from the potential neuronal termini to the soma centre. An online confidence score is computed to decide if a tracing iteration should be stopped and discarded from the final reconstruction. The performance improvements comparing with the previous methods are mainly introduced by a more accurate estimation of the traced area and the confidence controlled back-tracking algorithm. The proposed algorithm supports large-scale batch-processing by requiring only one user specified parameter for background segmentation. We bench tested the proposed algorithm on the images obtained from both the DIADEM challenge and the BigNeuron challenge. Our proposed algorithm achieved the state-of-the-art results. [ABSTRACT FROM AUTHOR]

In this paper, visual servoing based on photometric moments is advocated. A direct approach is chosen by which the extraction of geometric primitives, visual tracking and image matching steps of a conventional visual servoing pipeline can be bypassed. A vital challenge in photometric methods is the change in the image resulting from the appearance and disappearance of portions of the scene from the camera field of view during the servo. To tackle this issue, a general model for the photometric moments enhanced with spatial weighting is proposed. The interaction matrix for these spatially weighted photometric moments is derived in an analytical form. The correctness of the modeling, effectiveness of the proposed strategy in handling the exogenous regions, and improved convergence domain are demonstrated with a combination of simulation and the experimental results. [ABSTRACT FROM AUTHOR]

Blind source separation (BSS) aims to discover the underlying source signals from a set of linear mixture signals without any prior information of the mixing system, which is a fundamental problem in signal and image processing field. Most of the state-of-the-art algorithms have independently handled the decompositions of mixture signals. In this paper, we propose a new algorithm named multi-task sparse model to solve the BSS problem. Source signals are characterized via sparse techniques. Meanwhile, we regard the decomposition of each mixture signal as a task and employ the idea of multi-task learning to discover connections between tasks for the accuracy improvement of the source signal separation. Theoretical analyses on the optimization convergence and sample complexity of the proposed algorithm are provided. Experimental results based on extensive synthetic and real-world data demonstrate the necessity of exploiting connections between mixture signals and the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a novel rate-control algorithm based on the region adaptive R- $\lambda $ model is proposed for depth maps coding. First, in order to obtain an accurate rate control for depth maps coding, a modified frame level bit allocation method based on coding bits statistical distribution of depth maps is proposed. Second, considering that different areas in a depth map have an imparity effect on virtual view rendering, the blocks of the depth map are divided into two types, namely, interested blocks for virtual view rending (IBV) and noninterested blocks for virtual view rending (NIBV). Then, two different R- $\lambda $ models are derived for IBV and NIBV, respectively. The optimal bitrates for IBV and NIBV are determined by solving an optimization problem. After that, based on the regional R- $\lambda $ models, the optimal Lagrange multipliers are calculated for both IBV and NIBV. Finally, the largest coding unit (LCU) level rate control is performed by adaptively adjusting the Lagrange multiplier to avoid blocking artifacts and smooth the quality of coding. Experimental results demonstrate that the proposed method can achieve considerable BD-PSNR gains compared with the unified rate-quantization model and conventional R- $\lambda $ model-based algorithms in terms of rendered virtual views quality. [ABSTRACT FROM AUTHOR]

Object recognition is widely used in vision computing for various applications. Traditional CPU and application specific integrated circuit for vision computing cannot provide high performance and enough flexibility, which limit the use of vision systems. In this paper, a hierarchical and parallel pipelined heterogeneous chip for object recognition is proposed to achieve high flexibility, high performance, and area efficiency. In addition, a reformulation of 3D position estimation is proposed. The method uses single precision to achieve the short computing time and accuracy requirement. The hardware resource is small. Application-specific components, such as connected component information extractor and information extraction accelerator, are designed for high performance. Reconfiguration processors and application-specific instruction set processor are introduced to improve flexibility. These components are connected to hierarchical parallel buses. The chip is fabricated in 180-nm CMOS technology and occupies 72.25 mm2 with 1.09M bits on-chip memory. It delivers 204 GOPS + 665M FLOPS operations. The results show that this hierarchical and parallel pipelined heterogeneous chip is suitable for embedded vision systems. [ABSTRACT FROM AUTHOR]

Video surveillance systems have become extremely important recently. It has been observed that information extracted from a single spectrum video is often insufficient in adverse conditions like low illumination, shadowing, smoke, dust, unstable background, and camouflage. Real-time video processing systems also need to be very fast where future frames are often unavailable at the time of processing the current frame. In this paper, we propose a superpixel-based causal multisensor video fusion algorithm suitable for real-time surveillance tasks. We develop new superpixel level spatial and temporal saliency models. Novel superpixel level multiple fusion rules are also designed to obtain the fused output. Comprehensive comparisons with several existing works clearly indicate the benefit of our solution. [ABSTRACT FROM AUTHOR]

Compared with other applications in computer vision, convolutional neural networks (CNNs) have underperformed on pedestrian detection. A breakthrough was made very recently using sophisticated deep CNN (DCNN) models, with a number of handcrafted features or explicit occlusion handling mechanism. In this paper, we show that by reusing the convolutional feature maps of a DCNN model as image features to train an ensemble of boosted decision models, we are able to achieve the best reported accuracy without using specially designed learning algorithms. We empirically identify and disclose important implementation details. We also show that pixel labeling may be simply combined with a detector to boost the detection performance. By adding complementary handcrafted features such as optical flow, the DCNN-based detector can be further improved. We advance the state-of-the-art results by lowering the log-average miss rate from 11.7% to 8.9% on the Caltech data set and from 11.2% to 8.6% on the Inria data set. We also achieve a comparable result to state-of-the-art approaches on the KITTI data set. [ABSTRACT FROM AUTHOR]

Compressed sensing (CS) is the theory and practice of sub-Nyquist sampling of sparse signals of interest. Exact reconstruction may then be possible with much fewer than the Nyquist-required number of data. In this paper, we consider a multiview video system in which multiple cameras at different locations perform independent CS to simultaneously capture different views of a scene. At the decoder, we propose a disparity- and motion-compensated total variation minimization algorithm to jointly reconstruct the multiview video sequence. The experimental results show that the proposed joint reconstruction algorithm successfully exploits simultaneously intra-frame, inter-frame, and inter-view sparsity and significantly outperforms existing independent-view reconstruction, residue-view reconstruction, and motion-adaptive reconstruction algorithms. [ABSTRACT FROM AUTHOR]

Screen content coding plays an important role in many applications. Conventional reference picture management (RPM) strategies developed for natural content may not work well for screen content. This is because many regions in screen content remain static for a long time, causing a lot of repetitive contents to stay in the decoded picture buffer. The repetitive contents are not conducive to inter prediction, but still occupy valuable memory. This paper proposes a diversity-based RPM scheme for screen content coding. The concept of diversity is introduced for the reference picture set (RPS) to help formulate the RPM problem. By maximizing the diversity of RPS, more potentially better predictions are provided. Better compression performance can then be achieved. Meanwhile, the proposed scheme is nonnormative and compatible with existing video coding standards, such as High Efficiency Video Coding. The experimental results show that, for low delay screen content coding, the bit saving of the proposed scheme is 4.9% on average and up to 13.7%, without increasing encoding time. [ABSTRACT FROM AUTHOR]

Sparse modeling representative selection (SMRS) has been recently introduced for selecting the most relevant examples in data sets. SMRS exploits data self-representativeness coding in order to infer a coding matrix with block sparsity constraint. The relevance scores of samples are then derived from the estimated matrix of coefficients. Since SMRS is based on a linear model for data self-representation, it cannot always provide good relevant samples. Besides, most of its selected samples can be found in dense areas in input space. In this paper, we propose to overcome the SMRS method’s shortcomings that are related to the coding matrix estimation. We introduce two nonlinear data self-representativeness coding schemes that are based on Hilbert space and column generation. Experimental evaluation is carried out on summarizing a video movie and on summarizing training image data sets used for classification tasks. These experiments demonstrated that the proposed nonlinear methods can outperform state-of-the art selection methods. [ABSTRACT FROM AUTHOR]

3D video systems transmit depth maps in order to render synthesized views (SVs) at a receiver. To anticipate this purpose when processing a depth map, a sender-side depth processing algorithm (DPA), e.g. a depth encoder, can also render the SVs, compute their SV distortion (SVD), and adapt to it. This requires a low-complexity algorithm as computational resources are usually limited. We propose such an algorithm in this paper. First, we discuss a measure that relates a depth change to an SVD change using rendering. Then, we present an optimized process combining basic rendering steps, as warping, occlusion handling, interpolation, hole filling, and blending. Furthermore, we analyze which parts of an SV are affected by a depth change and modify the process to re-render only them. The resulting algorithm is significantly less complex than an unoptimized rendering-based variant and quantifies the SVD more accurately than existing estimation methods. The algorithm is used by the 3D-High Efficiency Video Coding reference software encoder as the main method for distortion computation and can also be used by other DPAs. [ABSTRACT FROM AUTHOR]

A smart device such as a television, tablet, and smartphone, often displays a compound image consisting of various types of sub-images, including texts and pictures. A text image displayed on a smart device is composed of red, green, and blue (RGB) sub-pixels having strong correlation among them. This paper proposes a new compression algorithm, called sub-pixel gradient compression (SPGC), that makes use of the correlation in a text in order to improve its compression efficiency. The initial step of the proposed algorithm converts a text into a de-colorized image in which RGB sub-pixels vary gradually. The de-colorization is a necessary step to further enhance the correlation among RGB sub-pixels, and thereby improving the compression efficiency. From the de-colorized text, the next step estimates the slopes of the gradual variations among sub-pixels and then encodes the slopes and their associated information as a bitstream. SPGC is a relatively simple algorithm because its main operation is the estimation of sub-pixel gradients and the precise estimation of the gradients makes it possible to reduce a degradation of the quality of the texts. Experimental results show that SPGC maintains lower complexity and a higher compression ratio than the compression standards including JPEG, JPEG-2000, H.264, or HEVC. [ABSTRACT FROM AUTHOR]

This paper presents a neural network-based method to generate multiple images with different exposures from a single input low dynamic range (LDR) image for improved high dynamic range (HDR) imaging. The proposed algorithm consists of three steps: 1) 2-D histogram estimation; 2) neural network-based LDR images estimation; and 3) generation of an optimal set of differently exposed images. The proposed method first generates image features by estimating a patched-based 2-D histogram. The extracted features are used in an input layer of the neural network, which plays a role to select an optimal set of LDR images. A set of LDR images is generated using a curvature-based contrast enhancement method. Experimental results show that the proposed method can generate an optimal set of LDR images using neural network and provide improved HDR images. In addition, the proposed method can be implemented as a preprocessing step in most existing HDR frameworks. The proposed HDR approach is considered as a single-input method that gives almost the same performance to multiple image-based HDR method. [ABSTRACT FROM PUBLISHER]

Bilateral filter is an edge-preserving smoother that has applications in image processing, computer vision, and computational photography. In the past, field-programmable gate array (FPGA) implementations of the filter have been proposed that can achieve high throughput using parallelization and pipelining. An inherent limitation with direct implementations is that their complexity scales as $O(\omega ^2)$ with the filter width $\omega$ . In this paper, we propose an FPGA implementation of a fast bilateral filter that requires just $O(1)$ operations for any arbitrary $\omega$. The attractive feature of the FPGA implementation is that it is both scalable and reconfigurable. To the best of our knowledge, this is the first scalable FPGA implementation of the bilateral filter. As an application, we use the FPGA implementation for image denoising. [ABSTRACT FROM PUBLISHER]

We consider the recovery of a continuous domain piecewise constant image from its nonuniform Fourier samples using a convex matrix completion algorithm. We assume the discontinuities/edges of the image are localized to the zero level set of a bandlimited function. This assumption induces linear dependencies between the Fourier coefficients of the image, which results in a two-fold block Toeplitz matrix constructed from the Fourier coefficients being low rank. The proposed algorithm reformulates the recovery of the unknown Fourier coefficients as a structured low-rank matrix completion problem, where the nuclear norm of the matrix is minimized subject to structure and data constraints. We show that the exact recovery is possible with high probability when the edge set of the image satisfies an incoherency property. We also show that the incoherency property is dependent on the geometry of the edge set curve, implying higher sampling burden for smaller curves. This paper generalizes recent work on the super-resolution recovery of isolated Diracs or signals with finite rate of innovation to the recovery of piecewise constant images. [ABSTRACT FROM AUTHOR]

A key challenge in feature correspondence is the difficulty in differentiating true and false matches at a local descriptor level. This forces adoption of strict similarity thresholds that discard many true matches. However, if analyzed at a global level, false matches are usually randomly scattered while true matches tend to be coherent (clustered around a few dominant motions), thus creating a coherence based separability constraint. This paper proposes a non-linear regression technique that can discover such a coherence based separability constraint from highly noisy matches and embed it into a correspondence likelihood model. Once computed, the model can filter the entire set of nearest neighbor matches (which typically contains over 90 percent false matches) for true matches. We integrate our technique into a full feature correspondence system which reliably generates large numbers of good quality correspondences over wide baselines where previous techniques provide few or no matches. [ABSTRACT FROM AUTHOR]

Many patch-based image denoising algorithms can be formulated as applying a smoothing filter to the noisy image. Expressed as matrices, the smoothing filters must be row normalized, so that each row sums to unity. Surprisingly, if we apply a column normalization before the row normalization, the performance of the smoothing filter can often be significantly improved. Prior works showed that such performance gain is related to the Sinkhorn–Knopp balancing algorithm, an iterative procedure that symmetrizes a row-stochastic matrix to a doubly stochastic matrix. However, a complete understanding of the performance gain phenomenon is still lacking. In this paper, we study the performance gain phenomenon from a statistical learning perspective. We show that Sinkhorn–Knopp is equivalent to an expectation-maximization (EM) algorithm of learning a Gaussian mixture model of the image patches. By establishing the correspondence between the steps of Sinkhorn–Knopp and the EM algorithm, we provide a geometrical interpretation of the symmetrization process. This observation allows us to develop a new denoising algorithm called Gaussian mixture model symmetric smoothing filter (GSF). GSF is an extension of the Sinkhorn–Knopp and is a generalization of the original smoothing filters. Despite its simple formulation, GSF outperforms many existing smoothing filters and has a similar performance compared with several state-of-the-art denoising algorithms. [ABSTRACT FROM PUBLISHER]

Moving object detection is about foreground and background separation based on motion detection. Detecting moving objects from similarly colored background (known as camouflage problem) has been a long-standing open question in this field. Discriminative modeling (DM), which focuses on enhancing the performance to distinguish foreground from background with discriminative features and well-designed classifiers, has been widely used for moving object detection. However, DM may tend to fail when encountering the camouflage problem, as the class separability in camouflaged areas is generally poor. In this paper, we propose a new strategy, camouflage modeling (CM), to identify camouflaged foreground pixels. In view of the fact that camouflage involves both foreground and background, we need to model both the background and the foreground, and compare them in a well-designed way in camouflage detection. Specifically, we develop a global model for the background, and an integration of global and local models for the foreground, respectively. Based on both background and foreground models, we introduce a factor to measure the degree of camouflage, and further identify truly camouflaged areas. In view of the fact that a moving object is usually composed of both camouflaged and noncamouflaged areas, CM and DM are fused in a Bayesian framework to perform complete object detection. Experiments are conducted on testing sequences to demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM PUBLISHER]

This paper develops distortion metrics for compressed synthetic aperture radar (SAR) imagery from change detection test statistics. These metrics are used to predict lossy image compression’s impact on change detection performance. The metrics do not require the intended change detection comparison image to provide these benefits. An SAR compression system leveraging the distortion metrics is proposed. The system generates a bad-pixel mask highlighting potential false alarms that are generated due to compression and are subsequently discarded in the change detection process. The proposed system’s performance is demonstrated through noncoherent change detection analysis after JPEG2000 and JPEG image compression. Similarly, a coherent change detection system is evaluated after JPEG2000 image compression. For noncoherent change detection at large compression ratios (CRs) using JPEG2000, the proposed system provides a 33% reduction in false alarms at a 0.1 probability of detection as well as the ability to maintain near-distortionless false alarm rates across a wide range of CRs. At a 0.1 probability of detection for coherent change detection, the system provides a 37% reduction in false alarms at modest CRs. The coherent change detection system is also demonstrated to maintain low false alarm rates across a range of CRs. [ABSTRACT FROM PUBLISHER]

Digital image processing (DIP) algorithms used in consumer electronics products have high computational complexities. Therefore, in this paper, we propose a novel low complexity 2D adaptive DIP algorithm. The proposed algorithm reduces computational complexities of 2D DIP algorithms by exploiting pixel correlations in input image without reducing quality of output image. We also designed a low energy 2D adaptive DIP hardware implementing the proposed algorithm. The proposed hardware is verified to work correctly on an FPGA board. It has significantly less energy consumption than original 2D DIP hardware. Therefore, it can be used especially in portable consumer electronics products. [ABSTRACT FROM AUTHOR]

In this paper, a graph-based nonlocal total variation method is proposed for unsupervised classification of hyperspectral images (HSI). The variational problem is solved by the primal-dual hybrid gradient algorithm. By squaring the labeling function and using a stable simplex clustering routine, an unsupervised clustering method with random initialization can be implemented. The effectiveness of this proposed algorithm is illustrated on both synthetic and real-world HSI, and numerical results show that the proposed algorithm outperforms other standard unsupervised clustering methods, such as spherical $K$ -means, nonnegative matrix factorization, and the graph-based Merriman–Bence–Osher scheme. [ABSTRACT FROM PUBLISHER]