Search

We consider the problem of minimizing the disturbance power at the output of the matched filter in a single antenna cognitive radar set-up. The aforementioned disturbance power can be shown to be an expectation of the slow-time ambiguity function (STAF) of the transmitted waveform over range-Doppler bins of interest. The design problem is known to yield a nonconvex quartic function of the transmit radar waveform. This STAF shaping problem becomes even more challenging in the presence of practical constraints on the transmit waveform such as the constant modulus constraint (CMC). Most existing approaches address the aforementioned challenges by suitably modifying or relaxing the design cost function and/or the CMC. In a departure from such methods, we develop a solution that involves direct optimization over the nonconvex complex circle manifold, i.e., the CMC set. We derive a new update strategy [quartic-gradient-descent (QGD)] that computes an exact gradient of the quartic cost and invokes principles of optimization over manifolds toward an iterative procedure with guarantees of monotonic cost function decrease and convergence. Experimentally, QGD can outperform state-of-the-art approaches for shaping the ambiguity function under the CMC while being computationally less expensive.

Vessel segmentation of retinal images is a key diagnostic capability in ophthalmology. This problem faces several challenges including low contrast, variable vessel size and thickness, and presence of interfering pathology such as micro-aneurysms and hemorrhages. Early approaches addressing this problem employed hand-crafted filters to capture vessel structures, accompanied by morphological post-processing. More recently, deep learning techniques have been employed with significantly enhanced segmentation accuracy. We propose a novel domain enriched deep network that consists of two components: 1) a representation network that learns geometric features specific to retinal images, and 2) a custom designed computationally efficient residual task network that utilizes the features obtained from the representation layer to perform pixel-level segmentation. The representation and task networks are {\em jointly learned} for any given training set. To obtain physically meaningful and practically effective representation filters, we propose two new constraints that are inspired by expected prior structure on these filters: 1) orientation constraint that promotes geometric diversity of curvilinear features, and 2) a data adaptive noise regularizer that penalizes false positives. Multi-scale extensions are developed to enable accurate detection of thin vessels. Experiments performed on three challenging benchmark databases under a variety of training scenarios show that the proposed prior guided deep network outperforms state of the art alternatives as measured by common evaluation metrics, while being more economical in network size and inference time., Accepted to IEEE TIP

Blind image deblurring remains a topic of enduring interest. Learning based approaches, especially those that employ neural networks have emerged to complement traditional model based methods and in many cases achieve vastly enhanced performance. That said, neural network approaches are generally empirically designed and the underlying structures are difficult to interpret. In recent years, a promising technique called algorithm unrolling has been developed that has helped connect iterative algorithms such as those for sparse coding to neural network architectures. In this article, we propose a neural network architecture based on this idea. We first present an iterative algorithm that may be considered as a generalization of the traditional total-variation regularization method in the gradient domain. We then unroll the algorithm to construct a neural network for image deblurring which we refer to as Deep Unrolling for Blind Deblurring (DUBLID). Key algorithm parameters are learned with the help of training images. Our proposed deep network DUBLID achieves significant practical performance gains while enjoying interpretability and efficiency at the same time. Extensive experimental results show that DUBLID outperforms many state-of-the-art methods and in addition is computationally faster.

It is common in anthropology and paleontology to address questions about extant and extinct species through the quantification of osteological features observable in micro-computed tomographic (micro-CT) scans. In cases where remains were buried, the grey values present in these scans may be classified as belonging to air, dirt, or bone. While various intensity-based methods have been proposed to segment scans into these classes, it is often the case that intensity values for dirt and bone are nearly indistinguishable. In these instances, scientists resort to laborious manual segmentation, which does not scale well in practice when a large number of scans are to be analyzed. Here we present a new domain-enriched network for three-class image segmentation, which utilizes the domain knowledge of experts familiar with manually segmenting bone and dirt structures. More precisely, our novel structure consists of two components: 1) a representation network trained on special samples based on newly designed custom loss terms, which extracts discriminative bone and dirt features, 2) and a segmentation network that leverages these extracted discriminative features. These two parts are jointly trained in order to optimize the segmentation performance. A comparison of our network to that of the current state-of-the-art U-NETs demonstrates the benefits of our proposal, particularly when the number of labeled training images are limited, which is invariably the case for micro-CT segmentation., 5 pages, 6 figures, accepted in 2020 54th Asilomar Conference on Signals, Systems, and Computers

Using low-frequency (UHF to L-band) ultrawideband synthetic aperture radar (SAR) technology for detecting buried and obscured targets, e.g., bomb or mine, has been successfully demonstrated recently. Despite promising recent progress, a significant open challenge is to distinguish obscured targets from other (natural and manmade) clutter sources in the scene. The problem becomes exacerbated in the presence of noisy responses from rough ground surfaces. In this paper, we present three novel sparsity-driven techniques, which not only exploit the subtle features of raw captured data, but also take advantage of the polarization diversity and the aspect angle dependence information from multichannel SAR data. First, the traditional sparse representation-based classification is generalized to exploit shared information of classes and various sparsity structures of tensor coefficients for multichannel data. Corresponding tensor dictionary learning models are consequently proposed to enhance classification accuracy. Finally, a new tensor sparsity model is proposed to model responses from multiple consecutive looks of objects, which is a unique characteristic of the data set we consider. Extensive experimental results on a high-fidelity electromagnetic simulated data set and radar data collected from the U.S. Army Research Laboratory side-looking SAR demonstrate the advantages of proposed tensor sparsity models.

We address the problem of designing a transmit beampattern for multiple-input multiple-output (MIMO) radar considering coexistence with wireless communication systems. The designed beampattern is able to manage the transmit energy in spatial directions as well as in spectral frequency bands of interest by minimizing the deviation of the designed beampattern versus a desired one under a spectral constraint as well as the constant modulus constraint. While unconstrained beampattern design is straightforward, a key open challenge is jointly enforcing the spectral constraint in addition to the constant modulus constraint on the radar waveform. A new approach is proposed in this paper, which involves solving a sequence of constrained quadratic programs such that constant modulus is achieved at convergence. Furthermore, we show that each problem in the sequence has a closed form solution leading to analytical tractability. We evaluate the proposed beampattern with interference control (BIC) algorithm against the state-of-the-art MIMO beampattern design techniques and show that BIC achieves closeness to an idealized beampattern along with desired spectral shaping.

Deep learning has been recently shown to improve performance in the domain of synthetic aperture sonar (SAS) image classification over existing shallow learning solutions. Given the constant resolution with range of a SAS, it is no surprise that deep learning techniques perform so well; the image of the seafloor produced by a SAS system is almost photographic in quality. Despite the image quality benefits of SAS, there is still room for classification improvement particularly in reducing the number of false alarms. This work addresses this by tackling one facet of the classification pipeline: image enhancement. Specifically, we ask and address the following question: Can we train a deep neural network to simultaneously enhance and classify a SAS image? We will respond in the affirmative as we introduce a new deep learning architecture tackling the problem, Data Adaptive Enhancement and Classification Network (DA-ECNet). DA-ECNet is a deep learning architecture which combines image enhancement as part of the classification procedure eliminating the need for a fixed state-of-the-art despeckling algorithm or enhancement module. Additionally, we train both image enhancement and classification jointly resulting in data adaptive image enhancement. Experiments on a challenging real-world dataset reveal that the proposed DA-ECNet outperforms state of the art deep learning as well as traditional feature based methods for SAS image classification.

The emergence of deep learning methods that complement traditional model-based methods has helped achieve a new state-of-the-art for image dehazing. Many recent methods design deep networks that either estimate the haze-free image (J) directly or estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t) followed by using the inverse of the haze model to estimate the dehazed image. However, both kinds of methods fail in dealing with non-homogeneous haze images where some parts of the image are covered with denser haze and the other parts with shallower haze. In this work, we develop a novel neural network architecture that can take benefits of the aforementioned two kinds of dehazed images simultaneously by estimating a new quantity - a spatially varying weight map (w). w can then be used to combine the directly estimated J and the results obtained by the inverse model. In our work, we utilize a shared DenseNet-based encoder, and four distinct DenseNet-based decoders that estimate J, A, t, and w jointly. A channel attention structure is added to facilitate the generation of distinct feature maps of different decoders. Furthermore, we propose a novel dilation inception module in the architecture to utilize the non-local features to make up the missing information during the learning process. Experiments performed on challenging benchmark datasets of NTIRE'20 and NTIRE'18 demonstrate that the proposed method -namely, AtJwD- can outperform many state-of-the-art alternatives in the sense of quality metrics such as SSIM, especially in recovering images under non-homogeneous haze.

This paper reviews the NTIRE 2020 Challenge on NonHomogeneous Dehazing of images (restoration of rich details in hazy image). We focus on the proposed solutions and their results evaluated on NH-Haze, a novel dataset consisting of 55 pairs of real haze free and nonhomogeneous hazy images recorded outdoor. NH-Haze is the first realistic nonhomogeneous haze dataset that provides ground truth images. The nonhomogeneous haze has been produced using a professional haze generator that imitates the real conditions of haze scenes. 168 participants registered in the challenge and 27 teams competed in the final testing phase. The proposed solutions gauge the state-of-the-art in image dehazing., CVPR Workshops Proceedings 2020

Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model-based methods and have helped define a new state of the art in achievable dehazed image quality. However, most deep learning-based methods usually design a regression network as a black-box tool to either estimate the dehazed image and/or the physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse haze model may then be used to estimate the dehazed image. In this work, we proposed a Depth-aware Dehazing using Reinforcement Learning system, denoted as DDRL. DDRL generates the dehazed image in a near-to-far progressive manner by utilizing the depth-information from the scene. This contrasts with the most recent learning-based methods that estimate these parameters in one pass. In particular, DDRL exploits the fact that the haze is less dense near the camera and gets increasingly denser as the scene moves farther away from the camera. DDRL consists of a policy network and a dehazing (regression) network. The policy network estimates the current depth for the dehazing network to use. A novel policy regularization term is introduced for the policy network to generate the policy sequence following the near-to-far order. Based on extensive tests over three benchmark test sets, DDRL demonstrates vastly enhanced dehazing results, particularly when training is limited.

High Dynamic Range (HDR) reconstruction is the process of producing an HDR image from a set of Standard Dynamic Range (SDR) images with different exposure times. This is a particularly challenging problem when relative camera or object motion exists between the available SDR images. Recently, deep learning methods, specifically those based on convolutional neural networks (CNNs) have been developed for HDR and shown to achieve unprecedented quality gains. Invariably an image alignment phase precedes the CNN mapping and merging. In practice, this alignment step greatly increases the computational burden of deep HDR methods often rendering them unsuitable for real-time composition. We propose a new deep HDR technique that does not need any explicit alignment of SDR images. Instead, a novel attention mask is developed that enables the network to focus on parts of the scene with considerable motion. Further, a dense merger is proposed that leads to an economical network. Evaluation over benchmark databases reveals that the proposed AttenDense network achieves high quality HDR results with significantly reduced computation time than state of the art. Further, the incorporation of domain knowledge (development of a custom attention mask) allows a more graceful decay in performance in the face of limited training.

We consider the problem of designing sequences with good auto- and cross-correlation properties for multiple-input multiple-output (MIMO) radar systems. This design problem aims to minimize a polynomial function of the transmit waveforms. The problem becomes more challenging in the presence of practical constraints on the waveform such as the constant modulus constraint (CMC). The aforementioned challenge has been addressed in the literature by approximating the cost function and/or constraints, i.e. the CMC. In this work, we develop a new algorithm that deals with the exact non-convex cost function and CMC. In particular, we develop a new update method (Correlation-Gradient-Descent (CGD)) that employs the exact gradient of the cost function to design such sequences with guarantees of monotonic cost function decrease and convergence. Our method further enables descent directly over the CMC by invoking principles of optimization over manifolds. Experimentally, CGD is evaluated against state-of-the-art methods for designing uni-modular sequences with good correlation properties. Results reveal that CGD can outperform these methods while being computationally less expensive.

In this paper, we propose a new algorithm that designs a transmit beampattern for Multiple-Input Multiple-Output (MIMO) radar considering coexistence with other wireless systems. This design process is conducted by minimizing the deviation of the generated beampattern against an idealized one while enforcing the waveform elements to be constant modulus and in the presence of spectral restrictions, which translates to a quadratic waveform constraint. This leads to a hard nonconvex optimization problem due to simultaneous presence of the constant modulus constraint (CMC) and spectral constraint (SpecC). In this work, we employ the geometrical structure of CMC, that is we redefine this constraint as an intersection of two sets. This definition allows us to handle the design problem in more tractable way. The proposed Iterative Beampattern with Spectral design (IBS) algorithm solves a sequence of convex problems such that constant modulus is achieved at convergence. Furthermore, we show that at convergence the obtained solution satisfies the Karush-Kuhn-Tucker (KKT) conditions of the aforementioned non-convex problem. Finally, we evaluate the proposed algorithm over challenging simulated scenarios, and show that it outperforms the state of the art competing methods.

Deep neural networks provide unprecedented performance gains in many real world problems in signal and image processing. Despite these gains, future development and practical deployment of deep networks is hindered by their blackbox nature, i.e., lack of interpretability, and by the need for very large training sets. An emerging technique called algorithm unrolling or unfolding offers promise in eliminating these issues by providing a concrete and systematic connection between iterative algorithms that are used widely in signal processing and deep neural networks. Unrolling methods were first proposed to develop fast neural network approximations for sparse coding. More recently, this direction has attracted enormous attention and is rapidly growing both in theoretic investigations and practical applications. The growing popularity of unrolled deep networks is due in part to their potential in developing efficient, high-performance and yet interpretable network architectures from reasonable size training sets. In this article, we review algorithm unrolling for signal and image processing. We extensively cover popular techniques for algorithm unrolling in various domains of signal and image processing including imaging, vision and recognition, and speech processing. By reviewing previous works, we reveal the connections between iterative algorithms and neural networks and present recent theoretical results. Finally, we provide a discussion on current limitations of unrolling and suggest possible future research directions.

This paper deals with the joint design of Multiple-Input Multiple-Output (MIMO) radar transmit waveform and receive filter to enhance multiple targets detectability in the presence of signal-dependent (clutter) and independent disturbance. The worst-case Signal-to-Interference-Noise-Ratio (SINR) over multiple targets is explicitly maximized. To ensure hardware compatibility and the coexistence between MIMO radar and other wireless systems, constant modulus and spectral restrictions on the waveform are incorporated in our design. A max-min non-convex optimization problem emerges as a function of the transmit waveform, which we solve via a novel polynomial-time iterative procedure that involves solving a sequence of convex problems with constraints that evolve with every iteration. We provide analytical guarantees of monotonic cost function improvement with proof of convergence to a solution that satisfies the KarushKuhnTucker (KKT) conditions. By simulating challenging practical scenarios, we evaluate the proposed algorithm against the state-of-the-art methods in terms of the achieved SINR value and the computational complexity.

Multiple-input multiple-output (MIMO) radar systems allow each antenna element to transmit a different waveform. This waveform diversity can be exploited to enhance the beampattern design, in particular, effective management of radar radiation power in directions of interest. We address the problem of designing a beampattern for MIMO radar, which in turn is determined by the transmit waveform. While unconstrained design is straightforward, a key open challenge is enforcing the constant modulus constraint on the radar waveform. It is well known that the problem of minimizing deviation of the designed beampattern from an idealized one subject to the constant modulus constraint constitutes a hard nonconvex problem. Existing methods that address constant modulus invariably lead to a stiff tradeoff between analytical tractability (achieved by relaxations and approximations) and realistic design that exactly achieves constant modulus but is computationally burdensome. A new approach is proposed in our paper, which involves solving a sequence of convex equality constrained quadratic programs, each of which has a closed form solution and such that constant modulus is achieved at convergence. We further prove that the converged solution satisfies the Karush–Kuhn–Tucker optimality conditions of the aforementioned hard nonconvex problem. We evaluate the proposed successive closed forms (SCF) algorithm against the state-of-the art MIMO beampattern design techniques in both narrowband and wideband setups and show that the SCF breaks the tradeoff between desirable performance and the associated computation cost.

High dynamic range (HDR) image synthesis from multiple low dynamic range (LDR) exposures continues to be actively researched. The extension to HDR video synthesis is a topic of significant current interest due to potential cost benefits. For HDR video, a stiff practical challenge presents itself in the form of accurate correspondence estimation of objects between video frames. In particular, loss of data resulting from poor exposures and varying intensity make conventional optical flow methods highly inaccurate. We avoid exact correspondence estimation by proposing a statistical approach via maximum a posterior (MAP) estimation, and under appropriate statistical assumptions and choice of priors and models, we reduce it to an optimization problem of solving for the foreground and background of the target frame. We obtain the background through rank minimization and estimate the foreground via a novel multiscale adaptive kernel regression technique, which implicitly captures local structure and temporal motion by solving an unconstrained optimization problem. Extensive experimental results on both real and synthetic datasets demonstrate that our algorithm is more capable of delivering high-quality HDR videos than current state-of-the-art methods, under both subjective and objective assessments. Furthermore, a thorough complexity analysis reveals that our algorithm achieves better complexity-performance trade-off than conventional methods.

Recent covariance estimation methods for radar space-time adaptive processing exploit practical constraints such as the rank of clutter subspace and the condition number of disturbance covariance to estimate accurate covariance even when training is not generous. While rank and condition number are very effective constraints, often practical nonidealities make it difficult to know them precisely using physical models. Therefore, we propose a method to determine constraints in covariance estimation for radar space-time adaptive processing via an expected likelihood approach. We analyze three cases of constraints: 1) a rank constraint, 2) both rank and noise power constraints, and 3) a condition number constraint. In each case, we formulate precise constraint determination as an optimization problem. For each of the three cases, we derive new analytical results which allow for computationally efficient, practical ways of determining these constraints with formal proofs. Through experimental results from a simulation model and the KASSPER data set, we show that the estimator with optimal constraints obtained by the expected likelihood approach outperforms state-of-the-art alternatives.

We propose an optimization-based power-constrained red-green–blue (RGB)-to-red-green–blue-white (RGBW) conversion algorithm for emissive RGBW displays. We measure the perceived color distortion using a color difference model in a perceptually uniform color space, and compute the power consumption for displaying an RGBW pixel on an emissive display. The central contribution of this paper is to formulate the optimization problem to minimize the color distortion subject to a constraint on the power consumption. Subsequently, we solve the optimization problem efficiently to convert an image in real time. Furthermore, based on the properties of the human visual system, we extend the proposed algorithm to image-dependent conversion that can preserve spatial detail in an input image. The simulation results show that the proposed algorithm provides a significantly less color distortion than the conventional methods, while providing a graceful tradeoff with the amount of power consumed. Specifically, it is shown that the power consumption can be reduced by up to 20%, while providing about 50% less color distortion than the conventional algorithms. In addition, a subjective evaluation on a real RGBW display is performed, which reveals the merits of the proposed image-dependent conversion for improving the perceptual quality over state-of-the-art techniques.

High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion \color{black} as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix(and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance., Accepted to IEEE transactions on Image Processing

Vessel segmentation of retinal images is a key diagnostic capability in ophthalmology. Early approaches addressing this problem employed hand-crafted filters to capture vessel structures, accompanied by morphological processing. More recently, deep learning techniques have been employed to significantly enhance segmentation accuracy. We propose a novel domain enriched deep network that consists of two components: 1) a representation network which learns geometric (specifically curvilinear) features that are tailored to retinal images, followed by 2) a task network that utilizes the features obtained from the representation layer to perform pixel-level segmentation. The representation and task networks are learned jointly for any given training set. To obtain effective representation filters, we develop a new orientation constraint that enables geometric diversity of curvilinear features. A multi-scale extension is further developed to enhance segmentation of thin vessels. Experiments performed on two challenging benchmark databases reveal that the proposed regularized deep network can outperform state of the art alternatives as measured by common evaluation metrics. Further, the proposed method exhibits a more graceful decay in performance as training data is reduced.

Objective: The diversity of tissue structure in histopathological images makes feature extraction for classification a challenging task. Dictionary learning within a sparse representation-based classification (SRC) framework has been shown to be successful for feature discovery. However, there exist stiff practical challenges: 1) computational complexity of SRC can be onerous in the decision stage since it involves solving a sparsity constrained optimization problem and often over a large number of image patches; and 2) images from distinct classes continue to share several geometric features. We propose a novel analysis–synthesis model learning with shared features algorithm (ALSF) for classifying such images more effectively. Methods: In the ALSF, a joint analysis and synthesis learning model is introduced to learn the classifier and the feature extractor at the same time. Unlike SRC, no explicit optimization is needed in the inference phase leading to much reduced computation. Crucially, we introduce the learning of a low-rank shared dictionary and a shared analysis operator, which more accurately represents both similarities and differences in histopathological images from distinct classes. We also develop an extension of ALSF with a sparsity constraint, whose presence or absence facilitates a cost–performance tradeoff. Results: The ALSF is evaluated on three challenging and well-known datasets: 1) spleen tissue images; 2) brain tumor images; and 3) breast cancer tissue dataset, provided by different organizations. Conclusion: Experimental results demonstrate both complexity and performance benefits of the ALSF over state-of-the-art alternatives. Significance: Modeling shared features with appropriate quantitative constraints lead to significantly improved classification in histopathology.

This paper reviews the first NTIRE challenge on video deblurring (restoration of rich details and high frequency components from blurred video frames) with focus on the proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed dynamic motion blurs while Track 2 had additional MPEG video compression artifacts. Each competition had 109 and 93 registered participants. Total 13 teams competed in the final testing phase. They gauge the state-of-the-art in video deblurring problem.

Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model-based methods and have helped define a new state of the art in achievable dehazed image quality. Yet, practical challenges remain in dehazing of real-world images where the scene is heavily covered with dense haze, even to the extent that no scene information can be observed visually. Many recent dehazing methods have addressed this challenge by designing deep networks that estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse of the haze model may then be used to estimate the dehazed image. In this work, we develop two novel network architectures to further this line of investigation. Our first model, denoted as At-DH, designs a shared DenseNet based encoder and two distinct DensetNet based decoders to jointly estimate the scene information viz. A and t respectively. This in contrast to most recent efforts (include those published in CVPR'18) that estimate these physical parameters separately. As a natural extension of At-DH, we develop the AtJ-DH network, which adds one more DenseNet based decoder to jointly recreate the haze-free image along with A and t. The knowledge of (ground truth) training dehazed/clean images can be exploited by a custom regularization term that further enhances the estimates of model parameters A and t in AtJ-DH. Experiments performed on challenging benchmark image datasets of NTIRE'19 and NTIRE'18 demonstrate that At-DH and AtJ-DH can outperform state-of-the-art alternatives, especially when recovering images corrupted by dense haze.

Single image dehazing has gained much attention recently. A typical learning based approach uses example hazy and clean image pairs to train a mapping between the two. Of the learning based methods, those based on deep neural networks have shown to deliver state of the art performance. An important aspect of recovered image quality is the color information, which is severely compromised when the image is corrupted by very dense haze. While many different network architectures have been developed for recovering dehazed images, an explicit attention to recovering individual color channels with a design that ensures their quality has been missing. Our proposed work, focuses on this issue by developing a novel network structure that comprises of: a common DenseNet based feature encoder whose output branches into three distinct DensetNet based decoders to yield estimates of the R, G and B color channels of the image. A subsequent refinement block further enhances the final synthesized RGB/color image by joint processing of these color channels. Inspired by its structure, we call our approach the One-To-Three Color Enhancement Dehazing (123-CEDH) network. To ensure the recovery of physically meaningful and high quality color channels, the main network loss function is further regularized by a multi-scale structural similarity index term as well as a term that enhances color contrast. Experiments reveal that 123-CEDH has the ability to recover color information at early training stages (i.e. in the first few epochs) vs. other highly competitive methods. Validation on the benchmark datasets of the NTIRE'19 and NTIRE'18 dehazing challenges reveals the 123-CEDH to be one of the Top-3 methods based on results released in the NTIRE'19 competition.

Deep learning methods, in particular, trained convolutional neural networks (CNNs) have recently been shown to produce compelling results for single image super-resolution (SR). Invariably, a CNN is learned to map the low resolution (LR) image to its corresponding high resolution (HR) version in the spatial domain. We propose a novel network structure for learning the SR mapping function in an image transform domain, specifically the discrete cosine transform (DCT). As the first contribution, we show that DCT can be integrated into the network structure as a convolutional DCT (CDCT) layer. With the CDCT layer, we construct the DCT deep SR (DCT-DSR) network. We further extend the DCT-DSR to allow the CDCT layer to become trainable (i.e., optimizable) . Because this layer represents an image transform, we enforce pairwise orthogonality constraints and newly formulated complexity order constraints on the individual basis functions/filters. This orthogonally regularized deep SR network (ORDSR) simplifies the SR task by taking advantage of image transform domain while adapting the design of transform basis to the training image set. The experimental results show ORDSR achieves state-of-the-art SR image quality with fewer parameters than most of the deep CNN methods. A particular success of ORDSR is in overcoming the artifacts introduced by bicubic interpolation. A key burden of deep SR has been identified as the requirement of generous training LR and HR image pairs; ORSDR exhibits a much more graceful degradation as training size is reduced with significant benefits in the regime of limited training. Analysis of memory and computation requirements confirms that ORDSR can allow for a more efficient network with faster inference.

The ability of Multiple-Input Multiple-Output (MIMO) radar systems to adapt waveforms across antennas allows flexibility in the transmit beampattern design. In cognitive radar, a popular cost function is to minimize the deviation against an idealized beampattern (which is arrived at with knowledge of the environment). The optimization of the transmit beampattern becomes particularly challenging in the presence of practical constraints on the transmit waveform. One of the hardest of such constraints is the non-convex constant modulus constraint, which has been the subject of much recent work. In a departure from most existing approaches, we develop a solution that involves direct optimization over the non-convex complex circle manifold. That is, we derive a new projection, descent, and retraction (PDR) update strategy that allows for monotonic cost function improvement while maintaining feasibility over the complex circle manifold (constant modulus set). For quadratic cost functions (as is the case with beampattern deviation), we provide analytical guarantees of monotonic cost function improvement along with proof of convergence to a local minima. We evaluate the proposed PDR algorithm against other candidate MIMO beampattern design methods and show that PDR can outperform competing wideband beampattern design methods while being computationally less expensive. Finally, orthogonality across antennas is incorporated in the PDR framework by adding a penalty term to the beampattern cost function. Enabled by orthogonal waveforms, robustness to target direction mismatch is also demonstrated.

While neural networks have achieved vastly enhanced performance over traditional iterative methods in many cases, they are generally empirically designed and the underlying structures are difficult to interpret. The algorithm unrolling approach has helped connect iterative algorithms to neural network architectures. However, such connections have not been made yet for blind image deblurring. In this paper, we propose a neural network architecture that advances this idea. We first present an iterative algorithm that may be considered a generalization of the traditional total-variation regularization method on the gradient domain, and subsequently unroll the half-quadratic splitting algorithm to construct a neural network. Our proposed deep network achieves significant practical performance gains while enjoying interpretability at the same time. Experimental results show that our approach outperforms many state-of-the-art methods., Comment: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

We study the problem of image alignment for panoramic stitching. Unlike most existing approaches that are feature-based, our algorithm works on pixels directly, and accounts for errors across the whole images globally. Technically, we formulate the alignment problem as rank-1 and sparse matrix decomposition over transformed images, and develop an efficient algorithm for solving this challenging non-convex optimization problem. The algorithm reduces to solving a sequence of subproblems, where we analytically establish exact recovery conditions, convergence and optimality, together with convergence rate and complexity. We generalize it to simultaneously align multiple images and recover multiple homographies, extending its application scope towards vast majority of practical scenarios. Experimental results demonstrate that the proposed algorithm is capable of more accurately aligning the images and generating higher quality stitched images than state-of-the-art methods., Comment: Accepted for publication in IEEE Transactions on Image Processing

Detection of cell nuclei in microscopic images is a challenging research topic, because of limitations in cellular image quality and diversity of nuclear morphology, i.e. varying nuclei shapes, sizes, and overlaps between multiple cell nuclei. This has been a topic of enduring interest with promising recent success shown by deep learning methods. These methods train for example convolutional neural networks (CNNs) with a training set of input images and known, labeled nuclei locations. Many of these methods are supplemented by spatial or morphological processing. We develop a new approach that we call Shape Priors with Convolutional Neural Networks (SP-CNN) to perform significantly enhanced nuclei detection. A set of canonical shapes is prepared with the help of a domain expert. Subsequently, we present a new network structure that can incorporate `expected behavior' of nucleus shapes via two components: {\em learnable} layers that perform the nucleus detection and a {\em fixed} processing part that guides the learning with prior information. Analytically, we formulate a new regularization term that is targeted at penalizing false positives while simultaneously encouraging detection inside cell nucleus boundary. Experimental results on a challenging dataset reveal that SP-CNN is competitive with or outperforms several state-of-the-art methods., Accepted paper to 2018 IEEE International Conference on Image Processing (ICIP 2018)

The capability of Multiple-Input Multiple-Output (MIMO) radar systems to adapt waveforms across antennas allows flexibility in transmit beampattern design. In cognitive radar, a popular cost function is to minimize deviation against an idealized beampattern (which is arrived at with knowledge of the environment). The optimization of transmit beampattern becomes particularly challenging in the presence of practical constraints on the transmit waveform. In this work, we study two important but difficult non-convex constraints: constant modulus and orthogonality of waveform code across transmit antennas. Individually, incorporating these constraints continues to be challenging and they have rarely been jointly studied because each set is individually a (non-convex) manifold. We develop a new approach by leveraging the theory of optimization over manifolds: we derive a new projection, descent and retraction (PDR) update strategy that allows for monotonic cost function improvement while maintaining feasibility over the complex circle manifold (constant modulus set). The approach allows a tractable extension to incorporate orthogonality as a penalty term with the cost function. We provide analytical guarantees of monotonic cost function improvement along with proof of convergence. Experimentally, we show that PDR can outperform state of the art wideband beampattern design methods while being computationally tractable. Robustness to target direction mismatch (enabled by orthogonal waveforms) is also demonstrated.

The authors address the problem of designing a waveform for multiple-input multiple-output (MIMO) radar under the important practical constraints of constant modulus and waveform similarity. Incorporating these constraints in an analytically tractable manner is a longstanding open challenge. This is due to the fact that the optimization problem that results from signal-to-interference-plus-noise ratio (SINR) maximization subject to these constraints is a hard non-convex problem. The authors develop a new analytical approach that involves solving a sequence of convex quadratically constrained quadratic programing (QCQP) problems, which they prove converges to a sub-optimal solution. Because an improvement in SINR results via solving each problem in the sequence, they call the method Successive QCQP Refinement (SQR). Furthermore, the proposed SQR method can be easily extended to incorporate emerging requirements of spectral coexistence, as shown briefly in this paper. The authors evaluate SQR against other candidate techniques with respect to SINR performance, beam pattern, and pulse compression properties in a variety of scenarios. Results show that SQR outperforms state-of-the-art methods that also employ constant modulus and/or similarity constraints while being computationally less burdensome.

High resolution magnetic resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware, cost and processing constraints. Recently, deep learning methods have been shown to produce compelling state of the art results for image super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image superresolution. Our contributions are then incorporating these priors in an analytically tractable fashion in the learning of a convolutional neural network (CNN) that accomplishes the super-resolution task. This is particularly challenging for the low rank prior, since the rank is not a differentiable function of the image matrix (and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed {\em feedback} layer at the output of the network. Experiments performed on two publicly available MR brain image databases exhibit promising results particularly when training imagery is limited., Comment: Accepted to IEEE ICIP 2018

Deep learning methods, in particular trained Convolutional Neural Networks (CNNs) have recently been shown to produce compelling state-of-the-art results for single image Super-Resolution (SR). Invariably, a CNN is learned to map the low resolution (LR) image to its corresponding high resolution (HR) version in the spatial domain. Aiming for faster inference and more efficient solutions than solving the SR problem in the spatial domain, we propose a novel network structure for learning the SR mapping function in an image transform domain, specifically the Discrete Cosine Transform (DCT). As a first contribution, we show that DCT can be integrated into the network structure as a Convolutional DCT (CDCT) layer. We further extend the network to allow the CDCT layer to become trainable (i.e. optimizable). Because this layer represents an image transform, we enforce pairwise orthogonality constraints on the individual basis functions/filters. This Orthogonally Regularized Deep SR network (ORDSR) simplifies the SR task by taking advantage of image transform domain while adapting the design of transform basis to the training image set. Experimental results show ORDSR achieves state-of-the-art SR image quality with fewer parameters than most of the deep CNN methods.

Automated histopathological image analysis offers exciting opportunities for the early diagnosis of several medical conditions including cancer. There are however stiff practical challenges: 1.) discriminative features from such images for separating diseased vs. healthy classes are not readily apparent, and 2.) distinct classes, e.g. healthy vs. stages of disease continue to share several geometric features. We propose a novel Analysis-synthesis model Learning with Shared Features algorithm (ALSF) for classifying such images more effectively. In ALSF, a joint analysis and synthesis learning model is introduced to learn the classifier and the feature extractor at the same time. In this way, the computation load in patch-level based image classification can be much reduced. Crucially, we integrate into this framework the learning of a low rank shared dictionary and a shared analysis operator, which more accurately represents both similarities and differences in histopathological images from distinct classes. ALSF is evaluated on two challenging databases: (1) kidney tissue images provided by the Animal Diagnosis Lab (ADL) at the Pennsylvania State University and (2) brain tumor images from The Cancer Genome Atlas (TCGA) database. Experimental results confirm that ALSF can offer benefits over state of the art alternatives., Comment: 2018 ISBI conference accepted paper

Classifying buried and obscured targets of interest from other natural and manmade clutter objects in the scene is an important problem for the U.S. Army. Targets of interest are often represented by signals captured using low-frequency (UHF to L-band) ultra-wideband (UWB) synthetic aperture radar (SAR) technology. This technology has been used in various applications, including ground penetration and sensing-through-the-wall. However, the technology still faces a significant issue regarding low-resolution SAR imagery in this particular frequency band, low radar cross sections (RCS), small objects compared to radar signal wavelengths, and heavy interference. The classification problem has been firstly, and partially, addressed by sparse representation-based classification (SRC) method which can extract noise from signals and exploit the cross-channel information. Despite providing potential results, SRC-related methods have drawbacks in representing nonlinear relations and dealing with larger training sets. In this paper, we propose a Simultaneous Decomposition and Classification Network (SDCN) to alleviate noise inferences and enhance classification accuracy. The network contains two jointly trained sub-networks: the decomposition sub-network handles denoising, while the classification sub-network discriminates targets from confusers. Experimental results show significant improvements over a network without decomposition and SRC-related methods.

Single image super-resolution (SR) via deep learning has recently gained significant attention in the literature. Convolutional neural networks (CNNs) are typically learned to represent the mapping between low-resolution (LR) and high-resolution (HR) images/patches with the help of training examples. Most existing deep networks for SR produce high quality results when training data is abundant. However, their performance degrades sharply when training is limited. We propose to regularize deep structures with prior knowledge about the images so that they can capture more structural information from the same limited data. In particular, we incorporate in a tractable fashion within the CNN framework, natural image priors which have shown to have much recent success in imaging and vision inverse problems. Experimental results show that the proposed deep network with natural image priors is particularly effective in training starved regimes.

There are many real-world classification problems wherein the issue of data imbalance (the case when a data set contains substantially more samples for one/many classes than the rest) is unavoidable. While under-sampling the problematic classes is a common solution, this is not a compelling option when the large data class is itself diverse and/or the limited data class is especially small. We suggest a strategy based on recent work concerning limited data problems which utilizes a supplemental set of images with similar properties to the limited data class to aid in the training of a neural network. We show results for our model against other typical methods on a real-world synthetic aperture sonar data set. Code can be found at github.com/JohnMcKay/dataImbalance., Submitted to IGARSS 2018, 4 Pages, 8 Figures

Feature extraction from infrared (IR) images remains a challenging task. Learning based methods that can work on raw imagery/patches have therefore assumed significance. We propose a novel multi-task extension of the widely used sparse-representation-classification (SRC) method in both single and multi-view set-ups. That is, the test sample could be a single IR image or images from different views. When expanded in terms of a training dictionary, the coefficient matrix in a multi-view scenario admits a sparse structure that is not easily captured by traditional sparsity-inducing measures such as the $l_0$-row pseudo norm. To that end, we employ collaborative spike and slab priors on the coefficient matrix, which can capture fairly general sparse structures. Our work involves joint parameter and sparse coefficient estimation (JPCEM) which alleviates the need to handpick prior parameters before classification. The experimental merits of JPCEM are substantiated through comparisons with other state-of-art methods on a challenging mid-wave IR image (MWIR) ATR database made available by the US Army Night Vision and Electronic Sensors Directorate., Comment: 4 pages, 3 figures, conference paper

Blind image deblurring is a particularly challenging inverse problem where the blur kernel is unknown and must be estimated en route to recover the deblurred image. The problem is of strong practical relevance since many imaging devices such as cellphone cameras, must rely on deblurring algorithms to yield satisfactory image quality. Despite significant research effort, handling large motions remains an open problem. In this paper, we develop a new method called Blind Image Deblurring using Row-Column Sparsity (BD-RCS) to address this issue. Specifically, we model the outer product of kernel and image coefficients in certain transformation domains as a rank-one matrix, and recover it by solving a rank minimization problem. Our central contribution then includes solving {\em two new} optimization problems involving row and column sparsity to automatically determine blur kernel and image support sequentially. The kernel and image can then be recovered through a singular value decomposition (SVD). Experimental results on linear motion deblurring demonstrate that BD-RCS can yield better results than state of the art, particularly when the blur is caused by large motion. This is confirmed both visually and through quantitative measures., Accepted to IEEE Signal Processing Letters, December 2017

Ambiguity function shaping continues to be one of the most challenging open problems in cognitive radar. Analytically, a complex quartic function should be optimized as a function of the radar waveform code. Practical considerations further require that the waveform be constant modulus, which exacerbates the issue and leads to a hard non-convex problem. We develop a new approach called Adaptive Sequential Refinement (ASR) to suppress the clutter returns for a desired range-Doppler, i.e. ambiguity function response. ASR solves the aforementioned optimization problem in a unique iterative manner such that the formulation is updated depending on the iteration index. We establish formally that: 1.) the problem in each step of the iteration has a closed form solution, and 2.) monotonic decrease of the cost function until convergence is guaranteed. Experimental validation shows that ASR produces a radar waveform with higher Signal to Interference Ratio (SIR) and superior ambiguity function shaping than state of the art alternatives even as its computational burden is orders of magnitude lower.

This paper reviews the first challenge on single image super-resolution (restoration of rich details in an low resolution image) with focus on proposed solutions and results. A new DIVerse 2K resolution image dataset (DIV2K) was employed. The challenge had 6 competitions divided into 2 tracks with 3 magnification factors each. Track 1 employed the standard bicubic downscaling setup, while Track 2 had unknown downscaling operators (blur kernel and decimation) but learnable through low and high res train images. Each competition had ∽100 registered participants and 20 teams competed in the final testing phase. They gauge the state-of-the-art in single image super-resolution.

Recent advances have seen a surge of deep learning approaches for image super-resolution. Invariably, a network, e.g. a deep convolutional neural network (CNN) or auto-encoder is trained to learn the relationship between low and high-resolution image patches. Recognizing that a wavelet transform provides a "coarse" as well as "detail" separation of image content, we design a deep CNN to predict the "missing details" of wavelet coefficients of the low-resolution images to obtain the Super-Resolution (SR) results, which we name Deep Wavelet Super-Resolution (DWSR). Out network is trained in the wavelet domain with four input and output channels respectively. The input comprises of 4 sub-bands of the low-resolution wavelet coefficients and outputs are residuals (missing details) of 4 sub-bands of high-resolution wavelet coefficients. Wavelet coefficients and wavelet residuals are used as input and outputs of our network to further enhance the sparsity of activation maps. A key benefit of such a design is that it greatly reduces the training burden of learning the network that reconstructs low frequency details. The output prediction is added to the input to form the final SR wavelet coefficients. Then the inverse 2d discrete wavelet transformation is applied to transform the predicted details and generate the SR results. We show that DWSR is computationally simpler and yet produces competitive and often better results than state-of-the-art alternatives.

Sonar imaging has seen vast improvements over the last few decades due in part to advances in synthetic aperture Sonar (SAS). Sophisticated classification techniques can now be used in Sonar automatic target recognition (ATR) to locate mines and other threatening objects. Among the most promising of these methods is sparse reconstruction-based classification (SRC) which has shown an impressive resiliency to noise, blur, and occlusion. We present a coherent strategy for expanding upon SRC for Sonar ATR that retains SRC's robustness while also being able to handle targets with diverse geometric arrangements, bothersome Rayleigh noise, and unavoidable background clutter. Our method, pose corrected sparsity (PCS), incorporates a novel interpretation of a spike and slab probability distribution towards use as a Bayesian prior for class-specific discrimination in combination with a dictionary learning scheme for localized patch extractions. Additionally, PCS offers the potential for anomaly detection in order to avoid false identifications of tested objects from outside the training set with no additional training required. Compelling results are shown using a database provided by the United States Naval Surface Warfare Center., 14 Pages, 16 Figures, Accepted TGARS

Sparsity constrained single image super-resolution (SR) has been of much recent interest. A typical approach involves sparsely representing patches in a low-resolution (LR) input image via a dictionary of example LR patches, and then using the coefficients of this representation to generate the high-resolution (HR) output via an analogous HR dictionary. However, most existing sparse representation methods for super resolution focus on the luminance channel information and do not capture interactions between color channels. In this work, we extend sparsity based super-resolution to multiple color channels by taking color information into account. Edge similarities amongst RGB color bands are exploited as cross channel correlation constraints. These additional constraints lead to a new optimization problem which is not easily solvable; however, a tractable solution is proposed to solve it efficiently. Moreover, to fully exploit the complementary information among color channels, a dictionary learning method is also proposed specifically to learn color dictionaries that encourage edge similarities. Merits of the proposed method over state of the art are demonstrated both visually and quantitatively using image quality metrics.

Although a lot of progress has been made over the years, one critical challenge still facing low-frequency (UHF to L-band) ultra-wideband (UWB) synthetic aperture radar (SAR) technology is the discrimination of buried and obscured targets of interest from other natural and manmade clutter objects in the scene. The key issues are i) low-resolution SAR imagery for this frequency band, ii) targets of interests being typically small compared to the radar signal wavelengths, iii) targets having low radar cross sections (RCS), and iv) very noisy SAR imagery (e.g. target responses buried in responses from cluttered environment). In this paper, we consider the problem of discriminating and classifying buried targets of interest (buried metal and plastic mines, 155-mm unexploded ordinance [UXO], etc.) from other natural and manmade clutter objects (soda can, rocks, etc.) in the presence of noisy responses from the rough ground surfaces for low-frequency UWB 2-D SAR images. We generalize the traditional sparse representation-based classification (SRC) to a model with capability of using the information of the shared class, and implement multichannel classification problems by exploiting structures of sparse coefficients using various techniques. Here, we employ an electromagnetic (EM) SAR database generated using the finite-difference, time-domain (FDTD) software, which is based on a full-wave computational EM method.

In multiple array radar systems, waveform diversity can be exploited to enhance the beampattern design. However, in a spectrally crowded environment, the spectral shape of radar waveforms must be compatible with civilian and other wireless communication services. We address the problem of designing a beampattern for MIMO radar under a spectral interference constraint. The problem essentially reduces to a constrained design of the radar waveform, where a key open challenge is jointly enforcing the spectral interference constraint in addition to the constant modulus constraint (CMC). A new approach is proposed in our work, which involves solving a sequence of convex Constrained Quadratic Programs, each of which has a closed form solution and such that constant modulus is achieved at convergence. New analytical results are provided in support of the proposed Beampattern design under constant modulus and Interference Constraints (BIC). We evaluate the proposed BIC algorithm against state of the art MIMO beampattern design methods that enforce constant modulus and show that BIC achieves a high fidelity radar beampattern while maintaining low spectral interference in the desired bands.

Any image recovery algorithm attempts to achieve the highest quality reconstruction in a timely manner. The former can be achieved in several ways, among which are by incorporating Bayesian priors that exploit natural image tendencies to cue in on relevant phenomena. The Hierarchical Bayesian MAP (HB-MAP) is one such approach which is known to produce compelling results albeit at a substantial computational cost. We look to provide further analysis and insights into what makes the HB-MAP work. While retaining the proficient nature of HB-MAP's Type-I estimation, we propose a stochastic approximation-based approach to Type-II estimation. The resulting algorithm, fast stochastic HB-MAP (fsHBMAP), takes dramatically fewer operations while retaining high reconstruction quality. We employ our fsHBMAP scheme towards the problem of tomographic imaging and demonstrate that fsHBMAP furnishes promising results when compared to many competing methods., Comment: 5 Pages, 4 Figures, Conference (Accepted to Asilomar 2017)

Recently deep learning methods have been applied to image super-resolution (SR). Typically, these approaches involve training a single convolutional neural network that is trained to perform resolution enhancement. We propose a new low-complexity but effective algorithm called Superresolution with Coupled Backpropagation (SR-CBP) which builds two Coupled Auto-encoder Networks (CAN), resp. the high-resolution (HR) and low-resolution (LR) networks, that capture the features of both high and low resolution images. The two networks in CAN have the ability to self-reconstruct its own input. Specifically, SR-CBP allows joint training of the LR and HR networks to have middle layer representations that agree for a pair of images (high-resolution image and its corresponding low-resolution version). For an LR input image, its middle layer representation obtained via the trained LR network can be fed into the HR network to generate the SR result. Preliminary experiments show that SR-CBP can produce better results than state of the art single image superresolution methods based on sparse representations. The memory and storage requirements of CAN are lesser than existing deep learning based SR methods.

Despite the fact that different objects possess distinct class-specific features, they also usually share common patterns. This observation has been exploited partially in a recently proposed dictionary learning framework by separating the particularity and the commonality (COPAR). Inspired by this, we propose a novel method to explicitly and simultaneously learn a set of common patterns as well as class-specific features for classification with more intuitive constraints. Our dictionary learning framework is hence characterized by both a shared dictionary and particular (class-specific) dictionaries. For the shared dictionary, we enforce a low-rank constraint, i.e. claim that its spanning subspace should have low dimension and the coefficients corresponding to this dictionary should be similar. For the particular dictionaries, we impose on them the well-known constraints stated in the Fisher discrimination dictionary learning (FDDL). Further, we develop new fast and accurate algorithms to solve the subproblems in the learning step, accelerating its convergence. The said algorithms could also be applied to FDDL and its extensions. The efficiencies of these algorithms are theoretically and experimentally verified by comparing their complexities and running time with those of other well-known dictionary learning methods. Experimental results on widely used image datasets establish the advantages of our method over state-of-the-art dictionary learning methods., Accepted version