Search

Hydrocephalus is the leading indication for pediatric neurosurgical care worldwide. Identification of postinfectious hydrocephalus (PIH) verses non-postinfectious hydrocephalus (NPIH), as well as the pathogen involved in PIH is crucial for developing an appropriate treatment plan. Accurate identification requires clinical diagnosis by neuroscientists and microbiological analysis, which are time-consuming and expensive. In this study, we develop a domain enriched AI method for CT-based infection diagnosis in hydrocephalic imagery. State-of-the-art (SOTA) convolutional neural network (CNN) approaches form an attractive neural engineering solution for addressing this problem as pathogen-specific features need discovery. Yet black-box deep networks often need unrealistic abundant training data and are not easily interpreted. In this paper, a novel brain attention regularizer (BAR) is proposed, which encourages the CNN to put more focus inside brain regions in its feature extraction and decision making. Our approach is then extended to a hybrid 2D/3D network that mines inter-slice information. A new strategy of regularization is also designed for enabling collaboration between 2D and 3D branches. Our proposed method achieves SOTA results on a CURE Children's Hospital of Uganda dataset with an accuracy of 95.8% in hydrocephalus classification and 84% in pathogen classification. Statistical analysis is performed to demonstrate that our proposed methods obtain significant improvements over the existing SOTA alternatives. . Such attention regularized learning has particularly pronounced benefits in regimes where training data may be limited, thereby enhancing generalizability. To the best of our knowledge, our findings are unique among early efforts in interpretable AI-based models for classification of hydrocehpalus and underlying pathogen using CT scans.

Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentation. Open challenges remain in scenarios where labeled 3D segmentation maps for training are severely limited, as is often the case in practice, and in ensuring robustness to noise. Inspired by the observation that 3D vessel structures project onto 2D image slices with informative and unique edge profiles, we propose a novel deep 3D vessel segmentation network guided by edge profiles. Our network architecture comprises a shared encoder and two decoders that learn segmentation maps and edge profiles jointly. 3D context is mined in both the segmentation and edge prediction branches by employing bidirectional convolutional long-short term memory (BCLSTM) modules. 3D features from the two branches are concatenated to facilitate learning of the segmentation map. As a key contribution, we introduce new regularization terms that: a) capture the local homogeneity of 3D blood vessel volumes in the presence of biomarkers; and b) ensure performance robustness to domain-specific noise by suppressing false positive responses. Experiments on benchmark datasets with ground truth labels reveal that the proposed approach outperforms state-of-the-art techniques on standard measures such as DICE overlap and mean Intersection-over-Union. The performance gains of our method are even more pronounced when training is limited. Furthermore, the computational cost of our network inference is among the lowest compared with state-of-the-art.

OBJECTIVE Hydrocephalus in infants, particularly that with a postinfectious etiology, is a major public health burden in Sub-Saharan Africa. The authors of this study aimed to determine whether surgical treatment of infant postinfectious hydrocephalus in Uganda results in sustained, long-term brain growth and improved cognitive outcome. METHODS The authors performed a trial at a single center in Mbale, Uganda, involving infants (age < 180 days old) with postinfectious hydrocephalus randomized to endoscopic third ventriculostomy plus choroid plexus cauterization (ETV+CPC; n = 51) or ventriculoperitoneal shunt (VPS; n = 49). After 2 years, they assessed developmental outcome with the Bayley Scales of Infant Development, Third Edition (BSID-III), and brain volume (raw and normalized for age and sex) with CT scans. RESULTS Eighty-nine infants were assessed for 2-year outcome. There were no significant differences between the two surgical treatment arms in terms of BSID-III cognitive score (p = 0.17) or brain volume (p = 0.36), so they were analyzed together. Raw brain volumes increased between baseline and 2 years (p < 0.001), but this increase occurred almost exclusively in the 1st year (p < 0.001). The fraction of patients with a normal brain volume increased from 15.2% at baseline to 50.0% at 1 year but then declined to 17.8% at 2 years. Substantial normalized brain volume loss was seen in 21.3% patients between baseline and year 2 and in 76.7% between years 1 and 2. The extent of brain growth in the 1st year was not associated with the extent of brain volume changes in the 2nd year. There were significant positive correlations between 2-year brain volume and all BSID-III scores and BSID-III changes from baseline. CONCLUSIONS In Sub-Saharan Africa, even after successful surgical treatment of infant postinfectious hydrocephalus, early posttreatment brain growth stagnates in the 2nd year. While the reasons for this finding are unclear, it further emphasizes the importance of primary infection prevention and mitigation strategies along with optimizing the child’s environment to maximize brain growth potential.

A significant research problem of recent interest is the localization of targets like vessels, surgical needles, and tumors in photoacoustic (PA) images. To achieve accurate localization, a high photoacoustic signal-to-noise ratio (SNR) is required. However, this is not guaranteed for deep targets, as optical scattering causes an exponential decay in optical fluence with respect to tissue depth. To address this, we develop a novel deep learning method designed to explicitly exhibit robustness to noise present in photoacoustic radio-frequency (RF) data. More precisely, we describe and evaluate a deep neural network architecture consisting of a shared encoder and two parallel decoders. One decoder extracts the target coordinates from the input RF data while the other boosts the SNR and estimates clean RF data. The joint optimization of the shared encoder and dual decoders lends significant noise robustness to the features extracted by the encoder, which in turn enables the network to contain detailed information about deep targets that may be obscured by noise. Additional custom layers and newly proposed regularizers in the training loss function (designed based on observed RF data signal and noise behavior) serve to increase the SNR in the cleaned RF output and improve model performance. To account for depth-dependent strong optical scattering, our network was trained with simulated photoacoustic datasets of targets embedded at different depths inside tissue media of different scattering levels. The network trained on this novel dataset accurately locates targets in experimental PA data that is clinically relevant with respect to the localization of vessels, needles, or brachytherapy seeds. We verify the merits of the proposed architecture by outperforming the state of the art on both simulated and experimental datasets., Accepted by IEEE Transactions on Medical Imaging

Glioma, characterized by neoplastic growth in the brain, is a life-threatening condition that, in most cases, ultimately leads to death. Typical analysis of glioma development involves observation of brain tissue in the form of a histology slide under a microscope. Although brain histology images have much potential for predicting patient outcomes such as overall survival (OS), they are rarely used as the sole predictors due challenges presented by unique characteristics of brain tissue histology. However, utilizing histology in predicting overall survival can be useful for treatment and quality-of-life for patients with early-stage glioma. In this study, we investigate the use of deep learning models on histology slides combined with simple descriptor data (age and glioma subtype) as a predictor of (OS) in patients with low-grade glioma (LGG). Using novel clinical data, we show that models which are more attentive to discriminative features of the image will confer better predictions than generic models (82.7 and 65.3 AUC RFD-Net and Baseline VGG16 model, respectively). Additionally, we show that adding age and subtype information to a histology image-based model may provide greater robustness in the model than using the image alone (3.8 and 4.3 stds for RFD-Net and Baseline VGG16 model with 3-fold CV, respectively), while a model based on image and age but not subtype may confer the best predictive results (83.7 and 82.0 AUC for RFD-Net + age and RFD-Net + age + subtype, respectively). Clinical relevance- This study establishes important criteria for deep learning models which predict OS using histology and basic clinical data from LGG patients.

Deep convolutional neural networks (DCNNs) have recently emerged as a powerful tool to deliver breakthrough performances in various image analysis and processing applications. However, DCNNs lack a strong theoretical foundation and require massive amounts of training data. More recently, the deep scattering network (DSN), a variant of DCNNs, has been proposed to address these issues. DSNs inherit the hierarchical structure of DCNNs, but replace data-driven linear filters with predefined fixed multi-scale wavelet filters, which facilitate an in-depth understanding of DCNNs and also offer the state-of-the-art performance in image classification. Unfortunately, DSNs suffer from a major drawback: they are suitable for stationary image textures but not non-stationary image textures, since 2D wavelets are intrinsically linear translation-invariant filters in the Fourier transform domain. The objective of this paper is to overcome this drawback using the fractional wavelet transform (FRWT) which can be viewed as a bank of linear translation-variant multi-scale filters and thus may be well suited for non-stationary texture analysis. We first propose the fractional wavelet scattering transform (FRWST) based upon the FRWT. Then, we present a generalized structure for the DSN by cascading fractional wavelet convolutions and modulus operators. Basic properties of this generalized DSN are derived, followed by a fast implementation of the generalized DSN as well as their practical applications. The theoretical derivations are validated via computer simulations.

In this article, we propose a new algorithm that designs a transmit beampattern for multiple-input multiple-output (MIMO) radar considering coexistence with other wireless systems. This design process is conducted by minimizing the deviation of the generated beampattern (which in turn is a function of the transmit waveform) against an idealized one while enforcing the waveform elements to be constant modulus and in the presence of spectral restrictions. This leads to a hard nonconvex optimization problem primarily due to the presence of the constant modulus constraint (CMC). In this article, we exploit the geometrical structure of CMC, i.e., we redefine this constraint as an intersection of two sets (one convex and other nonconvex). This new perspective allows us to solve the nonconvex design problem via a tractable method called iterative beampattern with spectral design (IBS). In particular, the proposed IBS algorithm develops and solves a sequence of convex problems such that constant modulus is achieved at convergence. Crucially, we show that at convergence the obtained solution satisfies the Karush–Kuhn–Tucker conditions of the aforementioned nonconvex problem. Finally, we evaluate the proposed algorithm over challenging simulated scenarios, and show that it outperforms the state-of-the-art competing methods.

The low resolution of infrared (IR) images makes feature extraction for classification of a challenging work. Learning-based methods, therefore, are preferred to be used on such raw imagery. In this article, in order to avoid difficulties in feature extraction, a novel multitask extension of the widely used sparse-representation-classification (SRC) method is proposed in both single and multiview set-ups. That is, the test sample could be a single IR image or images from different views. In both single-view and multiview scenarios, we try to employ collaborative spike and slab priors. This is because the traditional sparsity-inducing measures such as the $l_{0}$ -row pseudonorm makes it hard to capture the sparse structure of the coefficient matrix when expanded in terms of a training dictionary, and the priors are proved to be able to capture fairly general sparse structures. Furthermore, a joint prior and sparse coefficient estimation method (JPCEM) is proposed for the first time in this article in order to alleviate the need to handpick prior parameters required before classification. Multiple experiments are conducted on a synthetic Comanche Forward Looking IR (FLIR) Automatic Target Recognition (ATR) database collected by Army Research Lab and a challenging mid-wave IR (MWIR) image ATR database made available by the U.S. Army Night Vision and Electronic Sensors Directorate. The final results substantiate the merits of the proposed JPCEM through comparisons with other state-of-the-art methods, including both the ones based on SRC and the ones constructed using deep learning frameworks.

The nineteen papers in this special section focus on domain enriched learning for medical imaging. In recent years, learning based methods have emerged to complement traditional model and feature based methods for a variety of medical imaging problems such as image formation, classification and segmentation, quality enhancement etc. In the case of deep neural networks, many solutions have achieved unprecedented performance gains and have defined a new state of the art. Despite the progress, compelling open challenges remain. One such key challenge is that many learning frameworks (notably deep learning) are purely data-driven approaches and their performance depends strongly on the quantity and quality of training image data available. When training is limited or noisy, the performance drops sharply. Deep neural networks based approaches additionally face the challenge of often not being straightforward to interpret. Fortunately, exciting recent progress has emerged in enriching learning frameworks with domain knowledge and signal structure. As a couple of representative examples: in image reconstruction problems, this may involve using statistical/structural image priors; for image segmentation, shape and anatomical knowledge (conveyed by an expert) may be leveraged, etc. This special issue brings together contributions that combine signal, image priors and other flavors of domain knowledge with machine learning methods for solving many diverse medical imaging problems.

We consider the problem of minimizing the disturbance power at the output of the matched filter in a single antenna cognitive radar set-up. The aforementioned disturbance power can be shown to be an expectation of the slow-time ambiguity function (STAF) of the transmitted waveform over range-Doppler bins of interest. The design problem is known to yield a nonconvex quartic function of the transmit radar waveform. This STAF shaping problem becomes even more challenging in the presence of practical constraints on the transmit waveform such as the constant modulus constraint (CMC). Most existing approaches address the aforementioned challenges by suitably modifying or relaxing the design cost function and/or the CMC. In a departure from such methods, we develop a solution that involves direct optimization over the nonconvex complex circle manifold, i.e., the CMC set. We derive a new update strategy [quartic-gradient-descent (QGD)] that computes an exact gradient of the quartic cost and invokes principles of optimization over manifolds toward an iterative procedure with guarantees of monotonic cost function decrease and convergence. Experimentally, QGD can outperform state-of-the-art approaches for shaping the ambiguity function under the CMC while being computationally less expensive.

Vessel segmentation of retinal images is a key diagnostic capability in ophthalmology. This problem faces several challenges including low contrast, variable vessel size and thickness, and presence of interfering pathology such as micro-aneurysms and hemorrhages. Early approaches addressing this problem employed hand-crafted filters to capture vessel structures, accompanied by morphological post-processing. More recently, deep learning techniques have been employed with significantly enhanced segmentation accuracy. We propose a novel domain enriched deep network that consists of two components: 1) a representation network that learns geometric features specific to retinal images, and 2) a custom designed computationally efficient residual task network that utilizes the features obtained from the representation layer to perform pixel-level segmentation. The representation and task networks are {\em jointly learned} for any given training set. To obtain physically meaningful and practically effective representation filters, we propose two new constraints that are inspired by expected prior structure on these filters: 1) orientation constraint that promotes geometric diversity of curvilinear features, and 2) a data adaptive noise regularizer that penalizes false positives. Multi-scale extensions are developed to enable accurate detection of thin vessels. Experiments performed on three challenging benchmark databases under a variety of training scenarios show that the proposed prior guided deep network outperforms state of the art alternatives as measured by common evaluation metrics, while being more economical in network size and inference time., Accepted to IEEE TIP

Blind image deblurring remains a topic of enduring interest. Learning based approaches, especially those that employ neural networks have emerged to complement traditional model based methods and in many cases achieve vastly enhanced performance. That said, neural network approaches are generally empirically designed and the underlying structures are difficult to interpret. In recent years, a promising technique called algorithm unrolling has been developed that has helped connect iterative algorithms such as those for sparse coding to neural network architectures. In this article, we propose a neural network architecture based on this idea. We first present an iterative algorithm that may be considered as a generalization of the traditional total-variation regularization method in the gradient domain. We then unroll the algorithm to construct a neural network for image deblurring which we refer to as Deep Unrolling for Blind Deblurring (DUBLID). Key algorithm parameters are learned with the help of training images. Our proposed deep network DUBLID achieves significant practical performance gains while enjoying interpretability and efficiency at the same time. Extensive experimental results show that DUBLID outperforms many state-of-the-art methods and in addition is computationally faster.

This paper deals with the joint design of Multiple-Input Multiple-Output (MIMO) radar transmit waveform and receive filter to enhance multiple targets detectability in the presence of signal-dependent (clutter) and independent disturbance. The worst-case Signal-to-Interference-Noise-Ratio (SINR) over multiple targets is explicitly maximized. To ensure hardware compatibility and the coexistence between MIMO radar and other wireless systems, constant modulus and spectral restrictions on the waveform are incorporated in our design. A max-min non-convex optimization problem emerges as a function of the transmit waveform, which we solve via a novel polynomial-time iterative procedure that involves solving a sequence of convex problems with constraints that evolve with every iteration. The overall algorithm follows an alternate optimization over the receive filter and transmit waveform. For the problem of waveform optimization (which is our central contribution), we provide analytical guarantees of monotonic cost function improvement with proof of convergence to a solution that satisfies the KarushKuhnTucker (KKT) conditions. We also develop extensions that address the well-known waveform similarity constraint. By simulating challenging practical scenarios, we evaluate the proposed algorithm against the state-of-the-art methods in terms of the achieved SINR value and the computational complexity. Overall, we show that our proposal outperforms state of the art competing methods while providing the most favorable performance-complexity balance.

Image relighting has emerged as a problem of significant research interest inspired by augmented reality applications. Physics-based traditional methods, as well as black box deep learning models, have been developed. The existing deep networks have exploited training to achieve a new state of the art; however, they may perform poorly when training is limited or does not represent problem phenomenology, such as the addition or removal of dense shadows. We propose a model which enriches neural networks with physical insight. More precisely, our method generates the relighted image with new illumination settings via two different strategies and subsequently fuses them using a weight map (w). In the first strategy, our model predicts the material reflectance parameters (albedo) and illumination/geometry parameters of the scene (shading) for the relit image (we refer to this strategy as intrinsic image decomposition (IID)). The second strategy is solely based on the black box approach, where the model optimizes its weights based on the ground-truth images and the loss terms in the training stage and generates the relit output directly (we refer to this strategy as direct). While our proposed method applies to both one-to-one and any-to-any relighting problems, for each case we introduce problem-specific components that enrich the model performance: 1) For one-to-one relighting we incorporate normal vectors of the surfaces in the scene to adjust gloss and shadows accordingly in the image. 2) For any-to-any relighting, we propose an additional multiscale block to the architecture to enhance feature extraction. Experimental results on the VIDIT 2020 and the VIDIT 2021 dataset (used in the NTIRE 2021 relighting challenge) reveals that our proposal can outperform many state-of-the-art methods in terms of well-known fidelity metrics and perceptual loss., Rank second in NTIRE 2021 One-to-one depth guided image relighting challenge, accepted by CVPRW 2021

This work reviews the results of the NTIRE 2021 Challenge on Non-Homogeneous Dehazing. The proposed techniques and their results have been evaluated on a novel dataset that extends the NH-Haze datset. It consists of additional 35 pairs of real haze free and nonhomogeneous hazy images recorded outdoor. The nonhomogeneous haze has been introduced in the outdoor scenes by using a a professional setup that imitates the real conditions of haze scenes. 327 participants registered in the challenge and 23 teams competed in the final testing phase. The proposed solutions gauge the state-of-the-art in image dehazing.

Image relighting is attracting increasing interest due to its various applications. From a research perspective, im-age relighting can be exploited to conduct both image normalization for domain adaptation, and also for data augmentation. It also has multiple direct uses for photo montage and aesthetic enhancement. In this paper, we review the NTIRE 2021 depth guided image relighting challenge.We rely on the VIDIT dataset for each of our two challenge tracks, including depth information. The first track is on one-to-one relighting where the goal is to transform the illumination setup of an input image (color temperature and light source position) to the target illumination setup. In the second track, the any-to-any relighting challenge, the objective is to transform the illumination settings of the in-put image to match those of another guide image, similar to style transfer. In both tracks, participants were given depth information about the captured scenes. We had nearly 250 registered participants, leading to 18 confirmed team sub-missions in the final competition stage. The competitions, methods, and final results are presented in this paper.

Using low-frequency (UHF to L-band) ultrawideband synthetic aperture radar (SAR) technology for detecting buried and obscured targets, e.g., bomb or mine, has been successfully demonstrated recently. Despite promising recent progress, a significant open challenge is to distinguish obscured targets from other (natural and manmade) clutter sources in the scene. The problem becomes exacerbated in the presence of noisy responses from rough ground surfaces. In this paper, we present three novel sparsity-driven techniques, which not only exploit the subtle features of raw captured data, but also take advantage of the polarization diversity and the aspect angle dependence information from multichannel SAR data. First, the traditional sparse representation-based classification is generalized to exploit shared information of classes and various sparsity structures of tensor coefficients for multichannel data. Corresponding tensor dictionary learning models are consequently proposed to enhance classification accuracy. Finally, a new tensor sparsity model is proposed to model responses from multiple consecutive looks of objects, which is a unique characteristic of the data set we consider. Extensive experimental results on a high-fidelity electromagnetic simulated data set and radar data collected from the U.S. Army Research Laboratory side-looking SAR demonstrate the advantages of proposed tensor sparsity models.

We address the problem of designing a transmit beampattern for multiple-input multiple-output (MIMO) radar considering coexistence with wireless communication systems. The designed beampattern is able to manage the transmit energy in spatial directions as well as in spectral frequency bands of interest by minimizing the deviation of the designed beampattern versus a desired one under a spectral constraint as well as the constant modulus constraint. While unconstrained beampattern design is straightforward, a key open challenge is jointly enforcing the spectral constraint in addition to the constant modulus constraint on the radar waveform. A new approach is proposed in this paper, which involves solving a sequence of constrained quadratic programs such that constant modulus is achieved at convergence. Furthermore, we show that each problem in the sequence has a closed form solution leading to analytical tractability. We evaluate the proposed beampattern with interference control (BIC) algorithm against the state-of-the-art MIMO beampattern design techniques and show that BIC achieves closeness to an idealized beampattern along with desired spectral shaping.

ImportancePost-infectious hydrocephalus in infants is a major public health burden in sub-Saharan Africa.ObjectiveTo determine long-term brain growth and cognitive outcome after surgical treatment of infant post-infectious hydrocephalus in Uganda.DesignProspective follow-up of a previously randomized cohort.SettingSingle center in Mbale, Uganda.ParticipantsInfants (InterventionsEndoscopic or shunt surgery.Main outcomesBayley Scales of Infant Development (BSID-3) and brain volume on computed tomography (raw and normalized for age and sex) at 2 years after treatment.ResultsEighty-nine infants were assessed for 2-year outcome. There were no significant differences between the two surgical treatment arms, so they were analyzed together. Raw brain volumes increased between baseline and 24 months (median change=361 cc, IQR=293 to 443, pConclusions and RelevanceIn sub-Saharan Africa, even after successful surgical treatment of infant post-infectious hydrocephalus, post-treatment brain growth stagnates in the second year. While the reasons for this are unclear, this emphasizes the importance of primary infection prevention strategies along with optimizing the child’s environment to maximize brain growth potential.Trial RegistrationClinicalTrials.gov number, NCT01936272KEY POINTSQuestionWhat is the brain growth and cognitive trajectory of infants treated for post-infectious hydrocephalus in Uganda?FindingsIn this prospective follow-up of a cohort of 89 infants, early normalization of brain volume after treatment was followed by brain growth stagnation in the second year, with many falling back into the sub-normal range. Poor brain growth was associated with poor cognitive outcome.MeaningSuccessful surgical treatment of hydrocephalus is not sufficient to allow for adequate brain growth and cognitive development. Interventions aimed at primary infection prevention and reducing comorbidities are needed to improve brain growth potential.

Deep learning has been recently shown to improve performance in the domain of synthetic aperture sonar (SAS) image classification over existing shallow learning solutions. Given the constant resolution with range of a SAS, it is no surprise that deep learning techniques perform so well; the image of the seafloor produced by a SAS system is almost photographic in quality. Despite the image quality benefits of SAS, there is still room for classification improvement particularly in reducing the number of false alarms. This work addresses this by tackling one facet of the classification pipeline: image enhancement. Specifically, we ask and address the following question: Can we train a deep neural network to simultaneously enhance and classify a SAS image? We will respond in the affirmative as we introduce a new deep learning architecture tackling the problem, Data Adaptive Enhancement and Classification Network (DA-ECNet). DA-ECNet is a deep learning architecture which combines image enhancement as part of the classification procedure eliminating the need for a fixed state-of-the-art despeckling algorithm or enhancement module. Additionally, we train both image enhancement and classification jointly resulting in data adaptive image enhancement. Experiments on a challenging real-world dataset reveal that the proposed DA-ECNet outperforms state of the art deep learning as well as traditional feature based methods for SAS image classification.

Image dehazing is one of the most challenging imaging inverse problems. Although deep learning methods produce compelling results, one of the most crucial practical challenge is that of non-homogeneous haze, which remains an open problem. To address this challenge, we propose 3 models that are inspired by ensemble techniques. First, we propose a DenseNet based single-encoder four-decoders structure denoted as EDN-3J, wherein among the four decoders, three of them output estimates of dehazed images (J 1 , J 2 , J 3 ) that are then weighted and combined via weight maps learned by the fourth decoder. In our second model called EDN-AT, the single-encoder four-decoders structure is maintained while three decoders are transformed to jointly estimate two physical inverse haze models that share a common transmission map t with two distinct ambient light maps (A 1 , A 2 ). The two inverse haze models are then weighted and combined for the final dehazed image. To endow two sub-models flexibility and to induce capability of modeling non-homogeneous haze, we apply attention masks to ambient lights. Both the weight maps and attention maps are generated from the fourth decoder. Finally, in contrast to the above two ensemble models, we propose an encoder-decoder-U-net structure called EDN-EDU, which is a sequential hierarchical ensemble of two different dehazing networks with different modeling capacities. Experiments performed on challenging benchmark image datasets of NTIRE'20 and NTIRE'19 demonstrate that the proposed models outperform many state-of-the-art methods and this fact is particularly demonstrated in the NTIRE-2020 contest where the EDN-AT model achieves the best result in the sense of the perceptual quality metric LPIPS.

The emergence of deep learning methods that complement traditional model-based methods has helped achieve a new state-of-the-art for image dehazing. Many recent methods design deep networks that either estimate the haze-free image (J) directly or estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t) followed by using the inverse of the haze model to estimate the dehazed image. However, both kinds of methods fail in dealing with non-homogeneous haze images where some parts of the image are covered with denser haze and the other parts with shallower haze. In this work, we develop a novel neural network architecture that can take benefits of the aforementioned two kinds of dehazed images simultaneously by estimating a new quantity - a spatially varying weight map (w). w can then be used to combine the directly estimated J and the results obtained by the inverse model. In our work, we utilize a shared DenseNet-based encoder, and four distinct DenseNet-based decoders that estimate J, A, t, and w jointly. A channel attention structure is added to facilitate the generation of distinct feature maps of different decoders. Furthermore, we propose a novel dilation inception module in the architecture to utilize the non-local features to make up the missing information during the learning process. Experiments performed on challenging benchmark datasets of NTIRE'20 and NTIRE'18 demonstrate that the proposed method -namely, AtJwD- can outperform many state-of-the-art alternatives in the sense of quality metrics such as SSIM, especially in recovering images under non-homogeneous haze.

ObjectThe study of brain size and growth has a long and contentious history, yet normal brain volume development has yet to be fully described. In particular, the normal brain growth and cerebrospinal fluid (CSF) accumulation relationship is critical to characterize because it is impacted in numerous conditions of early childhood where brain growth and fluid accumulation are affected such as infection, hemorrhage, hydrocephalus, and a broad range of congenital disorders. This study aims to describe normal brain volume growth, particularly in the setting of cerebrospinal fluid accumulation.MethodsWe analyzed 1067 magnetic resonance imaging (MRI) scans from 505 healthy pediatric subjects from birth to age 18 to quantify component and regional brain volumes. The volume trajectories were compared between the sexes and hemispheres using Smoothing Spline ANOVA. Population growth curves were developed using Generalized Additive Models for Location, Scale, and Shape.ResultsBrain volume peaked at 10-12 years of age. Males exhibited larger age-adjusted total brain volumes than females, and body size normalization procedures did not eliminate this difference. The ratio of brain to CSF volume, however, revealed a universal age-dependent relationship independent of sex or body size.ConclusionsThese findings enable the application of normative growth curves in managing a broad range of childhood disease where cognitive development, brain growth, and fluid accumulation are interrelated.

Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model-based methods and have helped define a new state of the art in achievable dehazed image quality. However, most deep learning-based methods usually design a regression network as a black-box tool to either estimate the dehazed image and/or the physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse haze model may then be used to estimate the dehazed image. In this work, we proposed a Depth-aware Dehazing using Reinforcement Learning system, denoted as DDRL. DDRL generates the dehazed image in a near-to-far progressive manner by utilizing the depth-information from the scene. This contrasts with the most recent learning-based methods that estimate these parameters in one pass. In particular, DDRL exploits the fact that the haze is less dense near the camera and gets increasingly denser as the scene moves farther away from the camera. DDRL consists of a policy network and a dehazing (regression) network. The policy network estimates the current depth for the dehazing network to use. A novel policy regularization term is introduced for the policy network to generate the policy sequence following the near-to-far order. Based on extensive tests over three benchmark test sets, DDRL demonstrates vastly enhanced dehazing results, particularly when training is limited.

High Dynamic Range (HDR) reconstruction is the process of producing an HDR image from a set of Standard Dynamic Range (SDR) images with different exposure times. This is a particularly challenging problem when relative camera or object motion exists between the available SDR images. Recently, deep learning methods, specifically those based on convolutional neural networks (CNNs) have been developed for HDR and shown to achieve unprecedented quality gains. Invariably an image alignment phase precedes the CNN mapping and merging. In practice, this alignment step greatly increases the computational burden of deep HDR methods often rendering them unsuitable for real-time composition. We propose a new deep HDR technique that does not need any explicit alignment of SDR images. Instead, a novel attention mask is developed that enables the network to focus on parts of the scene with considerable motion. Further, a dense merger is proposed that leads to an economical network. Evaluation over benchmark databases reveals that the proposed AttenDense network achieves high quality HDR results with significantly reduced computation time than state of the art. Further, the incorporation of domain knowledge (development of a custom attention mask) allows a more graceful decay in performance in the face of limited training.

We consider the problem of designing sequences with good auto- and cross-correlation properties for multiple-input multiple-output (MIMO) radar systems. This design problem aims to minimize a polynomial function of the transmit waveforms. The problem becomes more challenging in the presence of practical constraints on the waveform such as the constant modulus constraint (CMC). The aforementioned challenge has been addressed in the literature by approximating the cost function and/or constraints, i.e. the CMC. In this work, we develop a new algorithm that deals with the exact non-convex cost function and CMC. In particular, we develop a new update method (Correlation-Gradient-Descent (CGD)) that employs the exact gradient of the cost function to design such sequences with guarantees of monotonic cost function decrease and convergence. Our method further enables descent directly over the CMC by invoking principles of optimization over manifolds. Experimentally, CGD is evaluated against state-of-the-art methods for designing uni-modular sequences with good correlation properties. Results reveal that CGD can outperform these methods while being computationally less expensive.

In this paper, we propose a new algorithm that designs a transmit beampattern for Multiple-Input Multiple-Output (MIMO) radar considering coexistence with other wireless systems. This design process is conducted by minimizing the deviation of the generated beampattern against an idealized one while enforcing the waveform elements to be constant modulus and in the presence of spectral restrictions, which translates to a quadratic waveform constraint. This leads to a hard nonconvex optimization problem due to simultaneous presence of the constant modulus constraint (CMC) and spectral constraint (SpecC). In this work, we employ the geometrical structure of CMC, that is we redefine this constraint as an intersection of two sets. This definition allows us to handle the design problem in more tractable way. The proposed Iterative Beampattern with Spectral design (IBS) algorithm solves a sequence of convex problems such that constant modulus is achieved at convergence. Furthermore, we show that at convergence the obtained solution satisfies the Karush-Kuhn-Tucker (KKT) conditions of the aforementioned non-convex problem. Finally, we evaluate the proposed algorithm over challenging simulated scenarios, and show that it outperforms the state of the art competing methods.

Micro-computer tomography (μCT) is increasingly used in Anthropology and Palaeontology to quantify the external and internal osteological characteristics of extant/extinct species. One of the challenging tasks on such data is the accurate segmentation of images into bone and non-bone classes. Many intensity-based segmentation approaches have been proposed to overcome this issue, moving from global-thresholding to robust (semi)automatic methods. However, researchers often resort to laborious manual segmentation when the intensity levels of bone and non-bone material are extremely similar. Recently, deep learning methods have been shown to outperform traditional approaches for image segmentation. Here we propose a novel domain enriched deep network architecture that combines the benefits of deep learning with expert knowledge of bone structure via two components - 1) a representation network capable of extracting features that are more responsive to bone structures and less responsive to non-bone structures, and 2) a segmentation network that utilizes the features obtained from the representation network to perform segmentation. Effective representation filters are obtained through a robust discriminative-features constraint that enables the discovery of novel features and enhances segmentation accuracy. Experiments performed on challenging μCT images of archaeological bones reveal practical merits of our proposal over state-of-the-art alternatives.

This paper deals with the joint design of Multiple-Input Multiple-Output (MIMO) radar transmit waveform and receive filter to enhance multiple targets detectability in the presence of signal-dependent (clutter) and independent disturbance. The worst-case Signal-to-Interference-Noise-Ratio (SINR) over multiple targets is explicitly maximized. To ensure hardware compatibility and the coexistence between MIMO radar and other wireless systems, constant modulus and spectral restrictions on the waveform are incorporated in our design. A max-min non-convex optimization problem emerges as a function of the transmit waveform, which we solve via a novel polynomial-time iterative procedure that involves solving a sequence of convex problems with constraints that evolve with every iteration. We provide analytical guarantees of monotonic cost function improvement with proof of convergence to a solution that satisfies the KarushKuhnTucker (KKT) conditions. By simulating challenging practical scenarios, we evaluate the proposed algorithm against the state-of-the-art methods in terms of the achieved SINR value and the computational complexity.

Postinfectious hydrocephalus in infants is a major health problem in sub-Saharan Africa. The conventional treatment is ventriculoperitoneal shunting, but surgeons are usually not immediately available to revise shunts when they fail. Endoscopic third ventriculostomy with choroid plexus cauterization (ETV-CPC) is an alternative treatment that is less subject to late failure but is also less likely than shunting to result in a reduction in ventricular size that might facilitate better brain growth and cognitive outcomes.We conducted a randomized trial to evaluate cognitive outcomes after ETV-CPC versus ventriculoperitoneal shunting in Ugandan infants with postinfectious hydrocephalus. The primary outcome was the Bayley Scales of Infant Development, Third Edition (BSID-3), cognitive scaled score 12 months after surgery (scores range from 1 to 19, with higher scores indicating better performance). The secondary outcomes were BSID-3 motor and language scores, treatment failure (defined as treatment-related death or the need for repeat surgery), and brain volume measured on computed tomography.A total of 100 infants were enrolled; 51 were randomly assigned to undergo ETV-CPC, and 49 were assigned to undergo ventriculoperitoneal shunting. The median BSID-3 cognitive scores at 12 months did not differ significantly between the treatment groups (a score of 4 for ETV-CPC and 2 for ventriculoperitoneal shunting; Hodges-Lehmann estimated difference, 0; 95% confidence interval [CI], -2 to 0; P=0.35). There was no significant difference between the ETV-CPC group and the ventriculoperitoneal-shunt group in BSID-3 motor or language scores, rates of treatment failure (35% and 24%, respectively; hazard ratio, 0.7; 95% CI, 0.3 to 1.5; P=0.24), or brain volume (z score, -2.4 and -2.1, respectively; estimated difference, 0.3; 95% CI, -0.3 to 1.0; P=0.12).This single-center study involving Ugandan infants with postinfectious hydrocephalus showed no significant difference between endoscopic ETV-CPC and ventriculoperitoneal shunting with regard to cognitive outcomes at 12 months. (Funded by the National Institutes of Health; ClinicalTrials.gov number, NCT01936272 .).

Multiple-input multiple-output (MIMO) radar systems allow each antenna element to transmit a different waveform. This waveform diversity can be exploited to enhance the beampattern design, in particular, effective management of radar radiation power in directions of interest. We address the problem of designing a beampattern for MIMO radar, which in turn is determined by the transmit waveform. While unconstrained design is straightforward, a key open challenge is enforcing the constant modulus constraint on the radar waveform. It is well known that the problem of minimizing deviation of the designed beampattern from an idealized one subject to the constant modulus constraint constitutes a hard nonconvex problem. Existing methods that address constant modulus invariably lead to a stiff tradeoff between analytical tractability (achieved by relaxations and approximations) and realistic design that exactly achieves constant modulus but is computationally burdensome. A new approach is proposed in our paper, which involves solving a sequence of convex equality constrained quadratic programs, each of which has a closed form solution and such that constant modulus is achieved at convergence. We further prove that the converged solution satisfies the Karush–Kuhn–Tucker optimality conditions of the aforementioned hard nonconvex problem. We evaluate the proposed successive closed forms (SCF) algorithm against the state-of-the art MIMO beampattern design techniques in both narrowband and wideband setups and show that the SCF breaks the tradeoff between desirable performance and the associated computation cost.

High dynamic range (HDR) image synthesis from multiple low dynamic range (LDR) exposures continues to be actively researched. The extension to HDR video synthesis is a topic of significant current interest due to potential cost benefits. For HDR video, a stiff practical challenge presents itself in the form of accurate correspondence estimation of objects between video frames. In particular, loss of data resulting from poor exposures and varying intensity make conventional optical flow methods highly inaccurate. We avoid exact correspondence estimation by proposing a statistical approach via maximum a posterior (MAP) estimation, and under appropriate statistical assumptions and choice of priors and models, we reduce it to an optimization problem of solving for the foreground and background of the target frame. We obtain the background through rank minimization and estimate the foreground via a novel multiscale adaptive kernel regression technique, which implicitly captures local structure and temporal motion by solving an unconstrained optimization problem. Extensive experimental results on both real and synthetic datasets demonstrate that our algorithm is more capable of delivering high-quality HDR videos than current state-of-the-art methods, under both subjective and objective assessments. Furthermore, a thorough complexity analysis reveals that our algorithm achieves better complexity-performance trade-off than conventional methods.

Recent covariance estimation methods for radar space-time adaptive processing exploit practical constraints such as the rank of clutter subspace and the condition number of disturbance covariance to estimate accurate covariance even when training is not generous. While rank and condition number are very effective constraints, often practical nonidealities make it difficult to know them precisely using physical models. Therefore, we propose a method to determine constraints in covariance estimation for radar space-time adaptive processing via an expected likelihood approach. We analyze three cases of constraints: 1) a rank constraint, 2) both rank and noise power constraints, and 3) a condition number constraint. In each case, we formulate precise constraint determination as an optimization problem. For each of the three cases, we derive new analytical results which allow for computationally efficient, practical ways of determining these constraints with formal proofs. Through experimental results from a simulation model and the KASSPER data set, we show that the estimator with optimal constraints obtained by the expected likelihood approach outperforms state-of-the-art alternatives.

We propose an optimization-based power-constrained red-green–blue (RGB)-to-red-green–blue-white (RGBW) conversion algorithm for emissive RGBW displays. We measure the perceived color distortion using a color difference model in a perceptually uniform color space, and compute the power consumption for displaying an RGBW pixel on an emissive display. The central contribution of this paper is to formulate the optimization problem to minimize the color distortion subject to a constraint on the power consumption. Subsequently, we solve the optimization problem efficiently to convert an image in real time. Furthermore, based on the properties of the human visual system, we extend the proposed algorithm to image-dependent conversion that can preserve spatial detail in an input image. The simulation results show that the proposed algorithm provides a significantly less color distortion than the conventional methods, while providing a graceful tradeoff with the amount of power consumed. Specifically, it is shown that the power consumption can be reduced by up to 20%, while providing about 50% less color distortion than the conventional algorithms. In addition, a subjective evaluation on a real RGBW display is performed, which reveals the merits of the proposed image-dependent conversion for improving the perceptual quality over state-of-the-art techniques.

Vessel segmentation of retinal images is a key diagnostic capability in ophthalmology. Early approaches addressing this problem employed hand-crafted filters to capture vessel structures, accompanied by morphological processing. More recently, deep learning techniques have been employed to significantly enhance segmentation accuracy. We propose a novel domain enriched deep network that consists of two components: 1) a representation network which learns geometric (specifically curvilinear) features that are tailored to retinal images, followed by 2) a task network that utilizes the features obtained from the representation layer to perform pixel-level segmentation. The representation and task networks are learned jointly for any given training set. To obtain effective representation filters, we develop a new orientation constraint that enables geometric diversity of curvilinear features. A multi-scale extension is further developed to enhance segmentation of thin vessels. Experiments performed on two challenging benchmark databases reveal that the proposed regularized deep network can outperform state of the art alternatives as measured by common evaluation metrics. Further, the proposed method exhibits a more graceful decay in performance as training data is reduced.

Objective: The diversity of tissue structure in histopathological images makes feature extraction for classification a challenging task. Dictionary learning within a sparse representation-based classification (SRC) framework has been shown to be successful for feature discovery. However, there exist stiff practical challenges: 1) computational complexity of SRC can be onerous in the decision stage since it involves solving a sparsity constrained optimization problem and often over a large number of image patches; and 2) images from distinct classes continue to share several geometric features. We propose a novel analysis–synthesis model learning with shared features algorithm (ALSF) for classifying such images more effectively. Methods: In the ALSF, a joint analysis and synthesis learning model is introduced to learn the classifier and the feature extractor at the same time. Unlike SRC, no explicit optimization is needed in the inference phase leading to much reduced computation. Crucially, we introduce the learning of a low-rank shared dictionary and a shared analysis operator, which more accurately represents both similarities and differences in histopathological images from distinct classes. We also develop an extension of ALSF with a sparsity constraint, whose presence or absence facilitates a cost–performance tradeoff. Results: The ALSF is evaluated on three challenging and well-known datasets: 1) spleen tissue images; 2) brain tumor images; and 3) breast cancer tissue dataset, provided by different organizations. Conclusion: Experimental results demonstrate both complexity and performance benefits of the ALSF over state-of-the-art alternatives. Significance: Modeling shared features with appropriate quantitative constraints lead to significantly improved classification in histopathology.

This paper reviews the first NTIRE challenge on video deblurring (restoration of rich details and high frequency components from blurred video frames) with focus on the proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed dynamic motion blurs while Track 2 had additional MPEG video compression artifacts. Each competition had 109 and 93 registered participants. Total 13 teams competed in the final testing phase. They gauge the state-of-the-art in video deblurring problem.

Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model-based methods and have helped define a new state of the art in achievable dehazed image quality. Yet, practical challenges remain in dehazing of real-world images where the scene is heavily covered with dense haze, even to the extent that no scene information can be observed visually. Many recent dehazing methods have addressed this challenge by designing deep networks that estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse of the haze model may then be used to estimate the dehazed image. In this work, we develop two novel network architectures to further this line of investigation. Our first model, denoted as At-DH, designs a shared DenseNet based encoder and two distinct DensetNet based decoders to jointly estimate the scene information viz. A and t respectively. This in contrast to most recent efforts (include those published in CVPR'18) that estimate these physical parameters separately. As a natural extension of At-DH, we develop the AtJ-DH network, which adds one more DenseNet based decoder to jointly recreate the haze-free image along with A and t. The knowledge of (ground truth) training dehazed/clean images can be exploited by a custom regularization term that further enhances the estimates of model parameters A and t in AtJ-DH. Experiments performed on challenging benchmark image datasets of NTIRE'19 and NTIRE'18 demonstrate that At-DH and AtJ-DH can outperform state-of-the-art alternatives, especially when recovering images corrupted by dense haze.

Single image dehazing has gained much attention recently. A typical learning based approach uses example hazy and clean image pairs to train a mapping between the two. Of the learning based methods, those based on deep neural networks have shown to deliver state of the art performance. An important aspect of recovered image quality is the color information, which is severely compromised when the image is corrupted by very dense haze. While many different network architectures have been developed for recovering dehazed images, an explicit attention to recovering individual color channels with a design that ensures their quality has been missing. Our proposed work, focuses on this issue by developing a novel network structure that comprises of: a common DenseNet based feature encoder whose output branches into three distinct DensetNet based decoders to yield estimates of the R, G and B color channels of the image. A subsequent refinement block further enhances the final synthesized RGB/color image by joint processing of these color channels. Inspired by its structure, we call our approach the One-To-Three Color Enhancement Dehazing (123-CEDH) network. To ensure the recovery of physically meaningful and high quality color channels, the main network loss function is further regularized by a multi-scale structural similarity index term as well as a term that enhances color contrast. Experiments reveal that 123-CEDH has the ability to recover color information at early training stages (i.e. in the first few epochs) vs. other highly competitive methods. Validation on the benchmark datasets of the NTIRE'19 and NTIRE'18 dehazing challenges reveals the 123-CEDH to be one of the Top-3 methods based on results released in the NTIRE'19 competition.

Controlling the range-Doppler response, i.e. Ambiguity Function (AF) continues to be of great interest in cognitive radar. The design problem is known to be a nonconvex quartic function of the transmit radar waveform. This AF shaping problem becomes even more challenging in the presence of practical constraints on the transmit waveform such as the Constant Modulus Constraint (CMC). Most existing approaches address the aforementioned challenges by suitably modifying or relaxing the design cost function and/or CMC. In a departure from such methods, we develop a solution that involves direct optimization over the non-convex complex circle manifold, i.e. the CMC set. We derive a new update strategy (Quartic-Gradient-Descent (QGD)) that computes an exact gradient of the quartic cost and invokes principles of optimization over manifolds towards an iterative procedure with guarantees of monotonic cost function decrease and convergence. Experimentally, QGD can outperform state of the art approaches for shaping the ambiguity function under CMC while being computationally less expensive.

Outline ❶Part I: Review of Radar STAP and Convex Optimization Principles ❷Part II: Constrained Covariance Estimation ❸Part III: Optimization for Practical Waveform Design