Search

Junction plays an important role in biomedical research such as retinal biometric identification, retinal image registration, eye-related disease diagnosis and neuron reconstruction. However, junction detection in original biomedical images is extremely challenging. For example, retinal images contain many tiny blood vessels with complicated structures and low contrast, which makes it challenging to detect junctions. In this paper, we propose an O-shape Network architecture with Attention modules (Attention O-Net), which includes Junction Detection Branch (JDB) and Local Enhancement Branch (LEB) to detect junctions in biomedical images without segmentation. In JDB, the heatmap indicating the probabilities of junctions is estimated and followed by choosing the positions with the local highest value as the junctions, whereas it is challenging to detect junctions when the images contain weak filament signals. Therefore, LEB is constructed to enhance the thin branch foreground and make the network pay more attention to the regions with low contrast, which is helpful to alleviate the imbalance of the foreground between thin and thick branches and to detect the junctions of the thin branch. Furthermore, attention modules are utilized to introduce the feature maps of LEB to JDB, which can establish a complementary relationship and further integrate local features and contextual information between these two branches. The proposed method achieves the highest average F1-scores of 0.82, 0.73 and 0.94 in two retinal datasets and one neuron dataset, respectively. The experimental results confirm that Attention O-Net outperforms other state-of-the-art detection methods, and is helpful for retinal biometric identification.

Hyperspectral anomaly detection, which is aimed at locating anomaly, has received widespread attention. In this article, a new anomaly detector, named local spatial constraint and total variation (LSC-TV), is proposed for hyperspectral imagery. In anomaly detection methods based on low-rank representation, background pixels are usually considered to have a global low-dimensional structure. However, the complex background distribution in hyperspectral images (HSIs) means that this global low-dimensional structure rarely occurs. In LSC-TV, the effective local spatial information is extracted by superpixel segmentation, and the regularization based on the F-norm is used to force the background within the same superpixel to show uniform spectral features. Moreover, each pixel is given a penalty based on the degree of anomaly determined during model iteration, while the anomaly is not considered by the background constraint. In addition, the background pixels in the neighborhood often show a high correlation, whereas the anomaly does not possess this feature. Nonisotropic TV is introduced into the proposed LSC model using the correlation of first-order neighborhoods to make it easier for anomalies to be separated. The proposed LSC-TV method and current state-of-the-art methods are tested on a set of simulated data and four sets of real data. The experimental results demonstrate that the proposed method is superior to the comparative method in terms of both color map detection and quantitative evaluation.

Scene recovery is a fundamental imaging task with several practical applications, including video surveillance and autonomous vehicles, etc. In this paper, we provide a new real-time scene recovery framework to restore degraded images under different weather/imaging conditions, such as underwater, sand dust and haze. A degraded image can actually be seen as a superimposition of a clear image with the same color imaging environment (underwater, sand or haze, etc.). Mathematically, we can introduce a rank-one matrix to characterize this phenomenon, i.e., rank-one prior (ROP). Using the prior, a direct method with the complexity O(N) is derived for real-time recovery. For general cases, we develop ROP

Boron-based hypergolic ionic liquids (HILs) have gained increasing attention in the field of propellants due to the low toxicity, high energy density, and short ignition delay time. However, the performance of propellants based on boron-based HILs is still inferior to hydrazine derivatives, restricting their widespread applications as a rocket propellant. To boost the propellant performance of boron-based HILs, constantly engineering their chemical structures is highly necessary. The conventional approaches modify the anions and cations of the ionic liquids based on experiences and heuristics, and then experimentally verify the physicochemical properties of the synthesized compounds. However, such a trial-and-error design cycle is biased by practitioners’ preferences and the variations of experimental conditions. Meanwhile, it is expensive and tedious to measure all relevant properties of the ionic liquids by experiments. To solve these problems, we propose a protocol of combining machine learning and density functional theory (DFT) calculations to predict boron-based HILs’ properties such as ignition delay time, thermal stability, and energy storage capacity. With well-curated data, the experimental results show that the machine learning approaches can satisfactorily predict the chemical properties of boron-based HILs. We also use the Shapley value to find that (1) the thermal correction to enthalpy is critical for ignition delay, (2) HOMO, electronic energy and entropy values have a major impact on decomposition temperature, and (3) electronic energy also has a significant impact on the specific impulse.

Image denoising is a challenging inverse problem due to complex scenes and information loss. Recently, various methods have been considered to solve this problem by building a well-designed convolutional neural network (CNN) or introducing some hand-designed image priors. Different from previous works, we investigate a new framework for image denoising, which integrates edge detection, edge guidance, and image denoising into an end-to-end CNN model. To achieve this goal, we propose a multilevel edge features guided network (MLEFGN). First, we build an edge reconstruction network (Edge-Net) to directly predict clear edges from the noisy image. Then, the Edge-Net is embedded as part of the model to provide edge priors, and a dual-path network is applied to extract the image and edge features, respectively. Finally, we introduce a multilevel edge features guidance mechanism for image denoising. To the best of our knowledge, the Edge-Net is the first CNN model specially designed to reconstruct image edges from the noisy image, which shows good accuracy and robustness on natural images. Extensive experiments clearly illustrate that our MLEFGN achieves favorable performance against other methods and plenty of ablation studies demonstrate the effectiveness of our proposed Edge-Net and MLEFGN. The code is available at https://github.com/MIVRC/MLEFGN-PyTorch .

Image demosaicking and denoising are the first two key steps of the color image production pipeline. The classical processing sequence has for a long time consisted of applying denoising first, and then demosaicking. Applying the operations in this order leads to oversmoothing and checkerboard effects. Yet, it was difficult to change this order, because once the image is demosaicked, the statistical properties of the noise are dramatically changed and hard to handle by traditional denoising models. In this paper, we address this problem by a hybrid machine learning method. We invert the traditional color filter array (CFA) processing pipeline by first demosaicking and then denoising. Our demosaicking algorithm, trained on noiseless images, combines a traditional method and a residual convolutional neural network (CNN). This first stage retains all known information, which is the key point to obtain faithful final results. The noisy demosaicked image is then passed through a second CNN restoring a noiseless full-color image. This pipeline order completely avoids checkerboard effects and restores fine image detail. Although CNNs can be trained to solve jointly demosaicking-denoising end-to-end, we find that this two-stage training performs better and is less prone to failure. It is shown experimentally to improve on the state of the art, both quantitatively and in terms of visual quality., 28 pages, 40 figures

Blind image deblurring is a conundrum because there are infinitely many pairs of latent image and blur kernel. To get a stable and reasonable deblurred image, proper prior knowledge of the latent image and the blur kernel is urgently required. Different from the recent works on the statistical observations of the difference between the blurred image and the clean one, our method is built on the surface-aware strategy arising from the intrinsic geometrical consideration. This approach facilitates the blur kernel estimation due to the preserved sharp edges in the intermediate latent image. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods on deblurring the text and natural images. Moreover, our method can achieve attractive results in some challenging cases, such as low-illumination images with large saturated regions and impulse noise. A direct extension of our method to the non-uniform deblurring problem also validates the effectiveness of the surface-aware prior.

We present a probabilistic semi-supervised learning (SSL) framework based on sparse graph structure learning. Different from existing SSL methods with either a predefined weighted graph heuristically constructed from the input data or a learned graph based on the locally linear embedding assumption, the proposed SSL model is capable of learning a sparse weighted graph from the unlabeled high-dimensional data and a small amount of labeled data, as well as dealing with the noise of the input data. Our representation of the weighted graph is indirectly derived from a unified model of density estimation and pairwise distance preservation in terms of various distance measurements, where latent embeddings are assumed to be random variables following an unknown density function to be learned, and pairwise distances are then calculated as the expectations over the density for the model robustness to the data noise. Moreover, the labeled data based on the same distance representations are leveraged to guide the estimated density for better class separation and sparse graph structure learning. A simple inference approach for the embeddings of unlabeled data based on point estimation and kernel representation is presented. Extensive experiments on various data sets show promising results in the setting of SSL compared with many existing methods and significant improvements on small amounts of labeled data.

Morphology reconstruction of neurons from 3-D microscopic images is essential to neuroscience research. However, many reconstructions may contain errors and ambiguities because of the crossover neuronal fibers. In this article, an automatic algorithm is proposed for the detection and separation of crossover structures and is applied to neuron tracing for improving the neuron reconstruction results. First, a spherical-patches extraction (SPE)-Net is employed to detect the 3-D neuron crossover points and locate the crossover structures in neuron volumetric images. Second, a multiscale upgraded ray-shooting model (MSURS) is proposed to obtain robust results at different scales with high confidence and is employed to extract the crossover neuronal structure features. Then, a crossover structure separation (CSS) method is developed to eliminate the false connections of crossover structures and generate deformed separated neuronal fibers based on the extracted features to replace the original neurites signals. Experiments demonstrate that the SPE-Net for crossover point detection achieves average precision and recall rates of 73.89% and 79.66%, respectively, and demonstrate the proposed CSS method can improve 20.46% the performance of the reconstructions on average. The results confirm that the proposed method can effectively improve the neuron tracing results in volumetric images.

Imbalanced data classification, as a challenging task, has drawn a significant interest in numerous scientific areas. One popular strategy to balance the instance quantities between two classes is oversampling via generating synthetic instances. However, it still suffers from two key issues: where and how many synthetic instances should be generated. In this paper, we propose a Local distribution-based Adaptive Minority Oversampling method (LAMO) to deal with the imbalance classification problem. LAMO first identifies the informative borderline minority instances as sampling seeds according to their neighbors and the corresponding class distribution. Then, LAMO captures the local distribution of each seed according to its Euclidean distances from the nearest majority instance and nearest minority instance.Finally, LAMO generates synthetic instances around seeds via a Gaussian Mixture Model (GMM). For each component of GMM, the mixing coefficient and bandwidth are adaptively set with the aid of seeds’ local distribution. Extensive experiments have been conducted on both simulated and real data sets under varying the imbalance ratio and data size. By comparing with the state-of-the-art oversampling methods, the proposed LAMO obtains promising results in terms of several widely used evaluation metrics.

Convolutional neural networks (CNNs) have been applied to many image processing tasks and achieve great successes. In order to extract common features, every pixel in an image shares the same filters. However, pixels in different regions of an image varies dramatically and shared filters may lose some important local information. Rather than shared filters, smart filters which can be adapted to image context should be designed to better remove noise which occurs randomly in noisy image. Meanwhile, current CNN architectures compute the loss of each color channel independently, regardless of the potential color information. In this letter, we proposed a pixel-attention convolutional neural network (PACNN) with color correlation loss for the color image denoising task. The pixel-attention mechanism could generate pixel-wise attention maps which help remove random noise. The color correlation loss exploits color correlation to further improve denoising performance on color noisy images. The experimental results on several standard datasets demonstrate the state-of-the-art (SOTA) performance and the superiority of the proposed method.

The digital reconstruction of neurons is essential to various neuroscientific studies. Due to the existence of gaps and ambiguities in neuron images, the neuron tracing results generated by most automatic reconstruction algorithms may be incomplete, resulting in false negatives (FNs), which need to be repaired in proof editing. However, the automatic proof-editing methods for repairing FN branches have rarely been explored. In this study, we propose a proof-editing algorithm for automatically detecting and repairing the FN branches of the initial reconstruction, which is based on a multiscale upgraded ray (MUR)-shooting model and an MOST-based repairer. The MUR detects the FN branch and the corresponding branch direction vector by analyzing the multiscale intensity distribution features around a topological feature point. The topological feature points contain the junction points detected from the neuron image and the tip nodes extracted from the initial reconstruction. The MOST-based repairer is proposed to prevent the redundant reconstructions by assigning the detected branch direction vector as the initial tracing direction, which rejects the nodes returning to the traced area. The experimental results demonstrate clearly that the proposed method can reduce 20% of the false-negative rate at most. The experimental results confirm that the proposed method is extremely helpful for generating faithful reconstructions.

Training deep neural networks can be difficult. For classical neural networks, the initialization method by Xavier and Yoshua which is later generalized by He, Zhang, Ren and Sun can facilitate stable training. However, with the recent development of new layer types, we find that the above mentioned initialization methods may fail to lead to successful training. Based on these two methods, we will propose a new initialization by studying the parameter space of a network. Our principal is to put constrains on the growth of parameters in different layers in a consistent way. In order to do so, we introduce a norm to the parameter space and use this norm to measure the growth of parameters. Our new method is suitable for a wide range of layer types, especially for layers with parameter-sharing weight matrices.

Orientation estimation is an important task in three-dimensional cryo-EM image reconstruction. By applying the common line method, the orientation estimation task can be formulated as a least squares (LS) problem or a least un-squared deviation (LUD) problem with orthogonality constraint. However, the non-convexity of the orthogonality constraint introduces numerical difficulties to the orientation estimation. The conventional approach is to reformulate the orthogonality constrained minimization problem into a semi-definite programming problem using convex relaxation strategies. In this paper, we consider a direct way to solve the constrained minimization problem without relaxation. We focus on the weighted LS problem because the LUD problem can be reformulated into a sequence of weighted LS problems using the iteratively re-weighted LS approach. As a classical approach, the projected gradient descent (PGD) method has been successfully applied to solve the convex constrained minimization problem. We apply the PGD method to the minimization problem with orthogonality constraint and show that the constraint set is a generalized prox-regular set, and it satisfies the norm compatibility condition. We also demonstrate that the objective function of the minimization problem satisfies the restricted strong convexity and the restricted strong smoothness over a constraint set. Therefore, the sequence generated by the PGD method converges when the initial conditions are satisfied. Experimental results show that the PGD method significantly outperforms the semi-definite relaxation methods from a computation standpoint, and the mean square error is almost the same or smaller.

The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this challenge, we propose a hierarchical perception adversarial learning framework (HP-ALF). HP-ALF can perceive the image information in the hierarchical mechanism: image-level perception and patch-level perception. The former can reduce the visual perception difference in the entire image, and thus achieve aliasing artifact removal. The latter can reduce this difference in the regions of the image, and thus recover fine details. Specifically, HP-ALF achieves the hierarchical mechanism by utilizing multilevel perspective discrimination. This discrimination can provide the information from two perspectives (overall and regional) for adversarial learning. It also utilizes a global and local coherent discriminator to provide structure information to the generator during training. In addition, HP-ALF contains a context-aware learning block to effectively exploit the slice information between individual images for better reconstruction performance. The experiments validated on three datasets demonstrate the effectiveness of HP-ALF and its superiority to the comparative methods., Comment: 15 pages, 13 figures, IEEE TMI

In this paper, we investigate the challenging task of removing haze from a single natural image. The analysis on the haze formation model shows that the atmospheric veil has much less relevance to chrominance than luminance, which motivates us to neglect the haze in the chrominance channel and concentrate on the luminance channel in the dehazing process. Besides, the experimental study illustrates that the YUV color space is most suitable for image dehazing. Accordingly, a variational model is proposed in the Y channel of the YUV color space by combining the reformulation of the haze model and the two effective priors. As we mainly focus on the Y channel, most of the chrominance information of the image is preserved after dehazing. The numerical procedure based on the alternating direction method of multipliers (ADMM) scheme is presented to obtain the optimal solution. Extensive experimental results on real-world hazy images and synthetic dataset demonstrate clearly that our method can unveil the details and recover vivid color information, which is competitive among many existing dehazing algorithms. Further experiments show that our model also can be applied for image enhancement.

Semantic segmentation has achieved great progress by effectively fusing features of contextual information. In this article, we propose an end-to-end semantic attention boosting (SAB) framework to adaptively fuse the contextual information iteratively across layers with semantic regularization. Specifically, we first propose a pixelwise semantic attention (SAP) block, with a semantic metric representing the pixelwise category relationship, to aggregate the nonlocal contextual information. In addition, we improve the computation complexity of SAP block from O(n²) to O(n) for images with size n. Second, we present a categorywise semantic attention (SAC) block to adaptively balance the nonlocal contextual dependencies and the local consistency with a categorywise weight, overcoming the contextual information confusion caused by the feature imbalance within intra-category. Furthermore, we propose the SAB module to refine the segmentation with SAC and SAP blocks. By applying the SAB module iteratively across layers, our model shrinks the semantic gap and enhances the structure reasoning by fully utilizing the coarse segmentation information. Extensive quantitative evaluations demonstrate that our method significantly improves the segmentation results and achieves superior performance on the PASCAL VOC 2012, Cityscapes, PASCAL Context, and ADE20K datasets.

Gaussian process regression (GPR) has been a well-known machine learning method for various applications such as uncertainty quantifications (UQ). However, GPR is inherently a data-driven method, which requires sufficiently large dataset. If appropriate physics constraints (e.g. expressed in partial differential equations) can be incorporated, the amount of data can be greatly reduced and the accuracy further improved. In this work, we propose a hybrid data driven-physics constrained Gaussian process regression framework. We encode the physics knowledge with Boltzmann-Gibbs distribution and derive our model through maximum likelihood (ML) approach. We apply deep kernel learning method. The proposed model learns from both data and physics constraints through the training of a deep neural network, which serves as part of the covariance function in GPR. The proposed model achieves good results in high-dimensional problem, and correctly propagate the uncertainty, with very limited labelled data provided., Comment: 16 pages, 10 figures

Recently, deep convolution neural networks (CNNs) steered face super-resolution methods have achieved great progress in restoring degraded facial details by jointly training with facial priors. However, these methods have some obvious limitations. On the one hand, multi-task joint learning requires additional marking on the dataset, and the introduced prior network will significantly increase the computational cost of the model. On the other hand, the limited receptive field of CNN will reduce the fidelity and naturalness of the reconstructed facial images, resulting in suboptimal reconstructed images. In this work, we propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks, which uses the multi-scale connected encoder-decoder architecture as the backbone. Specifically, we first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block, to promote the consistency of local facial detail and global facial structure restoration simultaneously. Then, we design an efficient Feature Refinement Module (FRM) to enhance the encoded features. Finally, to further improve the restoration of fine facial details, we present a Multi-scale Feature Fusion Unit (MFFU) to adaptively fuse the features from different stages in the encoder procedure. Extensive evaluations on various datasets have assessed that the proposed CTCNet can outperform other state-of-the-art methods significantly. Source code will be available at https://github.com/IVIPLab/CTCNet., Comment: IEEE Transactions on Image Processing, 12 figures, 9 tables

An ongoing outbreak of atypical pneumonia caused by the 2019 novel coronavirus (SARS-CoV-2) is hitting Wuhan City and has spread to other provinces/cities of China and overseas. It very urgent to forecast the future course of the outbreak. Here, we provide an estimate of the potential total number of confirmed cases in mainland China by applying Boltzmann-function based regression analyses. We found that the cumulative number of confirmed cases from Jan 21 to Feb 14, 2020 for mainland China, Hubei Province, Wuhan City and other provinces were all well fitted with the Boltzmann function (R2 being close to 0.999). The potential total number of confirmed cases in the above geographic regions were estimated at 95% confidence interval (CI) as 79589 (71576, 93855), 64817 (58223, 77895), 46562 (40812, 57678) and 13956 (12748, 16092), respectively. Notably, our results suggest that the number of daily new confirmed cases of SARS-CoV-2 in mainland China (including Hubei Province) will become minimal between Feb 28 and Mar 10, 2020, with 95% CI. In addition, we found that the data of cumulative confirmed cases of 2003 SARS-CoV in China and Worldwide were also well fitted to the Boltzmann function. To our knowledge this is the first study revealing that the Boltzmann function is suitable to simulate epidemics. The estimated potential total number of confirmed cases and key dates for the SARS-CoV-2 outbreak may provide certain guidance for governments, organizations and citizens to optimize preparedness and response efforts.

The task of single image super-resolution (SISR) is a highly ill-posed inverse problem since reconstructing the high-frequency details from a low-resolution image is challenging. Most previous CNN-based super-resolution (SR) methods tend to directly learn the mapping from the low-resolution image to the high-resolution image through some complex convolutional neural networks. However, the method of blindly increasing the depth of the network is not the best choice because the performance improvement of such methods is marginal but the computational cost is huge. A more efficient method is to integrate the image prior knowledge into the model to assist the image reconstruction. Indeed, the soft-edge has been widely applied in many computer vision tasks as the role of an important image feature. In this paper, we propose a Soft-edge assisted Network (SeaNet) to reconstruct the high-quality SR image with the help of image soft-edge. The proposed SeaNet consists of three sub-nets: a rough image reconstruction network (RIRN), a soft-edge reconstruction network (Edge-Net), and an image refinement network (IRN). The complete reconstruction process consists of two stages. In Stage-I, the rough SR feature maps and the SR soft-edge are reconstructed by the RIRN and Edge-Net, respectively. In Stage-II, the outputs of the previous stages are fused and then fed to the IRN for high-quality SR image reconstruction. Extensive experiments show that our SeaNet converges rapidly and achieves excellent performance under the assistance of image soft-edge. The code is available at https://gitlab.com/junchenglee/seanet-pytorch .

Cauchy noise, as a typical non-Gaussian noise, appears frequently in many important fields, such as radar, medical, and biomedical imaging. In this letter, we focus on image recovery under Cauchy noise. Instead of the celebrated total variation or low-rank prior, we adopt a novel deep-learning-based image denoiser prior to effectively remove Cauchy noise with blur. To preserve more detailed texture and better balance between the receptive field size and the computational cost, we apply the multi-level wavelet convolutional neural network (MWCNN) to train this denoiser. We use the forward-backward splitting (FBS) method to handle the proposed model, which can be implemented efficiently without introducing auxiliary variables. Moreover, the multi-noise-levels strategy is employed to train a series of denoisers to restore the image corrupted by Cauchy noise and blur. Numerical experiments demonstrate clearly that our method has better performance than the existing image restoration methods for removing Cauchy noise in terms of the quantitative index and visual quality.

In this paper, we propose a new three-stage segmentation framework based on a convex variant of the Mumford--Shah model and the intensity inhomogeneity information of an image. The first stage in o...

Inspired by the work of Daubechies and Teschke (Appl Comput Harmon Anal 19(1):1–16. https://doi.org/10.1016/j.acha.2004.12.004 , 2005), we propose an image deblurring and denoising method based on fractional-order model with simultaneous decomposition. We use fractional-order derivative as the regularization term of cartoon part to avoid blocky effect. We replace the BV regularization term by $$B^\beta _q(L_p(\varOmega ))$$ term, and $$B^{-1}_1(L_1(\varOmega ))$$ term for the regularization of texture part. To promote sparsity, we add a nonconvex regularization term which is the weighted difference of $$l_1$$ -norm and $$l_2$$ -norm based on wavelet frame to the regularization term. The model can be solved by alternating direction method of multipliers (ADMM). The comparative experimental results show that the capability of preserving the edges and textural details of our algorithms. Our fractional-order algorithms are superior to that of traditional integer-order algorithms especially for images with texture.