Search

Hashing is an efficient method for nearest neighbor search in large-scale data space by embedding high-dimensional feature descriptors into a similarity preserving Hamming space with a low dimension. However, large-scale high-speed retrieval through binary code has a certain degree of reduction in retrieval accuracy compared to traditional retrieval methods. We have noticed that multi-view methods can well preserve the diverse characteristics of data. Therefore, we try to introduce the multi-view deep neural network into the hash learning field, and design an efficient and innovative retrieval model, which has achieved a significant improvement in retrieval performance. In this paper, we propose a supervised multi-view hash model which can enhance the multi-view information through neural networks. This is a completely new hash learning method that combines multi-view and deep learning methods. The proposed method utilizes an effective view stability evaluation method to actively explore the relationship among views, which will affect the optimization direction of the entire network. We have also designed a variety of multi-data fusion methods in the Hamming space to preserve the advantages of both convolution and multi-view. In order to avoid excessive computing resources on the enhancement procedure during retrieval, we set up a separate structure called memory network which participates in training together. The proposed method is systematically evaluated on the CIFAR-10, NUS-WIDE and MS-COCO datasets, and the results show that our method significantly outperforms the state-of-the-art single-view and multi-view hashing methods. [ABSTRACT FROM AUTHOR]

Three-dimensional multi-modal data are used to represent 3D objects in the real world in different ways. Features separately extracted from multimodality data are often poorly correlated. Recent solutions leveraging the attention mechanism to learn a joint-network for the fusion of multimodality features have weak generalization capability. In this paper, we propose a hamming embedding sensitivity network to address the problem of effectively fusing multimodality features. The proposed network called HamNet is the first end-to-end framework with the capacity to theoretically integrate data from all modalities with a unified architecture for 3D shape representation, which can be used for 3D shape retrieval and recognition. HamNet uses the feature concealment module to achieve effective deep feature fusion. The basic idea of the concealment module is to re-weight the features from each modality at an early stage with the hamming embedding of these modalities. The hamming embedding also provides an effective solution for fast retrieval tasks on a large scale dataset. We have evaluated the proposed method on the large-scale ModelNet40 dataset for the tasks of 3D shape classification, single modality and cross-modality retrieval. Comprehensive experiments and comparisons with state-of-the-art methods demonstrate that the proposed approach can achieve superior performance. [ABSTRACT FROM AUTHOR]

A predefined permission source region (PSR) is commonly used in bioluminescence tomography (BLT) as a priori information to reduce the ill-posedness of source reconstruction. However, the PSR is usually hard to determine and may miss the actual source region. Simultaneously, a linear or nonlinear based reconstruction method commonly have a relatively poor performance in the terms of low photon rate and strong noise. Here, we propose an alternative statistical-based-reconstruction framework with a cluster-algorithm-optimized PSR for BLT. In the alternative reconstruction framework, a global reconstruction is first conducted overall the solving domain to generate a rough estimation of source distribution by using a global based maximum likelihood expectation maximum (MLEM) method. Subsequently, a k-means cluster method is used to learn and find likely source support regions, which generates the optimal PSR. Finally, a local MLEM based reconstruction is performed to refine the source distribution on the optimal PSR. We first verify the performance of the proposed method with a series of numerical simulations, and then explore the potential by imaging in situ liver cancer in living mouse. [ABSTRACT FROM AUTHOR]

To better understand the acid-temperature coupling mechanism, it is critical to quantify the coupled effects of acid and temperature on the damage characteristics of sandstone. In this paper, macroscopic mechanical properties and microstructure changes of sandstone samples under different acid-temperature conditions are obtained by uniaxial compression experiments and nuclear magnetic resonance (NMR) measurements. Stress–strain curves and failure modes, changes in porosity, T2 (transverse relaxation time) spectra and fractal characteristics under different acid-temperature conditions are analyzed. Damage mechanism of acid-temperature coupling is discussed and damage variables caused by acid and temperature are defined, respectively. The results show that the effects of acid and temperature on the failure modes of sandstone samples are different. Based on the T2 spectral morphology of samples, the pores are classified into micro pores (< 0.1 μm), meso pores (0.1–1 μm) and macro pores (≥ 1 μm). With the increase of acidity, the sum of micro pores and meso pores gradually increase while macro pores gradually decrease. With increasing temperature, micro pores tend to decrease while meso pores and macro pores increase gradually. The fractal dimension of macro pores is largest, followed by the overall pore structure and meso pores. The damage variable caused by physical effect Dp is greater than that caused by chemical effect Dc. There is an exponential decreasing relationship between σc, E and total damage variable D. Highlights: Quantification of coupled effects of acid-temperature on the damage characteristics of sandstone. Changes of microscopic characteristics on the effects of acid-temperature coupling. Definition of damage variables separately for the effects of acid and temperature. [ABSTRACT FROM AUTHOR]

A 400W near-threshold supply class-C voltage controlled oscillator (VCO) with amplitude feedback loop and auto amplitude control (AAC) is proposed in this paper. The amplitude feedback loop and AAC ensure the robust startup of the proposed VCO and automatically adapts it to the class-C mode in steady state. Consequently, ultra-low power can be achieved in AAC mode and low phase noise, high swing can be achieved in AAC off mode. The proposed VCO with AAC gets ultra-low power consumption by limiting the oscillating amplitude and driving the proposed VCO into the deep Class-C mode. Additionally, the peak value detector is employed in this work to boost the controlling voltage of capacitors bank. Thus, a low on resistance of switch transistors is obtained, which increases the Q value of capacitors bank. The simulated phase noise is 124.5dBc/Hz at 1MHz offset with the 1.16GHz oscillation frequency. In this case, the figure-of-merit including tuning range (FOMT) of proposed VCO is 195dBc/Hz. The proposed VCO is fabricated in SMIC 40nm CMOS process and consumes 0.62mA from 0.65V supply. The measured phase noise is 109dBc/Hz and FOMT is 179dBc/Hz. [ABSTRACT FROM AUTHOR]

A low-voltage feedforward ring voltage-controlled oscillator (FRVCO) withstanding process variation and supply-voltage noise is proposed in this paper. In this design, the variation of transistors’ threshold voltages and supply voltage noise are detected by a compensating voltage generated circuit, and the load capacitance of the FRVCO is tuned by the compensating voltage to reduce the variation of oscillating frequency. Simulation results demonstrate that the centre oscillating frequency difference between ‘FF’ and ‘SS’ corners is reduced from 38.3% to 1.5% by tuning the voltage from 0 to 650 mV. Moreover, with a supply voltage change of ±0.5%, the frequency variation is smaller than ±0.06%. The 0.65-V 600-MHz FRVCO has been fabricated using the TSMC 130-nm CMOS process. [ABSTRACT FROM AUTHOR]

Limited memory size is considered as a major bottleneck in data centers for intelligent urban computing. It is shown that there exist a large number of duplicated pages when various processes working with big data are hosted in data centers. Memory deduplication aims to automatically eliminate duplicate data in memory. It is an efficient technique to reduce memory requirement. In memory deduplication, pages with same content are detected and merged into a single copy. Recently, several system-level techniques have been proposed to address this issue, in which content-based page sharing (CBPS) is most widely used, since CBPS can be performed transparently in the hypervisor layer without any modification to guest operating systems of data center. In this paper, we survey several techniques for memory deduplication. We also classify these techniques based on their characteristics to highlight their similarities and differences. The aim of this paper is to provide insights to researchers into working of memory deduplication techniques and also to motivate them to propose better intelligent urban computing systems. [ABSTRACT FROM AUTHOR]

Web-scale image understanding is drawing more and more attention from the computer vision and multimedia domain. To solve the key problem of visual polysemia and concept polymorphism in the image understanding, this paper proposes a semantic dictionary to describe the images on the level of semantic. The semantic dictionary characterizes the probability distribution between visual appearances and semantic concepts, and the learning procedure of semantic dictionary is formulated into a minimization optimization problem. Mixed-norm regularization is adopted to solve the above optimization for learning the concept membership distribution of visual appearance. Furthermore, to improve the generalization ability of the semantic description, we propose the semantic expansion technology, where a concept transferring matrix is learnt to quantize the implicit relevancy among the concepts. Finally, the distributed framework on the basis of the semantic dictionary is constructed to speed up the large scale image understanding. The semantic dictionary is validated in the tasks of large scale semantic image search and image annotation. [ABSTRACT FROM AUTHOR]

Large scale image understanding is a challenging but significant task to comprehend image contents on the internet. The de-facto standard methods based on machine learning or computer vision still suffer from a phenomenon of visual polysemia and concept polymorphism (VPCP). To resolve the VPCP, semantic dictionary has been proposed to characterize the membership distribution between visual appearances and semantic concepts. In this paper, we propose an online semantic dictionary learning algorithm on the base of both locality sensitive hashing (LSH) and stochastic approximations, which can scale up to large scale datasets with millions of training samples and speedup the efficiency of follow-up processing. With the help of the LSH-based semantic dictionary, we develop an extension of the spatial pyramid matching (SPM) kernel method by generalizing the dictionary as a basic semantic description. The efficiency of our approach is validated in the experiments of web-scale semantic image search and image classification on the ImageNet dataset and Caltech-256 dataset. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) provides superior coding efficiency than previous video coding standards at the cost of increasing encoding complexity. The complexity increase of motion estimation (ME) procedure is rather significant, especially when considering the complicated partitioning structure of HEVC. To fully exploit the coding efficiency brought by HEVC requires a huge amount of computations. In this paper, we analyze the ME structure in HEVC and propose a parallel framework to decouple ME for different partitions on many-core processors. Based on local parallel method (LPM), we first use the directed acyclic graph (DAG)-based order to parallelize coding tree units (CTUs) and adopt improved LPM (ILPM) within each CTU (DAGILPM), which exploits the CTU-level and prediction unit (PU)-level parallelism. Then, we find that there exist completely independent PUs (CIPUs) and partially independent PUs (PIPUs). When the degree of parallelism (DP) is smaller than the maximum DP of DAGILPM, we process the CIPUs and PIPUs, which further increases the DP. The data dependencies and coding efficiency stay the same as LPM. Experiments show that on a 64-core system, compared with serial execution, our proposed scheme achieves more than 30 and 40 times speedup for $1920\times 1080$ and $2560\times 1600$ video sequences, respectively. [ABSTRACT FROM AUTHOR]

Clothing style analysis is a critical step for understanding images of people. To automatically identify the style of clothing that people wear is a challenging task due to various poses of person and large variations for even the same clothing category. Suit as one of the clothing style is a key element in many important activities. In this paper, we propose a novel suits detection method for images of people in unconstrained environments. In order to cope with various human poses, human pose estimation is incorporated. By analyzing the style of clothing, we propose the color features, shape features and statistical features for suits detection. Experiments with four popular classifiers have been conducted to demonstrate that the proposed features are effective and robust. Comparative experiments with Bag of Words (BoW) method demonstrate that the proposed features are superior to BoW which is a popular method for object detection. The proposed method has achieved promising performance over our dataset, which is a challenging web image set with various human poses and diverse styles of clothing. [ABSTRACT FROM AUTHOR]

In traditional image retrieval, images are commonly represented using Bag-of-visual-Words (BoW) built from image local features. However, the lack of spatial and structural information suppresses its performance in applications. In this paper, we introduce a multi-cues description by fusing structural, content and spatial information for partial-duplicate image retrieval. Firstly, we propose a rotation-invariant Local Self-Similarity Descriptor (LSSD), which captures the internal structural layouts in the local textural self-similar regions around interest points. Then, based on the spatial pyramid model, we make use of both LSSD and SIFT to construct an image representation with multi-cues. Finally, we formulate the Semi-Relative Entropy as the distance metric. Comparison experiments with state-of-the-art methods on four popular databases show the efficiency and effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Highlights: [•] A novel approach for ROI detection in pornographic images is put forward. [•] A ROI-based codebook algorithm is proposed. [•] A hybrid approach of pornographic image detection is explored. [Copyright &y& Elsevier]

Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin. [ABSTRACT FROM AUTHOR]

Human brain is a complex yet economically organized system, where a small portion of critical hub regions support the majority of brain functions. The identification of common hub nodes in a population of networks is often simplified as a voting procedure on the set of identified hub nodes across individual brain networks, which ignores the intrinsic data geometry and partially lacks the reproducible findings in neuroscience. Hence, we propose a first-ever group-wise hub identification method to identify hub nodes that are common across a population of individual brain networks. Specifically, the backbone of our method is to learn common graph embedding that can represent the majority of local topological profiles. By requiring orthogonality among the graph embedding vectors, each graph embedding as a data element is residing on the Grassmannian manifold. We present a novel Grassmannian manifold optimization scheme that allows us to find the common graph embeddings, which not only identify the most reliable hub nodes in each network but also yield a population-based common hub node map. Results of the accuracy and replicability on both synthetic and real network data show that the proposed manifold learning approach outperforms all hub identification methods employed in this evaluation. [ABSTRACT FROM AUTHOR]

Due to the instability of the hyperspectral imaging system and the atmospheric interference, hyperspectral images (HSIs) often suffer from losing the image information of areas with different shapes, which significantly degrades the data quality and further limits the effectiveness of methods for subsequent tasks. Although mainstream deep learning-based methods have achieved promising inpainting performance, the complicated ground object distributions increase the difficulty of HSIs inpainting in practice. In addition, spectral redundancy and complex texture details are two main challenges for deep neural network-based inpainting methods. To address the above issues, we propose a Global-Local Constraints-based Spectral Adaptive Network (GLCSA-Net) for HSI inpainting. To reduce the redundancy of spectral information, a multi-frequency channel attention module is designed to strengthen the essential channels and suppress the less significant ones, which calculates adaptive weight coefficients by converting feature maps to the frequency domain. Furthermore, we propose to constrain the generation of missing areas from both global and local perspectives, by fully leveraging the HSI texture information, so that the overall structure information and regional texture consistency of the original HSI can be maintained. The proposed method has been extensively evaluated on the Indian Pines and FCH datasets. The promising results indicate that GLCSA-Net outperforms the state-of-the-art methods in quantitative and qualitative assessments. [ABSTRACT FROM AUTHOR]

The core purpose of deep metric learning is to construct an embedding space, where objects belonging to the same class are gathered together and the ones from different classes are pushed apart. Most existing approaches typically insist to inter-class characteristics, e.g., class-level information or instance-level similarity, to obtain semantic relevance of data points and get a large margin between different classes in the embedding space. However, the intra-class characteristics, e.g., local manifold structure or relative relationship within the same class, are usually overlooked in the learning process. Hence the output embeddings have limitation in retrieving a good ranking result if existing multiple positive samples. And the local data structure of embedding space cannot be fully exploited since lack of relative ranking information. As a result, the model is prone to overfitting on a train set and get low generalization on the test set (unseen classes) when losing sight of intra-class variance. This paper presents a novel self-supervised synthesis ranking auxiliary framework, which captures intra-class characteristics as well as inter-class characteristics for better metric learning. Our method designs a synthetic samples generation of polar coordinates to generate measurable intra-class variance with different strength and diversity in the latent space, which can simulate the various local structure change of intra-class in the initial data domain. And then formulates a self-supervised learning procedure to fully exploit this property and preserve it in the embedding space. As a result, the learned embedding space not only keeps inter-class discrimination but also owns subtle intra-class diversity, leading to better global and local embedding structures. Extensive experiments on five benchmarks show that our method significantly improves and outperforms the state-of-the-art methods on the performances of both retrieval and ranking by 2%-4% (personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending an email to pubs-permissions@ieee.org). [ABSTRACT FROM AUTHOR]

Background: Mounting evidence shows that the neuropathological burdens manifest preference in affecting brain regions during the dynamic progression of Alzheimer's disease (AD). Since the distinct brain regions are physically wired by white matter fibers, it is reasonable to hypothesize the differential spreading pattern of neuropathological burdens may underlie the wiring topology, which can be characterized using neuroimaging and network science technologies.Objective: To study the dynamic spreading patterns of neuropathological events in AD.Methods: We first examine whether hub nodes with high connectivity in the brain network (assemble of white matter wirings) are susceptible to a higher level of pathological burdens than other regions that are less involved in the process of information exchange in the network. Moreover, we propose a novel linear mixed-effect model to characterize the multi-factorial spreading process of neuropathological burdens from hub nodes to non-hub nodes, where age, sex, and APOE4 indicators are considered as confounders. We apply our statistical model to the longitudinal neuroimaging data of amyloid-PET and tau-PET, respectively.Results: Our meta-data analysis results show that 1) AD differentially affects hub nodes with a significantly higher level of pathology, and 2) the longitudinal increase of neuropathological burdens on non-hub nodes is strongly correlated with the connectome distance to hub nodes rather than the spatial proximity.Conclusion: The spreading pathway of AD neuropathological burdens might start from hub regions and propagate through the white matter fibers in a prion-like manner. [ABSTRACT FROM AUTHOR]

Cross-view geo-localization is to spot images of the same geographic target from different platforms, e.g., drone-view cameras and satellites. It is challenging in the large visual appearance changes caused by extreme viewpoint variations. Existing methods usually concentrate on mining the fine-grained feature of the geographic target in the image center, but underestimate the contextual information in neighbor areas. In this work, we argue that neighbor areas can be leveraged as auxiliary information, enriching discriminative clues for geo-localization. Specifically, we introduce a simple and effective deep neural network, called Local Pattern Network (LPN), to take advantage of contextual information in an end-to-end manner. Without using extra part estimators, LPN adopts a square-ring feature partition strategy, which provides the attention according to the distance to the image center. It eases the part matching and enables the part-wise representation learning. Owing to the square-ring partition design, the proposed LPN has good scalability to rotation variations and achieves competitive results on three prevailing benchmarks, i.e., University-1652, CVUSA and CVACT. Besides, we also show the proposed LPN can be easily embedded into other frameworks to further boost performance. [ABSTRACT FROM AUTHOR]

Thanks to the powerful modeling capabilities of multi-head self attention (MSA), transformers have shown significant performance gains in vision tasks. However, as transformers require heavy computation, more efficient designs are required. In this paper, we present an efficient transformer architecture named GoLDFormer for image restoration. GoLDFormer extends the capability of window-based self-attention through two core designs. First, We propose a globally-enhanced window-based transformer block (G-WTB), which applies transposed attention to a compressed window representation rather than the spatial features, thus establishing connections between all windows with less computational complexity. Second, since the interactions between image content and window attention weights can be interpreted as spatially varying convolution, we introduce an adaptive filter structure into transformer models and propose a deformable filtering block (DFB) to enable cross-window connections. By adjusting the shape of the generated filters in the DFB, we can balance the computational costs and the degree of adjacent window interaction. Extensive experiments on several image restoration tasks demonstrate that GoLDFormer achieves competitive results against recent methods but with optimal computational costs. • We propose GoLDFormer, including two blocks, to enhance W-MSA globally and locally. • Globally-enhanced attention block captures global dependencies between all windows. • Deformable filtering block captures local dependencies by modifying filter size. • Our GoLDFormer achieves competitive results against recent lightweight methods. [ABSTRACT FROM AUTHOR]

Cross modal retrieval can retrieve images through a text query and vice versa. In recent years, cross modal retrieval has attracted extensive attention. The purpose of most now available cross modal retrieval methods is to find a common subspace and maximize the different modal correlation. To generate specific representations consistent with cross modal tasks, this paper proposes a novel cross modal retrieval framework, which integrates feature learning and latent space embedding. In detail, we proposed a deep CNN and a shallow CNN to extract the feature of the samples. The deep CNN is used to extract the representation of images, and the shallow CNN uses a multi‐dimensional kernel to extract multi‐level semantic representation of text. Meanwhile, we enhance the semantic manifold by constructing cross modal ranking and within‐modal discriminant loss to improve the division of semantic representation. Moreover, the most representative samples are selected by using online sampling strategy, so that the approach can be implemented on a large‐scale data. This approach not only increases the discriminative ability among different categories, but also maximizes the relativity between different modalities. Experiments on three real word datasets show that the proposed method is superior to the popular methods. [ABSTRACT FROM AUTHOR]

Automatically identifying objects similar to the surroundings is a complex and difficult task in real-world scenarios. In addition to the high intrinsic similarity between camouflaged objects and their backgrounds, these objects are usually diverse in scale and blurred in appearance. And the deceptive nature of the camouflaged objects introduces lots of noise into the features and generates inaccurate segmentation map extracted by deep learning model. We tackle these problems by proposing a novel dynamic interactive refinement network (DIRNet), which aims to make the features exploit effective details and semantics together as well as discard interference information. Specifically, we utilize bilateral interaction module (BIM) to interact with foreground and background information to conduct contextual exploration, which can capture more meaningful details and refine the confusion. Additionally, in the purpose of retaining the appropriate information and erasing noise, we design an adjacent aggregation interaction module (AAIM) to integrate the adjacent multi-level features with attention coefficients for each layer. The final results are obtained through the dynamic refinement of the BIM and AAIM. Extensive quantitative and qualitative experiments on four public benchmark datasets demonstrate that our proposed DIRNet is an effective COD framework and outperforms 14 state-of-the-art models. [ABSTRACT FROM AUTHOR]

RGB-T salient object detection (SOD) has made significant progress in recent years. However, most existing works are based on heavy models, which are not applicable to mobile devices. Additionally, there is still room for improvement in the design of cross-modal feature fusion and cross-level feature fusion. To address these issues, we propose a lightweight cross-modal information mutual reinforcement network for RGB-T SOD. Our network consists of a lightweight encoder, the cross-modal information mutual reinforcement (CMIMR) module, and the semantic-information-guided fusion (SIGF) module. To reduce the computational cost and the number of parameters, we employ the lightweight module in both the encoder and decoder. Furthermore, to fuse the complementary information between two-modal features, we design the CMIMR module to enhance the two-modal features. This module effectively refines the two-modal features by absorbing previous-level semantic information and inter-modal complementary information. In addition, to fuse the cross-level feature and detect multiscale salient objects, we design the SIGF module, which effectively suppresses the background noisy information in low-level features and extracts multiscale information. We conduct extensive experiments on three RGB-T datasets, and our method achieves competitive performance compared to the other 15 state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Most person re-identification (re-ID) approaches are based on supervised learning, which requires manually annotated data. However, it is not only resource-intensive to acquire identity annotation but also impractical for large-scale data. To relieve this problem, we propose a cross-camera unsupervised approach that makes use of unsupervised style-transferred images to jointly optimize a convolutional neural network (CNN) and the relationship among the individual samples for person re-ID. Our algorithm considers two fundamental facts in the re-ID task, i.e., variance across diverse cameras and similarity within the same identity. In this paper, we propose an iterative framework which overcomes the camera variance and achieves across-camera similarity exploration. Specifically, we apply an unsupervised style transfer model to generate style-transferred training images with different camera styles. Then we iteratively exploit the similarity within the same identity from both the original and the style-transferred data. We start with considering each training image as a different class to initialize the Convolutional Neural Network (CNN) model. Then we measure the similarity and gradually group similar samples into one class, which increases similarity within each identity. We also introduce a diversity regularization term in the clustering to balance the cluster distribution. The experimental results demonstrate that our algorithm is not only superior to state-of-the-art unsupervised re-ID approaches, but also performs favorably compared with other competing unsupervised domain adaptation methods (UDA) and semi-supervised learning methods. [ABSTRACT FROM AUTHOR]

Frame rate up-conversion (FRUC) usually suffers from unreliable motion vectors due to the absence of the current frame to be interpolated. In addition, since the majority of video sequences are usually compressed by various coding standards to reduce the data volume, the quality of the generated frames in the FRUC will be further impaired. To address this problem, we proposed two FRUC algorithms based on deep residual network. We first present a deep residual network for the FRUC (DRNFRUC), which consists of feature extraction, feature recursive analysis, and image restoration parts with a skip connection between the input and the output of the network. The proposed DRNFRUC takes the result of an arbitrary existing FRUC method as the input and is able to significantly reduce the edge blurring and blocking artifacts when the motion of the block is violent. In addition, we proposed a deep residual network with weighted convolutional motion compensation (DRNWCMC) for the FRUC, where the convolution operations can be embedded into the motion compensation interpolation (MCI) in any existing MCI-based FRUC method. In DRNWCMC, we first devise two convolutional neural networks corresponding to the forward and backward motion compensated frames, respectively. And then, the adaptive interpolation coefficients for motion compensation are designed as two $1\times1$ convolutional kernels. Finally, the interpolation result of WCMC is fed into another convolutional neural network to further improve the performance. All the parameters involved in the DRNWCMC are trained simultaneously under the same cost function. The experimental results show that the two proposed algorithms can remarkably improve both the objective and subjective quality of the interpolated frames. [ABSTRACT FROM AUTHOR]

Time-interleaving has been a popular choice for multi-GHz analog-to-digital converters (ADCs). Unfortunately, inherent defects such as offset, gain, timing-skew mismatches among sub-ADCs degrade overall performance seriously. At present, the method for eliminating offset and gain mismatch is fairly straightforward, however, calibration for timing-skew is still in a state of exploration. An efficient digital-mixing-based method to estimate timing skew in time-interleaved analog-to-digital converters (TI-ADCs) is proposed and verified. The timing skew can be estimated by digital-mixing and inverse sine function processing. The proposed method can greatly alleviate the effects of phase ambiguity. It is also insensitive to offset mismatch. A four-channel TI-ADC behavioral system of 10-bits 2 GS/s has been simulated to verify the performance of the method. Simulation results demonstrate that the method greatly improves the TI-ADC′s performance and is effective up to the severe timing skew mismatch. [ABSTRACT FROM AUTHOR]

Bokeh effect is usually used to highlight major contents in an image. Limited by the small sensors, cameras on smartphones are less sensitive to the depth information and cannot directly produce bokeh effect as pleasant as digital single lens reflex cameras. To address this problem, a depth-guided deep filtering network, called DDFN, is proposed in this study. Specifically, the focused region detection block is designed to detect the salient areas, and the depth estimated block is introduced to estimate depth maps from full-focus images. Further, combining depth maps and focused features, an adaptive rendering block is proposed to synthesize bokeh effect with adaptive cross 1-D filters. Both quantitative and qualitative evaluations on the public datasets demonstrate that the proposed model performs favorably against state-of-the-art methods in terms of rendering effects and has lower computational cost, e.g., 24.07 dB PSNR on EBB! dataset and 0.45 s inference times for a 512 × 768 image on a Snapdragon 865 mobile processor. [ABSTRACT FROM AUTHOR]

High‐efficiency video coding (HEVC) is the next generation standard of video coding. The deblocking filter (DF) constitutes a significant part of the HEVC decoder complexity. A three‐step parallel framework (TPF) is proposed for the H.264/AVC DF, which is also suitable for HEVC except the third step. The third step of the TPF is replaced with a directed acyclic graph‐based order. Experiments show that the proposed method dramatically accelerates more than the state‐of‐the‐art parallel method. [ABSTRACT FROM AUTHOR]

Accumulating evidences have shown that plenty of miRNAs play fundamental and important roles in various biological processes and the deregulations of miRNAs are associated with a broad range of human diseases. However, the mechanisms underlying the dysregulations of miRNAs still have not been fully understood yet. All the previous computational approaches can only predict binary associations between diseases and miRNAs. Predicting multiple types of disease-miRNA associations can further broaden our understanding about the molecular basis of diseases in the level of miRNAs. In this study, the model of Restricted Boltzmann machine for multiple types of miRNA-disease association prediction (RBMMMDA) was developed to predict four different types of miRNA-disease associations. Based on this model, we could obtain not only new miRNA-disease associations, but also corresponding association types. To our knowledge, RBMMMDA is the first model which could computationally infer association types of miRNA-disease pairs. Leave-one-out cross validation was implemented for RBMMMDA and the AUC of 0.8606 demonstrated the reliable and effective performance of RBMMMDA. In the case studies about lung cancer, breast cancer, and global prediction for all the diseases simultaneously, 50, 42, and 45 out of top 100 predicted miRNA-disease association types were confirmed by recent biological experimental literatures, respectively. [ABSTRACT FROM AUTHOR]

A background calibration technique is proposed to correct bit weights in pipelined-successive-approximation-register (SAR) analogue-to-digital converters (ADCs). By splitting the second stage, the input signal interference is mostly removed, thereby greatly enhancing the convergence speed of the algorithm. Besides, the dither signal assists to eliminate mismatch issues between the partially split ADCs, thus relaxing the analogue overheads. According to the simulation, after calibration, the spurious-free-dynamic-range and signal-to-noise-and-distortion-ratio are improved from 53.2 to 88.2 dB and 49.5 to 75.2 dB, respectively. The calibration algorithm converges with about only 600 K samples. [ABSTRACT FROM AUTHOR]

Increasing evidence has indicated that plenty of lncRNAs play important roles in many critical biological processes. Developing powerful computational models to construct lncRNA functional similarity network based on heterogeneous biological datasets is one of the most important and popular topics in the fields of both lncRNAs and complex diseases. Functional similarity network consturction could benefit the model development for both lncRNA function inference and lncRNA-disease association identification. However, little effort has been attempted to analysis and calculate lncRNA functional similarity on a large scale. In this study, based on the assumption that functionally similar lncRNAs tend to be associated with similar diseases, we developed two novel lncRNA functional similarity calculation models (LNCSIM). LNCSIM was evaluated by introducing similarity scores into the model of Laplacian Regularized Least Squares for LncRNA-Disease Association (LRLSLDA) for lncRNA-disease association prediction. As a result, new predictive models improved the performance of LRLSLDA in the leave-one-out cross validation of various known lncRNA-disease associations datasets. Furthermore, some of the predictive results for colorectal cancer and lung cancer were verified by independent biological experimental studies. It is anticipated that LNCSIM could be a useful and important biological tool for human disease diagnosis, treatment, and prevention. [ABSTRACT FROM AUTHOR]

A shaping technique is proposed to remedy the attenuation‐factor error in the split capacitor‐digital‐to‐analogue converter (CDAC) of a successive‐approximation‐register (SAR) ADC. By feeding back the previous quantised value of the least significant bit part to the most significant bit part, the inaccurate attenuation‐factor error caused by mismatches and parasitic components is shaped to high frequency. The in‐band dynamic performance is significantly improved without any calibration. Simulation results show that the proposed attenuation‐factor shaping technique improves the spurious‐free‐dynamic‐range by 10 dB, whereas the number of capacitors increases only 1.2%. [ABSTRACT FROM AUTHOR]

A difference‐equalisation based method is presented to estimate timing skew adaptively in time‐interleaved analogue‐to‐digital converters (ADCs). By digital zero‐crossing detection and calculating the mean of the absolute difference between adjacent sub‐ADC's outputs, the timing mismatch information for each channel can be identified easily. Simulation results demonstrate that the proposed method can estimate timing skew accurately. Furthermore, the proposed method requires far fewer samples to estimation than the previous methods. [ABSTRACT FROM AUTHOR]

In this paper, we developed a novel distributed and privatized framework for miRNA-disease association prediction (DPFMDA) to predict potential miRNA-disease associations. DPFMDA assumes that diseases and miRNAs can be represented as feature matrices. Through independent factorization, collaborative analysis and matrix re-factorization, feature matrices are estimated and potential associations in miRNA-disease association matrix are predicted. The proposed computational model is tested and verified by leave-one-out cross validation (LOOCV). The AUCs of global and local LOOCV are 0.8859 and 0.8267 respectively, which outperform the state-of-the-art computational models. DPFMDA is further evaluated with case studies of four important human complex diseases. We test Esophageal Neoplasms and Prostate Neoplasms using DPFMDA in the HMDD v2.0 database. For Breast Neoplasms, we use the HMDD v2.0 database which removes all the known miRNAs-disease associations on Breast Neoplasms. For Hepatocellular Carcinoma, HMDD v1.0 database is used as the input of DPFMDA and predicted results are verified with HMDD v2.0, dbDEMC and miR2Disease databases. The results show that 90% (Esophageal Neoplasms), 88% (Prostate Neoplasms), 98% (Breast Neoplasms) and 86% (Hepatocellular Carcinoma) of top 50 predicted miRNA-disease associations are confirmed by recent experimental studies respectively. It is anticipated that DPFMDA would be an effective and extensible method for potential miRNA-disease association prediction. [ABSTRACT FROM AUTHOR]

Image restoration methods based on convolutional neural networks have shown great success in the literature. However, since most of networks are not deep enough, there is still some room for the performance improvement. On the other hand, though some models are deep and introduce shortcuts for easy training, they ignore the importance of location and scaling of different inputs within the shortcuts. As a result, existing networks can only handle one specific image restoration application. To address such problems, we propose a novel adaptive residual network (ARN) for high-quality image restoration in this paper. Our ARN is a deep residual network, which is composed of convolutional layers, parametric rectified linear unit layers, and some adaptive shortcuts. We assign different scaling parameters to different inputs of the shortcuts, where the scaling is considered as part parameters of the ARN and trained adaptively according to different applications. Due to the special construction of ARN, it can solve many image restoration problems and have superior performance. We demonstrate its capabilities with three representative applications, including Gaussian image denoising, single image super resolution, and JPEG image deblocking. Experimental results prove that our model greatly outperforms numerous state-of-the-art restoration methods in terms of both peak signal-to-noise ratio and structure similarity index metrics, e.g., it achieves 0.2–0.3 dB gain in average compared with the second best method at a wide range of situations. [ABSTRACT FROM AUTHOR]

Compared with traditional image classification, fine-grained visual categorization is a more challenging task, because it targets to classify objects belonging to the same species, e.g., classify hundreds of birds or cars. In the past several years, researchers have made many achievements on this topic. However, most of them are heavily dependent on the artificial annotations, e.g., bounding boxes, part annotations, and so on. The requirement of artificial annotations largely hinders the scalability and application. Motivated to release such dependence, this paper proposes a robust and discriminative visual description named Automated Bi-level Description (AutoBD). “Bi-level” denotes two complementary part-level and object-level visual descriptions, respectively. AutoBD is “automated,” because it only requires the image-level labels of training images and does not need any annotations for testing images. Compared with the part annotations labeled by the human, the image-level labels can be easily acquired, which thus makes AutoBD suitable for large-scale visual categorization. Specifically, the part-level description is extracted by identifying the local region saliently representing the visual distinctiveness. The object-level description is extracted from object bounding boxes generated with a co-localization algorithm. Although only using the image-level labels, AutoBD outperforms the recent studies on two public benchmark, i.e., classification accuracy achieves 81.6% on CUB-200–2011 and 88.9% on Car-196, respectively. On the large-scale Birdsnap data set, AutoBD achieves the accuracy of 68%, which is currently the best performance to the best of our knowledge. [ABSTRACT FROM PUBLISHER]

With the rapid growth of the number of images on the Internet, it has become more necessary to ensure the content security of images. The key problem is retrieving relevant images from the large database. Binary embedding is an effective way to improve the efficiency of calculating similarities for image content security as binary code is storage efficient and fast to compute. It tries to convert real-valued signatures into binary codes while preserving similarity of the original data, and most binary embedding methods quantize each projected dimension to one bit (presented as 0/1). As a consequence, it greatly decreases the discriminability of original signatures. In this paper, we first propose a novel triple-bit quantization strategy to solve the problem by assigning 3-bit to each dimension. Then, asymmetric distance algorithm is applied to re-rank candidates obtained from Hamming space for the final nearest neighbors. For simplicity, we call the framework triple-bit quantization with asymmetric distance (TBAD). The inherence of TBAD is combining the best of binary codes and real-valued signatures to get nearest neighbors quickly and concisely. Moreover, TBAD is applicable to a wide variety of embedding techniques. Experimental comparisons on BIGANN set show that the proposed method can achieve remarkable improvement in query accuracy compared to original binary embedding methods. [ABSTRACT FROM AUTHOR]

This paper proposes a 3D laser scanning technology for the pinhole camera with lens distortion. On the basis of the pinhole camera mathematical model, we improve the extraction algorithm of Harris corner points, and introduce the professional 2D RAC calibration, 3D reconstruction, and 3D volume calculation. With the help of stepper motor, and the camera, we can capture a series of images and obtain the object 3D surface information. Furthermore, we use the line laser with Gaussian distribution as the measurement illuminant, and select the samples at the peak of intensity distribution. This brings both high accuracy and speed of measurement. [ABSTRACT FROM AUTHOR]

Traditional machine learning sequence models, such as RNN and LSTM, can solve sequential data problems with the use of internal memory states. However, the neuron units and weights are shared at each time step to reduce computational costs, limiting their ability to learn time-varying relationships between model inputs and outputs. In this context, this paper proposes two methods to characterize the dynamic relationships in real-world sequential data, namely, the internal time-varying sequence model (ITV model) and the external time-varying sequence model (ETV model). Our methods were designed with an automated basis expansion module to adapt internal or external parameters at each time step without requiring high computational complexity. Extensive experiments performed on synthetic and real-world data demonstrated superior prediction and classification results to conventional sequence models. Our proposed ETV model is particularly effective at handling long sequence data. [ABSTRACT FROM AUTHOR]

Motivation Light-field microscopy (LFM) is a compact solution to high-speed 3D fluorescence imaging. Usually, we need to do 3D deconvolution to the captured raw data. Although there are deep neural network methods that can accelerate the reconstruction process, the model is not universally applicable for all system parameters. Here, we develop AutoDeconJ, a GPU-accelerated ImageJ plugin for 4.4× faster and more accurate deconvolution of LFM data. We further propose an image quality metric for the deconvolution process, aiding in automatically determining the optimal number of iterations with higher reconstruction accuracy and fewer artifacts. Results Our proposed method outperforms state-of-the-art light-field deconvolution methods in reconstruction time and optimal iteration numbers prediction capability. It shows better universality of different light-field point spread function (PSF) parameters than the deep learning method. The fast, accurate and general reconstruction performance for different PSF parameters suggests its potential for mass 3D reconstruction of LFM data. Availability and implementation The codes, the documentation and example data are available on an open source at: https://github.com/Onetism/AutoDeconJ.git. Supplementary information Supplementary data are available at Bioinformatics online. [ABSTRACT FROM AUTHOR]

The reconstruction of the floorplan for a building requires the creation of a two-dimensional floorplan from a 3D model. This task is widely employed in interior design and decoration. In reality, the structures of indoor environments are complex with much clutter and occlusions, making it difficult to reconstruct a complete and accurate floorplan. It is well known that a suitable dataset is a key point to drive an effective algorithm, while existing datasets of floorplan reconstruction are synthetic and small. Without reliable accumulations of real datasets, the robustness of methods to real scene reconstruction is weakened. In this paper, we first annotate a large-scale realistic benchmark, which contains RGBD image sequences and 3D models of 80 indoor scenes with more than 10,000 square meters. We also introduce a framework for the floorplan reconstruction with mesh-based point cloud normalization. The loose-Manhattan constraint is performed in our optimization process, and the optimal floorplan is reconstructed via constraint integer programming. The experimental results on public and our own datasets demonstrate that the proposed method outperforms FloorNet and Floor-SP. [ABSTRACT FROM AUTHOR]

Real-time 3D scene reconstruction has attracted a great amount of attention in the fields of augmented reality, virtual reality and robotics. Previous works usually assumed slow sensor motions to avoid large interframe differences and strong image blur, but this limits the applicability of the techniques in real cases. In this study, we propose an end-to-end 3D reconstruction system that combines color, depth and inertial measurements to achieve a robust reconstruction with fast sensor motions. We involved an extended Kalman filter (EKF) to fuse RGB-D-IMU data and jointly optimize feature correspondences, camera poses and scene geometry by using an iterative method. A novel geometry-aware patch deformation technique is proposed to adapt the changes in patch features in the image domain, leading to highly accurate feature tracking with fast sensor motions. In addition, we maintained the global consistency of the reconstructed model by achieving loop closure with submap-based depth image encoding and 3D map deformation. The experiments revealed that our patch deformation method improves the accuracy of feature tracking, that our improved loop detection method is more efficient than the original method and that our system possesses superior 3D reconstruction results compared with the state-of-the-art solutions in handling fast camera motions. [ABSTRACT FROM AUTHOR]

Object detection is one of the most challenging tasks in computer vision. With the advances in semiconductor devices and chip technology, hardware accelerators have been widely used. Field-programmable gate arrays (FPGAs) are a highly flexible hardware platform that allows customized reconfiguration of the integrated circuit, which has the potential to improve the efficiency of object detection accelerators. However, few reviews summarize FPGA-based object detection accelerators. Also, there is no general principle for realizing object detection according to FPGA characteristics. In this paper, the current hardware accelerators are introduced and compared. Then, the typical deep learning-based object detectors are summarized. Next, the questions of "Why choose FPGA," "The design goals of FPGA accelerators" and "The design methods for FPGA accelerators" are discussed in detail. Finally, the challenges of object detection algorithms, hardware, and co-design are presented. In addition, an online platform (https://github.com/vivian13maker/) is constructed to provide specific information on all advanced works. [ABSTRACT FROM AUTHOR]

Conventional direction-of-arrival (DOA) estimation algorithms are sensitive to array imperfections and outliers, making it challenging to realize accurate estimates in real applications. Facing the challenge, we propose a robust variational Bayesian inference based DOA estimation algorithm using the linear sparse array in this paper, where accurate DOA estimation with increased number of degrees of freedom (DOFs) is realized. With the mixture of von Mises model, a prior-grid scheme is further proposed to alleviate the computational burden introduced by the Bayesian variational inference framework. Since there is no restriction on the prior knowledge of the number of sources, the proposed algorithm is friendly to actual scenarios. Simulation results demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Rate control is a key issue in the research of video coding optimization. In this work, we propose a statistical modeling based frame-level distortion estimation model and a distortion dependency based rate control optimization algorithm for Versatile Video Coding (VVC). VVC has incorporated into more advanced coding tools, resulting in the distribution of transform coefficient more sharper. In response to this phenomenon, the TCD of VVC is modeled by the probability density function (PDF) of Cauchy distribution and generalized Gaussian distribution (GGD) individually. Then, a sliding window-based frame-level distortion model is proposed, which is used for derivation of inter-frame distortion dependency factor. According to the derived distortion dependency factor, a frame-level rate control optimization algorithm is presented to update the coding parameters in practical coding process. Experiments have shown that the proposed method could effectively improve rate–distortion performance, achieving an average rate distortion performance improvement of 0.72% in sequences blow 4K resolution, and 1.47% in 4K sequences in low delay P frame (LDP) configuration. • The TCDs of I, B and P frames are modeled by the probability density functions of generalized Gaussian distribution and Cauchy distribution. • The mapping relationship between coding distortion and the quantization step is further investigated. A frame-level distortion–quantization (D–Q) model is then established with a sliding window strategy. • The infer-frame distortion dependency is analyzed and applied to the optimization of frame-level bit allocation. Experimental results show that the proposed RC optimization algorithm could improve the RD performance significantly in LDP configuration. [ABSTRACT FROM AUTHOR]

Multi-source Unsupervised Domain Adaptation (MUDA) transfers knowledge learned from multiple labeled source domains to an unlabeled target domain by minimizing the domain shift between multiple source domains and the target domain. Recent studies on MUDA have focused on aligning the distribution of each pair of source and target domains in separate feature spaces to reduce their domain shift. However, these approaches suffer from two main shortcomings. First, they usually focus on the global domain shift which lacks consideration of the joint distribution of category-corresponded subdomains. Second, out-of-distribution samples far from the sample center are hard to align by the global domain alignment. Therefore, we propose a novel Deep Joint Semantic Adaptive Network (DJSAN) for MUDA. Specifically, a new maximum mean discrepancy-based metric, Joint Semantic Maximum Mean Discrepancy (JSMMD), is proposed, which can uniformly optimize the cross-domain joint distribution of category-corresponded subdomains on multiple task-specific layers. Moreover, to deal with the out-of-distribution hard samples, we propose an across-domain data augmentation method called Source-Target Domain Mixing (STDMix) to enhance the robustness of the model, which synthesizes the source domain and target domain into a new domain at a fixed ratio and utilizes information entropy to provide reliable pseudo-labels for samples in the target domain. Experimental results on three public datasets, i.e., Office-31, Digits-five, and Office-Home, show that our proposed method achieves improvements of 0.3%, 1.8%, and 2.7% in average accuracy, respectively. • We propose a simple and effective multi-source unsupervised domain adaptation method for classification. • We propose a metric to align the cross-domain joint distribution of category-corresponded subdomains on multiple task-specific layers. • We propose a cross-domain data augmentation method to handle out- of-distribution hard samples. • Experiment results show that our proposed method achieves state-of-the-art performance on multiple domain adaptation datasets. [ABSTRACT FROM AUTHOR]

As the latest generation of video coding standards, versatile video coding (VVC) introduces several new coding tools in transform coding to concentrate the energy of residual blocks. In this work, we propose a regular multiplier (RM) based hardware architecture for 2D transform that can process different transform sizes and types using a unified architecture. This architecture consists of a 1D row transform, a transpose memory, and a 1D column transform, allowing for processing 16 coefficients per cycle. For the 1D row/column transform, the unified calculation structure consists of 512 regular multipliers supporting various transform sizes and types. For the transpose memory, we design an SRAM-based diagonal storage approach, along with a FIFO for storing the block information. Compared with the state-of-the-art MCM-based design, the proposed RM-based design shows a 30.9% reduction in area while can operate at a higher frequency. [ABSTRACT FROM AUTHOR]

The goal of No-Reference Image Quality Assessment (NR-IQA) is to endow computers with a human-like ability to evaluate an image's quality without comparison to a reference. Current deep learning-based methods mainly work in the spatial domain to measure the quality, heavily rely on semantic information, and less on the degradation of the image itself, struggling to accurately judge the quality of an image in similar scenes. In this paper, we propose a novel degradation priors learning architecture to address the NR-IQA task by leveraging learnable degradation priors, along with semantic features. The multi-task learning strategy is introduced to ensure our model could obtain accurate degradation priors for the NR-IQA task. Extensive experiments on public benchmarks demonstrate that our approach outperforms state-of-the-art solutions. Besides we also collect an additional dataset namely ReD-1K to illustrate the superiority of our approach to judge the image quality in similar scenes. • We propose a novel architecture involving degradation priors and semantic features for NR-IQA. • A multi-task learning framework is proposed for NR-IQA, intergating semantic features and frequency domain degradation features. • We collect a new dataset namely ReD-1K, which consists of 537 pairs of degraded and non-degraded real images. [ABSTRACT FROM AUTHOR]

This paper addresses the issue of noise interference in road defect detection, caused by various environmental factors or acquisition equipment. In this article, we add three different levels of salt & pepper noise to the road defect dataset and propose a novel anti-noise dual-branch network (ADNet). The proposed ADNet leverages two backbone networks equipped with the dual-branch interaction (DI) modules to learn the defect information from noise and clear images for improving noise immunity. Then, the weighted feature representation (WFR) module is designed to extract more context-aware cues from the multi-level feature. Additionally, the region perception unit is proposed, where channel-spatial attention optimization (CSAO) module extracts more defect region information by utilizing the attention mechanism and multi-scale refinement (MR) optimizes the boundary information with the U-Net structure. Extensive experimental results demonstrate that the proposed method outperforms state-of-the-art methods, making it a promising solution for detecting road defects in noisy environments. [ABSTRACT FROM AUTHOR]