Search

Existing crowd counting approaches predominantly perform well on the training-testing protocol. However, due to large style discrepancies not only among images but also within a single image, they suffer from obvious performance degradation when applied to unseen domains. In this paper, we aim to design a generalizable crowd counting framework which is trained on a source domain but can generalize well on the other domains. To reach this, we propose a gated ensemble learning framework. Specifically, we first propose a diverse fine-grained style attention model to help learn discriminative content feature representations, allowing for exploiting diverse features to improve generalization. We then introduce a channel-level binary gating ensemble model, where diverse feature prior, input-dependent guidance and density grade classification constraint are implemented, to optimally select diverse content features to participate in the ensemble, taking advantage of their complementary while avoiding redundancy. Extensive experiments show that our gating ensemble approach achieves superior generalization performance among four public datasets. Codes are publicly available at https://github.com/wdzhao123/DCSL. [ABSTRACT FROM AUTHOR]

Remote sensing data distribution generally exposes the long-tailed characteristic. This will limit the object recognition performance of existing deep models when they are trained with such unbalanced data. In this article, we propose a novel hierarchical distillation framework (HDF) to address the long-tailed object recognition in aerial images. First, we notice that not only student model should learn feature representations from teachers but also teacher models should learn feature representations from each other. Therefore, we build hierarchical teacher-wise distillation (HTWD) to improve the feature representations of the teacher models trained with middle and tail data, which is achieved by distilling the feature representations of the teacher model trained with head data. Second, we notice that the feature representations of the middle and tail classes cannot be effectively distilled from the teacher to the student since too little middle and tail data can be used to learn. Thus, we propose self-calibrated sampling (SCS) learning that enforces the student to strengthen the learning of the middle and tail data, thereby improving the student’ feature learning ability. Extensive experiments on two widely used DOTA and FGSC-23 datasets demonstrate the superior performance of the proposed method compared with state-of-the-art methods. Model and code are publicly available at https://github.com/wdzhao123/T2FTS. [ABSTRACT FROM AUTHOR]

Defocus blur detection (DBD) for natural images is a challenging vision task especially in the presence of homogeneous regions and gradual boundaries. In this paper, we propose a novel image-scale-symmetric cooperative network (IS2CNet) for DBD. On one hand, in the process of image scales from large to small, IS2CNet gradually spreads the recept of image content. Thus, the homogeneous region detection map can be optimized gradually. On the other hand, in the process of image scales from small to large, IS2CNet gradually feels the high-resolution image content, thereby gradually refining transition region detection. In addition, we propose a hierarchical feature integration and bi-directional delivering mechanism to transfer the hierarchical feature of previous image scale network to the input and tail of the current image scale network for guiding the current image scale network to better learn the residual. The proposed approach achieves state-of-the-art performance on existing datasets. Codes and results are available at: https://github.com/wdzhao123/IS2CNet. [ABSTRACT FROM AUTHOR]

Fine-grained object classification (FGOC) focuses on identifying subcategories of objects, which is crucial in military and civilian. Existing FGOC methods primarily focus on high-resolution aerial images, limiting their application on low-resolution (LR) FGOC that is a more realistic setting, especially on resource-constrained satellite devices. It is more challenging to deal with LR FGOC since objects’ details are blurred or missing. Addressing this issue, we make the first attempt to explore LR FGOC and propose a novel pipeline based on two technical insights: 1) feature balance strategy discriminatively integrates super-resolution weak and strong detailed presentations into coarse features of LR aerial images, achieving a feature balance to avoid that the weak detailed presentations are inhibited by the strong ones and 2) iterative interaction mechanism alternately refines feature details of the discriminative ship regions and optimizes the performance of FGOC. Moreover, we build a low-resolution fine-grained object (LFS) dataset to promote further study and evaluation. Extensive experiments on the proposed LFS dataset and the other three object datasets of DOTA, FS23, and HRSC2016 demonstrate that our method outperforms state-of-the-art algorithms. Dataset and code are publicly available at https://github.com/wdzhao123/FBNet. [ABSTRACT FROM AUTHOR]

This article presents a CMOS image sensor (CIS) with column-parallel single-shot compressive sensing (CS) for always-on Internet-of-Things (IoT) application, which achieves an energy efficiency of 51 pJ/pixel, while maintaining high image quality of PSNR > 33.7 dB and SSIM > 0.89. This is enabled by an energy-efficient encoder, which replaces a densely populated CS encoding matrix with a highly sparse pseudo-diagonal one. Since the proposed column-parallel CS encoder can be implemented directly at pixel outputs with an energy-efficient switched-capacitor matrix multiplier, data compression is achieved prior to the pixel digitization, thereby greatly reducing ADC power, data size, and I/O power. The energy efficiency of the image sensor is further improved by using dynamic single-slope ADCs. A prototype VGA image sensor implemented in a 110-nm CMOS process consumes only 0.7 mW at 45 frames/s. The corresponding energy per pixel (51 pJ/pixel) amounts to more than $5\times $ improvement over the previous low-energy benchmark for CS image sensors. [ABSTRACT FROM AUTHOR]

This article presents an enhanced interstage gain error shaping (GES) technique that adopts a digital error feedback (DEF) method to address the truncation error in the prior implementation, which can extend the interstage gain error tolerance by five times. The proposed DEF technique does not introduce additional errors as it operates purely in the digital domain. This article also proposes a first-order passive quantization noise shaping (NS) technique that reduces the input-pair ratio of the two-input-pair comparator by 2.7 times, thus alleviating the noise penalty caused by using a multiple-input-pair comparator. A prototype analog-to-digital converter (ADC) equipped with the proposed techniques in a 40-nm CMOS technology achieves a 77.1-dB signal-to-noise-and-distortion ratio (SNDR) over 6.25-MHz bandwidth while operating at 100 MS/s and consuming 1.38 mW. It achieves a 173.7-dB Schreier figure of merit (FoM). [ABSTRACT FROM AUTHOR]

As any analog-to-digital converter (ADC) with a front-end sample-and-hold (S/H) circuit, successive approximation register (SAR) ADC suffers from a fundamental signal-to-noise ratio (SNR) challenge: its sampling kT/C noise. To satisfy the SNR requirement, the input capacitor size has to be sufficiently large, leading to a great burden for the design of the ADC input driver and reference buffer. This article presents an SAR ADC with a kT/C noise-cancellation technique. It enables the substantial reduction of ADC input capacitor size but without the large kT/C noise penalty. It greatly relaxes the requirement for ADC input driver and reference buffer. Built in 40-nm CMOS, a prototype 13-bit ADC has only 240-fF input capacitance and occupies a small area of 0.005 mm2. Operating at 40 MS/s, it achieves a 69-dB signal-to-noise-and-distortion ratio (SNDR) across the Nyquist frequency band while consuming 591 $\mu \text{W}$ of power. [ABSTRACT FROM AUTHOR]

This article presents a second-order noise-shaping (NS) successive approximation register (SAR) analog-to-digital converter (ADC) with a process, voltage, and temperature (PVT)-robust closed-loop dynamic amplifier. The proposed closed-loop dynamic amplifier combines the merits of closed-loop architecture and dynamic operation, realizing robustness, high accuracy, and high energy-efficiency simultaneously. It is embedded in the loop filter of an NS SAR design, enabling the first fully dynamic NS-SAR ADC that realizes sharp noise transfer function (NTF) while not requiring any gain calibration. Fabricated in 40-nm CMOS technology, the prototype ADC achieves an SNDR of 83.8 dB over a bandwidth of 625 kHz while consuming only $107~\mu \text{W}$. It results in an SNDR-based Schreier figure-of-merit (FoM) of 181.5 dB. [ABSTRACT FROM AUTHOR]

This article presents an incremental two-step capacitance-to-digital converter (CDC) with a time-domain $\Delta \Sigma $ modulator (TD $\Delta \Sigma \text{M}$). Unlike the classic two-step CDCs, this work replaces the operational transconductance amplifier (OTA)-based active- RC integrator by a voltage-controlled oscillator (VCO)-based integrator, which is mostly digital and low-power. Featuring the infinite dc gain and intrinsic quantization in phase domain, this TD $\Delta \Sigma \text{M}$ enables a CDC design achieving 76-dB SNDR while requiring only a first-order loop, and a low oversampling ratio (OSR) of 15. Fabricated in 40-nm CMOS technology, the prototype CDC achieves a resolution of 0.29 fF while dissipating only 0.083 nJ/conversion, which improves the energy efficiency by over two times comparing to the similar performance designs. [ABSTRACT FROM AUTHOR]

Defocus blur detection (DBD) is aimed to estimate the probability of each pixel being in-focus or out-of-focus. This process has been paid considerable attention due to its remarkable potential applications. Accurate differentiation of homogeneous regions and detection of low-contrast focal regions, as well as suppression of background clutter, are challenges associated with DBD. To address these issues, we propose a multi-stream bottom-top-bottom fully convolutional network (BTBNet), which is the first attempt to develop an end-to-end deep network to solve the DBD problems. First, we develop a fully convolutional BTBNet to gradually integrate nearby feature levels of bottom to top and top to bottom. Then, considering that the degree of defocus blur is sensitive to scales, we propose multi-stream BTBNets that handle input images with different scales to improve the performance of DBD. Finally, a cascaded DBD map residual learning architecture is designed to gradually restore finer structures from the small scale to the large scale. To promote further study and evaluation of the DBD models, we construct a new database of 1100 challenging images and their pixel-wise defocus blur annotations. Experimental results on the existing and our new datasets demonstrate that the proposed method achieves significantly better performance than other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Recent state-of-the-art methods on focus region detection (FRD) rely on deep convolutional networks trained with costly pixel-level annotations. In this study, we propose a FRD method that achieves competitive accuracies but only uses easily obtained bounding box annotations. Box-level tags provide important cues of focus regions but lose the boundary delineation of the transition area. A recurrent constraint network (RCN) is introduced for this challenge. In our static training, RCN is jointly trained with a fully convolutional network (FCN) through box-level supervision. The RCN can generate a detailed focus map to locate the boundary of the transition area effectively. In our dynamic training, we iterate between fine-tuning FCN and RCN with the generated pixel-level tags and generate finer new pixel-level tags. To boost the performance further, a guided conditional random field is developed to improve the quality of the generated pixel-level tags. To promote further study of the weakly supervised FRD methods, we construct a new dataset called FocusBox, which consists of 5000 challenging images with bounding box-level labels. Experimental results on existing datasets demonstrate that our method not only yields comparable results than fully supervised counterparts but also achieves a faster speed. [ABSTRACT FROM AUTHOR]

This article presents a capacitively coupled voltage-controlled oscillator (VCO)-based sensor readout featuring a hybrid phase-locked loop (PLL)- $\Delta \Sigma $ modulator structure. It leverages phase-locking and phase-frequency detector (PFD) array to concurrently perform quantization and dynamic element matching (DEM), much-reducing hardware/power compared with the existing VCO-based readouts’ counting scheme. A low-cost in-cell data-weighted averaging (DWA) scheme is presented to enable a highly linear tri-level digital-to-analog converter (DAC). Fabricated in 40-nm CMOS, the prototype readout achieves 78-dB SNDR in 10-kHz bandwidth, consuming 4.68 $\mu \text{W}$ and 0.025-mm2 active area. With 172-dB Schreier figure of merit, its efficiency advances the state-of-the-art VCO-based readouts by $50\times $. [ABSTRACT FROM AUTHOR]

This article presents a reference ripple cancellation technique for high-speed successive approximation register analog-to-digital converters (SAR ADCs) to address the reference voltage settling issue. Unlike prior techniques that aim to minimize the reference ripple, this article proposes a new perspective: it provides an extra path for the full-sized reference ripple to couple to the comparator but with an opposite polarity, so that the effect of the reference ripple is canceled out, thus ensuring an accurate conversion result. To verify the proposed technique, a prototype 10-bit 120-MS/s SAR ADC is fabricated in a 40-nm CMOS process. The proposed ripple cancellation technique improves the signal-to-noise and distortion ratio (SNDR) by 8 dB and reduces the worst case integrated non-linearity (INL)/differential non-linearity (DNL) by ten times. Overall, the ADC achieves an SNDR of 55 dB with only 3-pF reference decoupling capacitor. [ABSTRACT FROM AUTHOR]

This article presents a power-efficient purely voltage-controlled oscillator (VCO)-based second-order continuous-time (CT) $\Delta \Sigma $ analog-to-digital converter (ADC), featuring a modified digital phase-locked loop (DPLL) structure. The proposed ADC combines a VCO with a switched-ring oscillator (SRO)-based time-to-digital converter (TDC), which enables second-order noise shaping without any operational transconductance amplifiers (OTAs). The nonlinearity of the front-end VCO is mitigated by putting it inside a closed loop. An array of phase/frequency detectors (PFDs) is used to relax the requirement on the VCO center frequency and thus reduces the VCO power and noise. The proposed architecture also realizes an intrinsic tri-level data-weighted averaging (DWA). A prototype chip is fabricated in a 40-nm CMOS process. The proposed ADC achieves a peak signal-to-noise-and-distortion ratio (SNDR) of 69.4 dB over 5.2-MHz bandwidth, while operating at the 260 MS/s and consuming 0.86 mW from a 1.1-V supply. [ABSTRACT FROM AUTHOR]

This article presents a two-step analog-to-digital converter (ADC) that operates its first-stage successive approximation register (SAR) ADC in the continuous-time (CT) domain. It avoids the front-end sample-and-hold (S/H) circuit and its associated sampling noise. Hence, the proposed ADC allows the input capacitor size to be substantially reduced without incurring large sampling noise penalty. With input ac coupling, the first-stage CT-SAR can simultaneously perform input tracking and SAR quantization. Its conversion error is minimized by accelerating the SAR speed and providing redundancy. A floating-inverter-based (FIB) dynamic amplifier (DA) is used as the inter-stage amplifier and acts as a low-pass filter for the first-stage residue. To verify the proposed techniques, a 13-bit prototype ADC is built in 40-nm CMOS process. Its input capacitor is only 120 fF, which is over 20 times smaller than what would be needed in a classic Nyquist ADC with the S/H circuit. Operating at 2 MS/s, it achieves 72-dB signal-to-noise-and-distortion-ratio (SNDR) at the Nyquist rate while consuming only 25 $\mu \text{W}$ of power and 0.01 mm2 of area. [ABSTRACT FROM AUTHOR]

Common non-focused areas are often present in multi-focus images due to the limitation of the number of focused images. This factor severely degrades the fusion quality of multi-focus images. To address this problem, we propose a novel end-to-end multi-focus image fusion with a natural enhancement method based on deep convolutional neural network (CNN). Several end-to-end CNN architectures that are specifically adapted to this task are first designed and researched. On the basis of the observation that low-level feature extraction can capture low-frequency content, whereas high-level feature extraction effectively captures high-frequency details, we further combine multi-level outputs such that the most visually distinctive features can be extracted, fused, and enhanced. In addition, the multi-level outputs are simultaneously supervised during training to boost the performance of image fusion and enhancement. Extensive experiments show that the proposed method can deliver superior fusion and enhancement performance than the state-of-the-art methods in the presence of multi-focus images with common non-focused areas, anisotropic blur, and misregistration. [ABSTRACT FROM AUTHOR]

Most existing image fusion methods assume that at least one input image contains high-quality information at any place of an observed scene. Thus, these fusion methods will fail if every input image is degraded. To address this issue, this study proposes a novel fusion framework that integrates image fusion based on spectral total variation (TV) method and image enhancement. For spatially varying multiscale decompositions generated by the spectral TV framework, this study verifies that the decomposition components can be modeled efficiently by tailed $\alpha$-stable-based random variable distribution (TRD) rather than the commonly used Gaussian distribution. Consequently, salience and match measures based on TRD are proposed to fuse each sub-band decomposition. The spatial intensity information is also adopted to fuse the remainder of the image decomposition components. A sub-band adaptive gain function family based on TV spectrum and space variation is constructed for fused multiscale decompositions to enhance fused image simultaneously. Finally, numerous experiments with various multisensor image pairs are conducted to evaluate the proposed method. Experimental results show that even if the input images are degraded, the fused image obtained by the proposed method achieves significant improvement in terms of edge details and contrast while extracting the main features of the input images, thereby achieving better performance compared with the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Medical image fusion aims at integrating information from multimodality medical images to obtain a more complete and accurate description of the same object, which provides an easy access for image-guided medical diagnostic and treatment. Unfortunately, medical images are often corrupted by noise in acquisition or transmission, and the noise signal is easily mistaken for a useful characterization of the image, making the fusion effect drop significantly. Thus, the existence of noise presents a great challenge for most of traditional image fusion methods. To address this problem, an effective variation model for multimodality medical image fusion and denoising is proposed. First, a multiscale alternating sequential filter is exploited to extract the useful characterizations (e.g., details and edges) from noisy input medical images. Then, a recursive filtering-based weight map is constructed to guide the fusion of main features of input images. Additionally, total variation (TV) constraint is developed by constructing an adaptive fractional order $p$ based on the local contrast of fused image, further effectively suppressing noise while avoiding the staircase effect of the TV. The experimental results indicate that the proposed method performs well with both noisy and normal medical images, outperforming conventional methods in terms of fusion quality and noise reduction. [ABSTRACT FROM PUBLISHER]