Search

Image quality assessment (IQA) is an important task of image processing and has diverse applications, such as image super-resolution reconstruction, image transmission and monitoring systems. This paper proposes a perceptual hashing algorithm with complementary color wavelet transform (CCWT) and compressed sensing (CS) for reduced-reference (RR) IQA. The CCWT is exploited to decompose input color image into different sub-bands. Since the calculation of CCWT uses all color channels without discarding any information, the distortions introduced by digital operations on color channels are preserved in the CCWT sub-bands. The block-based CS is used to extract features from the CCWT sub-bands. As the Euclidean distance between the block-based CS features is slightly influenced by content-preserving operations, perceptual features constructed by Euclidean distances are robust, discriminative and compact. Hash sequence is finally determined by quantifying the perceptual features. Effectiveness of the proposed hashing is verified by various experiments on four open image databases. Experimental results demonstrate that the proposed hashing is superior to some state-of-the-art algorithms in terms of classification and RR IQA application. [ABSTRACT FROM AUTHOR]

In this paper, we conduct an extensive study on the use of pre-trained convolutional neural networks (CNNs) for omnidirectional image quality assessment (IQA). To cope with the lack of available IQA databases, transfer learning from seven pre-trained CNN models is investigated over retraining on standard 2D databases. In addition, we explore the influence of various image representations and training strategies on the model’s performance. A comparison of the use of projected versus radial content, and multichannel CNN versus patch-wise training is also covered. The experimental results on two publicly available databases are used to draw conclusions about which strategy best fits the visual quality prediction and at which computational cost. The analysis shows that retraining CNN models on 2D IQA databases improves the prediction accuracy. The latter and the required computational time are found to be significantly affected by the training strategy. Cross-database evaluations demonstrate that the nature and variety of the content impact the generalization ability of the models. Finally, we show that conclusions coming from other image processing communities may not hold for IQA. The provided discussion shall provide insights and recommendations when using pre-trained CNNs for omnidirectional IQA. [ABSTRACT FROM AUTHOR]

Even if Application-Specific Integrated Circuits (ASIC) have proven to be a relevant choice for integrating inference at the edge, they are often limited in terms of applicability. In this paper, we demonstrate that an ASIC neural network accelerator dedicated to image processing can be applied to multiple tasks of different levels: image classification and compression, while requiring a very limited hardware. The key component is a reconfigurable, mixed-precision (3b/2b/1b) encoder that takes advantage of proper weight and activation quantizations combined with convolutional layer structural pruning to lower hardware-related constraints (memory and computing). We introduce an automatic adaptation of linear symmetric quantizer scaling factors to perform quantized levels equalization, aiming at stabilizing quinary and ternary weights training. In addition, a proposed layer-shared Bit-Shift Normalization significantly simplifies the implementation of the hardware-expensive Batch Normalization. For a specific configuration in which the encoder design only requires 1Mb, the classification accuracy reaches 87.5% on CIFAR-10. Besides, we also show that this quantized encoder can be used to compress image patch-by-patch while the reconstruction can performed remotely, by a dedicated full-frame decoder. This solution typically enables an end-to-end compression almost without any block artifacts, outperforming patch-based state-of-the-art techniques employing a patch-constant bitrate. [ABSTRACT FROM AUTHOR]

As two practical and important image processing tasks, color demosaicking (CDM) and super-resolution (SR) have been studied for decades. However, most literature studies these two tasks independently, ignoring the potential benefits of a joint solution. In this paper, aiming at efficient and effective joint demosaicking and super-resolution (JDSR), a well-designed two-stage convolutional neural network (CNN) architecture is proposed. For the first stage, by making use of the sampling-pattern information, a pattern-aware feature extraction (PFE) module extracts features directly from the Bayer-sampled low-resolution (LR) image, while keeping the resolution of the extracted features the same as the input. For the second stage, a dual-branch feature refinement (DFR) module effectively decomposes the features into two components with different spatial frequencies, on which different learning strategies are applied. On each branch of the DFR module, the feature refinement unit, namely, densely-connected dual-path enhancement blocks (DDEB), establishes a sophisticated nonlinear mapping from the LR space to the high-resolution (HR) space. To achieve strong representational power, two paths of transformations and the channel attention mechanism are adopted in DDEB. Extensive experiments demonstrate that the proposed method is superior to the sequential combination of state-of-the-art (SOTA) CDM and SR methods. Moreover, with much smaller model size, our approach also surpasses other SOTA JDSR methods. [ABSTRACT FROM AUTHOR]

Image classification is a fundamental artificial intelligence task that labels images into one of some predefined classes. However, training complex image classification models requires a large amount of computation resources and data in order to reach state-of-the-art performance. This demand drives the growth of distributed deep learning, where multiple agents cooperatively train global models with their individual datasets. Among such learning systems, decentralized learning is particularly attractive, as it can improve the efficiency and fault tolerance by eliminating the centralized parameter server, which could be the single point of failure or performance bottleneck. Although the agents do not need to disclose their training image samples, they exchange parameters with each other at each iteration, which can put them at the risk of data privacy leakage. Past works demonstrated the possibility of recovering training images from the exchanged parameters. One common defense direction is to adopt Differential Privacy (DP) to secure the optimization algorithms such as Stochastic Gradient Descent (SGD). Those DP-based methods mainly focus on standalone systems, or centralized distributed learning. How to enforce and optimize DP protection in decentralized learning systems is unknown and challenging, due to their complex communication topologies and distinct learning characteristics. In this paper, we design TOP- DP, a novel solution to optimize the differential privacy protection of decentralized image classification systems. The key insight of our solution is to leverage the unique features of decentralized communication topologies to reduce the noise scale and improve the model usability. (1) We enhance the DP-SGD algorithm with this topology-aware noise reduction strategy, and integrate the time-aware noise decay technique. (2) We design two novel learning protocols (synchronous and asynchronous) to protect systems with different network connectivities and topologies. We formally analyze and prove the DP requirement of our proposed solutions. Experimental evaluations demonstrate that our solution achieves a better trade-off between usability and privacy than prior works. To the best of our knowledge, this is the first DP optimization work from the perspective of network topologies. [ABSTRACT FROM AUTHOR]

Image quality assessment (IQA) plays a central role in many image processing algorithms and systems. Although many popular IQA models achieves high performance on existing released databases, they are still not well accepted in practical applications due to the not-always satisfactory accuracy on real-world data and situations. In this paper, we revisit the IQA research, and point out an ignored but interesting problem in IQA: the coarse-grained (i.e., when quality variation is sufficiently big, as the setting of most IQA databases up to date) statistical results evaluated on existing databases mask the fine-grained differentiation. Accordingly, we present a survey on image quality assessment from a new perspective: fine-grained image quality assessment (FG-IQA). Recent FG-IQA research on five major kinds of images is introduced, and some popular IQA methods are analyzed from FG-IQA perspective. The potential problems for current IQA research based on existing coarse-grained databases are analyzed and the necessity of more FG-IQA research is justified. Finally, we discuss some challenges and possible directions for future works in FG-IQA. [ABSTRACT FROM AUTHOR]

As an efficient watermark attack method, geometric distortions destroy the synchronization between the watermark encoder and decoder. Local geometric distortion is a considerable challenge in the watermarking field. Although many geometric distortion resilient watermarking schemes have been proposed, few perform well against local geometric distortions, such as random bending attacks (RBAs). To address this problem, this paper proposes a novel watermark synchronization process and a corresponding watermarking scheme. In our scheme, the watermark bits are represented by random patterns. The message is encoded to obtain a watermark unit, and the watermark unit is flipped to generate a symmetrical watermark. Then, the symmetrical watermark is additively embedded into the spatial domain of the host image. In watermark extraction, we first obtain the theoretical mean-square error minimized estimation of the watermark. Then, an autoconvolution function is applied to this estimation to detect the symmetry and obtain a watermark unit map. According to this map, the watermark can be accurately synchronized, and then extraction can be performed. Experimental results demonstrate the excellent robustness of the proposed watermarking scheme to local geometric distortions, global geometric distortions, common image processing operations, and some kinds of combined attacks. [ABSTRACT FROM AUTHOR]

Color demosaicing is an essential step in the camera image processing pipeline, especially on hardware platforms for real-time video applications. Though many demosaicing algorithms have been proposed in the past two decades, there remains a substantial gap between industrial needs and academic research. On one hand, industry requires a high perceptual quality, artifact-free, and low-cost demosaicing algorithm that is noniterative and small window based ready to be implemented on a hardware platform with a limited line buffer. On the other hand, academia is targeting high PSNR/SSIM, with the computation cost and line buffer receiving second priority, and often a frame buffer and iterative operation are used. The cost, large line buffer and iteration requirement make most existing demosaicing algorithms inapplicable on hardware platforms. In this paper, we introduce a novel low-cost demosaicing algorithm to narrow the gap. We keep the operation window size and computation cost as the first priority and achieve both high PSNR/SSIM and visual perceptual quality compared to previous state-of-the-art methods on standard test datasets. We fully investigate the a priori knowledge of natural scene raw images and find several key cues that are beneficial to demosaicing. Our demosaicing algorithm is a smart fusion of these useful cues. Furthermore, we solve typical demosaicing issues that occur in many traditional methods, including false color artifacts, T-section closing, and zippering artifacts. The proposed method has no learning stage, no iteration operation, a small line buffer and a limited number of parameters, so it can easily be applied to hardware platforms. [ABSTRACT FROM AUTHOR]

Watermarking neural networks is a quite important means to protect the intellectual property (IP) of neural networks. In this paper, we introduce a novel digital watermarking framework suitable for deep neural networks that output images as the results, in which any image outputted from a watermarked neural network must contain a certain watermark. Here, the host neural network to be protected and a watermark-extraction network are trained together, so that, by optimizing a combined loss function, the trained neural network can accomplish the original task while embedding a watermark into the outputted images. This work is totally different from previous schemes carrying a watermark by network weights or classification labels of the trigger set. By detecting watermarks in the outputted images, this technique can be adopted to identify the ownership of the host network and find whether an image is generated from a certain neural network or not. We demonstrate that this technique is effective and robust on a variety of image processing tasks, including image colorization, super-resolution, image editing, semantic segmentation and so on. [ABSTRACT FROM AUTHOR]