Search

In this paper, we propose a spatial–spectral feature fusion model with a predictive feature weighting mechanism and demonstrate its applications to the problems of hyperspectral image classification and segmentation. To address these problems, we learn a set of 1-D convolutional local spectral filters and 2-D spatial–spectral filters that feed features into a fusion module, in an end-to-end fashion. We propose a lightweight predictive feature weighting component embedded in the fusion model and consider four design fusion options, i.e., by adding or concatenating features with equal or predicted weights. For the pixel classification task, the training input consists of image patches with labeled central pixels, whereas for the spatial segmentation task, it includes the label maps of image regions. The proposed networks have been evaluated on the Indian Pines, Pavia University, and Houston University for the classification problem and the SpaceNet data set for the spatial segmentation problem. The quantitative results favor the proposed approach over the state-of-the-art methods across all the four data sets. [ABSTRACT FROM AUTHOR]

This paper presents an adaptive image retargeting approach using saliency-based continuous seam carving to efficiently display images on small screens. A multiscale contrast-based saliency map is first generated and used as the energy map for conventional discrete seam carving, and a reasonable number of seams are adaptively extracted. Then the reduced dimensions allocated to continuous seam carving (CSC) and possible scaling are determined by the analysis of the energy curve of the extracted seams. The proposed CSC assigns to each extracted seam a reserving ratio in accordance with the corresponding seam energy and the reduced dimension, and the retargeted image is generated using an efficient scheme of mapping and resampling based on the reserving ratio map. Experimental results demonstrate better retargeting performance of the proposed approach under different reduction ratios. [ABSTRACT FROM AUTHOR]

Video caption aims to generate descriptive sentences about the video, and the most critical problem is how to achieve accurate word prediction with standardized and coherent syntax structure, which requires the model to thoroughly understand video content and precisely map them into corresponding sentence components. Many existing methods usually fuse different video features into a single visual feature for generating sentences. However, they ignore the word dataset prior information in the annotations (such as Part-Of-Speech) and they also ignore the association between sentence components and types of visual features. To solve these problems, we propose a POS-trends dynamic-aware model (PDA) to fully exploit the word dataset prior information in the captions to predict POS tag, so as to assist generating captions. We propose a POS feature extraction (PFE) module to use different filters to extract different POS-trends features, predict POS tags and fuse visual features. Furthermore, we propose a visual-dynamic-aware (VDA) module to dynamically adjust the mapping way of words and supplement the visual information into the local features. The fusion features provide directional visual information to generate correct words, and the predicted POS tags to guide the decoding process to generate a more standardized and coherent syntax structure. A large number of experiments based on MSVD, MSR-VTT and VATEX demonstrated that our method outperforms the state-of-the-art methods in BLEU-4, ROUGE-L, METEOR, CIDEr. Code can be available at: https://github.com/WangLanxiao/PDA-for-video-caption. [ABSTRACT FROM AUTHOR]

Instance segmentation has made great progress in recent years. However, current mainstream detection-based methods ignore the process of distinguishing different instances in the same detected region, making it hard to segment the correct instance when a detected region contains multiple instances. To address this problem, we propose an Instance-level Context Attention Network (ICANet) to generate more discriminative features for different instances based on the context from their respective instance scopes, called instance-level context. Specifically, given a detected region, we first propose an instance attention module to obtain the specific attention maps that focus on the relationships between pixel pairs from the same instance by learning an embedding space. With this type of attention map, the features from the same instance achieve mutual gains, while the features from different instances become more discriminative. Then, we propose a spatial attention module to incorporate spatial information and enhance feature representations based on both feature and spatial relations. Moreover, to obtain clearer attention maps, we further propose a weight clipping strategy to filter out the noise by cutting off the lower weight. We perform extensive experiments to verify the effectiveness of the proposed method. As reported in the results, our method steadily outperforms the baseline by over 1.5% on the COCO dataset using different backbones and 3.7% on the Cityscapes dataset, which demonstrates the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Class Activation Maps (CAMs) visualize the pixels within an image that contribute to classify the image to a certain category, and can be used to localize the object regions from images that benefit to many tasks such as image segmentation and object detection. However, the object regions highlighted by the existing methods are usually small and local. We believe such drawback lies in the image-level labels, because the the image-level label prefers the network to capture the common regions intra-class and the discriminative region inter-class. Due to the variations of objects intra-class and the common region widely shared inter class, the regions based on image-level labels are usually small and local. Based on such observation, we propose a new strategy to use a set of lower-level labels called component labels to replace the image-level labels. The advantage is that the component regions have small feature variation intra-class and are non-overlapping inter-class, which leads to better CAMs generation through their simple combination. Furthermore, since the component labels are also shared by unknown classes, the proposed CAMs generation method can be easily extended to unknown classes, which facilitate the improvement of tasks related to new class processing, such as few-shot segmentation and detection. Specifically, the component labels are set based on the WordNet hierarchy firstly, which can also provides the relationships of classes. Besides, graph convolution networks (GCNs) are used as the classifiers, which can exactly describe not only the component, but also their structural relationships. Based on the component features, a feature fusion module is also designed to merge local component features into the global feature. Better CAM is finally obtained. The experiment section shows the effectiveness of our component labels in terms of better subjective and objective results compared with the existing CAM generation methods. Furthermore, we also show good generalization of our component label on unknown classes. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a novel reduced-reference (RR) image quality assessment (IQA) is proposed by depicting the subband statistical characteristics in the reorganized discrete cosine transform (RDCT) domain. First, the block-based DCT coefficients are reorganized into a three-level coefficient tree, resulting in ten RDCT subbands. For the intra RDCT subband characteristic, the coefficient distribution of each RDCT subband is modeled by the generalized Gaussian density (GGD) function. The city-block distance (CBD) is employed to measure the modeling error between the actual distribution and the fitted GGD curve. For the inter RDCT subband characteristic, the mutual information (MI) is utilized to depict the dependencies between coefficient pairs in related RDCT subbands. Moreover, a frequency ratio descriptor (FRD) calculated in the RDCT domain is proposed to depict how the image energy distributes among different frequency components. The FRD values computed from both the reference and distorted images are jointly considered to derive a novel mutual masking strategy for simulating the texture masking property of the human visual system (HVS). By considering the GGD modeling of intra RDCT subband, MI of inter RDCT subbands, and FRD of the image, the proposed RR IQA is developed. Experimental results demonstrate that a small number of RR features is sufficient to represent the reference image for the perceptual quality analysis. The proposed method can outperform the state-of-the-art RR IQAs, and even the full-reference (FR) PSNR and SSIM. [Copyright &y& Elsevier]

Abstract: In this paper, we propose an automatic human body segmentation system which mainly consists of human body detection and object segmentation. Firstly, an automatic human body detector is designed to provide hard constraints on the object and background for segmentation. And a coarse-to-fine segmentation strategy is employed to deal with the situation of partly detected object. Secondly, background contrast removal (BCR) and self-adaptive initialization level set (SAILS) are proposed to solve the tough segmentation problems of the high contrast at object boundary and/or similar colors existing in the object and background. Finally, an object updating scheme is proposed to detect and segment new object when it appears in the scene. Experimental results demonstrate that our body segmentation system works very well in the live video and standard sequences with complex background. [Copyright &y& Elsevier]

Abstract: AVS1-P2 is the newest video standard of Audio Video coding Standard (AVS) workgroup of China, which provides close performance to H.264/AVC main profile with lower complexity. In this paper, a platform-independent software package with macroblock-based (MB-based) architecture is proposed to facilitate AVS video standard implementation on embedded system. Compared with the frame-based architecture, which is commonly utilized for PC platform oriented video applications, the MB-based decoder performs all of the decoding processes, except the high-level syntax parsing, in a set of MB-based buffers with adequate size for saving the information of the current MB and the neighboring reference MBs to minimize the on-chip memory and to save the time consumed in on-chip/off-chip data transfer. By modifying the data flow and decoding hierarchy, simulating the data transfer between the on-chip memory and the off-chip memory, and modularizing the buffer definition and management for low-level decoding kernels, the MB-based system architecture provides over 80% reduction in on-chip memory compared to the frame-based architecture when decoding 720p sequences. The storage complexity is also analyzed by referencing the performance evaluation of the MB-based decoder. The MB-based decoder implementation provides an efficient reference to facilitate development of AVS applications on embedded system. The complexity analysis provides rough storage complexity requirements for AVS video standard implementation and optimization. [Copyright &y& Elsevier]

Abstract: Compressed video bit stream is very sensitive to transmission errors, due to error propagation resulting from variable-length coding, differential coding, and motion compensated prediction. There exist many error-resilient techniques that can significantly improve the reconstructed video quality by reducing the spatial and temporal error propagation. In this paper, an error-resilient coding scheme that can limit the spatial error propagation with little overhead is proposed. It disables intra AC prediction and groups appropriate fixed length coded (FLC) syntaxes in a video packet (or slice) together to form a new partition. With intra AC prediction disabled, errors occurred in these FLC syntaxes will not cause spatial error propagation. This error resilient coding scheme essentially classifies the syntaxes into two categories according to whether that syntax will cause spatial error propagation when an error occurs. We evaluate the efficiency of the error-resilient coding scheme by comparing the reconstructed PSNR of the proposed coding scheme with that of the MPEG-4 coding scheme over the AWGN channel and the Rayleigh channel. Depending on the video sequence, channel coding rate and channel condition, our proposed data-partitioning scheme can achieve up to 1.8dB PSNR gain. [Copyright &y& Elsevier]

Abstract: In this paper, we study joint source channel coding for bitplane based video coding over wireless channels. We consider using frame-level intra-mode to stop error propagation and using unequal error protection (UEP) to combat channel errors. Our focus is on how to optimally select coding modes and find UEP solutions for bitplane based video coding. In particular, we propose an overall end-to-end rate–distortion (R–D) function, which considers not only the source distortion and the channel distortion introduced in the current frame but also the propagated channel distortion from the previous frames. Based on this end-to-end R–D function, we are able to find the optimal solutions for both mode selection and UEP so that an optimal tradeoff between efficiency and robustness can be achieved. Experimental results demonstrate the significant performance gain. [Copyright &y& Elsevier]

Abstract: It is a challenging problem to design an efficient real-time video transport system for the present best-effort Internet because of the strict bandwidth, loss and time constraints and the lack of quality of service (QoS) guarantees from the present IP networks. This paper presents an end-to-end system architecture based on the MPEG-4 Delivery Multimedia Integration Framework (DMIF) for transporting real-time live video over the Internet. The key features of the system include: (1) MPEG-4 DMIF-based terminal architecture; (2) combination of an end-to-end feedback control mechanism and a rate-adaptive encoding algorithm for the best use of the Internet; (3) a robust and efficient packetization scheme for the MPEG-4 bit-stream by using the resynchronization marker approach specified in the MPEG-4 standard; and (4) efficient error control algorithms adopted at the end systems for visual quality enhancement. Simulation and test results using the actual Internet showed that our system is capable of utilizing the available network resource and achieve good perceptual quality at the application level. [Copyright &y& Elsevier]

Object proposals are used in two-stage detectors, such as R-CNN, to generate detection results, including category predictions and refined bounding-boxes. As a result, classification scores are assigned to refined bounding-boxes rather than object proposals. However, this procedure ignores the discrepancy of data distribution between object proposals and refined bounding-boxes. We consider this discrepancy could limit the detection accuracy. Specifically, the foreground/background imbalance on object proposals and inaccurate information from low-IoU proposals could hinder the category prediction. In this paper, we propose a detector called the Multi-Path Detection Calibration Network (PDC-Net) to address this problem. The key idea behind PDC-Net is calibrating detection results from R-CNN by considering the statistical discrepancy between object proposals and refined bounding-boxes. PDC-Net is built on Faster R-CNN. The core component in PDC-Net is the multi-path detection head, in which the base detector (from Faster R-CNN) generates detection results from object proposals and multiple calibration detectors fix incorrect outputs from the base detector using refined bounding-boxes. Experiments reveal that PDC-Net can boost detection results. Our method could reach 83.1% and 43.3% mAP respectively on PASCAL VOC and MSCOCO benchmarks, which is comparable to several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Rectified activation units make an important contribution to the success of deep neural networks in many computer vision tasks. In this paper, we propose a Parametric Deformable Exponential Linear Unit (PDELU) and theoretically verify its effectiveness for improving the convergence speed of learning procedure. By means of flexible map shape, the proposed PDELU could push the mean value of activation responses closer to zero, which ensures the steepest descent in training a deep neural network. We verify the effectiveness of the proposed method in the image classification task. Extensive experiments on three classical databases (i.e., CIFAR-10, CIFAR-100, and ImageNet-2015) indicate that the proposed method leads to higher convergence speed and better accuracy when it is embedded into different CNN architectures (i.e., NIN, ResNet, WRN, and DenseNet). Meanwhile, the proposed PDELU outperforms many existing shape-specific activation functions (i.e., Maxout, ReLU, LeakyReLU, ELU, SELU, SoftPlus, Swish) and the shape-adaptive activation functions (i.e., APL, PReLU, MPELU, FReLU). [ABSTRACT FROM AUTHOR]

Thanks to its storage and computation efficiency, hashing as a kind of nearest neighbor search method can facilitate massive data processing in recent vision and learning studies. Particularly, deep supervised hashing methods have significantly improved the retrieval performance compared with non-deep supervised hashing methods. However, most existing deep supervised hashing methods approximate the similarity between two images with the hamming distance between the outputs of the same hash function, i.e., the symmetric strategy. Consequently, it is typically time-consuming to train these symmetric hashing methods, and these methods can hardly take full advantage of the supervised information in the large-scale database. In this paper, we propose a novel discriminative deep m etric learning approach for a symmetric d iscrete h ashing (ADMH) approach for supervised hashing learning. ADMH integrates an asymmetric strategy with a deep metric learning method to learn the hash function for the query images and the discrete hash codes for database images directly. More specifically, we train a deep neural network to extract the features of the query images. Subsequently, we introduce a metric learning scheme for learning the discrete discriminative hash codes of the database images directly. Finally, the feature learning procedure for generating the hash codes of the query images and the discrete coding procedure for generating the hash codes of the database images are integrated into an end-to-end learning framework. Extensive experiments on various benchmark datasets show that the proposed asymmetric deep hashing method outperforms the existing hashing methods. [ABSTRACT FROM AUTHOR]

Head detection plays an important role in localizing and identifying persons from visual data. Most existing methods treat head detection as a specific form of object detection. Head detection is nontrivial due to the considerable difficulty in building the local and global information under conditions of unconstrained pose and orientation. To address these issues, this paper presents an effective adaptive relational network to capture context information, which is greatly helpful to suppress missed detection. We show that the fundamental contextual properties, such as the global shape priors from different heads and the local adjacent relationship between the head and shoulders, can be systematically quantified by visual operators. Specifically, we propose a two-step search algorithm to quantify the global intergroup conflict with adaptive scale, pose and viewpoint. Meanwhile, a structured feature module is introduced to capture the local relation of intraindividual stability. Finally, the global priors and local relation are integrated seamlessly into a single-stage head detector that is end-to-end trainable. An extensive ablation analysis demonstrates the effectiveness of our approach. We achieve state-of-the-art results on two challenging datasets, i.e., HollywoodHeads and Brainwash. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles are an essential component in the realization of Industry 4.0. With drones helping to improve industrial safety and efficiency in utilities, construction, and communication, there is an urgent need for drone-based intelligent applications. In this paper, we develop a unified framework to simultaneously detect and count vehicles from drone images. We first explore why the state-of-the-art detectors fail in highly dense drone scenes, which provides more appropriate insights. Then, we propose an effective loss to push the anchors toward matching the ground-truth boxes as much as possible, specifically designed for scale-adaptive anchor generation. Inspired by attention mechanisms in the human visual system, we maximize the mutual information between object classes and features by combining bottom-up cues with top-down attention mechanisms specifically designed for feature extraction. Finally, we build a counting layer with a regularized constraint related to the number of vehicles. Extensive experiments demonstrate the effectiveness of our approach. For both tasks, our proposed method achieves state-of-the-art results on all four challenging datasets. In particular, our results reduce error by a larger factor than previous methods. [ABSTRACT FROM AUTHOR]

The ultra high-definition (UHD) video format, which has recently become popular, aims to provide high spatial resolution, high temporal frame rate, high sample bit-depth, and wide pixel color gamut. Despite the continued development of global network capacities, it inevitably causes the increased bandwidth cost of catering to the requirement of delivering UHD video services. To address such challenges, this paper presents an improved super coding unit (SCU) method for UHD video coding in High Efficiency Video Coding (HEVC). Initially, the medium coding unit (MCU) is proposed to avoid unnecessary brute-force coding unit (CU) partitions of SCU. Furthermore, the SCU is proposed to be encoded by Direct-MCU and SCU-to-MCU modes: the Direct-MCU mode is intended to better adapt to the texture-rich region, which guarantees the compression efficiency by avoiding extra-size CU partition; the SCU-to-MCU mode is designed for the homogeneous region of UHD content, which saves the encoding time by skipping fine-grained CU partition search. Moreover, a learning-based fast SCU decision approach is proposed to speed up the determination process of Direct-MCU and SCU-to-MCU, where three representative handcrafted features are extracted. Experimental results show that our method achieves an affordable complexity and excellent coding efficiency (up to 7.30% Bjøntegaard Delta rate savings) in UHD video coding compared to recent HEVC reference software. [ABSTRACT FROM AUTHOR]

Referring Image Segmentation (RIS) aims to extract the object or stuff from an image according to the given natural language expression. As a representative multi-modal reasoning task, the main challenge of RIS lies in accurately understanding and aligning two types of heterogeneous data (i.e. image and text). Existing methods typically conduct this task via inexplicit cross-modal fusion toward the visual and linguistic features, which are separately extracted from different encoders and hard to capture accurate image–text alignment information due to their distinct latent representation structures. In this paper, we propose a Dual-Graph Hierarchical Interaction Network (DGHIN) to facilitate the explicit and comprehensive alignment between the image and text data. Firstly, two graphs are separately built for the initial visual and linguistic features extracted with different pre-trained encoders. By means of graph reasoning, we obtain a unified representation structure for different modalities to capture the intra-modal entities and their contexts, where each projected node incorporates the long-range dependencies into the latent representation. Then, the Hierarchical Interaction Module (HIM) is applied to the visual and linguistic graphs to extract comprehensive inter-modal correlations from the entity level and graph level, which not only capture the corresponding keywords and visual patches but also draws the whole sentence closer to the image region with the consistent context in the latent space. Extensive experiments on RefCOCO, RefCOCO+, G-Ref, and ReferIt demonstrate that the proposed DGHIN outperforms many state-of-the-art methods. Code is available at https://github.com/ZhaofengSHI/referring-DGHIN. • We propose a novel "part-unity-whole" paradigm for RIS. • We develop a DGHIN consisting of the DGM and a HIM. • DGM represents the image/text with a unified node and edge structure. • HIM captures comprehensive inter-modal correlations across the entity and graph level. • Experiments show that the proposed method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

In this paper, we propose a generative framework that unifies depth-based 3D facial pose tracking and face model adaptation on-the-fly, in the unconstrained scenarios with heavy occlusions and arbitrary facial expression variations. Specifically, we introduce a statistical 3D morphable model that flexibly describes the distribution of points on the surface of the face model, with an efficient switchable online adaptation that gradually captures the identity of the tracked subject and rapidly constructs a suitable face model when the subject changes. Moreover, unlike prior art that employed ICP-based facial pose estimation, to improve robustness to occlusions, we propose a ray visibility constraint that regularizes the pose based on the face model's visibility with respect to the input point cloud. Ablation studies and experimental results on Biwi and ICT-3DHP datasets demonstrate that the proposed framework is effective and outperforms completing state-of-the-art depth-based methods. [ABSTRACT FROM AUTHOR]

• A novel 3D face reconstruction framework which is robust to expression and pose changes. • A high level feature, which has robustness to lighting variations. • we provide 3DMM parameters and 3D meshes for a 3D face database. This paper proposes a novel model fitting algorithm for 3D facial expression reconstruction from a single image. Face expression reconstruction from a single image is a challenging task in computer vision. Most state-of-the-art methods fit the input image to a 3D Morphable Model (3DMM). These methods need to solve a stochastic problem and cannot deal with expression and pose variations. To solve this problem, we adopt a 3D face expression model and use a combined feature which is robust to scale, rotation and different lighting conditions. The proposed method applies a cascaded regression framework to estimate parameters for the 3DMM. 2D landmarks are detected and used to initialize the 3D shape and mapping matrices. In each iteration, residues between the current 3DMM parameters and the ground truth are estimated and then used to update the 3D shapes. The mapping matrices are also calculated based on the updated shapes and 2D landmarks. HOG features of the local patches and displacements between 3D landmark projections and 2D landmarks are exploited. Compared with existing methods, the proposed method is robust to expression and pose changes and can reconstruct higher fidelity 3D face shape. [ABSTRACT FROM AUTHOR]

The generation of object proposals plays an important role in object detection. Most existing methods produce object proposals by using bottom-up cues, such as closed contour or superpixel. In this paper, we propose a novel method to improve the ranking of object proposals by combining bottom-up cues with top-down information of objectivity. Firstly, we utilize the bottom-up method to generate initial object proposals of the given test image. Then we retrieve its top-k similar images from training images set. Considering both appearance and spatial similarity between initial object proposals and the ground truth bounding boxes of these top-k similar images, we obtain the top-down guided scores of initial object proposals. Finally, the refined score of each initial object proposal is modeled as a fusion of the bottom-up score and the top-down score. Experiments show that our method achieves better performance compared with the state-of-art on the Pascal VOC2007 dataset. [ABSTRACT FROM AUTHOR]

A multi-layer authentication scheme for HEVC compressed video is proposed. The combination of CU sizes, which is unique to HEVC and sensitive to video manipulation, is considered along with other elements in the HEVC coding standard to generate the authentication tag. Temporal dependency was enforced, where the authentication tag generated in one slice is embedded into its subsequent slice. By design, the authentication tag is repeatedly but selectively embedded into various elements in a HEVC video, including nonzero DCT coefficients, QP parameter values, and prediction modes, depending on the bit segment in the generated tag. Our scheme offers three layers of authentication to detect and localize the tampered regions in a HEVC video, as well as verifying the source/sender of the video using a shared secret key. Video sequences from various classes (resolutions) are considered to verify the performance of the proposed multi-layer authentication scheme. Results show that, at the expense of slight degradation in perceptual quality, the proposed scheme is robust against several common attacks. A functional comparison is performed between the proposed multi-layer authentication scheme and the conventional schemes. [ABSTRACT FROM AUTHOR]

The general purpose no reference image quality assessment (NR-IQA) is a challenging task, which faces two hurdles: (1) it is difficult to develop one quality aware feature which works well across different types of distortion and (2) it is hard to learn a regression model to approximate a complex distribution for all training samples in the feature space. In this paper, we propose a novel NR-IQA method that addresses these problems by introducing the multi-domain structural information and piecewise regression. The main motivation of our method is based on two points. Firstly, we develop a new local image representation which extracts the structural image information from both the spatial-frequency and spatial domains. This multi-domain description could better capture human vision property. By combining our local features with a complementary global feature, the discriminative power of each single feature could be further improved. Secondly, we develop an efficient piecewise regression method to capture the local distribution of the feature space. Instead of minimizing the fitting error for all training samples, we train the specific prediction model for each query image by adaptive online learning, which focuses on approximating the distribution of the current test image’s k -nearest neighbor (KNN). Experimental results on three benchmark IQA databases (i.e., LIVE II, TID2008 and CSIQ) show that the proposed method outperforms many representative NR-IQA algorithms. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a novel method is proposed to detect salient regions in images. To measure pixel-level saliency, joint spatial-color constraint is defined, i.e., spatial constraint (SC), color double-opponent (CD) constraint and similarity distribution (SD) constraint. The SC constraint is designed to produce global contrast with ability to distinguish the difference between “center and surround”. The CD constraint is introduced to extract intensive contrast of red-green and blue-yellow double opponency. The SD constraint is developed to detect the salient object and its background. A two-layer structure is adopted to merge the SC, CD and SD saliency into a saliency map. In order to obtain a consistent saliency map, the region-based saliency detection is performed by incorporating a multi-scale segmentation technique. The proposed method is evaluated on two image datasets. Experimental results show that the proposed method outperforms the state-of-the-art methods on salient region detection as well as human fixation prediction. [Copyright &y& Elsevier]

Recently, contrastive learning based on augmentation invariance and instance discrimination has made great achievements, owing to its excellent ability to learn beneficial representations without any manual annotations. However, the natural similarity among instances conflicts with instance discrimination which treats each instance as a unique individual. In order to explore the natural relationship among instances and integrate it into contrastive learning, we propose a novel approach in this paper, Relationship Alignment (RA for abbreviation), which forces different augmented views of current batch instances to main a consistent relationship with other instances. In order to perform RA effectively in existing contrastive learning framework, we design an alternating optimization algorithm where the relationship exploration step and alignment step are optimized respectively. In addition, we add an equilibrium constraint for RA to avoid the degenerate solution, and introduce the expansion handler to make it approximately satisfied in practice. In order to better capture the complex relationship among instances, we additionally propose Multi-Dimensional Relationship Alignment (MDRA for abbreviation), which aims to explore the relationship from multiple dimensions. In practice, we decompose the final high-dimensional feature space into a cartesian product of several low-dimensional subspaces and perform RA in each subspace respectively. We validate the effectiveness of our approach on multiple self-supervised learning benchmarks and get consistent improvements compared with current popular contrastive learning methods. On the most commonly used ImageNet linear evaluation protocol, our RA obtains significant improvements over other methods, our MDRA gets further improvements based on RA to achieve the best performance. The source code of our approach will be released soon.

Due to the lack of natural scene and haze prior information, it is greatly challenging to completely remove the haze from a single image without distorting its visual content. Fortunately, the real-world haze usually presents non-homogeneous distribution, which provides us with many valuable clues in partial well-preserved regions. In this paper, we propose a Non-Homogeneous Haze Removal Network (NHRN) via artificial scene prior and bidimensional graph reasoning. Firstly, we employ the gamma correction iteratively to simulate artificial multiple shots under different exposure conditions, whose haze degrees are different and enrich the underlying scene prior. Secondly, beyond utilizing the local neighboring relationship, we build a bidimensional graph reasoning module to conduct non-local filtering in the spatial and channel dimensions of feature maps, which models their long-range dependency and propagates the natural scene prior between the well-preserved nodes and the nodes contaminated by haze. To the best of our knowledge, this is the first exploration to remove non-homogeneous haze via the graph reasoning based framework. We evaluate our method on different benchmark datasets. The results demonstrate that our method achieves superior performance over many state-of-the-art algorithms for both the single image dehazing and hazy image understanding tasks. The source code of the proposed NHRN is available on https://github.com/whrws/NHRNet.

The design of robust and efficient cosegmentation algorithms is challenging because of the variety and complexity of the objects and images. In this paper, we propose a new cosegmentation model by incorporating a color reward strategy and an active contour model. A new energy function corresponding to the curve is first generated with two considerations: the foreground similarity between the image pairs and the background consistency in each of the image pair. Furthermore, a new foreground similarity measurement based on the rewarding strategy is proposed. Then, we minimize the energy function value via a mutual procedure which uses dynamic priors to mutually evolve the curves. The proposed method is evaluated on many images from commonly used databases. The experimental results demonstrate that the proposed model can efficiently segment the common objects from the image pairs with generally lower error rate than many existing and conventional cosegmentation methods.