Search

We present an unsupervised optical flow estimation method by proposing an adaptive pyramid sampling in the deep pyramid network. Specifically, in the pyramid downsampling, we propose a Content-Aware Pooling (CAP) module, which promotes local feature gathering by avoiding cross region pooling, so that the learned features become more representative. In the pyramid upsampling, we propose an Adaptive Flow Upsampling (AFU) module, where cross edge interpolation can be avoided, producing sharp motion boundaries. Equipped with these two modules, our method achieves the best performance for unsupervised optical flow estimation on multiple leading benchmarks, including MPI-Sintel, KITTI 2012 and KITTI 2015. Particularly, we achieve EPE=1.5 on KITTI 2012 and F1=9.67% KITTI 2015, which outperform the previous state-of-the-art methods by 16.7% and 13.1%, respectively. [ABSTRACT FROM AUTHOR]

Dictionary learning has achieved remarkable success on a wide range of machine learning-based applications. In this paper, a locality-adaptive structured dictionary learning (LASDL) algorithm for cross-domain recognition is proposed. In the LASDL, a projective structured double reconstruction strategy is developed to train class-oriented sub-dictionaries from the specific classes of cross-domain samples. The strategy benefits to make full advantage of the discriminative information of cross-domain data and bridge the distribution divergence between two different domains. Meanwhile, an adaptive geometrical structure preserving function is designed to not only preserve the local manifold structures spanned by the representation coefficient spaces of the source and target domains, but also impose a constraint that the coefficients should keep closer to their class centers, which is propitious to reduce the distribution divergence and make the representation more accurate. With the structured linear coding technique, the final cross-domain recognition can be efficiently performed by determining class-specific reconstruction error. The optimization of the proposed LASDL model can be efficiently solved by simple least square method and alternating direction method of multipliers (ADMM) algorithm. Extensive experimental results validate the superiority of the proposed algorithm in contrast to other state-of-the-art predecessors. [ABSTRACT FROM AUTHOR]

As an essential technique for data privacy protection, reversible data hiding in encrypted images (RDHEI) methods have drawn intensive research interest in recent years. In response to the increasing demand for protecting data privacy, novel methods that perform RDHEI are continually being developed. We propose two effective multi-MSB (most significant bit) replacement-based approaches that yield comparably high data embedding capacity, improve overall processing speed, and enhance reconstructed images’ quality. Our first method, Efficient Multi-MSB Replacement-RDHEI (EMR-RDHEI), obtains higher data embedding rates (DERs, also known as payloads) and better visual quality in reconstructed images when compared with many other state-of-the-art methods. Our second method, Lossless Multi-MSB Replacement-RDHEI (LMR-RDHEI), can losslessly recover original images after an information embedding process is performed. To verify the accuracy of our methods, we compared them with other recent RDHEI techniques and performed extensive experiments using the widely accepted BOWS-2 dataset. Our experimental results showed that the DER of our EMR-RDHEI method ranged from 1.2087 bit per pixel (bpp) to 6.2682 bpp with an average of 3.2457 bpp. For the LMR-RDHEI method, the average DER was 2.5325 bpp, with a range between 0.2129 bpp and 6.0168 bpp. Our results demonstrate that these methods outperform many other state-of-the-art RDHEI algorithms. Additionally, the multi-MSB replacement-based approach provides a clean design and efficient vectorized implementation. [ABSTRACT FROM AUTHOR]

The performance of person re-identification (re-ID) is easily affected by illumination variations caused by different shooting times, places and cameras. Existing illumination-adaptive methods usually require annotating cross-camera pedestrians on each illumination scale, which is unaffordable for a long-term person retrieval system. The cross-illumination person retrieval problem presents a great challenge for accurate person matching. In this paper, we propose a novel method to tackle this task, which only needs to annotate pedestrians on one illumination scale. Specifically, (i) we propose a novel Illumination Estimation and Restoring framework (IER) to estimate the illumination scale of testing images taken at different illumination conditions and restore them to the illumination scale of training images, such that the disparities between training images with uniform illumination and testing images with varying illuminations are reduced. IER achieves promising results on illumination-adaptive dataset and proving itself a proper baseline for cross-illumination person re-ID. (ii) we propose a Mixed Training strategy using both Original and Reconstructed images (MTOR) to further improve model performance. We generate reconstructed images that are consistent with the original training images in content but more similar to the restored images in style. The reconstructed images are combined with the original training images for supervised training to further reduce the domain gap between original training images and restored testing images. To verify the effectiveness of our method, some simulated illumination-adaptive datasets are constructed with various illumination conditions. Extensive experimental results on the simulated datasets validate the effectiveness of the proposed method. The source code is available at https://github.com/FadeOrigin/IUReId. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation crowd counting (UDACC) has been studied with practical research utility by getting rid of the labeling burden on large-scale dense crowds in the target domain. Current methods generalize well within the specific domain gap by directly aligning domain distributions or translating synthetic data to realistic images. However, it is difficult to define domain gaps among complex real-world datasets, in which the images vary greatly in style, density level and/or content. To tackle this problem, in this paper, we propose a Cross-Domain Attention Network (CDANet), which can effectively generalize the model to the unlabeled domain on both unsupervised synthetic-to-realistic and realistic-to-realistic crowd counting. Specifically, we propose a Cross-Domain Attention Module (CDAM) to learn domain-related information between the source and target domain, which extracts relations in cross-domain attentive information, thus enhancing crowd-informative features. Moreover, to make our CDAM invariant to domain shifts, we introduce a consistency penalty to ensure that the attention maps are consistent before and after the domain shifting. Thus our CDANet can pay attention to the shared counting information across domains, while remaining its invariant ability during domain adaptation. Extensive experiments on several common benchmarks for UDACC demonstrate that our CDANet gets competitive results on both unsupervised synthetic-to-realistic and realistic-to-realistic UDACC tasks. [ABSTRACT FROM AUTHOR]

Given a language query, temporal grounding task is to localize temporal boundaries of the described event in an untrimmed video. There is a long-standing challenge that multiple moments may be associated with one same video-query pair, termed label uncertainty. However, existing methods struggle to localize diverse moments due to the lack of multi-label annotations. In this paper, we propose a novel Diverse Temporal Grounding framework (DTG) to achieve diverse moment localization with only single-label annotations. By delving into the label uncertainty, we find the diverse moments retrieved tend to involve similar actions/objects, driving us to perceive these interest moments. Specifically, we construct soft multi-label through semantic similarity of multiple video-query pairs. These soft labels reveal whether multiple moments in the intra-videos contain similar verbs/nouns, thereby guiding interest moment generation. Meanwhile, we put forward a diverse moment regression network (DMRNet) to achieve multiple predictions in a single pass, where plausible moments are dynamically picked out from the interest moments for joint optimization. Moreover, we introduce new metrics that better reveal multi-output performance. Extensive experiments conducted on Charades-STA and ActivityNet Captions show that our method achieves state-of-the-art performance in terms of both standard and new metrics. [ABSTRACT FROM AUTHOR]

In this work, we introduce a new concept, named source-free open compound domain adaptation (SF-OCDA), and study it in semantic segmentation. SF-OCDA is more challenging than the traditional domain adaptation but it is more practical. It jointly considers (1) the issues of data privacy and data storage and (2) the scenario of multiple target domains and unseen open domains. In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model. The model is evaluated on the samples from the target and unseen open domains. To solve this problem, we present an effective framework by separating the training process into two stages: (1) pre-training a generalized source model and (2) adapting a target model with self-supervised learning. In our framework, we propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level, which can benefit the training of both stages. First, CPSS can significantly improve the generalization ability of the source model, providing more accurate pseudo-labels for the latter stage. Second, CPSS can reduce the influence of noisy pseudo-labels and also avoid the model overfitting to the target domain during self-supervised learning, consistently boosting the performance on the target and open domains. Experiments demonstrate that our method produces state-of-the-art results on the C-Driving dataset. Furthermore, our model also achieves the leading performance on CityScapes for domain generalization. [ABSTRACT FROM AUTHOR]

As a specific data hiding technique, reversible data hiding (RDH) has recently received extensive attention. By this technique, both the embedded data and the original cover image can be exactly extracted from the marked image. In our previous work, a general expansion-shifting model for RDH is proposed by introducing the so-called reversible embedding function (REF). With REF, RDH can be designed and the corresponding rate-distortion formulations can be established, providing an approach to optimize the reversible embedding performance. In this paper, by extending our previous work, optimal REF for one-dimensional histogram is investigated, and all optimal REF are derived in this case when the maximum modification to the cover pixel is limited as a small value. Moreover, based on the derived optimal REF for one-dimensional histogram and multiple histograms modification, a practical RDH scheme is presented and it is experimental verified better than some state-of-the-art algorithms in terms of capacity-distortion performance. [ABSTRACT FROM AUTHOR]

Recognition of thyroid glands and tissues of the neck is vital for screening related diseases in ultrasound videos. This task is subjective, challenging, and dependent on the experience of sonographer in current clinical practice. The purpose is to develop a fully automated thyroid gland and tissues of neck recognition framework to assist doctors in distinguishing the boundaries of different tissues. In this paper, we propose a novel deep learning framework that consists of a feature extraction network, region proposal network, object detection head, and spatial pyramid RoIAlign-based segmentation head. Designed spatial pyramid RoIAlign can efficiently capture local and global context features, and aggregates the multiple context information that makes the result much more reliable. A large dataset is constructed to train the proposed method. The performance is evaluated using the COCO metrics. The experimental results demonstrate that the proposed deep learning method can effectively realize the automatic recognition of the thyroid gland and tissues of neck in ultrasound videos. Considering the clinical practical application scenarios, we developed an automatic recognition system of thyroid and neck tissue based on edge computing, which can expediently assist doctors in distinguishing the boundaries between different tissues. [ABSTRACT FROM AUTHOR]

Frequent bad weather at sea severely damages the quality of visual images captured by imaging equipment. Ship instance segmentation in adverse weather conditions remains a major challenge because of poor visibility at sea. Existing approaches for instance segmentation are primarily designed for clear days and rarely consider the aforementioned severe weather. Blurred ship objects can easily cause missed ship detection and decrease the instance segmentation performance on ship images, especially in the case of frequent fog at sea. To this end, we propose a ship instance segmentation framework (IRDCLNet) based on Interference Reduction and Dynamic Contour Learning in foggy scenes. The Interference Reduction Module is proposed to reduce the interference caused by fog and solves the problem of missed ship detection. Meanwhile, we present Dynamic Contour Learning to predict the overall contour of the blurred ships to assist in mask prediction. To handle the scarcity of ocean data in foggy weather, we build the Foggy ShipInsseg dataset, which contains 5,739 real and simulated foggy ship images with 10,900 fine instance mask annotations. Experiments on the Foggy ShipInsseg dataset show that our IRDCLNet outperforms the Mask R-CNN and CondInst baselines and achieves the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Reversible data hiding (RDH) for color images has attracted increasing attention in recent years. Due to its effective utilization of the correlation between prediction errors, high-dimensional prediction-error expansion (PEE) can achieve much better performance for color image RDH than low-dimensional PEE. However, existing studies only focus on high-dimensional PEE with nonadaptive embedding. To further improve the embedding performance for color images, we propose a novel three-dimensional PEE method that is adaptive to image content. Double deep Q-network (DDQN), introduced to RDH for the first time, is adopted to find the optimal mapping paths for PEE. In addition, an action selection scheme is presented for DDQN to efficiently find the reversible mapping paths. Extensive experiments show that the proposed method outperforms existing color image RDH methods in image quality. [ABSTRACT FROM AUTHOR]

Most existing RGB-D salient detection models pay more attention to the quality of the depth images, while in some special cases, the quality of RGB images may even have greater impacts on saliency detection, which has long been ignored and underestimated. To address this problem, in this paper, we present a Bi-directional Progressive Guidance Network (BPGNet) for RGB-D salient object detection, where the qualities of both RGB and depth images are involved. Since it is usually difficult to determine which modality data have low quality in advance, a bi-directional framework based on progressive guidance (PG) strategy is employed to extract and enhance the unimodal features with the aid of another modality data via the alternative interactions between the saliency prediction results and the extracted features from the multi-modality input data. Specifically, the proposed PG strategy is achieved by using the proposed Global Context Awareness (GCA), Auxiliary Feature Extraction (AFE) and Cross-modality Feature Enhancement (CFE) modules. Benefiting from the proposed PG strategy, the disturbing information within the input RGB and depth images can be well suppressed, while the discriminative information within the input images gets enhanced. On top of that, a Fusion Prediction Module (FPM) is further designed to adaptively select those features with higher discriminability as well as enhancing the common information for the final saliency prediction. Experimental results demonstrate that our proposed model is comparable to those of state-of-the-art RGB-D SOD models. [ABSTRACT FROM AUTHOR]

Existing fine-grained image recognition methods are difficult to learn complete discriminative features from low-resolution (LR) data, because the original subtle inter-class distinctions become slimmer with the reduction of the image resolution. Besides, existing methods of LR fine-grained image recognition and general LR image recognition only consider the restoration and extraction of global discriminative features, ignoring unreliable local fine-grained details can be detrimental to final recognition. To address the above problems, we propose a multi-tasking framework, discriminative feature mining and enhancement network (DME-Net), for the LR fine-grained image recognition task, which aims to capture the reliable object descriptions from macro and micro perspectives, respectively. Macroscopically, we train the framework’s ability to recover and extract global discriminative features based on the whole images. Microscopically, we purposefully reinforce the framework’s ability to repair and capture the local discriminative details on the mined informative parts. To precisely excavate the most potential parts, we design an informative part mining (IPM) module, in which we firstly employ a part generation layer to predict several part masks that focus on different discriminative parts under the guidance of discrepancy loss and discriminant loss. Then we introduce a part selection (PS) submodule to further screen out a group of most informative parts from the predicted part masks according to their corresponding scores, which measure the semantic correlation degree of each part to the others. Experimental results on three benchmark datasets and one retail product dataset consistently show that our proposed framework can significantly boost the performance of the baseline model. Besides, extensive ablation studies are conducted, which further prove the effectiveness of each component of our designs. [ABSTRACT FROM AUTHOR]

Most of the existing blind omnidirectional image quality assessment (BOIQA) methods are based on data-driven approach where the end-to-end neural network or deep learning tools are mainly used for feature extraction. However, it usually lacks interpretability and is difficult to discover the perceptual mechanism behind. In this paper, from the perspective of perception modeling, we propose a novel multi-angle projection based BOIQA (MP-BOIQA) method. Considering the omnibearing and near eye display characteristics with head mounted display, multiple color cubemap projection images with respect to different viewpoints are grouped as the color omnidirectional distortion (COD) units so as to simulate the user’s viewing behavior in subjective quality assessment. In the designed multi-angle projection based feature extractor, tensor decomposition is implemented on each COD unit for dimensionality reduction, and piecewise exponential fitting is used to get the distribution of mean subtracted contrast normalized coefficients of the unit’s feature matrices in tensor domain. Finally, the extracted features are pooled with random forest. The experimental results on three omnidirectional image quality datasets show that the MP-BOIQA method can deliver highly competitive performance compared with some representative full-reference quality assessment methods, as well as some state-of-the-art BOIQA methods. [ABSTRACT FROM AUTHOR]

Traditional steganography methods are usually designed on a lossless channel; thus, messages are often not extracted correctly from an image transmitted over a lossy channel that includes attacks such as scaling. To address this issue, in recent years, the field of robust steganography has emerged. In this paper, the process of image scaling by interpolation is first observed and serves as the basis for proposing the idea of inverse interpolation. Subsequently, the idea of constructing an inverse interpolation equation set is proposed to solve the problem of intersectional blocks during the inverse interpolation process. Then, the scaling factor’s valid range of inverse interpolation is analyzed. Next, the inverse interpolation is successfully applied in robust image steganography. A method that combines antiscaling and antidetection is proposed. Afterward, actual tests on the top 9 mobile phone brands with 28 models and 2 social communication apps that are currently popular in China are done. The scaling factor’s valid range of the proposed method is verified to match the actual lossy channel. The experimental results show that the proposed method achieves a reliable extraction of embedded messages for common interpolation scaling attacks while maintaining high statistical detection resistance. [ABSTRACT FROM AUTHOR]

Joint photographic experts group (JPEG) compression is widely used in image processing and computer vision. Detecting double compressed JPEG images is a common problem in forensics and detecting compressed images with the same quantization matrix remains a challenging task. However, most existing methods were designed for detection in grayscale images and cannot fully use the unique characteristics of color images (such as the relationship between channels and color information). In addition, the performance of existing methods is unsatisfactory for low JPEG quality factors and in cross detection experiments. To solve these problems, we analyze the stability of a color image to obtain the convergence error and transposition error. According to the convergence characteristics of color JPEG images, the continuous compression by the same quantization matrix can make the JPEG image tend to be stable. The final stable state and the convergence process are determined by the number of compressions of the original image. Thus, continuously compressed JPEG images can be regarded as a continuous frame to obtain the convergence error. As the color image converges, its ability to resist interference decreases. To reflect the changes in anti-interference ability, the transposition operation is used to disturb the color JPEG image to obtain the transposition error. In addition, quaternion mapping is used to retain the relationship between continuously compressed JPEG images and enlarge the influence caused by transposition operation. In our experiments on several image databases, the proposed method outperforms existing methods in different settings. [ABSTRACT FROM AUTHOR]

Real-world scenes always exhibit objects with clutter backgrounds, posing great challenges for deep salient object detection models. In this paper, we propose salient object detection by engaging two saliency cues, i.e., the part-whole hierarchies and contrast cues, resulting in a PWHCNet. Specifically, two branches, which consists of a Dynamic Grouping Capsules (DGC) branch and a DenseHRNet branch, are put in place to learn the part-whole hierarchies and contrast cues, respectively. Moreover, to help highlight the whole salient object in complex scenes, a Background Suppression (BS) module is proposed to guide the shallow features of DenseHRNet with the aid of the part-whole relational cues captured by DGC. Subsequently, these two saliency cues are integrated via a Self-Channel and Mutual-Spatial (SCMS) attention mechanism. Experimental results on five benchmarks demonstrate that the proposed PWHCNet achieves state-of-the-art performance while obtaining the whole salient objects with fine details. [ABSTRACT FROM AUTHOR]

Most existing action recognition approaches directly leverage the video-level features to recognize human actions from videos. Although these methods have made remarkable progress, the accuracy is still unsatisfied. When the test video involves complex backgrounds and activities, existing methods usually suffer from a significant drop in accuracy. Human action is inherently a high-level concept. Merely applying a video classification model without a detailed semantic understanding of the video content, e.g., objects, scene context, object motions, object interactions, is inadequate to tackle the challenges for action recognition. Fine-level semantic understanding of videos generates elementary semantic concepts from the raw video data, such as the semantics of objects and background regions. It can be employed to bridge the gap between the raw video data and the high-level concept of human actions. In this work, we leverage dense semantic segmentation masks, which encode rich semantic details, provide extra information for the network training, and improve the performance of action recognition. We propose a novel deep architecture which is named as Dense Semantics-Assisted Convolutional Neural Networks (DSA-CNNs) to effectively utilize dense semantic information of video by a bottom-up attention way in the spatial stream, while by the way of branch fusion in the temporal stream. To verify the effectiveness of our approach, we conduct extensive experiments on publicly available datasets – UCF101, HMDB51, and Kinetics. The experimental results demonstrate that our approach substantially improves existing methods and achieves very competitive performance. It also shows that our approach is superior to other related methods that utilize extra information for action recognition. [ABSTRACT FROM AUTHOR]

An important cue for multi-pedestrian tracking in video is the consistent appearance of an individual for quite a while. In this paper, we address multi-pedestrian tracking by learning a robust appearance model from the paradigm of tracking by detection. To separate detections of different pedestrians while assembling detections of the same pedestrian, we take advantage of the cue of consistent appearance and exploit three types of evidence from the recent, past and near-future. Existing online approaches only exploit the detection-to-detection and sequence-to-detection metrics, which focus on the recent and past appearance patterns respectively, while the future pedestrian appearance is simply ignored. This drawback is remedied in this paper by further considering the sequence-to-sequence metric, which resorts to near-future appearance presentation. Adaptive combination weights are learned to fuse these three different metrics. Moreover, we propose a novel Focal Triplet Loss to make the model focus more on hard examples than the easy ones. We demonstrate that this can significantly enhance the discriminating power of the model compared with treating every sample equally. Effectiveness and efficiency of the proposed method is verified by conducting comprehensive ablation studies and comparing with many competitive (offline/online/near-online) counterparts on the MOT16 and MOT17 Challenges. [ABSTRACT FROM AUTHOR]

We present an effective blind image deblurring algorithm based on superpixel segmentation prior (SSP). The motivation of this work is an interesting observation that the more rough the segmentation boundaries are, the clearer the image will be. Intuitively, the blurry images have less image details, which results in more smooth segmentation boundaries. The clear images have more vivid textural details and obtain more rough boundaries. The segmentation roughness could be defined as the length of image segmentation boundaries. However, the segmentation boundary length is not differentiable, making it difficult to integrate into existing joint optimization framework. Therefore, we transform the segmentation boundary length into the segmentation entropy to guide the process of image deblurring. With the image becomes clearer, its boundary becomes more rough, while the segmentation entropy is much smaller. The analysis of relationship between segmentation entropy and segmentation boundaries is detailed. Benefiting from the convexity of segmentation entropy, we propose a novel algorithm by integrating half-quadratic split and gradient descent to alternately minimize energy function. Extensive experiments show that the proposed method achieves best performance with the state-of-the-art blind deblurring methods on natural and face image deblurring. [ABSTRACT FROM AUTHOR]

JPEG-domain enhancement improves the visual quality of JPEG images by directly manipulating the decoded DCT (discrete cosine transform) coefficients, which inevitably leads to mixed compression and enhancement artifacts. Existing forensic methods that merely consider JPEG artifacts are unsuitable to address such mixed artifacts and hence suffer a considerable performance decline in compression parameter estimation and lack the ability to estimate the enhancement parameter. This work attempts to explore the characterization of the mixed artifacts, and to further estimate both the enhancement and compression parameters of JPEG-domain enhanced images. First, a statistical likelihood function is proposed to characterize the periodicity of DCT coefficients, which can measure how well an enhanced image is de-enhanced back to its JPEG compressed version given the compression and enhancement parameters. The proposed likelihood function reaches its maximum if the parameters match their true values. Then, a forensic method of enhancement detection and parameter estimation is developed based on the proposed likelihood function for two kinds of classical JPEG-domain enhancement. Specifically, JPEG-domain enhanced images are detected by thresholding a scalar feature computed upon the likelihoods, and the enhancement and compression parameters are estimated by locating the maximal likelihood. In addition, mathematical proof of the de-enhancement feasibility is provided. Experimental results demonstrate that the proposed method outperforms the compared methods in both enhancement detection and parameter estimation. [ABSTRACT FROM AUTHOR]

In recent years, RGBT tracking has become a hot topic in the field of visual tracking, and made great progress. In this paper, we propose a novel Trident Fusion Network (TFNet) to achieve effective fusion of different modalities for robust RGBT tracking. In specific, to deploy the complementarity of features of all convolutional layers, we propose a recursive strategy to densely aggregate these features that yield robust representations of target objects in two modalities. Moreover, we design a trident architecture to integrate the fused features and both modality-specific features for robust target representations. There are three main advantages. First, retaining the classification layer of each modality is beneficial to enhance feature learning of single modality, and compared with aggregate branches, single-modality branches pay more attention to the mining of modal specific information. Second, when some modality is noisy or invalid, the modality-specific branches would capture more discriminative features for RGBT tracking. Finally, the integration of aggregation branches and single-modality branches is beneficial to the complementary learning of different modalities. In addition, we also introduce a feature pruning module in each branch to prune the redundant features and avoid network overfitting. Experimental results on four RGBT tracking benchmark datasets suggest that our tracker achieves superior performance against the state-of-the-art RGBT tracking methods. [ABSTRACT FROM AUTHOR]

In this work, we propose a new solution to 3D human pose estimation in videos. Instead of directly regressing the 3D joint locations, we draw inspiration from the human skeleton anatomy and decompose the task into bone direction prediction and bone length prediction, from which the 3D joint locations can be completely derived. Our motivation is the fact that the bone lengths of a human skeleton remain consistent across time. This promotes us to develop effective techniques to utilize global information across all the frames in a video for high-accuracy bone length prediction. Moreover, for the bone direction prediction network, we propose a fully-convolutional propagating architecture with long skip connections. Essentially, it predicts the directions of different bones hierarchically without using any time-consuming memory units (e.g. LSTM). A novel joint shift loss is further introduced to bridge the training of the bone length and bone direction prediction networks. Finally, we employ an implicit attention mechanism to feed the 2D keypoint visibility scores into the model as extra guidance, which significantly mitigates the depth ambiguity in many challenging poses. Our full model outperforms the previous best results on Human3.6M and MPI-INF-3DHP datasets, where comprehensive evaluation validates the effectiveness of our model. [ABSTRACT FROM AUTHOR]

Image fusion integrates a series of images acquired from different sensors, e.g., infrared and visible, outputting an image with richer information than either one. Traditional and recent deep-based methods have difficulties in preserving prominent structures and recovering vital textural details for practical applications. In this article, we propose a deep network for infrared and visible image fusion cascading a feature learning module with a fusion learning mechanism. Firstly, we apply a coarse-to-fine deep architecture to learn multi-scale features for multi-modal images, which enables discovering prominent common structures for later fusion operations. The proposed feature learning module requires no well-aligned image pairs for training. Compared with the existing learning-based methods, the proposed feature learning module can ensemble numerous examples from respective modals for training, increasing the ability of feature representation. Secondly, we design an edge-guided attention mechanism upon the multi-scale features to guide the fusion focusing on common structures, thus recovering details while attenuating noise. Moreover, we provide a new aligned infrared and visible image fusion dataset, RealStreet, collected in various practical scenarios for comprehensive evaluation. Extensive experiments on two benchmarks, TNO and RealStreet, demonstrate the superiority of the proposed method over the state-of-the-art in terms of both visual inspection and objective analysis on six evaluation metrics. We also conduct the experiments on the FLIR and NIR datasets, containing foggy weather and poor light conditions, to verify the generalization and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Most of the existing single object trackers track the target in a unitary local search window, making them particularly vulnerable to challenging factors such as heavy occlusions and out-of-view movements. Despite the attempts to further incorporate global search, prevailing mechanisms that cooperate local and global search are relatively static, thus are still sub-optimal for improving tracking performance. By further studying the local and global search results, we raise a question: can we allow more dynamics for cooperating both results? In this paper, we propose to introduce more dynamics by devising a dynamic attention-guided multi-trajectory tracking strategy. In particular, we construct dynamic appearance model that contains multiple target templates, each of which provides its own attention for locating the target in the new frame. Guided by different attention, we maintain diversified tracking results for the target to build multi-trajectory tracking history, allowing more candidates to represent the true target trajectory. After spanning the whole sequence, we introduce a multi-trajectory selection network to find the best trajectory that deliver improved tracking performance. Extensive experimental results show that our proposed tracking strategy achieves compelling performance on various large-scale tracking benchmarks. The project page of this paper can be found at https://sites.google.com/view/mt-track/. [ABSTRACT FROM AUTHOR]

In this paper, we tackle the problem of single image dehazing with a convolutional neural network. Within this network, we develop a multi-level fusion module to utilize both low-level and high-level features. The low-level features help to recover finer details, and the high-level features discover abstract semantics. They are complementary in the restoring of clear images. Moreover, a Residual Mixed-convolution Attention Module (RMAM) with an attention block is proposed to guide the network to focus on important features in the learning process. In this RMAM, group convolution, depth-wise convolution, and point-wise convolution are mixed, and thus it is much faster than its counterparts. With these two modules, we thus have an end-to-end network without explicitly estimating the atmospheric light intensity and the transmission map in the classical atmosphere scattering model. Both qualitative and quantitative experimental studies are carried out on public datasets including RESIDE, DCPDN-TestA, and the real-world dataset. The extensive results demonstrate both the effectiveness and efficiency of the proposed solution to single image dehazing. [ABSTRACT FROM AUTHOR]

Efficient representation and coding of fine-granular motion information is one of the key research areas for exploiting inter-frame correlation in video coding. Representative techniques towards this direction are affine motion compensation (AMC), decoder-side motion vector refinement (DMVR), and subblock-based temporal motion vector prediction (SbTMVP). Fine-granular motion information is derived at subblock level for all the three coding tools. In addition, the obtained inter prediction can be further refined by two optical flow-based coding tools, the bi-directional optical flow (BDOF) for bi-directional inter prediction and the prediction refinement with optical flow (PROF) exclusively used in combination with AMC. The aforementioned five coding tools have been extensively studied and finally adopted in the Versatile Video Coding (VVC) standard. This paper presents technical details of each tool and highlights the design elements with the consideration of typical hardware implementations. Following the common test conditions defined by Joint Video Experts Team (JVET) for the development of VVC, 5.7% bitrate reduction on average is achieved by the five tools. For test sequences characterized by large and complex motion, up to 13.4% bitrate reduction is observed. Additionally, visual quality improvement is demonstrated and analyzed. [ABSTRACT FROM AUTHOR]

In this paper, we study tracking by language that localizes the target box sequence in a video based on a language query. We propose a framework called GTI that decomposes the problem into three sub-tasks: Grounding, Tracking, and Integration. The three sub-task modules operate simultaneously and predict the box sequence frame-by-frame. “Grounding” predicts the referred region directly from the language query. “Tracking” localizes the target based on the history of the grounded regions in previous frames. “Integration” generates final predictions by synergistically combining grounding and tracking. With the “integration” task as the key, we explore how to indicate the quality of the grounded regions in each frame and achieve the desired mutually beneficial combination. To this end, we propose an “RT-integration” method that defines and predicts two scores to guide the integration: 1) R-score represents the Region correctness whether the grounding prediction accurately covers the target, and 2) T-score represents the Template quality whether the region provides informative visual cues to improve tracking in future frames. We present our real-time GTI implementation with the proposed RT-integration, and benchmark the framework on LaSOT and Lingual OTB99 with highly promising results. Moreover, we produce a disambiguated version of LaSOT queries to facilitate future tracking by language studies. [ABSTRACT FROM AUTHOR]

The success of convolutional neural network (CNN) based single image dynamic scene blind deblurring (SIDSBD) methods mainly stems from the multi-scale/multi-patch model and the designs of the encoder-decoder architecture, and the residual block structure, which make different contributions to SIDSBD. In this paper, we further exploit the advantages of the multi-scale model, the encoder-decoder module, and the residual block structure, respectively, and propose a novel multi-scale channel attention network (MSCAN) for effective single image dynamic scene blind deblurring. Different from existing multi-scale models, in our proposed network, each scale consists of multiple levels, in which a novel spatial pyramid pooling channel attention (SPPCA) strategy is proposed to adaptively rescale the channel-wise features by using both the global and local feature statistics for more powerful network representation. Extensive experiments on both the synthetic benchmark datasets and the real blurred images show that our method can produce better deblurring results than the state-of-the-art SIDSBD methods in terms of both qualitative evaluation and quantitative metrics. [ABSTRACT FROM AUTHOR]

Most of the existing coverless steganography approaches have poor robustness to geometric attacks, because these approaches use features of the entire image to map information, and these features are easy to be lost when being attacked. In order to improve the robustness against geometric attacks, we propose a coverless image steganography method based on multi-object recognition. In this scheme, we firstly use Faster RCNN to detect objects in the image data set, establish a mapping dictionary between object labels and binary sequence. Then we propose a novel mapping rule based on the filtered robust object labels for sequence generation. Therefore, an image can generate robust binary sequence through multi-objects recognition. In the transmission process, the transmitted image has not been modified, so our method can fundamentally resist steganalysis tools and avoid the attacker’s suspicions. In addition, the capacity and hiding rate of the proposed method are both considerable. Evaluations with under geometric attacks shows, on average, $3.1\times $ robustness increase over other five coverless steganography methods. Moreover, evaluations under ten noise attacks shows, on average, the robustness of our method is also excellent, which reaches 83%. [ABSTRACT FROM AUTHOR]

Visual relationship detection, as a challenging task used to find and distinguish interactions between object-pairs in one image, has received much attention recently. In this work, we devise a unified visual relationship detection framework with two types of correlation exploitation to address the combination explosion problem in the object-pairs proposing stage and the non-exclusive label problem in the predicate recognition stage. In the object-pairs proposing stage, with the exploitation of relative location correlation between two objects in one pair, one location-embedded rating module (LRM) is developed to effectively select plausible proposals. In the predicate recognition stage, one label-correlation graph module (LGM) is introduced to measure the implicit semantic correlation among predicates; and then assign discrete distributed labels to predicates to improve the precision of top-n recall. Experiments on the two widely used VRD and VG datasets show that our proposed method outperforms current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we address a problem of view-disturbing raindrop removal on a single image. In existing methods to tackle this problem, machine learning based ones seem promising but require elaborate pairwise images, i.e., the raindrop-degraded image and the corresponding clean image of the same scene, for training. To overcome this drawback, we propose a weakly supervised learning based model in the absence of pairwise training examples, which needs only a collection of images with image-level annotations indicating the presence/absence of raindrops for training. Specifically, we train a raindrop detector for highlighting regions of raindrops in a multi-task learning manner. Then, we propose an attention-based generative network for raindrop removal and introduce a weighted preservation loss to retain the non-raindrop details. Specially, our model can be mixedly trained with pairwise and unpaired samples, which enables us to conveniently adapt the model to a new domain. Experiments verify the effect of the proposed method. Especially, using only weakly-supervised learning, our method can achieve comparable results with state-of-the-art strongly-supervised learning methods. [ABSTRACT FROM AUTHOR]

Image dehazing has evolved into an attractive research field in the computer vision community in the past few decades. Previous traditional approaches attempt to design energy-based objective functions. However, they cannot accurately express the intrinsic characteristics of the images, posing weak adaptation ability for real-world complex scenarios. More recently, deep learning techniques for image dehazing have matured and become more reliable, showing outstanding performance. Nevertheless, these methods heavily depend on training data, restricting their application ranges. More importantly, both traditional and deep learning approaches all ignore a common issue, noises/artifacts always appear in the recovery process. To this end, a new Hadamard-Product (HP) model is proposed, which consists of a series of data-driven priors. Based on this model, we derive a Learnable Hadamard-Product-Propagation (LHPP) by cascading a series of principle-inspired guidance and recovery modules. In which, the principle-inspired guidance related to transmission is endowed the smoothness property, the other recovery module satisfies the distribution of natural images. The Hadamard-product-based propagations is generated in our developed learnable framework for the task of image dehazing. In this way, we can eliminate noises/artifacts in the recovery procedure to obtain the ideal outputs. Subsequently, since the generality of our HP model, we successfully extend our LHPP to settle low-light image enhancement and underwater image enhancement problems. A series of analytical experiments are performed to verify our effectiveness. Plenty of performance evaluations on three complex tasks fully reveal our superiority against multiple state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Supervised deep learning with pixel-wise training labels has great successes on multi-person part segmentation. However, data labeling at pixel-level is very expensive. To solve the problem, people have been exploring to use synthetic data to avoid the data labeling. Although it is easy to generate labels for synthetic data, the results are much worse compared to those using real data and manual labeling. The degradation of the performance is mainly due to the domain gap, i.e., the discrepancy of the pixel value statistics between real and synthetic data. In this paper, we observe that real and synthetic humans both have a skeleton (pose) representation. We found that the skeletons can effectively bridge the synthetic and real domains during the training. Our proposed approach takes advantage of the rich and realistic variations of the real data and the easily obtainable labels of the synthetic data to learn multi-person part segmentation on real images without any human-annotated labels. Through experiments, we show that without any human labeling, our method performs comparably to several state-of-the-art approaches which require human labeling on Pascal-Person-Parts and COCO-DensePose datasets. On the other hand, if part labels are also available in the real-images during training, our method outperforms the supervised state-of-the-art methods by a large margin. We further demonstrate the generalizability of our method on predicting novel keypoints in real images where no real data labels are available for the novel keypoints detection. Code and pre-trained models are available at https://github.com/kevinlin311tw/CDCL-human-part-segmentation. [ABSTRACT FROM AUTHOR]

Fully Convolutional Neural Network (FCN) has been widely applied to salient object detection recently by virtue of high-level semantic feature extraction, but existing FCN-based methods still suffer from continuous striding and pooling operations leading to loss of spatial structure and blurred edges. To maintain the clear edge structure of salient objects, we propose a novel Edge-guided Non-local FCN (ENFNet) to perform edge-guided feature learning for accurate salient object detection. In a specific, we extract hierarchical global and local information in FCN to incorporate non-local features for effective feature representations. To preserve good boundaries of salient objects, we propose a guidance block to embed edge prior knowledge into hierarchical feature maps. The guidance block not only performs feature-wise manipulation but also spatial-wise transformation for effective edge embeddings. Our model is trained on the MSRA-B dataset and tested on five popular benchmark datasets. Comparing with the state-of-the-art methods, the proposed method performance well on five datasets. [ABSTRACT FROM AUTHOR]

Weakly supervised semantic segmentation uses image-level labels to extract object regions. The existing methods focus on efficiently training CNN-based segmentation networks using the image-level labels. In contrast to the existing methods, this paper proposes a new fusion-based method, which first segments the foregrounds of each image by multiple group cosegmentation and then generates the semantic segmentation by combining the foregrounds. Specifically, a new CNN-based multiple group cosegmentation network is first proposed to segment foregrounds employing two cues, the discriminative cue and the local-to-global cue. Then, the fusion method is proposed to simply perform semantic segmentation based on the multiple group cosegmentation results. Experiments on the PASCAL VOC 2012 and MS COCO 2017 datasets demonstrate the effectiveness of the proposed method with mIoU values that are obviously larger than those of the existing methods. [ABSTRACT FROM AUTHOR]

To improve the performance of sketch-based image retrieval (SBIR) methods, most existing SBIR methods develop brand new SBIR methods. In fact, a re-ranking approach, which can refine the retrieval results of SBIR methods, is also beneficial. Inspired by this, in this paper, an SBIR re-ranking approach based on multi-clustering is proposed. In order to make the re-ranking approach invisible to users and adaptive to different types of image datasets, we made it an unsupervised method using blind feedback. Distinguished from the existing methods, this re-ranking approach uses the semantic information of three types of images: edge maps, object images (images with black background and natural images’ foreground objects) and natural images themselves. With the initial retrieval results of an SBIR method, our approach first does the clustering operation for three types of images. Then, we utilize the clustering results to generate a cluster score for each initial retrieval result. Finally, the cluster score is used to calculate the final retrieval scores for the initial retrieval results. The experiments on different SBIR datasets are conducted. Experimental results demonstrate that, by implementing our re-ranking approach, the retrieval accuracy of a variety of SBIR methods is increased. Furthermore, the comparisons between our re-ranking method and the existing re-ranking methods are given. [ABSTRACT FROM AUTHOR]

Underwater image enhancement is such an important low-level vision task with many applications that numerous algorithms have been proposed in recent years. These algorithms developed upon various assumptions demonstrate successes from various aspects using different data sets and different metrics. In this work, we setup an undersea image capturing system, and construct a large-scale Real-world Underwater Image Enhancement (RUIE) data set divided into three subsets. The three subsets target at three challenging aspects for enhancement, i.e., image visibility quality, color casts, and higher-level detection/classification, respectively. We conduct extensive and systematic experiments on RUIE to evaluate the effectiveness and limitations of various algorithms to enhance visibility and correct color casts on images with hierarchical categories of degradation. Moreover, underwater image enhancement in practice usually serves as a preprocessing step for mid-level and high-level vision tasks. We thus exploit the object detection performance on enhanced images as a brand new task-specific evaluation criterion. The findings from these evaluations not only confirm what is commonly believed, but also suggest promising solutions and new directions for visibility enhancement, color correction, and object detection on real-world underwater images. The benchmark is available at: https://github.com/dlut-dimt/Realworld-Underwater-Image-Enhancement-RUIE-Benchmark. [ABSTRACT FROM AUTHOR]

Considering that traditional image steganography technologies suffer from the potential risk of failure under lossy channels, an enhanced adaptive steganography with multiple robustness against image processing attacks is proposed, while maintaining good detection resistance. First, a robust domain constructing method is proposed utilizing robust element extraction and optimal element modification, which can be applied to both spatial and JPEG images. Then, a robust steganography is proposed based on “Robust Domain Constructing + RS-STC Codes,” combined with cover selection, robust cover extraction, message coding, and embedding with minimized costs. In addition, to provide a theoretical basis for message extraction integrity, the fault tolerance of the proposed algorithm is deduced using error model based on burst errors and decoding damage. Finally, on the basis of parameter discussion about robust domain construction, performance experiments are conducted, and the recommended coding parameters are given for lossy channels with different attacks using the analytic results for fault tolerance. A series of experimental results demonstrate that the proposed algorithm can extract embedded messages with significantly higher accuracy after different attacks, such as compression, noising, scaling and other attacks, compared with the state-of-the-art adaptive steganography, and robust watermarking algorithms, while maintaining good detection resistant performance. [ABSTRACT FROM AUTHOR]

Among various methods of reversible data hiding (RDH) in JPEG images, only rate-distortion, i.e. the image quality with given payload, is taken into consideration during algorithm designing. However, file size expansion is another important evaluation metric for JPEG RDH methods. Based on this situation, we propose a JPEG RDH method considering both the rate-distortion and the file size expansion at the same time while designing the algorithm. The multi-objective optimization strategy is utilized to realize the balance of the two objectives. Specifically, the cover signal is divided into several non-overlapping parts firstly, and after that, the embedding costs of each part are calculated. Next, the optimized combination of parts for embedding data is gained by multi-objective optimization. Experimental results show that the proposed algorithm outperforms the state-of-the-art methods in terms of rate-distortion and file size expansion performance. [ABSTRACT FROM AUTHOR]

Detection of double Joint Photographic Experts Group (JPEG) compression is an important part of image forensics. Although methods in the past studies have been presented for detecting the double JPEG compression with a different quantization matrix, the detection of double JPEG compression with the same quantization matrix is still a challenging problem. In this paper, an effective method to detect the recompression in the color images by using the conversion error, rounding error, and truncation error on the pixel in the spherical coordinate system is proposed. The randomness of truncation errors, rounding errors, and quantization errors result in random conversion errors. The pixel number of the conversion error is used to extract six-dimensional features. Truncation error and rounding error on the pixel in its three channels are mapped to the spherical coordinate system based on the relation of a color image to the pixel values in the three channels. The former is converted into amplitude and angles to extract 30-dimensional features and 8-dimensional auxiliary features are extracted from the number of special points and special blocks. As a result, a total of 44-dimensional features have been used in the classification by using the support vector machine (SVM) method. Thereafter, the support vector machine recursive feature elimination (SVMRFE) method is used to improve the classification accuracy. The experimental results show that the performance of the proposed method is better than the existing methods. [ABSTRACT FROM AUTHOR]

Sports video captioning refers to the task of automatically generating a textual description for sports events (football, basketball, or volleyball games). Although a great deal of previous work has shown promising performance in producing a coarse and a general description of a video but lack of professional sports knowledge, it is still quite challenging to caption a sports video with multiple fine-grained player’s actions and complex group relationship between players. In this paper, we present a novel hierarchical recurrent neural network-based framework with an attention mechanism for sports video captioning, in which a motion representation module is proposed to capture individual pose attribute and dynamical trajectory cluster information with extra professional sports knowledge, and a group relationship module is employed to design a scene graph for modeling players’ interaction by a gated graph convolutional network. Moreover, we introduce a new dataset called sports video captioning dataset-volleyball for evaluation. The proposed model is evaluated on three widely adopted public datasets and our collected new dataset, on which the effectiveness of our method is well demonstrated. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel supervised cross-modal hashing framework, namely Scalable disCRete mATrix faCtorization Hashing (SCRATCH). First, it utilizes collective matrix factorization on original features together with label semantic embedding, to learn the latent representations in a shared latent space. Thereafter, it generates binary hash codes based on the latent representations. During optimization, it avoids using a large $n\times n$ similarity matrix and generates hash codes discretely. Besides, based on different objective functions, learning strategy, and features, we further present three models in this framework, i.e., SCRATCH-o, SCRATCH-t, and SCRATCH-d. The first one is a one-step method, learning the hash functions and the binary codes in the same optimization problem. The second is a two-step method, which first generates the binary codes and then learns the hash functions based on the learned hash codes. The third one is a deep version of SCRATCH-t, which utilizes deep neural networks as hash functions. The extensive experiments on two widely used benchmark datasets demonstrate that SCRATCH-o and SCRATCH-t outperform some state-of-the-art shallow hashing methods for cross-modal retrieval. The SCRATCH-d also outperforms some state-of-the-art deep hashing models. [ABSTRACT FROM AUTHOR]

In real life, group activity recognition plays a significant and fundamental role in a variety of applications, e.g. sports video analysis, abnormal behavior detection, and intelligent surveillance. In a complex dynamic scene, a crucial yet challenging issue is how to better model the spatio-temporal contextual information and inter-person relationship. In this paper, we present a novel attentive semantic recurrent neural network (RNN), namely, stagNet, for understanding group activities and individual actions in videos, by combining the spatio-temporal attention mechanism and semantic graph modeling. Specifically, a structured semantic graph is explicitly modeled to express the spatial contextual content of the whole scene, which is further incorporated with the temporal factor through structural-RNN. By virtue of the “factor sharing” and “message passing” mechanisms, our stagNet is capable of extracting discriminative and informative spatio-temporal representations and capturing inter-person relationships. Moreover, we adopt a spatio-temporal attention model to focus on key persons/frames for improved recognition performance. Besides, a body-region attention and a global-part feature pooling strategy are devised for individual action recognition. In experiments, four widely-used public datasets are adopted for performance evaluation, and the extensive results demonstrate the superiority and effectiveness of our method. [ABSTRACT FROM AUTHOR]

RGB-thermal (RGB-T) object tracking, which has attracted much recent attention, uses thermal infrared information to assist object tracking with visible light information. However, it still faces many challenging problems, especially the background inclusion in the target bounding box which easily results in model drifting. To handle this problem, we propose a novel and general approach to learn a local-global multi-graph descriptor to suppress background effects for RGB-T tracking. Our approach relies on a novel graph learning algorithm. First, the object is represented with multiple graphs, with a set of multi-modal image patches as nodes, for the robustness to prevent deformation and partial occlusion. Second, we dynamically learn a joint graph over time with both local and global considerations using spatial smoothness and low-rank representation. In particular, we design a single unified alternating direction method of multipliers-based optimization framework to learn graph structure, edge weights, and node weights simultaneously. Third, we combine multi-graph information with corresponding graph node weights to form a robust object descriptor, and tracking is finally carried out by adopting the structured support vector machine. Extensive experiments conducted on the tracking benchmark data sets demonstrate the effectiveness of the proposed approach against the state-of-the-art RGB-T trackers. [ABSTRACT FROM AUTHOR]

In this paper, a novel forensics scheme for color image is proposed in color quaternion wavelet transform (CQWT) domain. Compared with discrete wavelet transform (DWT), contourlet wavelet transform, and local binary patterns, CQWT processes a color image as a unit, and so, it can provide more forensics information to identify the photograph (PG) and computer generated (CG) images by considering the quaternion magnitude and phase measures. Meanwhile, two novel quaternion central moments for color images, i.e., quaternion skewness and kurtosis, are proposed to extract forensics features. In the condition of the same statistical model as Farid’s model, the CQWT can boost the performance of the existing identification models. Compared with Farid’s model and Li’s model in 7500 PG and 7500 CG, the quaternion statistical features show a better classification performance. Results in the comparative experiments show that the classification accuracy of the CQWT improves by 19% more than Farid’s model, and the quaternion features approximately improve by 2% more than the traditional. [ABSTRACT FROM AUTHOR]

Action recognition with 3D skeleton sequences became popular due to its speed and robustness. The recently proposed convolutional neural networks (CNNs)-based methods show a good performance in learning spatio–temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN-based methods, there existed two problems that potentially limit the performance. First, previous skeleton representations were generated by chaining joints with a fixed order. The corresponding semantic meaning was unclear and the structural information among the joints was lost. Second, previous models did not have an ability to focus on informative joints. The attention mechanism was important for skeleton-based action recognition because different joints contributed unequally toward the correct recognition. To solve these two problems, we proposed a novel CNN-based method for skeleton-based action recognition. We first redesigned the skeleton representations with a depth-first tree traversal order, which enhanced the semantic meaning of skeleton images and better preserved the associated structural information. We then proposed the general two-branch attention architecture that automatically focused on spatio–temporal key stages and filtered out unreliable joint predictions. Based on the proposed general architecture, we designed a global long-sequence attention network with refined branch structures. Furthermore, in order to adjust the kernel’s spatio–temporal aspect ratios and better capture long-term dependencies, we proposed a sub-sequence attention network (SSAN) that took sub-image sequences as inputs. We showed that the two-branch attention architecture could be combined with the SSAN to further improve the performance. Our experiment results on the NTU RGB+D data set and the SBU kinetic interaction data set outperformed the state of the art. The model was further validated on noisy estimated poses from the subsets of the UCF101 data set and the kinetics data set. [ABSTRACT FROM AUTHOR]

Steganography detection based on Rich Model features is a hot research direction in steganalysis. However, rich model features usually result a large computation cost. To reduce the dimension of steganalysis features and improve the efficiency of steganalysis algorithm, differing from previous works that normally proposed new feature extraction algorithm, this paper proposes a general steganalysis feature selection method based on decision rough set $\alpha$ -positive region reduction. First, it is pointed out that decision rough set $\alpha$ -positive region reduction is suitable for steganalysis feature selection. Second, a quantization method of attribute separability is proposed to measure the separability of steganalysis feature components. Third, steganalysis feature components selection algorithm based on decision rough set $\alpha$ -positive region reduction is given; thus, stego images can be detected by the selected feature. The proposed method can significantly reduce the feature dimensions and maintain detection accuracy. Based on the BOSSbase-1.01 image database of 10 000 images, a series of feature selection experiments are carried on two kinds of typical rich model features (35263-D J+SRM feature and 17000-D GFR feature). The results show that even though these two kinds of features are reduced to approximately 8000-D, the detection performance of steganalysis algorithms based on the selected features are also maintained with that of original features, which will remarkably improve the efficiency of feature extraction and stego image detection. [ABSTRACT FROM AUTHOR]

This paper proposes a novel framework of reversible data hiding in encrypted JPEG bitstream. We first provide a JPEG encryption algorithm to encipher a JPEG image to a smaller size and keep the format compliant to JPEG decoders. After an image owner uploads the encrypted JPEG bitstreams to cloud storage, the server embeds additional messages into the ciphertext to construct a marked encrypted JPEG bitstream. During data hiding, we propose a combined embedding algorithm including two stages, the Huffman code mapping and the ordered histogram shifting. The embedding procedure is reversible. When an authorized user requires a downloading operation, the server extracts additional messages from the marked encrypted JPEG bitstream and recovers the original encrypted bitstream losslessly. After downloading, the user obtains the original JPEG bitstream by a direct decryption. The proposed framework outperforms previous works on reversible data hiding in encrypted images. First, since the tasks of data embedding/extraction and bitstream recovery are all accomplished by the server, the image owner and the authorized user are required to implement no extra operations except JPEG encryption or decryption. Second, the embedding payload is larger than state-of-the-art works. [ABSTRACT FROM AUTHOR]

This paper proposes a novel reversible data hiding scheme for encrypted images by using homomorphic and probabilistic properties of Paillier cryptosystem. In the proposed method, groups of adjacent pixels are randomly selected, and reversibly embedded into the rest of the image to make room for data embedding. In each group, there are a reference pixel and a few host pixels. Least significant bits (LSBs) of the reference pixels are reset before encryption and the encrypted host pixels are replaced with the encrypted reference pixel in the same group to form mirroring ciphertext groups (MCGs). In such a way, the modification on MCGs for data embedding will not cause any pixel oversaturation in plaintext domain and the embedded data can be directly extracted from the encrypted domain. In an MCG, the reference ciphertext pixel is kept unchanged as a reference while data hider embeds the encrypted additional data into the LSBs of the host ciphertext pixels by employing homomorphic multiplication. On the receiver side, the hidden ciphertext data can be retrieved by employing a modular multiplicative inverse operation between the marked host ciphertext pixels and their corresponding reference ciphertext pixels, respectively. After that, the hidden data are extracted promptly by looking for a one-to-one mapping table from ciphertext to plaintext. Data extraction and image restoration can be accomplished without any error after decryption. Compared with the existing works, the proposed scheme has lower computation complexity, higher security performance, and better embedding performance. The experiments on the standard image files also certify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]