Search

• An adaptive HEVC video steganography algorithm is designed, which can effectively resist the PU shift-based steganalysis. • The introduction of multivariate STC allows for smaller modifications with the same embedding capacity. • The superiority of this algorithm over the state-of-the-art video steganography algorithm is verified through extensive experiments. Calibration-based steganalysis methods are a fatal threat to current HEVC steganography algorithms. The current steganography algorithms based on prediction unit (PU) still lack resistance to this type of steganalysis. The reason is that the improvements in HEVC codec provide more potential steganographic space as well as introduce more detectable distortion. Therefore, a HEVC video steganography method that has resistance to PU shift-based steganalysis is proposed in this paper. First, the PU shift phenomenon that exists in PU-based steganography is introduced. Then the existing calibration-based steganalysis method is enhanced. According to the encoding rules of HEVC, the distortion cost function according to the difference of rate distortion cost (RD cost) during recompression is designed, which aims to obtain better resistance to calibration-based steganalysis. Experimental results demonstrate that the steganography algorithm designed in this paper provides excellent resistance ability for calibration-based steganalysis. [ABSTRACT FROM AUTHOR]

A watermark hidden in an image is retrieved differently from the original watermark due to the frequently used rounding approach. The simple rounding will cause numerous errors in the embedded watermark especially when it is large. A novel technique based on genetic algorithms (GAs) is presented in this paper to correct the rounding errors. The fundamental is to adopt a fitness function for choosing the best chromosome which determines the conversion rule of real numbers into integers during the cosine transformation. Experimental results show that the dramatic improvement in reducing the errors is successful when GAs are used in decision making. Furthermore, we develop an initial chromosome by comparing the difference between the original image and the rounded watermarked image to speed up the process. In additional to correct fragile-watermarking rounding errors, our algorithm can also be applied in robust watermarking to achieve higher watermarked image quality. [ABSTRACT FROM AUTHOR]

Highlights • The R-D relationship of the screen content video is related to video contents. On the basis of this fact, CTUs in screen content videos are classified into three types: T-CTUs, SI-CTUs and NI-CTUs according to their pixel activity and DCT coefficient distribution. • Disparate CTUs present vastly different R-D relationships. For this reason, instead of the unitive parameter update method in the R- λ model, the parameters used to code specific kind of CTU are updated according to the information of the last same kind of coded CTU rather than the last CTU. • Before assigning bits to each CTU, the bits allocated to text region, screen image region and nature image region are determined. When a scene change occurs, the bit allocation method which relied on the distortion and CTU number changes into the one only related to the number of CTUs. Abstract The popularity of 3D screen content videos like 3D animations calls for better coding performance of screen content videos. The exiting R- λ rate control model is designed to enable the coding performance of conventional nature videos, which can not accurately describe the Rate-Distortion (R-D) relationships of the screen content videos, especially for videos containing limited colors and sharper edges. This paper proposes a new rate control (RC) scheme which takes the divergent characteristics of text contents, screen images and nature images in screen content videos into consideration. In particular, the content-based rate control algorithm is developed at Coding Tree Unit (CTU) level, where CTUs are divided into three categories including Text-CTUs (T-CTUs), Screen Image-CTUs (SI-CTUs) and Nature Image-CTUs (NI-CTUs). Three R- λ models reflecting different R-D relationships of different CTUs are established, and the model parameters are updated on the basis of their own models. Furthermore, in view of the fact that continuity and mutability simultaneously exist in screen content videos, two bit allocation schemes for regions containing only one content are determined in this paper. Experimental results show that the proposed RC scheme achieves 0.749 dB BDPSNR increase under the Low Delay configuration and 0.637 dB BDPSNR increase under the Random Access configuration on average, compared to the default R- λ model in HEVC Screen Content Coding extension (HEVC-SCC) reference software. [ABSTRACT FROM AUTHOR]

The correlation filter based online tracking methods usually can achieve high real-time performance due to the leverage of the well-known FFT. However, they are also apt to generate the "corrupted training samples" in scenarios with complex background, which will trigger the model drift and deteriorate the tracking accuracy rapidly. The existing methods usually consider this problem from certain aspect and none of them has mined the potential of combining multiple formulas. In this paper, we propose the Output Constraint Transfer - Discriminative Scale Space Tracking (OCT-DSST) algorithm, which has taken full consideration of multiple channel feature, multiple filters, kernel trick, memory with incremental learning, and the self-supervision mechanism. We re-formulate the online tracking process by combining all formulas above in a unified framework. The so obtained adaptive learning rate can better exploit the feedback information coming from the intermediate tracking results, and effectively mitigate the corrupted sample problem. The experimental results on the OTB-50/100 and the VOT2016 datasets reveal that the proposed method is comparative to most state-of-the-arts algorithms, and can increase the accuracy by 2% and the success rate by 1.7%, compared to the traditional DSST method. [ABSTRACT FROM AUTHOR]

• An original classification of attacks on robust watermarks is proposed. • A review of watermarking schemes designed to counter targeted attacks is presented. • The main methods of countering targeted attacks are noted. • Promising research directions are discussed. Image watermarking is an effective and promising technology. Robust watermarks that are resistant to various attacks allow authors and owners of digital images to protect their rights to digital content, control its distribution and confirm its authenticity. Most of the modern algorithms for robust image watermarking aim to achieve resistance to a large number of different attacks. However, some authors develop algorithms designed to counter targeted attacks. The study of such schemes allows developers of watermarking algorithms to evaluate special means of counteracting various attacks, and then use them to create new robust schemes, both targeted and universal ones. In this paper, we present an overview of robust image watermarking schemes in terms of countering targeted attacks. We review the state-of-the-art in the field of attacking robust watermarks and propose a four-level classification of attacks that includes different levels of attack implementation, including an attacker's intent, characteristics of actions, the main target and an attack type. The proposed classification considers a watermark as an object of attack and summarizes various characteristics of attacks in a hierarchical manner. We analyze the means of countering common attacks such as image processing attacks, geometric attacks, print-scan and screen capture attacks, collusion attacks, and ambiguity attacks. Based on the results of our review, we highlight the most common methods of countering attacks and formulate promising areas of research in the field of methods for improving security of embedding schemes. [ABSTRACT FROM AUTHOR]

Researchers have been involved for decades in search of an efficient skin detection method. However, current methods have not overcome the significant challenges of skin detection, such as variation of illumination, various skin tones of different ethnic groups, and many others. This research proposed a clustering and region-growing-based skin detection method to overcome these limitations. Together with significant insight, these methods result in a more effective algorithm. The insight concerns the capability to dynamically define the number of clusters in a collection of pixels organized as images. In Clustering for most problem domains, the number of clusters is fixed prior and does not perform effectively over a wide variety of data contents. Therefore, this research paper proposed a skin detection method that validated the above findings. The proposed method assigns the number of clusters based on image properties and ultimately allows freedom from manual thresholds or other manual operations. The dynamic determination of clustering outcomes allows for greater automation of skin detection when dealing with uncertain real-world conditions. • Human skin tone is uniform, and clustering is an efficient method to detect human skin. • CIELAB color space is better for separating illumination from the image. • A small number of significant objects can occupy most of the pixel in an image. • Pixel connectivity of an image can automatically decide the number of clusters. • Clustering performs efficiently in Human skin detection. [ABSTRACT FROM AUTHOR]

Domain Generalization (DG) model is an important tool to improve the robustness of person re-identification algorithm, but the domain gap makes it difficult to transfer knowledge cross-domain effectively. To solve the above problems, this paper proposes a generalization model based on a Meta-Bond Domain Alignment (M−BDA) model. To learn a generalizable model, a meta -learning strategy is introduced to simulate the training–testing process of the domain generalization. Then a bond-domain module is constructed in the training to align the source domain, which can reduce the domain gap between the source domain and the target domain, and facilitate the knowledge transfer. Finally, the bond-domain loss is counted on the feature space of the bond-domain to prevent the generated bond-domain from overfitting to the source domain. The experimental results show that the proposed algorithm achieves better performance in DukeMTMC-ReID, Market1501 and MSMT17 alternating as the source and the target domain tasks. In the generalization experiments of Market-1501 → DukeMTMC-ReID, mAP and Rank-1 increased by 7.7 % and 3.6 %, respectively, and in the generalization experiments of DukeMTMC-ReID → Market-1501, mAP and Rank-1 are increased by 9.3 % and 3.5 %, respectively, which are significantly better than the newer representative algorithms. [ABSTRACT FROM AUTHOR]

Locally Sensitive Hashing (LSH) algorithms are classical algorithms commonly used on the c-Approximate Nearest Neighbor (c-ANN) search problem. When using Euclidean distance to measure sample similarity and solve the c-ANN problem, the traditional approach is to utilize the Exact Euclidean Locality Sensitive Hashing (E2LSH) algorithm based on the p-stable distribution. However, the uncertainty of the p-stable distribution causes the hash buckets constructed by the E2LSH algorithm to vary in queries. Therefore, this paper proposes the OLLSH algorithm based on the Weighted Majority algorithm in the Online-Learning framework, which selects the hash buckets with more stable query accuracy by weighted voting on the hash buckets generated by the E2LSH algorithm. Then, we conduct simulation experiments on synthetic dataset and four real data sets and conclude that the proposed OLLSH algorithm improves the accuracy compared to the original algorithm with the same memory usage. • The parameters of the E2LSH algorithm are analysed in detail. • The choice of hash buckets for the E2LSH algorithm is achieved with the WM algorithm. • The OLLSH algorithm saves memory by using less hash buckets during the query process. • The efficacy of the OLLSH algorithm was proved on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

Existing stereo matching algorithms suffer from issues such as susceptibility to distortion, weak noise resistance, and a high rate of mismatches in regions with weak textures and discontinuous disparities. To address these challenges, this paper proposes a stereo matching algorithm based on an improved census transform and minimum spanning tree (MST) cost aggregation. In the cost calculation phase, we employ a Gaussian-weighted transformation window and incorporate gradient and edge information to perform weighted fusion of the results. In the cost aggregation process, we introduce a collaborative adaptive window. Each pixel acquires information from the support window of the guided filter (GF) and other pixels within the MST. Furthermore, we integrate the SLIC superpixel segmentation algorithm into MST construction. These two components work synergistically to assign appropriate adaptive weights to pixels, facilitating coordinated cost volume aggregation. Different optimization methods are applied to address mismatched points of various types in post-disparity processing.Performance evaluation using the Middlebury dataset and KITTI dataset demonstrates that our proposed algorithm not only enhances matching accuracy in regions with discontinuous disparities and weak textures but also exhibits significantly improved robustness to interference. Additionally, the resulting disparity map displays smoother edges. • Cost Calculation Phase: Introducing a Gaussian-weighted transformation window, along with gradient and edge information, enables weighted fusion of results. • Cost Aggregation Process: Collaborative adaptive window introduction allows each pixel to gather information from the support window of the guided filter and neighboring pixels within the Minimum Spanning Tree(MST).Furthermore,the SLIC superpixel segmentation algorithm is integrated into the construction of the MST. • Post-Disparity Processing Optimization: Applying diverse optimization methods addresses mismatches among different types of points. [ABSTRACT FROM AUTHOR]

• A small target detection algorithm based on feature interaction and guided learning. • A feature guided alignment module is designed to obtain more effective features. • Gaussian distribution sample allocation strategy is introduced to obtain better samples. • Feature interaction and decoupling detection enhance the detection task capability. • A new regression loss function is used to optimize the localization ability to a certain extent. At present, for the diverse target sizes, different angles, and complex environment in aerial images, how to detect small objects effectively in aerial images is still a challenge task. In order to further improve the detection accuracy and reduce the missing rate, in this paper, we proposes a small target detection algorithm based on feature interaction and guided learning. Firstly, the feature guided alignment module is designed to deal with the problem of information loss and overlap. Secondly, the misallocation of small object samples is optimized by introducing gaussian sample allocation strategy. Then the interactive parallel detection header is designed to solve the detection task conflict problem. Finally, the new regression loss function is designed to enhance the localization ability of the target. The experimental results on the Tinyperson dataset and Visdrone dataset show that this model can improve the detection accuracy and reduce the missing rate of small targets effectively. [ABSTRACT FROM AUTHOR]

To overcome challenges in multiple-object tracking (MOT) tasks, recent algorithms use interaction cues alongside motion and appearance features. These algorithms use graph neural networks or transformers to extract interaction features that lead to high computation costs. In this paper, a novel interaction cue based on geometric features is presented aiming to detect occlusion and reidentify lost targets with low computational costs. Moreover, in the majority of algorithms, camera motion is considered negligible, which is a strong assumption that is not always true and can lead to identity (ID) switching or mismatching of targets. In this paper, a method for measuring camera motion is presented that efficiently reduces its effect on tracking. The proposed algorithm is evaluated on MOT17 and MOT20 datasets and achieves state-of-the-art performance on MOT17 with comparable results on MOT20. The code is also publicly available. 1 1 https://github.com/mhnasseri/for_tracking. • Cascade association based on the detection score. • A new similarity metric that considers detection's size. • A novel interaction model for occlusion handling and target re-identification. • A novel algorithm for estimating camera motion and removing it. • State-of-the-art results on MOT17 and comparable results on MOT20. [ABSTRACT FROM AUTHOR]

Depth image based rendering technology is essential for free viewpoint video systems. Because of the compression of depth images and limitations of rendering algorithms, various types of distortion might occur in the virtual viewpoints and cannot effectively be evaluated by traditional two-dimensional assessment methods. Hence, this paper proposes a method for virtual viewpoint quality assessment using the visual masking effect. First, shift is compensated for the distorted virtual viewpoint and then the compensated virtual viewpoint is objectively assessed. Next, according to human visual characteristics such as texture, magnitude, and distribution masking, the corresponding visual sensitivity map and visual masks are extracted. Finally, the visual masking and all factors are pooled to create the final quality score. As verified by the experimental results, the method proposed in this paper corresponds with the characteristics of human vision and can serve as a more effective method for assessing the quality of virtual viewpoints. [ABSTRACT FROM AUTHOR]

Rate control is of great significance for the High Efficiency Video Coding (HEVC). Due to the high efficiency and low complexity, the R-lambda model has been applied to the HEVC as the default rate control algorithm. However, the video content complexity, which can help improve the code efficiency and rate control performance, is not fully considered in the R-lambda model. To address this problem, an intra-frame rate control algorithm, which aims to provide improved and smooth video quality, is developed in this paper by jointly taking into consideration the frame-level content complexity between the encoded intra frames and the encoded inter frame, as well as the CTU-level complexity among different CTUs in texture–different regions for intra-frame. Firstly, in order to improve the rate control efficiency, this paper introduces a new prediction measure of content complexity for CTUs of intra-frame by jointly considering the inter-frame correlations between encoding intra frame and previous encoded inter frames as well as correlations between encoding intra frame and previous encoded intra frame. Secondly, a frame-level complexity-based bit-allocation-balancing method, by jointly considering the inter-frame correlation between intra frame and previous encoded inter frame, is brought up so that the smoothness of the visual quality can be improved between adjacent inter- and intra-frames. Thirdly, a new region-division and complexity-based CTU-level bit allocation method is developed to improve the objective quality and to reduce PSNR fluctuation among CTUs in intra-frame. In the end, related model parameters are updated during the encoding process to increase rate control accuracy. As a result, as can be seen from the extensive experimental results that compared with the state-of-the-art schemes, the video quality can be significantly improved. More specifically, up to 10.5% and on average 5.2% BD-Rate reduction was achieved compared to HM16.0 and up to 2.7% and an average of 2.0% BD-Rate reduction was achieved compared to state-of-the-art algorithm. Besides, a superior performance in enhancing the smoothness of quality can be achieved, which outperforms the state-of-the-art algorithms in term of flicker measurement, frame and CTU-wise PSNR, as well as buffer fullness. [ABSTRACT FROM AUTHOR]

Object drift determination is a crucial issue in long-term tracking. Most existing object drift determination criteria require manually selecting different thresholds on different datasets to determine whether the object is lost. In this case, choosing the appropriate threshold is a complex problem. An object drift determination network based on dual-template joint decision-making is proposed to address this issue. The proposed object drift determination network not only does not require selection of thresholds on different datasets, and can be used as a plug-and-play module of short-term visual tracking algorithm to achieve long-term visual tracking tasks with good generalization ability. The proposed object drift determination network is applied to four short-term baseline trackers and constructs four long-term visual tracking algorithms. Experimental results verify that all four improved algorithms significantly improve long-term visual tracking performance compared to the original algorithms. In addition, the determined speed of the object drift determination network proposed in this paper reaches 111 FPS, which has little effect on the long-term visual tracking speed. • New drift determination network without threshold selection is proposed. • New template update criterion is proposed to ensure the quality of templates. • Capable of acting as a plug-and-play module for most basic trackers. • The effectiveness of the proposed method is verified on five public datasets. [ABSTRACT FROM AUTHOR]

• A layout plane is a closed loop surrounded by layout keypoints. • The wall structure can be parametrization as an undirected graph. • Layout keypoint prediction is relative to human pose estimation. • Cuboid room assumption is limited in real life. Room layout estimation aims to predict the spatial structure of a room from a single image. Most existing methods relying on 2D cues are suitable for cuboid rooms, but non-cuboid rooms. 3D layout estimation methods can reconstruct 3D models of general rooms, but they are trained with depth information on high collection costs, consume large computation resources, and run slowly. This paper considers an undirected graph representation method of the general room layouts, which includes cuboid and non-cuboid rooms consisting of a single ceiling, a single floor, and multiple walls. To this end, we first predict the positions of the layout vertices and then use the network automatically learn the connection relationship between the vertices. The final layout is obtained through a simple layout inference post-processing algorithm. The experimental results both on cuboid and non-cuboid datasets validate the effectiveness and efficiency of our method. The code is available at https://github.com/Hui-Yao/2D-graph-layout-estimation. [ABSTRACT FROM AUTHOR]

Image segmentation is one of the fundamental steps in image analysis for object identification. The main goal of image segmentation is to recognize homogeneous regions within an image as distinct and belonging to different objects. Inspired by the idea of the packing problem, in this paper, we propose a fast O ( N α ( N ) ) -time algorithm for image segmentation by using Non-symmetry and Anti-packing Model and Extended Shading representation, which was called the NAMES-based algorithm, where N is the number of homogenous blocks and α ( N ) is the inverse of the Ackerman’s function and it is a very slowly growing function. We first put forward four extended Lemmas and two extended Theorems. Then, we present a new scanning method used to process each NAMES block. Finally, we propose a novel NAMES-based data structure used to merge two regions. With the same experimental conditions and the same time complexity, our proposed NAMES-based algorithm, which extends the popular hierarchical representation model to a new non-hierarchical representation model, has about 86.75% and 89.47% average execution time improvement ratio when compared to the Binary Partition Tree (BPT)-based algorithm and the Quadtree Shading (QS)-based algorithm which has about 55.4% execution time improvement ratio when the QS-based algorithm itself is compared to the previous fastest region segmentation algorithm by Fiorio and Gustedt whose O ( N 2 ) -time algorithm is run on the original N × N gray image. Further, the NAMES can improve the memory-saving by 28.85% (5.04%) and simultaneously reduce the number of the homogeneous blocks by 49.05% (36.04%) more than the QS (the BPT) whereas maintaining the satisfactory image quality. Therefore, by comparing our NAMES-based algorithm with the QS-based algorithm and the BPT-based algorithm, the experimental results presented in this paper show that the former has not only higher compression ratio and less number of homogenous blocks than the latter whereas maintaining the satisfactory image quality, but also can significantly improve the execution speed for image segmentation, and therefore it is a much more effective algorithm for image segmentation. [ABSTRACT FROM AUTHOR]

Image mosaicing, the process of obtaining a wider field-of-view of a scene from a sequence of partial views, has been an attractive research area because of its wide range of applications, including motion detection, resolution enhancement, monitoring global land usage, and medical imaging. A number of image mosaicing algorithms have been proposed over the last two decades. This paper provides an in-depth survey of the existing image mosaicing algorithms by classifying them into several groups. For each group, the fundamental concepts are first explained and then the modifications made to the basic concepts by different researchers are explained. Furthermore, this paper also discusses the advantages and disadvantages of all the mosaicing groups. [ABSTRACT FROM AUTHOR]

This paper introduces VEDAI: Vehicle Detection in Aerial Imagery a new database of aerial images provided as a tool to benchmark automatic target recognition algorithms in unconstrained environments. The vehicles contained in the database, in addition of being small, exhibit different variabilities such as multiple orientations, lighting/shadowing changes, specularities or occlusions. Furthermore, each image is available in several spectral bands and resolutions. A precise experimental protocol is also given, ensuring that the experimental results obtained by different people can be properly reproduce and compared. Finally, the paper also gives the performance of baseline algorithms on this dataset, for different settings of these algorithms, to illustrate the difficulties of the task and provide baseline comparisons. [ABSTRACT FROM AUTHOR]

In HEVC (High Efficiency Video Coding), linear interpolation of the boundary pixels is used as the predictor and pixels within the same PU (Prediction Unit) are in the same prediction direction. When the PU block is large and in high complexity, the prediction performance would worsen. Although many algorithms use the pixel-based weighted averaging or interpolation operations to perform the prediction which improves the performance of bitrate saving notably, there is still space for further improvement. This paper proposed an improved SAP (Sample-based Angular Prediction) algorithm based on adaptive directional prediction (ADSAP). The innovation of this paper lies in two aspects: it puts forward a new method to estimate the best prediction direction of current pixel and it introduces the concept of “frame-level pretreatment” which could greatly improve the encoding speed. Experimental results show that it could save the output bitrate by about 9.4158% when compared with the HEVC lossless intra prediction algorithm, which is better than other typical algorithms. Besides, the encoding and decoding time are reduced by about 11%. Moreover, when the minimum PU size gets larger, the increase of output bitrate of ADSAP is much less than that of HEVC and the conventional SAP algorithm, which makes the proposed algorithm quite suitable for the compression of high resolution videos. [ABSTRACT FROM AUTHOR]

Due to the limited capture range of common imaging sensors, a scene with high dynamic range usually cannot be well described by a single still image because some regions in it may be under-exposed or over-exposed. In this paper, a new multi-exposure fusion method based on dense scale invariant feature transform (SIFT) is presented. In our algorithm, the dense SIFT descriptor is first employed as the activity level measurement to extract local details from source images, and then adopted to remove ghosting artifacts when the captured scene is dynamic with moving objects. Furthermore, two popular weight distribution strategies for local contrast extraction, namely, “weighted-average” and “winner-take-all” are studied in this paper. The effects of these two strategies on the fusion results are compared and discussed. Experimental results demonstrate the effectiveness of the proposed method in terms of both visual quality and objective evaluation. [ABSTRACT FROM AUTHOR]

This paper proposes a novel face recognition method that improves Huang’s linear discriminant regression classification (LDRC) algorithm. The original work finds a discriminant subspace by maximizing the between-class reconstruction error and minimizing the within-class reconstruction error simultaneously, where the reconstruction error is obtained using Linear Regression Classification (LRC). However, the maximization of the overall between-class reconstruction error is easily dominated by some large class-specific between-class reconstruction errors, which makes the following LRC erroneous. This paper adopts a better between-class reconstruction error measurement which is obtained using the collaborative representation instead of class-specific representation and can be regarded as the lower bound of all the class-specific between-class reconstruction errors. Therefore, the maximization of the collaborative between-class reconstruction error maximizes each class-specific between-class reconstruction and emphasizes the small class-specific between-class reconstruction errors, which is beneficial for the following LRC. Extensive experiments are conducted and the effectiveness of the proposed method is verified. [ABSTRACT FROM AUTHOR]

The latest Versatile Video Coding (VVC) standard incorporates a series of effective and complex new intra coding tools, which obtains superior coding efficiency than the High Efficiency Video Coding (HEVC). However, this makes the intra coding more complicated and time-consuming. A fast algorithm for VVC from two aspects of fast mode decision and fast partition decision is proposed in this paper. For the fast mode decision, the relationship between bins with Histogram of Oriented Gradient (HOG) and intra modes is created for the mode selection, decreasing the planar modes for SATD and RDO. Moreover, we analyze the maximum bins to determine the final modes, and we use the modes of left and upper blocks as a reference for the current CU, which can early terminate RDO. Moreover, a two-step fast partition algorithm is proposed based on HOG for fast partition decision, in which two thresholds are investigated to control the uniformity of textures. The proposed fast algorithm is implemented on the VVC test model, and the experimental results show that it can achieve 69.07% time savings with only 2.96% BDBR increases averagely, which outperforms other relatively existing state-of-the-art methods. Moreover, to convince the universality of our algorithm, we further implement our method in Fraunhofer Versatile Video Encoder (VVenc) and Fraunhofer Versatile Video Decoder (VVdec), which have five settings to control the trade-off between encoding quality and efficiency for intra coding. The fast intra mode decision algorithm and fast partition algorithm decrease the complexity of intra coding for both VTM and VVenc, which shows the efficiency and universality of the proposed fast partition and fast mode decision algorithms. • The CUs' textures have strong correlations with the HOG operator. • A mapping table of VVC intra modes with HOG operator is created. • The HOG operator leads the intra mode decision to decline the probability and number of RDO operations. • The HOG operator controls the trade-off between BDBR and saving time with two VVC intra partition decision thresholds. • The VVC intra coding is accelerated with fast intra mode and fast partition decisions with HOG operator. [ABSTRACT FROM AUTHOR]

Most deep learning-based image enhancement algorithms have been developed based on the image-to-image translation approach, in which enhancement processes are difficult to interpret. In this paper, we propose a novel interpretable image enhancement algorithm that estimates multiple transformation functions to describe complex color mapping. First, we develop a histogram-based multiple transformation function estimation network (HMTF-Net) to estimate multiple transformation functions by exploiting both the spatial and statistical information of the input images. Second, we estimate pixel-wise weight maps, which indicate the contribution of each transformation function at each pixel, based on the local structures of the input image and the transformed images obtained by each transformation function. Finally, we obtain the enhanced image as the weighted sum of the transformed images using the estimated weight maps. Extensive experiments confirm the effectiveness of the proposed approach and demonstrate that the proposed algorithm outperforms state-of-the-art image enhancement algorithms for different image enhancement tasks. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, inspired by the idea of overlapping rectangular region coding of binary images, we extend the SDS design, which is based on overlapping representation from binary images to gray images based on the non-symmetry and anti-packing model (NAM). A novel gray image representation is proposed by using the overlapping rectangular NAM (RNAM) and the extended Gouraud shading approach, which is called ORNAM representation. Also, we present an ORNAM representation algorithm of gray images. The encoding and the decoding of the proposed algorithm can be performed in O(n log n) time and O(n) time, respectively, where n denotes the number of pixels in a gray image. The wrong decoding problem of the hybrid matrix R for the overlapping RNAM representation of gray images is solved by using the horizontal, vertical, and isolated matrices, i.e., H, V and I, respectively, which are used to identify the vertex types. Also, we put forward four criteria of anti-packing homogeneous blocks. In addition, by redefining a codeword set for the three vertices symbols, we also propose a new coordinate data compression procedure for coding the coordinates of all non-zone elements in the three matrices H, V and I. By taking some idiomatic standard gray images in the field of image processing as typical test objects, and by comparing our proposed ORNAM representation with the conventional S-Tree Coding (STC) representation, the experimental results in this paper show that the former has higher compression ratio and less number of homogeneous blocks than the latter whereas maintaining a satisfactory image quality, and therefore it is a better method to represent gray images. [Copyright &y& Elsevier]

Abstract: In this paper, we study the problem of feature extraction for pattern classification applications. RELIEF is considered as one of the best-performed algorithms for assessing the quality of features for pattern classification. Its extension, local feature extraction (LFE), was proposed recently and was shown to outperform RELIEF. In this paper, we extend LFE to the nonlinear case, and develop a new algorithm called kernel LFE (KLFE). Compared with other feature extraction algorithms, KLFE enjoys nice properties such as low computational complexity, and high probability of identifying relevant features; this is because KLFE is a nonlinear wrapper feature extraction method and consists of solving a simple convex optimization problem. The experimental results have shown the superiority of KLFE over the existing algorithms. [Copyright &y& Elsevier]

In this paper, a moving object detection algorithm is combined with a background estimate and a Bing (Binary Norm Gradient) object is proposed in video surveillance. A simple background estimation method is used to detect rough images of a group of moving foreground objects. The foreground setting in the foreground will estimate another set of candidate object windows, and the target (pedestrian/vehicle) from the intersection area comes from the first two steps. In addition, the time cost is reduced by the estimated area. Experiments on outdoor datasets show that the proposed method can not only achieve high detection rate (DR), but also reduce false alarm rate (FAR) and time cost. [ABSTRACT FROM AUTHOR]

Highlights • We model the RD cost comparison as a MDP for low complexity video coding. • We learn a CU early termination classifier with end-to-end actor-critic RL algorithm. • A CU early termination algorithm independent of depths is derived for whole CTU. Abstract In this paper, we propose a Reinforcement Learning (RL) based Coding Unit (CU) early termination algorithm for High Efficiency Video Coding (HEVC). RL is utilized to learn a CU early termination classifier independent of depths for low complexity video coding. Firstly, we model the process of CU decision as a Markov Decision Process (MDP) according to the Markov property of CU decision. Secondly, based on the MDP, a CU early termination classifier independent of depths is learned from trajectories of CU decision across different depths with the end-to-end actor-critic RL algorithm. Finally, a CU decision early termination algorithm is introduced with the learned classifier, so as to reduce computational complexity of CU decision. We implement the proposed scheme with different neural network structures. Two different neural network structures are utilized in the implementation of RL based video encoder, which are evaluated to reduce video coding complexity by 34.34 % and 43.33 %. With regard to Bj ø ntegaard delta peak signal-to-noise ratio and Bj ø ntegaard delta bit rate, the results are −0.033 dB and 0.85 % , −0.099 dB and 2.56 % respectively on average under low delay B main configuration, when compared with the HEVC test model version 16.5. [ABSTRACT FROM AUTHOR]

Abstract To protect the privacy of users, tables generally must be anonymized before publication. All existing anonymous methods have deficiencies. They do not consider the differences in attributes, or the optimization of information loss and time efficiency. his paper proposes a new method called KACM to realize k-anonymity. This method is mainly used for hybrid tables. The calculation of the distance between records considers the connection between quasi-identifier attributes and sensitive attributes, their effect on the sensitive privacy, and the information loss during the anonymity process. In the clustering process, the records with the minimum distance are always selected to add, and the clustering is individually controlled according to k to realize the equalization division of the equivalence class and reduce the total amount of distance calculation. Finally, the validity and practicability of the method are proved using theory and experiment. [ABSTRACT FROM AUTHOR]

Highlights • We present a new boosting technique for TGV based image denoising and inpainting. • The proposed technique first takes average of restored image with observed image. • It then operates the TGV algorithm with the average as input with dynamic parameter. • Algorithm is discussed on effect of parameters and compared with several methods. Abstract In this paper we present a new adaptive boosting technique for total generalized variation (TGV) based image denoising and inpainting. Instead of the strengthening and substracting steps in existing boosting techniques, the proposed technique is iteratively operated by two steps: the first step is to take average of restored image with observed image, and updated parameter; the second step is to operate the TGV restoration algorithm with the average and dynamic parameter. For each iteration, as the input contains more correct information, the restoration algorithm can produce signals with more details. We have solved our boosting TGV model by primal-dual method, and applied the boosting TGV technique for gray/color image denoising and inpainting. Our algorithms have been discussed about influence of parameters, computational cost and compared with several typical existing methods. Plenty of experimental results show that our method can produce images with more structures and prevent staircase artifacts effectively. [ABSTRACT FROM AUTHOR]

In response to the increased demand for high-resolution video, the new generation of video standards, High Efficiency Video Coding (HEVC) and its scalable extension (SHVC) have been finalized. The compression of HEVC/SHVC is efficiently improved and supports ultra-high resolution (UHD). Therefore, the coding complexity of HEVC/SHVC is much higher than those of previous standards. The framework of SHVC is based on HEVC and is divided into several types of scalable video. SHVC can be decoded into various video resolutions, frame rates and qualities, and only needs to be encoded once, but with higher complexity than HEVC. Thus, how to reduce the coding complexity of SHVC is the purpose of this paper. Our proposed algorithm accelerates the enhancement layer (EL) prediction by utilizing encoded Coding Unit (CU) sizes, prediction modes, motion vectors and Rate-Distortion Costs (RD-Costs) of the base layer (BL) and encoded CU sizes of the enhancement layer for quality scalability of SHVC. Experimental results show that the proposed algorithm can save lots of time while maintaining good video quality, and the performance is better than those of previous works. [ABSTRACT FROM AUTHOR]

In this paper we propose a novel approach for detecting and tracking objects in videos captured by moving cameras without any additional sensor. In such a video both the background and foreground change in each frame of the image sequence; making the separation of actual moving object from the background a challenging task. In this work, moving objects are detected as clusters of spatio-temporal blobs generated by spatio-temporal analysis of the image sequence using a three-dimensional Gabor filter and merged using Minimum Spanning Tree. Problem of data association during tracking is solved by Linear Assignment Problem and occlusion is handled by the application of Kalman filter. The major advantage of the proposed method is that, it does not require initialization or training on sample data to perform. Our algorithm demonstrated very satisfactory state-of-the-art result on benchmark videos. The performance of the algorithm is equivalent or superior to some benchmark algorithms. [ABSTRACT FROM AUTHOR]

Highlights • We present a novel view-based 3D object retrieval framework, which is deployed over a graph-based collaborative learning scheme to intelligently fuse multiple features. • The hypergraph based collaborative feature learning scheme to fuse complement descriptors from both the contour and the interior region of 3D object effectively. • The view-based 3D object retrieval is done via a greedy bipartite graph matching algorithm, which achieves highly accurate and efficient 3D object matching. • The interactive learning procedure with user feedbacks achieved better performance. Abstract 3D object retrieval has attractive extensive research focus in recent years. Among various schemes, view based 3D object retrieval is regarded as a promising direction. In this paper, we present a novel view-based 3D object retrieval framework, which is deployed over a graph-based collaborative learning scheme to intelligently fuse multiple features. In particular, we introduce a hypergraph based collaborative feature learning scheme to fuse complement descriptors from both the contour and the interior region of 3D object effectively. Then, the view-based 3D object retrieval is done via a greedy bipartite graph matching algorithm, which achieves highly accurate and efficient 3D object matching. With the above bipartite graph matching and feature concatenation, significant performance improvement is achieved in the 3D object retrieval task, on either widely-used benchmark datasets or open competitions like SHREC15 challenge. In both evaluations, the proposed graph-based collaborative feature learning scheme has beaten a serial of existing approaches and state-of-the-art schemes. [ABSTRACT FROM AUTHOR]

Highlights • A novel algorithm based on the sparse subspace is proposed. • The approach first learns a subset of the original training data to build a much correlated dictionary, and learns the reconstruction coefficients for each unlabeled data while considering the influence of its local neighbors. • The approach takes advantages of the linear regression techniques together with the weighted collaborative representation techniques to learn more discriminative representation coefficients for unlabeled data. • Experimental results demonstrate its effectiveness. Abstract This paper takes into account both unlabeled data and their local neighbors to learn their sparse representations, and proposes a face recognition approach named Weighted Locality Collaborative Representation Classifier based on sparse subspace (WLCRC). WLCRC firstly learns a subset of the original training data to build a much correlated dictionary, and then combines linear regression techniques together with weighted collaborative representation techniques to optimize the linear reconstruction of unlabeled data. It uses the newly built dictionary to learn the reconstruction coefficients for each unlabeled datum while considering the influence of its local neighbors. Classifications are performed according to the reconstruction residuals, and experimental results on benchmark datasets demonstrate that WLCRC is effective. [ABSTRACT FROM AUTHOR]

Highlights • Two joint image compression and encryption schemes based on JPEG are proposed. • A trade-off between encryption power and compression ability is considered. • The first encryption scheme has better compression performance. • The second encryption scheme has stronger security level. Abstract Joint image encryption and compression schemes have shown their great potential values in protecting compressed images. To achieve the protection, a trade-off between encryption power and compression ability needs to be considered. In this paper, we propose two new joint encryption and compression schemes, where one scheme emphasizes compression performance, another highlights protection performance. For a given plain-image, we first raster scan it into non-overlapping 16 × 16 blocks, then apply various encryption techniques to it. In the first scheme, encryption operations are conducted at the transformation stage and quantization stage of JPEG. As for the second scheme, we add the run/size and value (RSV) pairs' shuffling operation at JPEG's entropy coding stage after first scheme's encryption operations. Performance evaluations using various criteria are conducted to show that the first scheme has better compression efficiency, while the second scheme has better defense ability against the statistical attack. [ABSTRACT FROM AUTHOR]

Abstract: 3D reconstruction is a major problem in computer vision. This paper considers the problem of reconstructing 3D structures, given a 2D video sequence. This problem is challenging since it is difficult to identify the trajectory of each object point/pixel over time. Traditional stereo 3D reconstruction methods and volumetric 3D reconstruction methods suffer from the blank wall problem, and the estimated dense depth map is not smooth, resulting in loss of actual geometric structures such as planes. To retain geometric structures embedded in the 3D scene, this paper proposes a novel surface fitting approach for 3D dense reconstruction. Specifically, we develop an expanded deterministic annealing algorithm to decompose 3D point cloud to multiple geometric structures, and estimate the parameters of each geometric structure. In this paper, we only consider plane structure, but our methodology can be extended to other parametric geometric structures such as spheres, cylinders, and cones. The experimental results show that the new approach is able to segment 3D point cloud into appropriate geometric structures and generate accurate 3D dense depth map. [Copyright &y& Elsevier]

Abstract: This paper proposes a novel segmentation method based on the scale space techniques endowed with a feature stability approach. The novelty of the paper is the lifetime of the space-scale blobs measured not only by their presence and disappearance but by the stability of the features characterizing the objects of interest as well. Our numerical experiments show that the algorithm outperforms the conventional space scale algorithm applied to variable size and variable shape objects. The proposed algorithm can be used as a preprocessing step in object or pattern recognition applications to produce seeds for more accurate image segmentation methods such as the snakes or the level set techniques. [Copyright &y& Elsevier]

Graphical abstract Highlights • A novel distance metric learning (DML) using cost sensitive for CBIR is proposed. • Three kinds of costs of the minority class are adequately discussed. • The method of replacing the kernel function is proposed for CBIR. How to retrieve the desired images quickly and accurately from the large scale image database has become a hot topic in the field of multimedia research. Many content-based image retrieval (CBIR) technologies already exist, but they are not always satisfactory. In many applications, the CBIR model based on machine learning relies heavily on the distance metric between samples. Although the traditional distance metric methods are simple and convenient, it is not always appropriate for CBIR tasks. In this paper, a novel distance metric learning (DML) method based on cost sensitive learning (CSL) is studied, and then it is used in a large margin distribution learning machine (LDM) to replace the traditional kernel functions. The improved LDM also takes into account CSL, and which is called CS-DLDM. Finally, CS-DLDM model is applied to CBIR tasks for implementation classification. We compare the proposed CS-DLDM model with other classifiers based on CSL. The experimental results show that the proposed CS-DLDM model not only has satisfactory classification performance but also the lowest misclassification cost, can effectively avoid the class imbalance of sample. [ABSTRACT FROM AUTHOR]

Highlights • An improved face recognition is proposed using the fusion of global and local structures. • A combination of preprocessing techniques coupled with discriminative feature extractors. • Improved performance under poor illumination, partial occlusion and low resolution. • Our best recognition results using FRGC 2.0.4 indicated more than 28% improvement. In this paper an improved face recognition algorithm under degrading conditions is proposed. The proposed algorithm uses a combination of preprocessing techniques coupled with discriminative feature extractors to obtain the best distinctive features for classification. Preprocessing approach is the fusion of multi-scale Weber and enhanced complex wavelet transform. Combination of multiple feature extraction based on Gabor filters, block-based local phase quantization (LPQ) coupled with principal component analysis (PCA) proved to be very effective to improve correct rate of recognition. We have also used two known classifiers, extreme learning machine (ELM), and sparse classifier (SC), and fused their outputs to obtain best recognition rate. Experimental results show improved performance of proposed algorithm under poor illumination, partial occlusion and low-quality images in uncontrolled conditions. Our best recognition results using second version of face recognition grand challenge (FRGC 2.0.4) which is the most challenging database, indicated more than 28% improvement over previous works. [ABSTRACT FROM AUTHOR]

Highlights • A method for visual object tracking is proposed. • We propose a decomposition model in visual tracking. • The decomposition model is utilized in an integrated generative model and discriminative model tracking framework. • Alternating direction method of multipliers algorithm guarantees the convergence of the decomposition model. This paper presents a multi-task based object tracking algorithm via a collaborative model. Under the framework of particle filtering, we develop a multi-task sparse learning based generative and discriminative classifier model. In the generative model, we propose a histogram based subspace learning method which takes advantage of adaptive templates update. In the discriminative model, we introduce an effective method to compute the confidence value which assigns more weights to the foreground than the background. A decomposition model is employed to take the common features and outliers of each particle into consideration. The alternating direction method of multipliers (ADMM) algorithm guarantees the optimization problem can be solved robustly and accurately. Qualitative and quantitative comparisons with nine state-of-the-art methods demonstrate the effectiveness and efficiency of our method in handling various challenges during tracking. [ABSTRACT FROM AUTHOR]

Video clipping system is very important in many intelligent applications. In order to shorten the time of video and extract the framework of the video, many methods have been proposed. But these methods just considered videos without taking sound into account. As we know, sound is also an important information for image processing. For example, many sport match videos include rich sound of audience and commentator such as NBA. In addition, human pay more attention to some video clips of interest (VCOI) such as scoring time instead of pause. So in this paper, we propose a sound-based video clipping framework toward specific sports scenes. First, we convert sound of sport videos to sonogram. For some aesthetically-pleasing images (APIM) such as slam dunk or jump shot, a set of object patches are selected using BING feature. Then, these object patches are ordered by our active object patches ranking algorithm. After that, ordered object patches and sonogram are fed into CNN respectively to obtain patch-level deep feature. In order to obtain image-level deep representation, deep feature extracted from ordered object patches are aggregated statistically into a deep representation. Finally, probabilistic model is used to select VCOI and APIM. Experiments on some NBA basketball matches have shown the effectiveness of our video clipping framework. [ABSTRACT FROM AUTHOR]

Highlights • We propose an intensity-tracking-guided multi-frame 4D fusion pipeline. • The 4D fusion is for improving the quality of high-dynamic 3D reconstruction. • The fusion combines temporal inter-frame fusion with intra-frame filtering. • The algorithm reduces spatial noise and temporal fluctuations of 3D sequences. • The algorithm is applicable on static, dynamic, or deformable 3D objects. The availability of high-speed 3D video sensors has greatly facilitated 3D shape acquisition of dynamic and deformable objects, but high frame rate 3D reconstruction is always degraded by spatial noise and temporal fluctuations. This paper presents a simple yet powerful dynamic 3D reconstruction improvement algorithm based on intensity video guided multi-frame 4D fusion. Temporal tracking of intensity image points (of moving and deforming objects) allows registration of the corresponding 3D model points, whose 3D noise and fluctuations are then reduced by spatio-temporal multi-frame 4D fusion. We conducted simulated noise tests and real experiments on four 3D objects using a 1000 fps 3D video sensor. The results demonstrate that the proposed algorithm is effective at reducing 3D noise and is robust against intensity noise. It outperforms existing algorithms with good scalability on both stationary and dynamic objects. [ABSTRACT FROM AUTHOR]

Availability of low-cost range sensors and the development of relatively robust algorithms for the extraction of skeleton joint locations have inspired many researchers to develop human activity recognition methods using 3-D data. In this paper, an effective method for the recognition of human activities from the normalized joint trajectories is proposed. We represent the actions as multidimensional signals and introduce a novel method for generating action templates by averaging the samples in a “dynamic time” sense. Then, in order to deal with the variations in speed and style of performing actions, we warp the samples with action templates by an efficient algorithm and employ wavelet filters to extract meaningful spatiotemporal features. The proposed method is also capable of modeling the human-object interactions, by performing the template generation and temporal warping procedure via the joint and object trajectories simultaneously. Experimental evaluations on several challenging datasets demonstrates the effectiveness of our method compared to the state-of-the-arts as well as its robustness against different sources of noise. [ABSTRACT FROM AUTHOR]

Highlights • Information about image feature extraction methods is provided. • A survey on image clustering for various applicative areas is carried out. • Research gaps and challenges in image clustering are discussed. • Image characteristics along with suitable clustering approaches are provided. A huge amount of image data is being collected in real world sectors. Image data analytics provides information about important facts and issues of a particular domain. But, it is challenging to handle voluminous, unstructured and unlabeled image collection. Clustering provides groups of homogeneous unlabeled data. Therefore, it is used quite often to access the interesting data easily and quickly. Image clustering is a process of partitioning image data into clusters on the basis of similarities. Whereas, features extracted from images are used for the computation of similarities among them. In this paper, significant feature extraction approaches and clustering methods applied on the image data from nine important applicative areas are reviewed. Medical, 3D imaging, oceanography, industrial automation, remote sensing, mobile phones, security and traffic control are considered applicative areas. Characteristics of images, suitable clustering approaches for each domain, challenges and future research directions for image clustering are discussed. [ABSTRACT FROM AUTHOR]

Highlights • Study of sampling schemes for data augmentation, taking into account prior knowledge. • Investigations on features generation methods: space coverage and landmark importance. • Analysis of semantic-driven sampling schemes compared to conventional blind sampling. • Results show that semantic-driven samplings provide faster and better convergence. Automatic face landmarking has received a lot of attention in the past decades. It is now mature enough to be implemented in fully autonomous video systems. As cascade-of-regression based algorithms have become state of the art in such systems, two major (and still relevant) sources of interest have slowly faded away: the need for semantic-driven learning beyond ground truth annotation, and full video chain performance i.e. tracking efficiency, which in the case of said methods strongly relates to their robustness towards shape initialization before fitting. In this paper, we investigate how data sampling using face priors can affect their performance in terms of convergence and robustness. We propose new strategies based on said priors to overcome inconsistencies observed during cascade-of-regression learning on purely random sampling-based stages. We will show that simple choices can be easily integrated within regression-based face tracking systems to increase accuracy and robustness. [ABSTRACT FROM AUTHOR]

Highlights • We present a novel framework for soccer video event detection. • We propose an Adaptive Label-Consistent K-SVD algorithm. • The proposed method automatically extracts semantic concepts of each soccer event. • Experiments indicate that our approach outperforms state-of-the-art approaches. In this paper, we formulate the soccer video event detection task as a sparse representation problem by learning a supervised, discriminative and event-oriented dictionary based on learned weighted local features. To this end, we present a novel framework based on two ideas: First, we propose an approach for computing the representativeness of each video frame for each soccer event. Second, we propose an Adaptive Label-Consistent K-SVD (ALC-KSVD) algorithm to learn an event-oriented and discriminative dictionary based on the computed representativeness of frames to transfer video frames to a sparse space. To improve discrimination among frames of different events, we proposed a weighting method to identify local features that are more representative in each event category. Next, the representativeness score of each frame is calculated by aggregating the weighted local features within each frame. The calculated representativeness score of each frame indicates its belonging degree to each event. The representativeness score matrix, being a discriminative term, is combined with the reconstruction error to form an objective function to improve the discrimination ability in the sparse representation during the dictionary learning process. The obtained objective function is efficiently and optimally solved by the K-SVD algorithm. The representativeness score matrix, which is automatically calculated based on the training samples, defines an adaptive correspondence between the dictionary atoms and the labels of the frames. We demonstrate the effectiveness of the proposed framework on the detection and classification of several soccer events based on an extensive experimental investigation that was conducted using a large collection of video data. The experimental results indicate that our approach maintains good classification performance and outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Highlights • Two interaction mechanisms are allowed: fully-user-supervised (BnS) or machine-assisted (BnQ). • The manifold ranking is integrated with the query-based interactive mechanism. • The foreground-object-confidence update rule is efficient and effective. • The average response time per-interaction is quite short. • The initial bounding box can be imprecise. This paper presents an efficient and effective interactive segmentation scheme for extracting the region of a foreground object in an image. Our goal is to design an interactive segmentation algorithm that unifies the bounding-box-based, seed-based, and query-based interaction mechanisms for pursuing (i) high efficiency in simple interaction mechanism, (ii) few interaction rounds, and (iii) short response time. The proposed algorithm starts with a user-provided bounding box and obtains candidate background superpixels for inferring the foreground object. Our algorithm tolerates imprecise bounding boxes and provides two kinds of interactions for acquiring correct labels from the user. The user can either input the seed/scribble annotations or label the algorithm-queried regions. Our algorithm selects the most uncertain region as a query, and this query-based interaction mechanism reduces the burden of the user on deciding suitable annotation locations. The average response time per-interaction of our algorithm is merely 0.014 s. Our experiments demonstrate that the algorithm achieves an efficient unified scheme for interactive image segmentation. [ABSTRACT FROM AUTHOR]

Highlights • A novel method for video summarization based on keyframe extraction is proposed. • Keyframes are extracted using mean shift with global frame orientation features. • A new algorithm to remove the redundant frames of the video is proposed. • A new verification approach to objectively evaluate the video summary quality is proposed. • The proposed method achieves the state-of-the-art performances on several datasets. Video abstraction is an interesting topic that aims at briefly representing the entire video stream by producing a short summary either statically or dynamically. In this paper, we present an optimal static video summarization method based on keyframe extraction, termed as MSKVS. The proposed MSKVS has three major components: A new feature representation is exploited to describe the visual content of the video, a simple and fast algorithm is proposed to eliminate most similar and redundant frames, and an adaptive mean shift algorithm is used to select the most representative keyframes. We further develop a novel verification technique to measure the amount of information preserved by the produced summary and to make sure that it deserves to present the entire video stream regardless of human opinion impact. We report experimental results on six challenging datasets using different evaluation metrics, showing that MSKVS achieves state-of-the-art performances in a short computation time. [ABSTRACT FROM AUTHOR]

Highlights • We propose a weighted group-based simultaneous sparse coding model (WGSSC). • By introducing the SCESM prior, the sparse coefficients and weights of our model can be solved by AM algorithm. • The weighted nuclear norm minimization model can be seen a special case of our model. • We apply the proposed WGSSC-SCESM denoiser prior to more general IR problems under the ADMM framework. According to the nonlocal self-similarity property of natural images, group-based simultaneous sparse coding (GSSC) model assumes that nonlocal similar patches have similar sparse representations in a given dictionary and have been widely used in various image inverse problems. Inspired by the success of the GSSC mode in image restoration problems, this paper proposes a weighted group-based simultaneous sparse coding (WGSSC) model based on the spherically contoured exponential scale mixture (SCESM) prior for image restoration. Compared with traditional GSSC models, which often characterize the similar sparse coefficients by a common set of zero supports and lack spatial adaption and principled fashion, the proposed model considers the dependent relation and adaptivity of similar sparse coefficient, so it is more rational than the GSSC model. The similar sparse coefficients and scale variables can be jointly estimated by the alternating minimization algorithm with the SCESM prior. Based on the estimated sparse coefficients, we can reconstruct the clear patch group and obtain the global denoised image by averaging these patch groups. We refer this denoised method as WGSSC-SCESM based denoiser prior, which can be effectively plugged into general image restoration problems by the alternating direction method of multipliers (ADMM) techniques. Extensive experiments on various types of image restoration problems, e.g., image denoising, inpainting, deblurring and single image super-resolution, demonstrate that the proposed method outperforms many state-of-the-art restored methods in term of the objective and subjective metrics. [ABSTRACT FROM AUTHOR]

Graphical abstract Highlights • A feature visualization for CNNs method is exploited to improve co-segmentation. • Visualization is used as an auxiliary information to discriminate salient regions. • The visualization information is used as an extra energy term in the cost function. • Region occlusion sensitivity is introduced for feature visualization. • An adaptive strategy is designed to detect correct foreground/background regions. This paper addresses the co-segmentation problem using feature visualization for CNNs. Visualization is exploited as an auxiliary information to discriminate salient image regions (dubbed as “heat-regions”) from non-salient ones. Region occlusion sensitivity is proposed for feature visualization. The co-segmentation problem is formulated via a convex quadratic optimization which is initialized by the heat-regions. The information obtained through the visualization is considered as an extra energy term in the cost function. The results of the visualization demonstrate that there exist some heat-regions which are not productive in the co-segmentation. To detect helpful regions among them, an adaptive strategy in the form of an iterative algorithm is proposed according to the consistency among all images. Comparison experiments conducted on two benchmark datasets, iCoseg and MSRC, illustrate the superior performance of the proposed approach over state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Abstract: Point set registration is important for calibration of multiple cameras, 3D reconstruction and recognition, etc. The iterative closest point (ICP) algorithm is accurate and fast for point set registration in a same scale, but it does not handle the case with different scales. This paper instead introduces a novel approach named the scaling iterative closest point (SICP) algorithm which integrates a scale matrix with boundaries into the original ICP algorithm for scaling registration. At each iterative step of this algorithm, we set up correspondence between two m–D point sets, and then use a simple and fast iterative algorithm with the singular value decomposition (SVD) method and the properties of parabola incorporated to compute scale, rotation and translation transformations. The SICP algorithm has been proved to converge monotonically to a local minimum from any given parameters. Hence, to reach desired global minimum, good initial parameters are required which are successfully estimated in this paper by analyzing covariance matrices of point sets. The SICP algorithm is independent of shape representation and feature extraction, and thereby it is general for scaling registration of m–D point sets. Experimental results demonstrate its efficiency and accuracy compared with the standard ICP algorithm. [Copyright &y& Elsevier]