Search

Low-delay and error-resilient video coding is critical for real-time video communication over wireless networks. Intra-refresh coding, which embeds intra coded regions into inter frames can achieve a relatively smooth bit-rate and terminate the error propagation caused by the transmission loss. In this paper, we proposed a novel linear model for the intra-refresh cycle-size selection adapting to the network packet loss rates and the motions in the video content. We also analyze issues in designing the intra-refresh coding pattern and the refresh order, and propose a strategy which can adapt to different cycle-size and obtain better R-D performance compared with traditional random intra-refresh and vertical-partition intra-refresh. Experimental results show that the linear cycle-size selection model works effectively, where a 3dB improvement can be achieved compared with a fixed cycle-size. Also, with the proposed intra-refresh order, a 2.0% bitrate reduction is obtained in average compared with the vertical-partition intra-refresh.

We propose a framework, consisting of several algorithms to recognize human activities that involve manipulating objects. Our proposed algorithm identifies objects being manipulated and models high-level tasks being performed accordingly. Realistic settings for such tasks pose several problems for computer vision, including sporadic occlusion by subjects, non-frontal poses, and objects with few local features. We show how size and segmentation information derived from depth data can address these challenges using simple and fast techniques. In particular, we show how to robustly and without supervision find the manipulating hand, properly detect/recognize objects and properly use the temporal information to fill in the gaps between sporadically detected objects, all through careful inclusion of depth cues. We evaluate our approach on a challenging dataset of 12 kitchen tasks that involve 24 objects performed by 2 subjects. The entire framework yields 82%/84% precision (74%/83%recall) for task/object recognition. Our techniques outperform the state-of-the-art significantly in activity/object recognition.

Automatic object segmentation is a fundamentally difficult problem due to issues such as shadow, lighting, and semantic gaps. Edges play a critical role in object segmentation; however, it is almost impossible for the computer to know which edges correspond to object boundaries and which are caused by internal texture discontinuities. Active 3-D cameras, which provide streams of depth and RGB frames, are poised to become inexpensive and widespread. The depth discontinuities provide useful information for identifying object boundaries, which makes automatic object segmentation possible. However, the depth frames are extremely noisy. Also, the depth and RGB information often lose synchronization when the object is moving fast, due to different response time of the RGB and depth sensors. We show how to use the combined depth and RGB information to mitigate these problems and produce an accurate silhouette of the object. On a large dataset (24 objects with 1500 images), we provide both qualitative and quantitative evidences that our proposed techniques are effective.

H.264/AVC FRExt (Fidelity Range Extensions) and Motion JPEG 2000 are the current respective inter-frame and intra-frame coding standards for high resolution (HR) (e.g., 4096x2160) visual signals. It is commonly believed that an inter-frame method could achieve higher coding efficiency compared with an intra-frame one, due to the exploitation of video temporal redundancy. However, Motion JPEG 2000 has been selected as the digital cinema compression standard, and some existing work has demonstrated that JPEG 2000 is more suitable at HR situations. In this paper, we compare the rate-distortion (R-D) performance of these two different schemes and give more insight from both theoretical and experimental point of view. We derive an entropy-based R-D model to analyze the test results and the impact of residual entropy and quantization for inter-frame coding. Several extensions are introduced into H.264/AVC FRExt for HR video content for better performance. Experimental results show that these extensions lead to significantly higher coding efficiency and make our extended version more suitable for HR video coding

This paper proposes a region-based rate-control scheme for H.264/SVC. The inter-layer information is utilized to improve both the coding efficiency and the subjective quality of the moving region (MR). Firstly, we propose an MR extraction method by combining the inter-layer and inter-frame information. The difference between the current frame and the previous frame is used as the inter-frame information to detect moving areas. The motion information of the reference layer is used as the inter-layer information to combine with the inter-frame information to achieve a suitable MR extraction. Secondly, according to the coding result of the reference layer, we propose to sub-divide the non-MR into complex regions and flat regions based on a MacroBlock (MB) complexity measure. Experimental results demonstrate that the proposed scheme can obtain better quality both subjectively and objectively by using different weighting factors for the three regions.

This paper proposes a novel frame-level rate-control scheme for H.264/SVC for low-delay applications. The trend of the Mean Absolute Difference (MAD) (after motion compensation) of the reference layer is used as the inter-layer information for Inter-layer MAD prediction, which is combined with Intra-layer MAD prediction to predict the MAD of the enhancement layers (EL) more accurately. By considering the influence from both the texture and nontexture information, the varying picture complexity and feedback information from the actual encoding results are combined to achieve an accurate bit-allocation. A coarse-to-fine initial quantization parameter (QP) selection method is proposed to refine the initial QP of the EL according to the channel condition and the video sequence characteristics. Experimental results demonstrate that the proposed scheme can obtain high and smooth PSNR, and the output bit-rate is close to the target bit-rate. Also, the proposed scheme can avoid serious buffer fullness fluctuation and reduce skipped frames in the video coding.

Most automatic event detection methods for video surveillance detect target events based on features extracted in the pixel domain. However, in practice, surveillance videos are often compressed. It is desirable to perform automatic event detection in the compressed domain directly so that the video does not need to be decoded for analysis purpose. In this paper, we investigate the use of motion trajectories for video activity detection in the compressed domain. We show it is possible to extract reliable motion trajectories directly from compressed H.264 video streams. To overcome the problems caused by unreliable motion vectors, we propose to include the information from the compressed domain prediction residuals to make the tracking more robust. We use a real world application of detecting vacant or occupied parking spaces to demonstrate the effectiveness of our proposed approach. We also demonstrate the robustness of our approach to different encoder settings, and lighting conditions.

Inter-frame dependencies are usually ignored in video encoder coding parameter selection. This gives a non-optimal solution and degrades the compression performance. A mathematical model to estimate the importance of each pixel on the reconstructed video quality, called PixelRank, is developed in this paper. Theoretical analysis on the parameters used for PixelRank score calculation dealing with the video coding optimization problem is also given. The PixelRank algorithm tracks the importance of each pixel and distributes the PixelRank scores. With the PixelRank scores for all the pixels, MB-based quantization parameters are adjusted accordingly. Based on this technique, the rate can be allocated more accurately according to the importance of the pixels, thus achieving better overall rate-distortion performance. Compared to the non-optimized scheme in H.264/AVC, the proposed scheme can reduce 13.53% of the average bitrate and up to 25.17% of bitrate in the simulations.

This paper presents a rate-distortion optimized MPEG-4 FGS enhancement-layer truncation scheme. Our objective is to minimize the quality variation within each frame when the last transmitted enhancement layer is truncated according to the available network bandwidth. By properly redistributing the bit-budget to the enhancement layer that will be truncated, we can raise the quality of the whole frame uniformly. To achieve this goal, we use a trellis-search approach to decide which “1” bits will be truncated from the enhancement layer for each block. An operational rate-distortion optimization scheme based on the Lagrange Multiplier algorithm is adopted to decide the truncation criteria. Our proposed method can improve the visual quality and reduce the intra-frame quality variation both subjectively and objectively. Compared to previously reported straightforward truncation methods, our approach reduces the intra-frame quality variation significantly, and the decoded visual quality in terms of PSNR is also improved.

In H.264/MPEG-4 AVC encoding, computational complexity is the main concern for implementation and application. Motion estimation (ME) and Inter-mode decision (MD) consume a large portion of encoder computation power. ME is referred to the process to find the motion vector (MV) for a block, and MD is referred to the process to determine the best block-size for Inter-frame prediction. This paper proposes a new concept of introducing diversity into the MV search strategy. Based on the diversity concept, we introduce a new fast ME search strategy: adaptive diversity search strategy (ADSS), by combining the strength of different search strategies. We also investigate three fast ME strategies by MV merging and splitting for variable-size block motion estimation (VSBME), which explores the correlation of MVs of overlapping blocks. In addition, an Inter-mode decision is proposed by fast mode exclusion (FME). The experiment results show that the proposed fast ME can reduce the search points to about 4% of that by full search, and that the proposed FME can reduce the computation further, with negligible degradation in the rate-distortion (R-D) performance.

We propose a rate?distortion optimization algorithm that selects the quantization parameter and adjusts the level of the quantized DCT coefficients based on a rate?distortion measurement to achieve better video quality under a bit-rate constraint. The bit streams generated by the algorithm conform to the standards. The results show that the algorithm can achieve up to about 2.5-dB peak signal-to-noise ratio improvement compared to H.263+ TMN8 at the same bit rate.

Video transcoding is one of the key technologies in implementing dynamic adaptation of the bit-rate of a coded video bit-stream to the available bandwidth over various networks. Many fast transcoder architectures have been proposed to achieve fast processing. However, they suffer from quality degradation caused by the drift error. In this paper, we investigate the drift caused by the fast transcoder architectures for transcoding H.263 bit-streams. We discuss the limitations of the fast transcoder architectures and the flexibility that can be offered by a cascaded pixel-domain transcoder. Since the cascaded pixel-domain transcoder can achieve drift-free performance, we also propose methods to reduce the computational complexity of the drift-free cascaded pixel-domain transcoder.

A new region-based video coding technique, which combines region segmentation and geometric motion estimation, is proposed in this paper. The region segmentation algorithm based on both the histogram concavities and the probabilistic relaxation is applied to extract the significant regions. The merging of regions in the segmentation depends on the attributes of the regions, besides the motion vectors. A new approach based on the geometric features of objects and the scalable translation invariant rotation-to-shifting (STIRS) signatures is applied to the motion estimation of the global regions. Vector quantization (VQ) techniques are employed to solve the problems due to nonrigid object motions. With these techniques, the critical regions can be extracted with good performance and relatively low complexity. Furthermore, an uncovered/overlapped motion compensation method is presented. The integral algorithm is compared to Yokoyama's algorithm (Yokoyama et al., 1995, IEEE Trans. Circuit Systems Video Technol.5, 500?507). Our simulations show improved performance, in terms of both SNR improvement and reduction of computational requirements. For example, we achieve about 15% reduction of computational time and about 0.8 dB improvement in SNR for the “Miss America sequence.” The frame-to-frame SNR variation is also much smaller, which implies that the visual quality between frames is also very stable.

Computation reduction is important for the implementation of software video codecs. This paper describes a scheme for reducing the required computations in video encoders by skipping some of the discrete cosine transformation (DCT) and quantization calculations. We study the relationship between the mean absolute error (MAE) of the motion compensated difference blocks and the distribution of DCT coefficients. The relationship between the quantized DCT coefficients and the quantization level is also investigated. Based on the MAE of the motion compensated difference block and the quantization level, three options for the DCT calculation can be chosen: calculating all 8 × 8 DCT coefficients, calculating the DC coefficient only, or skipping the whole DCT computation. Simulation results show that the peak signal-to-noise ratio (PSNR) degradation is negligible with significant computation reduction achieved.