Search

Scanning of quantized transform coefficients is a very significant procedure in video coding. In H.264, it affects the coding efficiency of the following CABAC or CAVLC entropy coder directly. In this paper, we propose a novel edge-based predictive scanning scheme to improve the coding efficiency for inter-frame coding. This scheme includes three scanning pattern candidates. Besides zigzag pattern as defined in H.264, two alternative patterns are obtained by on-line training on frame level. Specifically, reference block is utilized to predict edge information in current 88 block. Based on predictive edge information, a suitable scanning pattern will be selected to scan the quantized coefficients of current 88 block. Since a similar prediction process can be done in the decoder side as well, no overhead is needed to be transmitted in the bit-stream. Experimental results show that the proposed edge-based predictive scanning scheme yields an average of 0.42% BD-bitrate reduction over the H.264 high profile. © 2011 IEEE.

Depth map is widely adopted and available in the 3D research area. Combining the depth map with the matting techniques is helpful to the original matte and depth image based rendering in 3D. Herein, in this paper, a novel adaptive depth map assisted matting approach with concise integration is presented and applied to achieve favorable matting results. In this approach, the Lagrange-multiplier-free closed form solution is firstly derived to reduce the computation complexity and to increase matting accuracy. Based on the work of Levin et al. on closed form matting, an improved alpha matte is then achieved by introducing an adaptive smoothness criterion which is the function of depth map variance. Finally, the matting system is capable of working in a full automatical way by generating the trimap from the depth information. Simulation results demonstrate that the proposed method is able to efficiently generate an alpha matte with an roughly user specified scribbles or an automatically generated trimap. © 2011 IEEE.

When encrypted data need to be sent to an untrusted computer for processing, it is highly desirable to have a homomorphic cryptosystem that allows the untrusted computer to process the encrypted signals totally without decryption such that, when the encrypted domain result is decrypted, the decrypted value is the same as the equivalent plaintext domain operation. These operations range from basic arithmetic operations to complicated transformations. In this paper, we analyze the existence of equivalent operations in the encrypted domain for four of the common bitwise operations: OR, AND, NOR and NAND in plaintext domain. We will show that such equivalent operations should not exist. For otherwise, if such operations exist, the RSA cryptosystems can be broken by a low-complexity attack. © 2011 IEEE.

In this paper, a new adaptive subspace learning model based on incremental nonparametric discriminant analysis (INDA) is proposed for visual tracking. Traditional subspace trackers focus on updating eigenvectors in handling with appearance variation of the target object, ignoring the non-target back-ground region during tracking. The INDA features take both of them into consideration, thereby promoting the tracking process in the ever-changing environment. Meanwhile, INDA relaxes the Gaussian assumption in Fisher discriminant analysis (FDA), so it can handle more general class distributions problem. The scatter matrices are also reformulated to update the subspace incrementally based on previous results. In conjunction with efficient feature extraction method, the system is real time capable. Numerous experiments show the superiority of our tracker over current states of art methods on several publicly available datasets.

When we design an anti-aliasing low pass filter, it is usually an IIR filter. We need to truncate the filter to an FIR filter. One may think that the more taps there are, the better the image quality is. However, we find that there exists an optimal value of tap number that will give the best visual quality. Filters with larger or smaller number of taps will degrade the image quality, due to the fact that the image contrast is reduced. In this paper we analyze this phenomenon using majorization theory and find that the image contrast can be formulated as a Schur convex function on filter coefficients. We also propose an effective method to choose the best filter so that the image contrast is maximized, so as to give best visual quality. © 2011 IEEE.

In advance of the imaging capturing technology, large amount of similar images are created. Instead of compressing each similar image individually, removing the inter-image redundancy would reduce the storage and transmission time. However, only a few set redundancy methods are proposed to deal with the problem. In this paper, a new method was derived from a theoretical model by extracting the low frequency in an image set. For the similar images, the values of their low frequency components are very close to that of their neighboring pixel in the spatial domain. In our model, a low frequency template is created and used as a prediction for each image to compute its residue. This model proves the reduction in the entropy and hence the bit rates. Experiments were conducted and proved there were up to 30% gains over the existing methods.

View synthesis offers a great flexibility in generating free viewpoint television (FTV) and 3D video (3DV). However, the depth-image-based view synthesis approach is very sensitive to errors in the camera parameters or poorly estimated depth maps (also called depth images). Because of these errors, three kinds of artifacts (blurring, contour, hole) are possibly introduced during the general synthesis process. Comparing to conventional methods which implement the view synthesis only in ideal case, in this paper, we propose to design an error compensation and reliability based view synthesis system where the potential errors are considered. The main contributions are highlighted as follows: Firstly, the camera parameter errors are compensated by a global homography transformation matrix. Secondly, the depth maps are classified into both reliable and unreliable regions and the reliability based weighting masks are built to blend synthesized images from two different views together. Finally, a reliability depth map based hole-filling technique is used to fill the existing holes. The experimental results demonstrate that these artifacts are efficiently reduced in the synthesized images. © 2011 IEEE.

In current video coding standard H.264/AVC, pixel prediction is applied to reduce spatial and temporal redundancy existed in video signal. Previously, it has been shown that better coding performance is achieved compared to H.264/AVC by combining inter and intra prediction to generate a more accurate prediction. In this paper, an improved combined prediction scheme is presented which allows the video codec to tune weighted coefficient for inter prediction and intra prediction adaptively to local signal characteristics. In order to avoid additional overhead signalling, statistics of already coded neighboring block is analyzed to predict the weighted coefficients of the combined prediction for current macroblock. Compared to H.264/AVC, coding performance is increased by up to 1.9%. And compared to the latest combined prediction method using spatial-invariant weighted coefficients, simulation results show that the proposed scheme achieves additional coding gain of up to 0.66%. © 2011 IEEE.

Single camera stereo system utilizes mirrors and a single camera for computational stereo, where the mirrors provide extra views needed for stereo and 3D reconstruction. In this paper, we investigate the basic epiploar geometric properties of the single camera stereo image and propose a novel image rectification technique to map the epipolar lines in the original image into the horizontally aligned lines in the rectified image. Besides, the rotation angles of the single camera corresponding to the planar mirror are derived during rectification. Experimental results show the robustness and accuracy of our method. © 2011 IEEE.

We present a novel framework for tree-structure embedded density estimation and its fast approximation for mode seeking. The proposed method could find diverse applications in. computer vision and feature space analysis. Given any undirected, connected and weighted graph, the density function is defined as a joint representation of the feature space and the distance domain on the graph's spanning tree. Since the distance domain of a tree is a constrained one, mode seeking can not be directly achieved by traditional mean shift in both domain, we address this problem by introducing node shifting with force competition and its fast approximation. Our work is closely related to the previous literature of nonparametric methods. One shall see, however; that the new formulation of this problem can lead to many advantages and new characteristics in its application, as will he illustrated later in this paper.

To achieve superior performance, rate-distortion optimized motion estimation (ME) for variable block size (RDO-VBSME) is often used in state-of-the-art video coding systems such as the H. 264 JM software. However, the complexity of RDO-VBSME is very high both for software and hardware implementations. In this paper, we propose a hardware-friendly ME algorithm called RDOMFS with a novel hardware-friendly rate-distortion (RD)-like cost function, and a hardware-friendly modified motion vector predictor. Simulation results suggest that the proposed RDOMFS can achieve essentially the same RD performance as RDO-VBSME in JM. We also propose a matching hardware architecture with a novel Smart Snake Scanning order which can achieve very high data re-use ratio and data throughout. It is also reconfigurable because it can achieve variable data re-use ratio and can process variable frame size. The design is implemented with TSMC 0.18 mu m CMOS technology and costs 103k gates. At a clock frequency of 63 MHz, the architecture achieves real-time 1920 x 1080 RDO-VBSME at 30 frames/s. At a maximum clock frequency of 250 MHz, it can process 4096 x 2160 at 30 frames/s.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the price of chrominance distortion. A major challenge is to suppress color fringing artifacts while maintaining sharpness. In [1], we proposed a novel human visual quality based (HVS) subpixel downsampling method. In this paper, we propose a hardware-friendly subpixel based downsampling scheme based on our previous work which can achieve similar performance as [1] but eliminate all floating point operations with limited bandwidth and much better performance than Direct Pixel based Downsampling (DPD) and Pixel-based downsampling with Anti-aliasing Filter (PDAF). We further propose a hardware architecture for our subpixel downsampling method which is highly data re-useable. We are the first few, if not the first, to implement subpixel downsampling method to video by hardware. The design is implemented with TSMC 0.18um CMOS technology and costs 244k gates. At a clock frequency of 63 MHz, the architecture achieves real-time 1920 × 1080 subpixel downsampling at 30fps. © 2011 IEEE.

A method and an apparatus relates to examining respective collections of coefficients out of a plurality of collections of coefficients according to a global coding order of the plurality of collections of coefficients, the respective collections of coefficients having respective priority levels as well as respective pluralities of coding units and truncation points which correspond to the coding units. In response to determining that an examined collection of coefficients out of the plurality of collections of coefficients has a priority level equal to a global priority level, encoding a first un-encoded coding unit according to a local coding order of the collection of coefficients and reducing the priority level of the collection of coefficients; and after each of the collections of coefficients in the plurality are examined, decreasing the global priority level. Repeating the examining, the encoding, the reducing and the decreasing for the plurality of collections of coefficients, and terminating the examining, the encoding, the reducing, and the decreasing in response to determining that a total amount of bits used is greater than a target bit rate.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the expense of chrominance distortion. In this paper, we formulate the subpixel-based down-sampling as a Min-Max problem (Min-Max Directional Error) which we call MMDE. Unfortunately, the solution of MMDE is computational intensive, especially for large images. We thus relax the MMDE by determining the maximum error based on HVS, which largely reduces the number of constraints. We call such relaxation as MMDE-VR (Visual Relaxation). Simulation results illustrate that MMDE-VR can effectively reduce visible color fringing artifacts while still maintaining sharpness. ©2010 IEEE.

3D Video has caught enormous attention in consumer market due to the progress in technical maturity of 3D displays, digital video broadcasts and computer vision algorithms. Video-plus-depth representation makes the concept of free viewpoint video feasible allowing the user to freely navigate within real world visual scenes. In this paper, we review the existing methodologies and 3D video processing framework. Previous image-based rendering techniques can be classified into three categories according to the amount of geometric information being used, of which DIBR combines the advantages of the two extremes (geometry based and image based representations). We then discuss the techniques for depth map creation, the method for rendering given the depth information and post-processing for corresponding artifacts in the multi-view adaptation. We also issue the quality measurement of the generated depth and synthesized view. In respect to the clear demand from industry and user side for such new types of visual media, there is still a large improving room in the 3D data representation and rendering quality of 3D perception.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a hardware friendly ME algorithm and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The proposed reconfigurable ME hardware is implemented in VHDL and mapped to a low cost Xilinx XC3S1500 FPGA. It works at 100MHz and is capable to process 1920 × 1080 of 30fps video format in real time and have very high data reuse ratio. © 2010 IEEE.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a novel hardware-oriented motion estimation algorithm called RD Optimized single-MVP-biased FS (RDOMFS), and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The design is implemented with TSMC 0.13um CMOS technology and costs 103k gates. At a clock frequency of 61 MHz, the architecture achieves real-time 1920 × 1080 RDO-VBSME at 30fps. © 2010 IEEE.

Error concealment at the video decoder is to recover erroneous picture region based on correctly decoded region in the same frame or the neighboring frames. However, the existing error concealment methods cannot give satisfactory result in some situations, e.g. when a whole frame is lost. In this paper, we propose a novel joint forward-backward concealment (JFBC) method. The key observation is that, a future frame can be recovered without using its reference frame when some error resilience tools, such as redundant picture, are used. Based on this observation, the proposed JFBC performs temporal error concealment along not only forward direction but also backward direction. Furthermore, the JFBC combines the forward concealment result and the backward concealment result by using linear minimum mean square error (LMMSE) estimation. The JFBC estimates for each pixel an error level such that the weights in LMMSE estimation are adaptive in pixel-level. In the experiments, the JFBC achieves considerable gain over the state-of-the-art temporal error concealment method. ©2010 IEEE.

Digital image forensics seeks to detect statistical traces left by image acquisition or post-processing in order to establish an images source and authenticity. Digital cameras acquire an image with one sensor overlayed with a color filter array (CFA), capturing at each spatial location one sample from the three necessary color channels. The missing pixels must be interpolated in a process known as demosaicking. This process is highly nonlinear and can vary greatly between different camera brands and models. Most practical algorithms, however, introduce correlations between the color channels, which are often different between algorithms. In this paper, we show how these correlations can be used to construct a characteristic map that is useful in matching an image to its source. Results show that our method employing inter-channel traces can distinguish between sophisticated demosaicking algorithms. It can complement existing classifiers based on inter-pixel correlations by providing a new feature dimension. © 2010 IEEE.

In this paper we give a step-by-step detailed analysis on the performance of shortest spanning tree (SST) and its revised version, recursive SST (RSST). We further propose a novel segmentation scheme based on recursive SST in the warped domain produced by density estimation. The proposed method is robust for variant natural image input and is easy to implement. Experimental results and comparisons with other methods have illustrated the effectiveness and robustness of the proposed method. © 2010 IEEE.

Error concealment at the video decoder is to recover erroneous picture region based on correctly decoded region in the same frame or the neighboring frames. However, error concealment cannot give satisfactory result in some cases, e.g. when a whole frame is lost. In this paper, we propose a novel backward error concealment method. While the existing temporal error concealment methods recover an erroneous frame by using its reference frame, the proposed method uses a refreshed future frame to recover the previous corrupted reference frame. This is based on the observation that a future frame can be recovered before its reference frame when some error resilience tools such as intra refresh or redundant picture are used. The advantage of the proposed method is that it can recover most pixels in the corrupted reference frame by inverse motion compensation without any error, since typically the refreshed future frame has both MVs and the residues. In the experiments, the proposed method achieves up to 1.0dB gain over the state-of-the-art temporal error concealment method. ©2010 IEEE.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the price of chrominance distortion. In this paper, we propose a new subpixel-based down-sampling scheme for which we design a two-dimensional image reconstruction model. Then, we formulate subpixel-based down-sampling as a MMSE problem and derive the optimal solution called MMSE-SD. To compare the performance of subpixel-based down-sampling methods, we propose novel objective measures for the apparent luminance resolution and chrominance distortion. Simulation results show that MMSE-SD can give sharper images compared with the conventional down-sampling methods, with little color fringing artifacts. ©2010 IEEE.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a novel hardware-oriented motion estimation algorithm called RD Optimized single-MVP-biased FS (RDOMFS), and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The design is implemented with TSMC 0.13um CMOS technology and costs 103k gates. At a clock frequency of 61 MHz, the architecture achieves real-time 1920 × 1080 RDO-VBSME at 30fps. © 2010 IEEE.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a hardware friendly ME algorithm and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The proposed reconfigurable ME hardware is implemented in VHDL and mapped to a low cost Xilinx XC3S1500 FPGA. It works at 100MHz and is capable to process 1920 × 1080 of 30fps video format in real time and have very high data reuse ratio. © 2010 IEEE.

We propose methods for generating a halftone image, in which each pixel takes one of two tone values. The generated image contains hidden data, which is present at data storage pixels chosen using a pseudo-random number generator. In a first case, the data is hidden within an existing halftone image by reversing the tone value at certain of the data storagepixels, and at pixels neighboring the data storage pixels. In a second case, the halftone image is generated from a greyscale image, and data is hidden during this conversion process.

There is disclosed a method of digital signal compression, coding and representation, and more particularly a method of video compression, coding and representation system that uses multi-frame motion estimation and includes both device and method aspects. The invention also provides a computer program product, such as a recording medium, carrying program instructions readable by a computing device to cause the computing device to carry out a method according to the invention.

We propose methods for generating a halftone image, in which each pixel takes one of two tone values. The generated image contains hidden data, which is present at data storage pixels chosen using a pseudo-random number generator. In a first case, the data is hidden within an existing halftone image by reversing the tone value at certain of the data storage pixels, and at pixels neighbouring the data storage pixels. In a second case, the halftone image is generated from a greyscale image, and data is hidden during this conversion process.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the expense of chrominance distortion. In this paper, we formulate the subpixel-based down-sampling as a Min-Max problem (Min-Max Directional Error) which we call MMDE. Unfortunately, the solution of MMDE is computational intensive, especially for large images. We thus relax the MMDE by determining the maximum error based on HVS, which largely reduces the number of constraints. We call such relaxation as MMDE-VR (Visual Relaxation). Simulation results illustrate that MMDE-VR can effectively reduce visible color fringing artifacts while still maintaining sharpness. ©2010 IEEE.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the price of chrominance distortion. In this paper, we propose a new subpixel-based down-sampling scheme for which we design a two-dimensional image reconstruction model. Then, we formulate subpixel-based down-sampling as a MMSE problem and derive the optimal solution called MMSE-SD. To compare the performance of subpixel-based down-sampling methods, we propose novel objective measures for the apparent luminance resolution and chrominance distortion. Simulation results show that MMSE-SD can give sharper images compared with the conventional down-sampling methods, with little color fringing artifacts. ©2010 IEEE.

Error concealment at the video decoder is to recover erroneous picture region based on correctly decoded region in the same frame or the neighboring frames. However, error concealment cannot give satisfactory result in some cases, e.g. when a whole frame is lost. In this paper, we propose a novel backward error concealment method. While the existing temporal error concealment methods recover an erroneous frame by using its reference frame, the proposed method uses a refreshed future frame to recover the previous corrupted reference frame. This is based on the observation that a future frame can be recovered before its reference frame when some error resilience tools such as intra refresh or redundant picture are used. The advantage of the proposed method is that it can recover most pixels in the corrupted reference frame by inverse motion compensation without any error, since typically the refreshed future frame has both MVs and the residues. In the experiments, the proposed method achieves up to 1.0dB gain over the state-of-the-art temporal error concealment method. ©2010 IEEE.

3D Video has caught enormous attention in consumer market due to the progress in technical maturity of 3D displays, digital video broadcasts and computer vision algorithms. Video-plus-depth representation makes the concept of free viewpoint video feasible allowing the user to freely navigate within real world visual scenes. In this paper, we review the existing methodologies and 3D video processing framework. Previous image-based rendering techniques can be classified into three categories according to the amount of geometric information being used, of which DIBR combines the advantages of the two extremes (geometry based and image based representations). We then discuss the techniques for depth map creation, the method for rendering given the depth information and post-processing for corresponding artifacts in the multi-view adaptation. We also issue the quality measurement of the generated depth and synthesized view. In respect to the clear demand from industry and user side for such new types of visual media, there is still a large improving room in the 3D data representation and rendering quality of 3D perception.

Digital image forensics seeks to detect statistical traces left by image acquisition or post-processing in order to establish an images source and authenticity. Digital cameras acquire an image with one sensor overlayed with a color filter array (CFA), capturing at each spatial location one sample from the three necessary color channels. The missing pixels must be interpolated in a process known as demosaicking. This process is highly nonlinear and can vary greatly between different camera brands and models. Most practical algorithms, however, introduce correlations between the color channels, which are often different between algorithms. In this paper, we show how these correlations can be used to construct a characteristic map that is useful in matching an image to its source. Results show that our method employing inter-channel traces can distinguish between sophisticated demosaicking algorithms. It can complement existing classifiers based on inter-pixel correlations by providing a new feature dimension. © 2010 IEEE.

Error concealment at the video decoder is to recover erroneous picture region based on correctly decoded region in the same frame or the neighboring frames. However, the existing error concealment methods cannot give satisfactory result in some situations, e.g. when a whole frame is lost. In this paper, we propose a novel joint forward-backward concealment (JFBC) method. The key observation is that, a future frame can be recovered without using its reference frame when some error resilience tools, such as redundant picture, are used. Based on this observation, the proposed JFBC performs temporal error concealment along not only forward direction but also backward direction. Furthermore, the JFBC combines the forward concealment result and the backward concealment result by using linear minimum mean square error (LMMSE) estimation. The JFBC estimates for each pixel an error level such that the weights in LMMSE estimation are adaptive in pixel-level. In the experiments, the JFBC achieves considerable gain over the state-of-the-art temporal error concealment method. ©2010 IEEE.

In this paper we give a step-by-step detailed analysis on the performance of shortest spanning tree (SST) and its revised version, recursive SST (RSST). We further propose a novel segmentation scheme based on recursive SST in the warped domain produced by density estimation. The proposed method is robust for variant natural image input and is easy to implement. Experimental results and comparisons with other methods have illustrated the effectiveness and robustness of the proposed method. © 2010 IEEE.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the price of chrominance distortion. In this paper, we propose a new subpixel-based down-sampling scheme for which we design a two-dimensional image reconstruction model. Then, we formulate subpixel-based down-sampling as a MMSE problem and derive the optimal solution called MMSE-SD. To compare the performance of subpixel-based down-sampling methods, we propose novel objective measures for the apparent luminance resolution and chrominance distortion. Simulation results show that MMSE-SD can give sharper images compared with the conventional down-sampling methods, with little color fringing artifacts. ©2010 IEEE.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the expense of chrominance distortion. In this paper, we formulate the subpixel-based down-sampling as a Min-Max problem (Min-Max Directional Error) which we call MMDE. Unfortunately, the solution of MMDE is computational intensive, especially for large images. We thus relax the MMDE by determining the maximum error based on HVS, which largely reduces the number of constraints. We call such relaxation as MMDE-VR (Visual Relaxation). Simulation results illustrate that MMDE-VR can effectively reduce visible color fringing artifacts while still maintaining sharpness. ©2010 IEEE.

Error concealment at the video decoder is to recover erroneous picture region based on correctly decoded region in the same frame or the neighboring frames. However, error concealment cannot give satisfactory result in some cases, e.g. when a whole frame is lost. In this paper, we propose a novel backward error concealment method. While the existing temporal error concealment methods recover an erroneous frame by using its reference frame, the proposed method uses a refreshed future frame to recover the previous corrupted reference frame. This is based on the observation that a future frame can be recovered before its reference frame when some error resilience tools such as intra refresh or redundant picture are used. The advantage of the proposed method is that it can recover most pixels in the corrupted reference frame by inverse motion compensation without any error, since typically the refreshed future frame has both MVs and the residues. In the experiments, the proposed method achieves up to 1.0dB gain over the state-of-the-art temporal error concealment method. ©2010 IEEE.

In this paper we give a step-by-step detailed analysis on the performance of shortest spanning tree (SST) and its revised version, recursive SST (RSST). We further propose a novel segmentation scheme based on recursive SST in the warped domain produced by density estimation. The proposed method is robust for variant natural image input and is easy to implement. Experimental results and comparisons with other methods have illustrated the effectiveness and robustness of the proposed method. © 2010 IEEE.

Error concealment at the video decoder is to recover erroneous picture region based on correctly decoded region in the same frame or the neighboring frames. However, the existing error concealment methods cannot give satisfactory result in some situations, e.g. when a whole frame is lost. In this paper, we propose a novel joint forward-backward concealment (JFBC) method. The key observation is that, a future frame can be recovered without using its reference frame when some error resilience tools, such as redundant picture, are used. Based on this observation, the proposed JFBC performs temporal error concealment along not only forward direction but also backward direction. Furthermore, the JFBC combines the forward concealment result and the backward concealment result by using linear minimum mean square error (LMMSE) estimation. The JFBC estimates for each pixel an error level such that the weights in LMMSE estimation are adaptive in pixel-level. In the experiments, the JFBC achieves considerable gain over the state-of-the-art temporal error concealment method. ©2010 IEEE.

3D Video has caught enormous attention in consumer market due to the progress in technical maturity of 3D displays, digital video broadcasts and computer vision algorithms. Video-plus-depth representation makes the concept of free viewpoint video feasible allowing the user to freely navigate within real world visual scenes. In this paper, we review the existing methodologies and 3D video processing framework. Previous image-based rendering techniques can be classified into three categories according to the amount of geometric information being used, of which DIBR combines the advantages of the two extremes (geometry based and image based representations). We then discuss the techniques for depth map creation, the method for rendering given the depth information and post-processing for corresponding artifacts in the multi-view adaptation. We also issue the quality measurement of the generated depth and synthesized view. In respect to the clear demand from industry and user side for such new types of visual media, there is still a large improving room in the 3D data representation and rendering quality of 3D perception.

Digital image forensics seeks to detect statistical traces left by image acquisition or post-processing in order to establish an images source and authenticity. Digital cameras acquire an image with one sensor overlayed with a color filter array (CFA), capturing at each spatial location one sample from the three necessary color channels. The missing pixels must be interpolated in a process known as demosaicking. This process is highly nonlinear and can vary greatly between different camera brands and models. Most practical algorithms, however, introduce correlations between the color channels, which are often different between algorithms. In this paper, we show how these correlations can be used to construct a characteristic map that is useful in matching an image to its source. Results show that our method employing inter-channel traces can distinguish between sophisticated demosaicking algorithms. It can complement existing classifiers based on inter-pixel correlations by providing a new feature dimension. © 2010 IEEE.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a hardware friendly ME algorithm and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The proposed reconfigurable ME hardware is implemented in VHDL and mapped to a low cost Xilinx XC3S1500 FPGA. It works at 100MHz and is capable to process 1920 × 1080 of 30fps video format in real time and have very high data reuse ratio. © 2010 IEEE.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a novel hardware-oriented motion estimation algorithm called RD Optimized single-MVP-biased FS (RDOMFS), and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The design is implemented with TSMC 0.13um CMOS technology and costs 103k gates. At a clock frequency of 61 MHz, the architecture achieves real-time 1920 × 1080 RDO-VBSME at 30fps. © 2010 IEEE.

We propose methods for generating a halftone image, in which each pixel takes one of two tone values. The generated image contains hidden data, which is present at data storage pixels chosen using a pseudo-random number generator. In a first case, the data is hidden within an existing halftone image by reversing the tone value at certain of the data storagepixels, and at pixels neighboring the data storage pixels. In a second case, the halftone image is generated from a greyscale image, and data is hidden during this conversion process.

There is disclosed a method of digital signal compression, coding and representation, and more particularly a method of video compression, coding and representation system that uses multi-frame motion estimation and includes both device and method aspects. The invention also provides a computer program product, such as a recording medium, carrying program instructions readable by a computing device to cause the computing device to carry out a method according to the invention.

We propose methods for generating a halftone image, in which each pixel takes one of two tone values. The generated image contains hidden data, which is present at data storage pixels chosen using a pseudo-random number generator. In a first case, the data is hidden within an existing halftone image by reversing the tone value at certain of the data storage pixels, and at pixels neighbouring the data storage pixels. In a second case, the halftone image is generated from a greyscale image, and data is hidden during this conversion process.

Human eyes have different sensitivity to different frequency components of image signals, typically, low frequency components are relatively more crucial to the perceptual quality of images than high frequency components. Based on this observation, we propose a novel sampling scheme for compressive sensing framework by designing a weighting scheme for the sampling matrix. By adjusting the weighting coefficients, we can tune the structure of the sampling matrix to favor the frequency components that are important to human perception, so that those components could be more precisely recovered in the reconstruction procedure. Experimental results reveal that our proposed scheme can greatly enhance the performance of compressive sensing framework in both PSNR and visual quality without increasing the complexity of the framework structure or computational procedure.

Compressive Sampling (CS), is an emerging theory which points us a promising direction of designing novel efficient data compression techniques. However, the conventional CS adopts a non-discriminated sampling scheme which usually gives poor performance on realistic complex signals. In this paper we propose a reweighted Compressive Sampling for image compression. It introduces a weighting scheme into the conventional CS framework whose coefficients arc determined in encoding side according to the statistics of image signals. Experimental results demonstrate that our proposed method notably outperforms the conventional Compressive Sampling framework in coding performance in the sense that the reconstruction quality is greatly enhanced with same number of measurements and computational complexity.

In this paper, a new adaptive subpixel-based downsampling scheme is proposed. Inside this scheme, we take full advantage of subpixels by adaptively choosing the sample directions based on edge information which has not been addressed before. Then, an adaptive filter is designed to suppress color fringing artifacts. Moreover, a good cut-off frequency is derived and deployed in our filter to obtain extra information. Simulation results illustrate that the proposed adaptive subpixel-based downsampling scheme successfully improves the resolution while efficiently removes visible color fringing artifacts. ©2009 IEEE.