Search

Textures contain a wealth of image information and are widely used in various fields such as computer graphics and computer vision. With the development of machine learning, the texture synthesis and generation have been greatly improved. As a very common element in everyday life, wallpapers contain a wealth of texture information, making it difficult to annotate with a simple single label. Moreover, wallpaper designers spend significant time to create different styles of wallpaper. For this purpose, this paper proposes to describe wallpaper texture images by using multi-label semantics. Based on these labels and generative adversarial networks, we present a framework for perception driven wallpaper texture generation and style transfer. In this framework, a perceptual model is trained to recognize whether the wallpapers produced by the generator network are sufficiently realistic and have the attribute designated by given perceptual description; these multi-label semantic attributes are treated as condition variables to generate wallpaper images. The generated wallpaper images can be converted to those with well-known artist styles using CycleGAN. Finally, using the aesthetic evaluation method, the generated wallpaper images are quantitatively measured. The experimental results demonstrate that the proposed method can generate wallpaper textures conforming to human aesthetics and have artistic characteristics. [ABSTRACT FROM AUTHOR]

To improve the viewer’s Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital representation formats of 3D models, the visual quality of which is quite sensitive to lossy operations like simplification and compression. Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality of distorted 3D models. However, most previous studies utilize full-reference (FR) metrics, which indicates they can not predict the quality level in the absence of the reference 3D model. Furthermore, few 3D-QA metrics consider color information, which significantly restricts their effectiveness and scope of application. In this paper, we propose a no-reference (NR) quality assessment metric for colored 3D models represented by both point cloud and mesh. First, we project the 3D models from 3D space into quality-related geometry and color feature domains. Then, the 3D natural scene statistics (3D-NSS) and entropy are utilized to extract quality-aware features. Finally, a support vector regression (SVR) model is employed to regress the quality-aware features into visual quality scores. Our method is validated on the colored point cloud quality assessment database (SJTU-PCQA), the Waterloo point cloud assessment database (WPC), and the colored mesh quality assessment database (CMDM). The experimental results show that the proposed method outperforms most compared NR 3D-QA metrics with competitive computational resources and greatly reduces the performance gap with the state-of-the-art FR 3D-QA metrics. The code of the proposed model is publicly available now at https://github.com/zzc-1998/NR-3DQA. [ABSTRACT FROM AUTHOR]

Sports data analysis is becoming increasingly large scale, diversified, and shared, but difficulty persists in rapidly accessing the most crucial information. Previous surveys have focused on the methodologies of sports video analysis from the spatiotemporal viewpoint instead of a content-based viewpoint, and few of these studies have considered semantics. This paper develops a deeper interpretation of content-aware sports video analysis by examining the insight offered by research into the structure of content under different scenarios. On the basis of this insight, we provide an overview of the themes particularly relevant to the research on content-aware systems for broadcast sports. Specifically, we focus on the video content analysis techniques applied in sportscasts over the past decade from the perspectives of fundamentals and general review, a content hierarchical model, and trends and challenges. Content-aware analysis methods are discussed with respect to object-, event-, and context-oriented groups. In each group, the gap between sensation and content excitement must be bridged using proper strategies. In this regard, a content-aware approach is required to determine user demands. Finally, this paper summarizes the future trends and challenges for sports video analysis. We believe that our findings can advance the field of research on content-aware video analysis for broadcast sports. [ABSTRACT FROM PUBLISHER]

Recently, region-based 3D video coding has been proposed. However, existing view synthesis distortion estimation (VSDE) methods are performed at the frame level. To guide the rate-distortion optimization process of region-based 3D video coding schemes, this paper proposes the first pixel-level VSDE (PL-VSDE) method. We first give the definition of the pixel-level view synthesis distortion. To estimate it, a backward prediction method is then developed, which starts from the pixels of interest (POIs) in the virtual view and finds their corresponding pixels in the reference view via a coarse-to-fine approach, denoted as coarse-to-fine backward prediction (CFBP) method. Additionally, the CFBP fully considers the details of 3D warping, the rounding operation and the warping competition in view synthesis, leading to improve accuracy of the prediction. Besides, a table-lookup method and a warping property are introduced to speed up the CFBP. After integrating the CFBP into the PL-VSDE, we can estimate the view synthesis distortion at the pixel level. Our method is carried out pixel-by-pixel independently, which is friendly for parallel processing. The experimental results demonstrate that our proposed method has significant advantages in both accuracy and efficiency compared with the state-of-the-art frame-level VSDE methods. [ABSTRACT FROM AUTHOR]

By providing angular and spatial information of light rays, light field images are widely used in many applications. To capture large field of view light field, existing approaches either stitch small field of view light fields, whose apertures are also small, or leverage specialized equipments, which are not accessible to ordinary users. In this paper, we present a method to extract fully 360° field of view panoramic light fields from densely sampled hand-held videos. Unlike previous works, our method handles the large exposure variation and motion blur problems, which are common in the panoramic capture and hand-held videos. To provide real-time rendering performance, our method applies light field sampling to the extracted panoramic light field. Based on the unstructured representation, our method can sample the light field without explicit parameterization. We formulate the sampling problem into a set multicover problem and solve it globally using integer linear programming. Compared with the greedy based method, our approach can use much less video frames to represent the panoramic light field, but still can get better rendering results. [ABSTRACT FROM AUTHOR]

Media technology, in particular video recording and playback, keeps improving to provide users with high-quality real and virtual visual content. In recent years, increasing the temporal sampling rate of videos and the refresh rate of displays has become one focus of technical innovation. This raises the question, how high the sampling and refresh rates should be? To answer this question, we determine the minimum temporal sampling rate at which a video should be presented to make temporal sampling imperceptible to viewers. Through a psychophysical study, we find that this minimum sampling rate depends on both the speed of the objects in the image plane and the exposure time of the recording camera. We propose a model to compute the required minimum sampling rate based on these two parameters. In addition, state-of-the-art video codecs employ motion vectors from which the local object movement speed can be inferred. Therefore, we present a procedure to compute the minimum sampling rate given an encoded video and camera exposure time. Since the object motion speed in a video may vary, the corresponding minimum frame rate is also varying. This is why the results of this paper are particularly applicable when used together with adaptive frame rate computer generated graphics or novel video communication solutions that drop insignificant frames. In our experiments, we show that videos played back at the minimum adaptive frame rate achieve an average bit rate reduction of 26% compared to constant frame rate playback, while perceptually no difference can be observed. [ABSTRACT FROM AUTHOR]

Human pose estimation from image or video is a basic issue in computer graphics and computer vision. The challenge of human pose estimation in video lies in the temporal coherency issue. The temporal consistency in video is the contents’ similarity shown in the video frames. In video, temporal consistency maintenance of human pose estimation is to obtain better long-term consistency. Great major methods for the long-term consistency are using the whole video optimization method, which makes very large computation and the absence of consistency before and after the articulated limbs. In this paper, a novel method for the maintenance of temporal consistency is proposed. We maintain the temporal consistency of the video by the structured space learning and halfway temporal evaluation methods. We adopt a three-stage multi-feature deep convolution network framework to generate the initial posture joints position data, and a long-term temporal coherence is propagated to the overall video at each stage. The long-term consistency is more appealing since it produces stable results over larger periods of time. Our method can achieve good temporal consistency and get accurate and stable human pose estimation results. Various experimental results demonstrated the superiority of our method. [ABSTRACT FROM AUTHOR]

This paper presents an advanced depth intra-coding approach for 3D video coding based on the High Efficiency Video Coding (HEVC) standard and the multiview video plus depth (MVD) representation. This paper is motivated by the fact that depth signals have specific characteristics that differ from those of natural signals, i.e., camera-view video. Our approach replaces conventional intra-picture coding for the depth component, targeting a consistent and efficient support of 3D video applications that utilize depth maps or polygon meshes or both, with a high depth coding efficiency in terms of minimal artifacts in rendered views and meshes with a minimal number of triangles for a given bit rate. For this purpose, we introduce intra-picture prediction modes based on geometric primitives along with a residual coding method in the spatial domain, substituting conventional intra-prediction modes and transform coding, respectively. The results show that our solution achieves the same quality of rendered or synthesized views with about the same bit rate as MVD coding with the 3D video extension of HEVC (3D-HEVC) for high-quality depth maps and with about 8% less overall bit rate as with 3D-HEVC without related depth tools. At the same time, the combination of 3D video with 3D computer graphics content is substantially simplified, as the geometry-based depth intra signals can be represented as a surface mesh with about 85% less triangles, generated directly in the decoding process as an alternative decoder output. [ABSTRACT FROM AUTHOR]

In past decades, saliency detection has received increasing attention from computer vision communities, for its potential usage in many vision-related tasks. However, finding representative and discriminative features to accurately locate salient regions from complex scenes remains a challenging problem. Recent research on primary visual cortex (V1) shows that vision neurons are sparsely connected to form a compact representation of natural scenes and different visual stimuli are processed separately according to their semantic importance. Inspired by the above characteristics of visual perception, in this paper we advance a novel saliency detection method via representative and discriminative dictionary learning. An assumption that salient and nonsalient information are sparsely coded under two separate dictionaries is cast on the problem and we propose to learn a compact background dictionary from the image itself for saliency estimation. Different from previous methods, our saliency cues are obtained via active learning strategies rather than artificially designed rules, and thus is more adaptive. Followed by this, a probabilistic inference model is deduced to fully excavate multisource information about the scenes for high-quality saliency map generation. This joint inference scheme takes both spatial and color space information into consideration and is proved to be quite effective in practice. Finally, to investigate the performance of the proposed model, some experiments are conducted on two benchmark data sets along with other 20 state-of-the-art saliency detection approaches. The experimental results show that our method outperforms its counterparts and can correctly detect salient regions, even when other methods fail. Besides, the usability of the proposed method in real application-based cases is verified by applying it to content-based image resizing and promising results are obtained. [ABSTRACT FROM PUBLISHER]

This paper studies the effects of front-end imager parameters on object detection performance and energy consumption. A custom version of histograms of oriented gradient (HOG) features based on 2-b pixel ratios is presented and shown to achieve superior object detection performance for the same estimated energy compared with conventional HOG features. A front-end hardware implementation capable of extracting these features at multiple scales is proposed, and a system-level energy analysis is performed. This energy analysis suggests a potential $19\times $ reduction in I/O energy and a $3.3\times $ reduction in back-end detection energy compared with conventional object detection pipelines. [ABSTRACT FROM PUBLISHER]

In the latest High Efficiency Video Coding (HEVC) development, i.e., HEVC screen content coding extensions (HEVC-SCC), a hash-based inter-motion search/block matching scheme is adopted in the reference test model, which brings significant coding gains to code screen content. However, the hash table generation itself may take up to half the encoding time and is thus too complex for practical usage. In this paper, we propose a hierarchical hash design and the corresponding block matching scheme to significantly reduce the complexity of hash-based block matching. The hierarchical structure in the proposed scheme allows large block calculation to use the results of small blocks. Thus, we avoid redundant computation among blocks with different sizes, which greatly reduces complexity without compromising coding efficiency. The experimental results show that compared with the hash-based block matching scheme in the HEVC-SCC test model (SCM)-6.0, the proposed scheme reduces about 77% of hash processing time, which leads to 12% and 16% encoding time savings in random access (RA) and low-delay B coding structures. The proposed scheme has been adopted into the latest SCM. A parallel implementation of the proposed hash table generation on graphics processing unit (GPU) is also presented to show the high parallelism of the proposed scheme, which achieves more than 30 frames/s for 1080p sequences and 60 frames/s for 720p sequences. With the fast hash-based block matching integrated into x265 and the hash table generated on GPU, the encoder can achieve 11.8% and 14.0% coding gains on average for RA and low-delay P coding structures, respectively, for real-time encoding. [ABSTRACT FROM PUBLISHER]

Conventional low-level feature-based saliency detection methods tend to use nonrobust prior knowledge and do not perform well in complex or low-contrast images. In this paper, to address these issues in existing methods, we propose a novel deep neural network (DNN)-based dense and sparse labeling (DSL) framework for saliency detection. DSL consists of three major steps, namely, dense labeling (DL), sparse labeling (SL), and deep convolutional (DC) network. The DL and SL steps conduct initial saliency estimations with macro object contours and low-level image features, respectively, which effectively approximate the location of the salient object and generate accurate guidance channels for the DC step; the DC step, on the other hand, takes in the results of DL and SL, establishes a six-channeled input data structure (including local superpixel information), and conducts accurate final saliency classification. Our DSL framework exploits the saliency estimation guidance from both macro object contours and local low-level features, as well as utilizing the DNN for high-level saliency feature extraction. Extensive experiments are conducted on six well-recognized public data sets against 16 state-of-the-art saliency detection methods, including ten conventional feature-based methods and six learning-based methods. The results demonstrate the superior performance of DSL on various challenging cases in terms of both accuracy and robustness. [ABSTRACT FROM PUBLISHER]

The H.264/AVC coding standard processes variable block size motion-compensated prediction with multiple reference frames to achieve a pronounced improvement in compression efficiency. Accordingly, the computation of motion estimation increases in proportion to the product of the number of reference frame and the number of intermode. The mathematical analysis in this paper illustrates that the motion-compensated prediction errors are mainly determined by the detailed textures in the source image. The image block being rich in textures contains numerous high-frequency signals, which make variable block size and multiple reference frame techniques essential. On the basis of rate-distortion theory, in this paper, the spatial homogeneity of an image block is made as a relative concept with respect to the current quantization step. For the homogenous block, its futile reference frames and intermodes can be eliminated efficiently. It is further revealed that the sum of absolute differences value of an image block is mainly determined by the sum of its edge gradient amplitude and the current quantization step. Consequently, the image content-based early termination algorithm is proposed, and it outperforms the original method adopted by JVT reference software. Moreover, the dynamic search range algorithm based on the edge gradient amplitude of source image block is analyzed. One eminent advantage of the proposed edge- based algorithms is their efficiency to the macroblock-pipelining architecture, and another desirable feature is their orthogonality to fast block-matching algorithms. Experimental results show that when these algorithms are integrated with hybrid unsymmetrical-cross multi-hexagongrid search, an averaged 31.4-60.0% motion estimation time can be saved, whereas the averaging BDPSNR loss is 0.0497 dB for all tested sequences. [ABSTRACT FROM AUTHOR]

Integer cosine transform (ICT) is adopted by H.264/AVC for its bit-exact implementation and significant complexity reduction compared to the discrete cosine transform (DCT) with an impact in peak sigal-to-noise ratio (PSNR) of less than 0.02 dB. In this paper, a new technique, named prescaled integer transform (PIT), is proposed. With PIT, while all the merits of ICT are kept, the implementation complexity of decoder is further reduced compared to corresponding conventional ICT, which is especially important and beneficial for implementation on low-end processors. Since not all PIT kernels are good in respect of coding efficiency, design rules that lead to good PIT kernels are considered in this paper. Different types of PIT and their target applications are examined. Both fixed block-size transform and adaptive block-size transform (ABT) schemes of PIT are also studied. Experimental results show that no penalty in performance is observed with PIT when the PIT kernels employed are derived from the design rules. Up to 0.2 dB of improvement in PSNR for all intra frame coding compared to H.264/AVC can be achieved and the subjective quality is also slightly improved when PIT scheme is carefully designed. Using the same concept, a variation of PIT, Post-scaled Integer Transform, can also be potentially designed to simplify the encoder in some special applications. PIT has been adopted in audio video coding standard (AVS), Chinese National Coding standard. [ABSTRACT FROM AUTHOR]

The accuracy of scene flow is restricted by several challenges such as occlusion and large displacement motion. When occlusion happens, the positions inside the occluded regions lose their corresponding counterparts in preceding and succeeding frames. Large displacement motion will increase the complexity of motion modeling and computation. Moreover, occlusion and large displacement motion are highly related problems in scene flow estimation, e.g., large displacement motion often leads to considerably occluded regions in the scene. An improved dense scene flow method based on red-green-blue-depth (RGB-D) data is proposed in this paper. To handle occlusion, we model the occlusion status for each point in our problem formulation, and jointly estimate the scene flow and occluded regions. To deal with large displacement motion, we employ an over-parameterized scene flow representation to model both the rotation and translation components of the scene flow, since large displacement motion cannot be well approximated using translational motion only. Furthermore, we employ a two-stage optimization procedure for this overparameterized scene flow representation. In the first stage, we propose a new RGB-D PatchMatch method, which is mainly applied in the RGB-D image space to reduce the computational complexity introduced by the large displacement motion. According to the quantitative evaluation based on the Middlebury data set, our method outperforms other published methods. The improved performance is also comprehensively confirmed on the real data acquired by Kinect sensor. [ABSTRACT FROM AUTHOR]

A 3-D multiview video gives users an experience that is different from that provided by a traditional video; however, it puts a huge burden on limited bandwidth resources. Mixed-resolution video in a multiview system can alleviate this problem by using different video resolutions for different views. However, to reduce visual uncomfortableness and to make this video format more suitable for free-viewpoint television, the low-resolution (LR) views need to be super-resolved to the target full resolution. In this paper, we propose a virtual-view-assisted super-resolution algorithm, where the inter-view similarity is used to determine whether the missing pixels in the super-resolved frame need to be filled by virtual-view pixels or by spatial interpolated pixels. The decision mechanism is steered by the texture characteristics of the neighbors of each missing pixel. Furthermore, the inter-view similarity is used, on the one hand, to enhance the quality of the virtual-view-copied pixels by compensating the luminance difference between different views and, on the other hand, to enhance the original LR pixels in the super-resolved frame by reducing their compression distortion. Thus, the proposed method can recover the details in regions with edges while maintaining good quality at smooth areas by properly exploiting the high-quality virtual-view pixels and the directional correlation of pixels. The experimental results demonstrate the effectiveness of the proposed approach with a peak signal-to-noise ratio gain of up to 3.85 dB. [ABSTRACT FROM AUTHOR]

Existing systems for 3-D reconstruction from multiple view video use controlled indoor environments with uniform illumination and backgrounds to allow accurate segmentation of dynamic foreground objects. In this paper, we present a portable system for 3-D reconstruction of dynamic outdoor scenes that require relatively large capture volumes with complex backgrounds and nonuniform illumination. This is motivated by the demand for 3-D reconstruction of natural outdoor scenes to support film and broadcast production. Limitations of existing multiple view 3-D reconstruction techniques for use in outdoor scenes are identified. Outdoor 3-D scene reconstruction is performed in three stages: 1) 3-D background scene modeling using spherical stereo image capture; 2) multiple view segmentation of dynamic foreground objects by simultaneous video matting across multiple views; and 3) robust 3-D foreground reconstruction and multiple view segmentation refinement in the presence of segmentation and calibration errors. Evaluation is performed on several outdoor productions with complex dynamic scenes including people and animals. Results demonstrate that the proposed approach overcomes limitations of previous indoor multiple view reconstruction approaches enabling high-quality free-viewpoint rendering and 3-D reference models for production. [ABSTRACT FROM AUTHOR]

This paper proposes a movable image-based rendering (M-IBR) system for improving the viewing freedom and environmental modeling capability of conventional static IBR systems. The system supports object-based rendering and 3-D reconstruction capability and consists of three main components. numeration="arabic" posttext=")" continuation="restarts"An improved video stabilization method to reduce the shaky motion frequently encountered in movable IBR systems. It employs local polynomial regression (LPR) to automatically select an appropriate bandwidth for smoothing the estimated motion. [ABSTRACT FROM PUBLISHER]

This paper explains the development of a highly efficient progressive 3-D mesh geometry coder based on the region adaptive transform in the spectral mesh compression method. A hierarchical set partitioning technique, originally proposed for the efficient compression of wavelet transform coefficients in high performance wavelet-based image coding methods, is proposed for the efficient compression of the coefficients of this transform. Experiments confirm that the proposed coder employing such a region adaptive transform has a high compression performance rarely achieved by other state of the art 3-D mesh geometry compression algorithms. A new, high-performance fixed spectral basis method is also proposed for reducing the computational complexity of the transform. Many-to-one mappings are employed to relate the coded irregular mesh region to a regular mesh whose basis is used. To prevent loss of compression performance due to the low-pass nature of such mappings, transitions are made from transform-based coding to spatial coding on a per region basis at high coding rates. Experimental results show the performance advantage of the newly proposed fixed spectral basis method over the original fixed spectral basis method in the literature that employs one-to-one mappings. [ABSTRACT FROM AUTHOR]

Watermark resistance to geometric attacks is an important issue in the image watermarking community. Most countermeasures proposed in the literature usually focus on the problem of global affine transforms such as rotation, scaling and translation (RST), but few are resistant to challenging cropping and random bending attacks (RBAs). The main reason is that in the existing watermarking algorithms, those exploited robust features are more or less related to the pixel position. In this paper, we present an image watermarking scheme by the use of two statistical features (the histogram shape and the mean) in the Gaussian filtered low-frequency component of images. The two features are: 1) mathematically invariant to scaling the size of images; 2) independent of the pixel position in the image plane; 3) statistically resistant to cropping; and 4) robust to interpolation errors during geometric transformations, and common image processing operations. As a result, the watermarking system provides a satisfactory performance for those content-preserving geometric deformations and image processing operations, including JPEG compression, lowpass filtering, cropping and RBAs. [ABSTRACT FROM AUTHOR]

Due to the complexity of H.264/AVC, it is very challenging to apply this standard to design a conversational video communication system. This problem is addressed in this paper by using region-of-interest (ROI) based bit allocation and computational power allocation schemes. In our system, the ROI is first detected by using the direct frame difference and skin-tone information. Several coding parameters including quantization parameter, candidates for mode decision, the number of referencing frames, accuracy of motion vectors and the search range of motion estimation are adaptively adjusted at the macroblock (MB) level according to the relative importance of each MB. Subsequently, the encoder could allocate more resources such as bits and computational power to the ROI, and the decoding complexity is also optimized at the encoder side by utilizing an ROI based rate-distortion-complexity (R-D-C) cost function. The encoder is thus simplified and decoding-friendly, and the overall subjective visual quality can also be improved. [ABSTRACT FROM AUTHOR]

The efficient progressive transmission of a textured 3-D model, which has a texture image mapped on a geometric mesh, requires careful organization of the bit stream to quickly present high-resolution visualization on the user's screen. Bit rates for the mesh and the texture need to be properly balanced so that the decoded model has the best visual fidelity during transmission. This problem is addressed in the paper, and a bit allocation framework is proposed. In particular, the relative importance of the mesh geometry and the texture image on visual fidelity is estimated using a fast quality measure (FQM). Optimal bit distribution between the mesh and the texture is then computed under the criterion of maximizing the quality measured by FQM. Both the quality measure and the bit allocation algorithm are designed with low computation complexity. Empirical studies show that not only the bit allocation framework maximizes the receiving quality of the textured 3-D model, but also it does not require sending additional information to indicate bit boundaries between the mesh and the texture in the multiplexed bit stream, which makes the streaming application more robust to bit errors that may occur randomly during transmission. [ABSTRACT FROM AUTHOR]

The algorithm of embedded block coding with optimization truncation (EBCOT) is one of the key techniques in JPEG2000 standard. The high-speed/performance hardware designs for EBCOT are critical to various high-resolution applications, such as digital cameras, digital video recorders, and HDTV, etc. This paper presents a detailed performance analysis for EBCOT Tier-1 and its bit plane parallel architecture design. By analyzing the bit plane parallel context modeling, we conclude that the difference between the output rate of context modeling module and that of arithmetic coding module degrades the performance of whole EBCOT parallel encoding. We propose two improved methods, referred to as data-pairs ordering (DPO) and flexible MQ (FMQ) coder. It solves the configuration problem between the parallel context modeling module and the sequent arithmetic coding module, takes full advantage of the bit plane parallel encoding technique, and improves the coding speed and efficiency of the EBCOT encoder significantly. The design of parallel EBCOT encoder is tested on the field-programmable gate array (FPGA) platform of Altera Company. The simulation results show that it can on average encode 54 million samples at 55-MHz working frequency. It is equivalent to encoding a 9000 × 6000 image per second or encoding 720 p (1280 × 720, 4:2:2) HDTV picture sequence nearly 30 frames per second. Compared with the conventional bit plane parallel architecture design, the proposed one can reduce execution time by 24%. [ABSTRACT FROM AUTHOR]

Prominences are clouds of relatively cool and dense gas in the solar atmosphere. In this paper, we present a new method to detect and characterize the prominence eruptions. The input is a sequence of consecutive Hα solar images, and the output is a list of prominence eruption events detected. We extract the limb events and measure their associated properties by applying image processing techniques. First, we perform image normalization and noise removal. Then, we isolate the limb objects and identify the prominence features. Finally, we apply pattern recognition techniques to classify the eruptive prominences. The characteristics of prominence eruptions, such as brightness, angular width, radial height and velocity are measured. The method presented can lead to automatic monitoring and characterization of solar events. [ABSTRACT FROM AUTHOR]

This paper presents a real-time high-definition 720p@30fps H.264/MPEG-4 AVC intra-frame codec IP suitable for digital video and digital still camera applications. The whole design is optimized in both the algorithm and architecture levels. In the algorithm level, we propose to remove the area-costly plane mode, and enhance the cost function to reduce hardware cost and to increase the processing speed while provide nearly the same quality. In the architecture design, in additional to the fast module implementation the process is arranged in the macroblock-level pipelining style together with three careful scheduling techniques to avoid the idle cycles and improve the data throughput. The whole codec design only needs 103 K gate count for a core size of 1.28 × 1.28 mm² and achieves real-time encoding and decoding at 117 and 25.5 MHz, respectively, when implemented by 0.18-μm CMOS technology. [ABSTRACT FROM AUTHOR]

A novel reversible data hiding algorithm, which can recover the original image without any distortion from the marked image after the hidden data have been extracted, is presented in this paper. This algorithm utilizes the zero or the minimum points of the histogram of an image and slightly modifies the pixel grayscale values to embed data into the image. It can embed more data than many of the existing reversible data hiding algorithms. It is proved analytically and shown experimentally that the peak signal-to-noise ratio (PSNR) of the marked image generated by this method versus the original image is guaranteed to be above 48 dB. This lower bound of PSNR is much higher than that of all reversible data hiding techniques reported in the literature. The computational complexity of our proposed technique is low and the execution time is short. The algorithm has been successfully applied to a wide range of images, including commonly used images, medical images, texture images, aerial images and all of the 1096 images in CorelDraw database. Experimental results and performance comparison with other reversible data hiding schemes are presented to demonstrate the validity of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In the domain of digital video coding, new technologies and solutions are emerging in a fast pace, targeting the needs of the evolving multimedia landscape. One of the questions that arises is how to assess these different video coding technologies in terms of compression efficiency. In this paper, several compression schemes are compared by means of peak signal-to-noise ratio (PSNR) and just noticeable difference (JND). The codecs examined are XviD 0.9.1 (conform to the MPEG-4 Visual Simple Profile), DivX 5.1 (implementing the MPEG-4 Visual Advanced Simple Profile), Windows Media Video 9, MC-EZBC and H.264/AVC AHM 2.0 (version JM 6.1 of the reference software, extended with rate control). The latter plays a key role in this comparison because the H.264/AVC standard can be considered as the de facto benchmark in the field of digital video coding. The obtained results show that H.264/AVC AHM 2.0 outperforms current proprietary and standards-based implementations in almost all cases. Another observation is that the choke of a particular quality metric can influence general statements about the relation between the different codecs. [ABSTRACT FROM AUTHOR]

One-bit transforms (1BTs) have been proposed for low-complexity block-based motion estimation by reducing the representation order to a single bit, and employing binary matching criteria. However, as a single bit is used in the representation of image frames, bad motion vectors are likely to be resolved in 1BT-based motion estimation algorithms particularly for small block sizes. It is proposed in this paper to utilize a two-bit transform (2BT) for block-based motion estimation. Image frames are converted into two-bit representations by a simple block-by-block two bit transform based on multithresholding with mean and linearly approximated standard deviation values. In order to avoid blocking effects at block boundaries during the block-by-block transformation while enabling the two-bit representation to be constructed according to local detail, threshold values are computed within a larger window surrounding the transforming block. The 2BT makes use of lower bit-depth and binary matching criteria properties of 1BTs to achieve low-complexity block motion estimation. The 2BT improves motion estimation accuracy and seriously reduces the amount of bad motion vectors compared to 1BTs, particularly for small block sizes. It is shown that the proposed 2BT-based motion estimation technique improves motion estimation accuracy in terms of peak signal-to-noise ratio of reconstructed frames and also results in visually more accurate frames subsequent to motion compensation compared to the IBT-based motion estimation approach. [ABSTRACT FROM AUTHOR]

In this paper, we propose a balanced multiple description video coding scheme which is based on the partitioning of the discrete cosine transform (DCT) coefficients. Our scheme splits the single-layered stream generated by a standard coder into two correlated and balanced (virtually identical rates and distortions) substreams. The optimization is in the redundancy-rate-distortion sense using Lagrangian relaxation for optimum allocation of redundancy among the blocks in the frame. A greedy algorithm to meet the equal distortions criterion and an algorithm to optimally achieve equal bit rates for the two descriptions are proposed. Our simulation results substantiate our claim of achieving balanced descriptions at high peak signal-to-noise ratios values for any specified redundancy and any target bit rate. Our design compiles with the existing motion-compensated DCT-based standards, i.e., each description can be independently decoded by any standard decoder. [ABSTRACT FROM AUTHOR]

This paper presents a method to discriminate pixel differences according to theft impact toward perceived visual quality. Noticeable local contrast changes are formulated firstly since contrast Is the basic sensory feature in the human visual system (HITS) perception. The analysis aims at quantifying the actual impact of such changes (further divided into increases and decreases on edges) in different signal contexts. An associated full-reference distortion metric proposed next provides better match with the HVS viewing. Experiments have used two independent visual data sets and the related subjective viewing results, and demonstrated the performance improvement of the proposed metric over the relevant existing ones with various video/images and under diversified test conditions. The proposed metric is particularly effective to visual signal with blurring and luminance fluctuations as the major artifacts, and brings about the fundamental improvement when sharpened image edges are involved. [ABSTRACT FROM AUTHOR]

Video segmentation has been an important and challenging issue for many video applications. Usually there are two different video segmentation approaches, i.e., shot-based segmentation that uses a set of key-frames to represent a video shot and object-based segmentation that partitions a video shot into objects and background. Representing a video shot at different semantic levels, two segmentation processes are usually implemented separately or independently for video analysis. In this paper, we propose a new approach to combine two video segmentation techniques together. Specifically, a combined key-frame extraction and object-based segmentation method is developed based state-of-the-art video segmentation algorithms and statistical clustering approaches. On the one hand, shot-based segmentation can dramatically facilitate and enhance object-based segmentation by using key-frame extraction to select a few key-frames for statistical model training. On the other hand, object-based segmentation can be used to improve shot-based segmentation results by using model-based key-frame refinement. The proposed approach is able to integrate advantages of these two segmentation methods and provide a new combined shot-based and object-based frame- work for a variety of advanced video analysis tasks. Experimental results validate effectiveness and flexibility of the proposed video segmentation algorithm. [ABSTRACT FROM AUTHOR]

Most modern computers or game consoles are equipped with powerful yet cost-effective graphics processing units (GPUs) to accelerate graphics operations. Though the graphics engines in these GPUs are specially designed for graphics operations, can we harness their computing power for more general nongraphics operations? The answer is positive. In this paper, we present our study on leveraging the GPUs graphics engine to accelerate the video decoding. Specifically, a video decoding framework that involves both the central processing unit (CPU) and the GPU is proposed. By moving the whole motion compensation feedback loop of the decoder to the GPU, the CPU and GPU have been made to work in parallel in a pipelining fashion. Several techniques are also proposed to overcome the GPUs constraints or to optimize the GPU computation. Initial experimental results show that significant speed-up can be achieved by utilizing the GPU power. We have achieved real-time playback of high definition video on a PC with an Intel Pentium III 667-MHz CPU and an nVidia GeForce3 GPU. [ABSTRACT FROM AUTHOR]

In this paper, a novel transform domain digital Watermarking scheme that uses visually meaningful binary image as watermark has been developed. The method embeds the water- mark information adaptively with localized embedding strength according to the noise sensitivity level of the host image. Fuzzy adaptive resonance theory (Fuzzy-ART) classification is used to identify appropriate locations for watermark insertion and its con- trol parameters add agility to the clustering results to thwart counterfeiting attacks. The scalability of visually recognizable watermark is exploited to devise a robust weighted recovery method with composite watermark. The proposed watermarking schemes can also be employed for oblivious detection. Unlike most oblivious watermarking schemes, our methods allow the use of visually meaningful image as watermark. For automation friendly verification, a normalized correlation metric that suits well with the statistical property of our methods is used. The experimental results show that the proposed techniques can survive several kinds of image processing attacks and the JPEG lossy compression. [ABSTRACT FROM AUTHOR]

This paper describes a new family of three-dimensional (3-D) representations for computer graphics and animation, called depth image-based representations (DIBR), which have been adopted into MPEG-4 Part16: Animation Framework extension (AFX). Idea of the approach is to build a compact and photorealistic representation of a 3-D object or scene without using polygonal mesh. Instead, images accompanied by depth values for each pixel are used. This type of representation allows us to build and render novel views of objects and scene with an interactive rate. There are many different methods for the image-based rendering with depths, and the DIBR format is designed to efficiently represent the information necessary for such methods. The main formats of the DIBR family are SimpleTexture (an image together with depth array), PointTexture (an image with multiple pixels along each line of sight), and Octreelmage (octree-like data structure together with a set of images containing viewport parameters). In order to store and transmit the DIBR object, we develop a compression algorithm and bitstream format for Octreelmage representation. [ABSTRACT FROM AUTHOR]

This paper proposes an efficient two-dimensional (2-D) discrete cosine and inverse discrete cosine transform (DCT/IDCT) core design. Adopting the row-column decomposition technique for computing 2-D DCT/IDCT, we formulate the one-dimensional (1-D) DCT/IDCT into cyclic convolution by properly arranging the input sequence, optimize the multiplications based on the concept of common subexpression sharing, and carry out the multiplications through carry-save adders (CSAs). Using cyclic convolution is helpful in exploiting the word-level data sharing in computing different DCT/IDCT outputs. Adopting the common subexpression sharing is beneficial to the bit-level data sharing in computing the outputs. As compared with some existing approaches of realizing DCT/IDCT, the proposed approach can save on average 20%∼33% in the delay-area product (gate-count * time-unit) based on a 0.35-μm CMOS technology under the data word-lengths ranging from 16∼24 b. Besides, we have also proposed an IP generator for designing the 2-D DCT/IDCT based on the proposed approach. It provides a design-automation environment with parameter configurations in designing a 2-D DCT/IDCT core that is suitable for most image and video compression applications. [ABSTRACT FROM AUTHOR]

The development of technologies and standards intended to enable and organize a variety of innovative services over the infrastructure that supports digital TV requires a parallel evolution of the techniques to manage its main information media: digital video. Particularly, as TV broadcasters confront the provision of dynamic value-added content-based services over the audio-visual information that is being transmitted, they require online annotation of the received digital channels. In order to achieve it, apart from high levels of success in video temporal segmentation--a great resource-consuming task previous to most video annotation techniques--a faster than real-time performance is crucial. This paper describes a software module for video temporal segmentation which is able to detect both abrupt transitions and all types of gradual transitions in real time, with levels of recall and precision not reported up to date for such a large and varied set of video sequences. [ABSTRACT FROM AUTHOR]

In this paper, we survey the techniques for image-based rendering (IBR) and for compressing image-based representations. Unlike traditional three-dimensional (3-D) computer graphics, in which 3-D geometry of the scene is known, IBR techniques render novel views directly from input images. IBR techniques ran be classified into three categories according to how much geometric information is used: rendering without geometry, rendering with implicit geometry (i.e., correspondence), and rendering with explicit geometry (either with approximate or accurate geometry). We discuss the characteristics of these categories and their representative techniques.IBR techniques demonstrate a surprising diverse range in their extent of use of images and geometry in representing 3-D scenes. We explore the issues in trading off the use of images and geometry by revisiting plenoptic-sampling analysis and the notions of view dependency and geometric proxies. Finally, we highlight compression techniques specifically designed for image-based representations. Such compression techniques are important in making IBR techniques practical. [ABSTRACT FROM AUTHOR]

This paper presents a system architecture for synthetic/natural hybrid coding toward future visual services. Scene-description capability, terminal architecture, and network architecture are discussed by taking account of the standardization activities: MPEG, VRML, ITU-T, and IETF. A consistent strategy to integrate scene-description capability and streaming technologies is demonstrated. Experimental results are also shown, in which synthetic/natural integration is successfully carried out [ABSTRACT FROM PUBLISHER]

An architecture of a matrix-vector multiplier (MVM) is devised, which is dedicated to MPEG-4 natural/synthetic video decoding. The MVM can perform the matrix-vector multiplication both in the inverse discrete cosine transform (IDCT) and in the geometrical transformation of three-dimensional computer graphics (3-D CG); or, specifically, it can achieve the multiplication of a 4×4 matrix by a four-tuple vector necessary in the one-dimensional IDCT for eight pixels and in the geometrical transformation for a point in a 3-D space. This paper describes a new architecture of this MVM and also shows the implementation result of a functional module composed of four MVMs with the use of 440-k transistors, which can operate at 20 MHz or less [ABSTRACT FROM PUBLISHER]