Search

Our aim in this study was to optimize and validate an adaptive denoising algorithm based on Block-Matching 3D, for reducing image noise and improving assessment of left ventricular function from low-radiation dose coronary CTA. In this paper, we describe the denoising algorithm and its validation, with low-radiation dose coronary CTA datasets from 7 consecutive patients. We validated the algorithm using a novel method, with the myocardial mass from the low-noise cardiac phase as a reference standard, and objective measurement of image noise. After denoising, the myocardial mass were not statistically different by comparison of individual datapoints by the students' t-test (130.9±31.3g in low-noise 70% phase vs 142.1±48.8g in the denoised 40% phase, p= 0.23). Image noise improved significantly between the 40% phase and the denoised 40% phase by the students' t-test, both in the blood pool (p

Visual analysis of three-dimensional (3D) Coronary Computed Tomography Angiography (CCTA) remains challenging due to large number of image slices and tortuous character of the vessels. We aimed to develop an accurate, automated algorithm for detection of significant and subtle coronary artery lesions compared to expert interpretation. Our knowledge-based automated algorithm consists of centerline extraction which also classifies 3 main coronary arteries and small branches in each main coronary artery, vessel linearization, lumen segmentation with scan-specific lumen attenuation ranges, and lesion location detection. Presence and location of lesions are identified using a multi-pass algorithm which considers expected or "normal" vessel tapering and luminal stenosis from the segmented vessel. Expected luminal diameter is derived from the scan by automated piecewise least squares line fitting over proximal and mid segments (67%) of the coronary artery, considering small branch locations. We applied this algorithm to 21 CCTA patient datasets, acquired with dual-source CT, where 7 datasets had 17 lesions with stenosis greater than or equal to 25%. The reference standard was provided by visual and quantitative identification of lesions with any ≥25% stenosis by an experienced expert reader. Our algorithm identified 16 out of the 17 lesions confirmed by the expert. There were 16 additional lesions detected (average 0.13/segment); 6 out of 16 of these were actual lesions with

The complexity model of the H.264 deblocking filter (DBF) is studied in this work. The DBF process consists of three main modules: 1) boundary strength computation, 2) edge detection, and 3) low-pass filtering. Complexities of all three are considered in the proposed model. DBF-based decoding complexity control is also investigated. It is shown experimentally that the proposed complexity model can provide fair estimation results. Besides, the H.264 encoder equipped with the complexity model and the DBF-based decoding complexity control algorithms can generate bit streams to save a significant amount of decoding complexity while offering quality similar to those generated by a typical H.264 encoder.

The performance of the motion-compensated prediction (MCP) in video coding is degraded by aliasing due to spatial sampling. To alleviate this problem in H.264/AVC, a low-pass filter is used by the fractional-pel motion estimation (FME) to suppress the aliasing component. However, the FME demands higher computational complexity on H.264/AVC encoding. In this work, we first perform a joint quantization and aliasing analysis on the H.264/AVC MCP process and show that the impact of the aliasing component can be alleviated by the quantization process. Then, we propose a fast motion estimation (ME) algorithm that uses the FME and the integer-pel motion estimation (IME) adaptively. The adaptive FME/IME algorithm examines the coding modes of the reference block for the current coding block, and then decides whether the FME or the IME should be applied to the current coding block. Experimental results show that the proposed adaptive FME/IME algorithm can help the encoder generate a bit stream at much lower computational complexity with small degradation in the coding gain as compared with a pure FME algorithm.

A novel multi-order-residual (MOR) coding approach is proposed for high-bit-rate video compression in this work. We show that a significant amount of short and medium-range correlations due to the use of a fine quantization parameter cannot be easily removed by the well known motion compensated prediction process in the high-bit- rate coding scenario. Consequently, the video coding performance degrades rapidly as quality increases. To address this issue, we propose a Multi-Order-Residual (MOR) coding scheme that encodes prediction residuals in multiple stages according to their correlation characteristics. Then, we adopt different compensation and coding schemes in different stages. Experimental results are given to show the proposed MOR scheme outperforms H.264/AVC by a significant margin.

Medical images often consist of low-contrast objects corrupted by random noise arising in the image acquisition process. Thus, image denoising is one of the fundamental tasks required by medical imaging analysis. In this work, we investigate an adaptive denoising scheme based on the nonlocal (NL)-means algorithm for medical imaging applications. In contrast with the traditional NL-means algorithm, the proposed adaptive NL-means (ANL-means) denoising scheme has three unique features. First, it employs the singular value decomposition (SVD) method and the K-means clustering (K-means) technique for robust classification of blocks in noisy images. Second, the local window is adaptively adjusted to match the local property of a block. Finally, a rotated block matching algorithm is adopted for better similarity matching. Experimental results from both additive white Gaussian noise (AWGN) and Rician noise are given to demonstrate the superior performance of the proposed ANL denoising technique over various image denoising benchmarks in term of both PSNR and perceptual quality comparison.

A novel video compression scheme that exploits the idea of second-order-residual (SOR) coding is proposed for high-bit-rate video applications in this work. We first study the limitation of today's high performance video coding standard, H.264/AVC, and show that it is not effective in the coding of small image features and variations for high-bit-rate video contents. For low to medium quality video streams, these small image features can be removed by the quantization process. However, when the quantization stepsize becomes small in high-bit-rate video, their existence degrades the rate-distortion coding performance significantly. To address this problem, we propose a coding scheme that decomposes the residual signals into two layers: the first-order-residual (FOR) and the second-order-residual (SOR). The FOR contains low frequency residuals while the SOR contains the high frequency residuals. We adopt the H.264/AVC for the FOR coding and propose two schemes, called SOR-freq and SOR-bp, for the SOR coding. It is shown by experimental results that the proposed FOR/SOR scheme outperforms H.264/AVC by a significant margin (with about 20% bit rate saving) in high-bit-rate video.

Coronary CT angiography (CTA) with multi-slice helical scanners is becoming the integral part of major diagnostic pathways for coronary artery disease. In addition, coronary CTA has demonstrated substantial potential in quantitative coronary plaque characterization. If serial comparisons of plaque progression or regression are to be made, accurate 3D volume registration of these volumes would be particularly useful. In this work, we propose a coronary CTA volume registration of the paired coronary CTA scans using feature-based non-rigid volume registration. We achieve this with a combined registration strategy, which uses the global rigid registration as an initialization, followed by local registration using non-rigid volume registration with a volume preserving constraint. We exploit the extracted coronary trees to help localize and emphasize the region of interest as unnecessary regions hinder registration process, which results in wrong registration result. The extracted binary masks of each coronary tree may not be the same due to initial segmentation errors, which could lead to subsequent bias in the registration process. Therefore we utilize a blur mask which is generated by convolving the Gaussian function with the binary coronary tree mask to include the neighboring vessel region into account. A volume preserving constraint is imposed so that the total volume of the binary mask before and after co-registration remains constant. To validate the proposed method, we perform experiments with data from 3 patients with available serial CT scans (6 scans in total) and measure the distance of anatomical landmarks between the registered serial scans of the same patient.

Automated image segmentation has been playing a critical role in medical image analysis. Recentl, Level Set methods have shown an efficacy and efficiency in various imaging modalities. In this paper, we present a novel segmentation approach to jointly delineate the boundaries of epi- and endocardium of the left ventricle on the Magnetic Resonance Imaging (MRI) images in a variational framework using level sets, which is in great demand as a clinical application in cardiology. One strategy to tackle segmentation under undesirable conditions such as subtle boundaries and occlusions is to exploit prior knowledge which is specific to the object to segment, in this case the knowledge about heart anatomy. While most left ventricle segmentation approaches incorporate a shape prior obtained by a training process from an ensemble of examples, we exploit a novel shape constraint using an implicit shape prior knowledge, which assumes shape similarity between epi- and endocardium allowing a variation under the Gaussian distribution. Our approach does not demand a training procedure which is usually subject to the training examples and is also laborious and time-consuming in generating the shape prior. Instead, we model a shape constraint by a statistical distance between the shape of epi- and endocardium employing signed distance functions. We applied this technique to cardiac MRI data with quantitative evaluations performed on 10 subjects. The experimental results show the robustness and effectiveness of our shape constraint within a Mumford-Shah segmentation model in the segmentation of left ventricle from cardiac MRI images in comparison with the manual segmentation results.

A high definition video coding technique using super-macroblocks is investigated in this work. Our research is motivated by the observation that the macroblock-based partition in H.264/AVC may not be efficient for high definition video since the maximum macroblock size of 16 x 16 is relatively small against the whole image size. In the proposed super-macboblock based video coding scheme, the original block size MxN in H.264 is scaled to 2Mx2N. Along with the super-macroblock prediction framework, a low-complexity 16 x 16 discrete cosine transform (DCT) is proposed. As compared with the 1D 8 x 8 DCT, only 16 additions are added for a 1D 16 points 16 x 16 DCT. Furthermore, an adaptive scheme is proposed for the selection the best coding mode and best transform size. It is shown by experimental results that the super-macroblock coding scheme can achieve a higher coding gain.

A new technique for film grain noise extraction, modeling and synthesis is proposed and applied to the coding of high definition video in this work. The film grain noise is viewed as a part of artistic presentation by people in the movie industry. On one hand, since the film grain noise can boost the natural appearance of pictures in high definition video, it should be preserved in high-fidelity video processing systems. On the other hand, video coding with film grain noise is expensive. It is desirable to extract film grain noise from the input video as a pre-processing step at the encoder and re-synthesize the film grain noise and add it back to the decoded video as a post-processing step at the decoder. Under this framework, the coding gain of the denoised video is higher while the quality of the final reconstructed video can still be well preserved. Following this idea, we present a method to remove film grain noise from image/video without distorting its original content. Besides, we describe a parametric model containing a small set of parameters to represent the extracted film grain noise. The proposed model generates the film grain noise that is close to the real one in terms of power spectral density and cross-channel spectral correlation. Experimental results are shown to demonstrate the efficiency of the proposed scheme.

A new complexity-scalable framework for motion estimation is proposed to efficiently reduce the motion-complexity of encoding process, with focus on long term memory motion-compensated prediction of the H.264 video coding standard in this work. The objective is to provide a complexity scalable scheme for the given motion estimation algorithm such that it reduces the encoding complexity to the desired level with insignificant penalty in rate-distortion performance. In principle, the proposed algorithm adaptively allocates available motion-complexity budget to macroblock based on estimated impact towards overall rate-distortion (RD) performance subject to the given encoding time limit. To estimate macroblock-wise tradeoff between RD coding gain (J) and motion-complexity (C), the correlation of J-C curve between current macroblock and collocated macroblock in previous frame is exploited to predict initial motion-complexity budget of current macroblock. The initial budget is adaptively assigned to each blocksize and block-partition successively and motion-complexity budget is updated at the end of every encoding unit for remaining ones. Based on experiment, proposed J

The problem of musical instrument recognition is investigated and its application to music segmentation is examined in this research. We propose a new framework for extracting features, in which audio frames are not placed uniformly as done traditionally. The equal spacing method inevitably leads to the consequence that some frames may contain a transition between notes and/or two notes may be included in one frame. Then, a music frame may consist of sounds from multiple instruments. Onset detection is integrated with frame location selection in this work to mitigate this phenomenon. This new framing scheme, called onsetaware framing scheme, provides comparable or better performance as compared with traditional methods. A new histogram-based feature is also presented and used with other common features in the musical instrument classification task. Feature reduction is adopted to reduce the dimensionality of the feature space. We conduct experiments on data sets of a different size with both synthesized and real musical signals. Finally, a simple segmentation method based on the musical instrument classification is proposed and demonstrated by an example.

Automatic music segmentation and structure analysis from audio waveforms based on a three-level hierarchy is examined in this research, where the three-level hierarchy includes notes, measures and parts. The pitch class profile (PCP) feature is first extracted at the note level. Then, a similarity matrix is constructed at the measure level, where a dynamic time warping (DTW) technique is used to enhance the similarity computation by taking the temporal distortion of similar audio segments into account. By processing the similarity matrix, we can obtain a coarse-grain music segmentation result. Finally, dynamic programming is applied to the coarse-grain segments so that a song can be decomposed into several major parts such as intro, verse, chorus, bridge and outro. The performance of the proposed music structure analysis system is demonstrated for pop and rock music.

A hybrid approach to cloud modeling and shading based on volumetric procedural modeling, particle modeling and point primitives is proposed in this work. The volumetric procedural modeling allows great flexibility in controlling object's level of detail, the particle modeling enables to model clouds piece by piece to facilitate interactive fly-in animation, and the point primitive facilitates the multi-resolution representation and animation of complex objects. A preliminary experimental result is given to demonstrate the superior performance of the proposed method.

A stereo video coding algorithm based on H.264 is presented in this work. The main difference between the multi-view and the single-view video coding algorithm is whether a disparity vector (DV) is used or not. For stereo video, we use binocular sequences that consist of left- and right-view sequences. The left-view sequence is coded by the conventional H.264 algorithm and used as a reference image for the prediction of the right-view sequence. The right-view sequence is coded with motion vectors (MVs) and DV. To code a forward prediction (P) frame or a bi-directional prediction (B) frame of the right-view sequence, we employ 2 or 3 prediction vectors. They are forward MV, backward MV and DV. For the coding of stereo sequences with DV and forward/backward MVs, we propose new macroblock (MB) type tables and a temporal prediction structure. Experimental results show that the proposed algorithm offers a higher peak signal to noise ratio (PSNR) than the H.264 simulcast method.

An operational rate control (RC) scheme based on two-stage encoding is studied in this research, where frame-layer rate control with a constant bit rate to achieve constant video quality is examined. In the first encoding stage, the R-D optimized mode decision and its associated motion estimation (RDO) as well as DCT/Q, IQ/IDCT and entropy coding are performed for all macroblocks (MBs) for a target frame using an initial quantization parameter (QP), which is the QP of its previous frame. In the second encoding stage, the residual signal from the first stage is encoded using several QP values around the initial QP. Given the target bits and distortion for the current frame, the residual signal is finally encoded using the QP determined by comparing target bits and distortion with actual bits and distortion. To reduce the additional coding complexity of the two-stage encoding, upper and lower bounds around target bits and distortion are employed to reduce the number of encoding required in the second stage. Experimental results are given to show the superior performance of two proposed rate control algorithms, where one targets at the constant bit rate while the other at constant quality.

An algorithm that integrates table indexing and quad-tree decomposition is proposed for cartoon image compression in this work. The proposed method includes 3 steps. First, colors in the color palette are selected based on the input image and a set of training images. Second, the input image is divided into blocks of size 16 by 16. The number of colors inside each block is checked. If the block has one uniform color or exactly two colors, no further processing is required. Otherwise, quad-tree decomposition will be performed for this block. The subdivision continues until all subblocks have either one or two colors. Then, the code for each subblock will be output in a depth-first order. If the subblock size reaches 2 x 2 and the number of colors in that block is still more than 2, no further subdivision is performed and a code that indicates colors of 4 pixels are output. Finally, to further reduce the size, the data part of the output stream is losslessly compressed by the LZW method. Experimental results are given to demonstrate the superior performance of the proposed method for cartoon image compression.

Stream switching among compressed video streams coded at different quality levels and bit rates is examined, and two enhanced stream switching schemes for H.264 video are proposed in this work. The flexibility of the original H.264 stream switching scheme is achieved at the cost of coding efficiency by introducing primary and secondary SP/SI pictures. In contrast, no modification is made to the original compressed streams in our enhanced schemes so that coding efficiency is maintained at switching points. When switching occurs, the two new schemes use a new picture type, called the difference picture (DIFF), in different ways to compensate the mismatch of reference frames. The difference picture can be coded efficiently, and it is easy to pick a good quantization step size to meet the quality requirement. It is shown by experimental results that the two new schemes outperform the original H.264 switching scheme in the sense that they can achieve prompt stream switching without noticeable quality degradation, and a small amount of bit rate overhead is demanded only when switching occurs.

A progressive lossless 3D geometry encoder using a hierarchical vertex set split method is presented in this work. Compared with prior art, the proposed coder has significantly better rate-distortion (R-D) performance at low bit-rates and provides visually pleasant intermediate meshes at all bit-rates. Given a 3D mesh, all its 3D vertices form an initial vertex set, which is split into several child vertex sets using the well-known Generalized Lloyd Algorithm (GLA). Each newly generated vertex set that contains more than one vertex is iteratively split so as to form a hierarchical structure. During the process of hierarchical vertex set split, a representative is calculated for each newly generated vertex set. Then, the representatives of all existing vertex sets form an approximation to the original 3D geometry. For each vertex set split, the number of child vertex sets is arithmetic encoded, and the offsets of the child representatives from their parent representative are sorted, quantized and arithmetic encoded. If a finer resolution is required for a vertex set containing only one vertex, the rectangloid cell containing that vertex can be further subdivided and coded iteratively. Experimental results are provided to demonstrate the superior performance of the proposed geometry coder.

Automatic authoring of MTV-style home video using music tempo and visual tempo analysis is investigated in this research. The music tempo is extracted using the onset analysis. The frame-level visual tempo is detected based on the motion degree between consecutive frames. The object-level visual tempo is performed based on the tension analysis of facial expression. Finally, the authoring methodology is presented, which consists of music and visual tempo matching to product MTV-Style video. Experiment results using baby home video are given to demonstrate the performance of the proposed algorithm.

The problem of identifying intense events via multimedia data mining in films is investigated in this work. Movies are mainly characterized by dialog, music, and sound effects. We begin our investigation with detecting interesting events through sound effects. Sound effects are neither speech nor music, but are closely associated with interesting events such as car chases and gun shots. In this work, we utilize low-level audio features including MFCC and energy to identify sound effects. It was shown in previous work that the Hidden Markov model (HMM) works well for speech/audio signals. However, this technique requires a careful choice in designing the model and choosing correct parameters. In this work, we introduce a framework that will avoid such necessity and works well with semi- and non-parametric learning algorithms.

An efficient search range prediction method is proposed to reduce the complexity of motion search in the H.264 video coding standard in this work. The main idea is to predict the temporal search range by modeling the relationship between the rate-distortion (RD) coding gain and the required computational complexity. The proposed algorithm first predicts the temporal search range to maximize the ratio of the expected RD coding gain and the normalized computational cost. Then, fast motion search is performed within the predicted search range with some early termination rule. Experimental results show that the proposed algorithm can save approximately 63-75% of the encoding complexity in motion estimation of H.264 (JM9.3) with negligible degradation in the RD performance.

A new grid system for dynamic fluid animation that controls the number of particles adaptively according to the viewing distance is proposed in this work. The proposed scalable grid system is developed in association with the semi-Lagrangian method to demonstrate the dynamic fluid behavior solved from the system of Navier-Stokes equations. It contains actual particles and virtual particles. To save computations, only actual particles are used to render images viewed from a standard viewpoint. When we zoom in, virtual particles are added to maintain the resolution of the fluid simulation and provide higher quality rendering. We implement a scalable computing procedure for the diffusion step in fluid simulation. The proposed scalable grid system can be incorporated into any semi-Lagrangian method that uses grid or voxel primitives.

An encryption scheme called the Randomized Dictionary Table (RDT), which embeds encryption into the LZ78 data compression method, is proposed and analyzed in this research. The basic idea is to construct multiple dictionaries with a different entry order and then randomly select one dictionary in each compression step according to a pseudo-random sequence. Our scheme incurs light computation overhead to encrypt the compressed data and does not impair the LZ compression ratio. Security analysis demonstrates that the proposed RDT scheme achieves high security strength under both the ciphertext only attack and the known/chosen plaintext attack.

Techniques for generating multi-resolution repositories of reusable XML subschemas are investigated in this research. There are three major components to achieve this goal: weighting XML elements, decomposing XML schemas, and clustering the decomposed subschemas. Two approaches are proposed to calculate the weights of XML elements. One is based on the analysis of links and their attributes while the other is based on the information accumulated from all reachable nodes. Elements with higher weights are selected as roots of reusable subschemas, which are further clustered into K groups according to the information amount carried to obtain multi-resolution repositories.

Online media server scheduling algorithms in distributed video-on-demand (VoD) systems are studied in this work. We first identify the failure rate and the server-side network bandwidth consumption as two main cost factors in a distributed VoD service model. The proposed distributed server scheduler consists of two parts: the request migration scheme and the dynamic content update strategy. By improving the random early migration (REM) scheme, we propose a cost-aware REM (CAREM) scheme to reduce the network bandwidth consumption due to the migration process. Furthermore, to accommodate the change in video popularity and/or client population, we use the server-video affinity to measure the potential server-side bandwidth cost after placing a specific video copy on that server. The dynamic content update strategy uses the server-video affinity to reconfigure video copies on media servers. We conduct extensive simulations to evaluate the performance of the proposed algorithm. It can be shown that CAREM together with the dynamic content update strategy can improve the system performance by reducing the request failure rate as well as the server bandwidth consumption.

In this work, we improve the H.264 error resilient coding scheme in [1, 2] with a hybrid scheme that generates alternative SP macroblocks utilizing both multiple reference frames [1] and the concealed versions of corrupted frames [2]. The new scheme is more robust and able to work effectively under different coding environments. Although the bit rate overhead introduced by [1] and [2] is acceptable for some applications, we use an adaptive coding and bit stream replacement mechanism to reduce the overhead furthermore to meet the strict bandwidth constraints. Specifically, two different versions of alternative SP macroblocks are coded using different quantization levels. They provide different levels of error resilient performance with different bit rate consumptions. When the sender attempts to replace the originally coded version of a target macroblock in the bit stream, it will select one proper version according to the importance of the macroblock. The importance of the macroblock is measured by its influence to the subjective quality of the current frame and its impact to subsequent frames. The implementation and the standard conformance of the proposed scheme are detailed in this work.

A multi-scale DCT-domain image registration technique for two MPEG video inputs is proposed in this work. Several edge detectors are first applied to the luminance component of DC coefficients to generate the so-called difference maps for each input image. Then, a threshold is selected for each difference map to filter out regions of lower activity. Following that, we estimate the displacement parameters by examining the difference maps of the two input images associated with the same edge detector. Finally, the ultimate displacement vector is calculated by averaging the parameters from all detectors. In order to reach higher quality of the output mosaic, 1D alignment is locally applied to pixels around the boundaries of displacement that is decided in the previous step. It is shown that the proposed method reduces the computation complexity dramatically as compared to pixel-based image registration techniques while reaching a satisfactory result in composition. Moreover, we discuss how the overlapping region affects the quality of alignment.

A highly adaptive deblocking algorithm is proposed for MPEG video in this research. In comparison with previous work in this area, the proposed deblocking filter improves in three aspects. First, the proposed algorithm is adaptive to the change of the quantization parameter (QP). Since blocking artifacts between two blocks encoded with different QPs tend to be more visible due to quality difference, filters should be able to adapt dynamically to the QP change between blocks. Second, the proposed algorithm classifies the block boundary into three different region modes based on local region characteristics. The three modes are active, smooth and dormant regions. The active region represents a complex region with details and high activities while the smooth and the dormant regions refer to moderately flat and extremely flat regions, respectively. By applying different filters of different strengths to each region mode, the proposed algorithm can minimize the undesirable blur so that both subjective and objective qualities improve for various types of sequences at a wide range of bitrates. Finally, the proposed algorithm also provides a way to determine the threshold values. The proposed adaptive deblocking algorithms require several thresholds in determining proper region modes and filters. Since the quality of image sequences after filtering depends largely on the threshold values, they have to be determined carefully. In the proposed algorithm, thresholds are determined adaptively to the strength of the blocking artifact and, as a result, to various encoding parameters such as QP, absolute difference between QPs, the coding type, and motion vectors. It is shown by experimental results that the proposed algorithm can achieve 0.2-0.4 dB gains for I- and P-frames, and 0.1-0.3 dB gains for the B-frame when bit streams are encoded using the TM5 rate control algorithm.

In this paper, a hierarchical prediction scheme for fast motion search is proposed in order to implement an effective encoder that pays a small amount of search cost to achieve the best match without little rate-distortion performance degradation. The proposed method has a hierarchical tree structure, where each node has two sub-tasks; namely, the reference frame selection and the spatial search region prediction. From now on, let’s call best motion vector by full search as true motion vector. Proposed fast motion search for multiple reference frames can be summarized as: Step1. Initial searching point in previous frame: In literature, majority of true motion vectors are distributed around the center of searching window in recent frames so we first search the previous frame. Diamond search fails to search the true motion when the initial point misleads search path to local minima. In order to solve this problem, initial points are searched based on multiresolution motion search. Step2. Perform diamond search for each initial points and compare the minimum distortion measure with stopping criterion which defines maximum searching points, minimum diamond size and threshold based on QP. If the stopping criterion is met, then motion search is finished with the motion vector having minimum distortion. Otherwise go to Step 3. Here, the diamond size depends on the motion activity between current frame (T) and previous frame (T-1). Step3. Find out search center in the frame indicated by motion trace (motion vector field obtained from previous encoding stage) for all best points in each diamonds in previous frame (T-1). Based on its motion activity, the diamond size is determined and checked the stopping criterion for all diamond in second previous frames. Iterate this step until the stopping condition is met.

The performance improvement of the in-loop deblocking filter module in the H.264/AVC video coding standard in embedded systems is studied in this research. A novel prediction scheme is presented in to reduce the complexity of the filter selection process and hence increase overall performance. We first examine the H.264/AVC deblocking filters by studying their correlation in terms of the filter type and pattern among a sequence of consecutive P frames and I frames. The experimental results show a high correlation of the filter skip rate and the filter pattern between different P frames and their leading I frame. Based on the correlation analysis, a binary history table predictor (the BHT predictor) and a complete history table predictor (the CHT predictor) are proposed to facilitate the deblocking filter selection process while maintaining good subjective and objective visual quality. We further present a hybrid filter prediction scheme that integrates both BHT and CHT to further improve prediction results.

The energy consumption profiling of the H.264 video decoder on VLIW embedded processors using the Trimaran simulator is conducted. Based on this study, we observe that the branch operations in the quarter-pixel (QP) interpolation and the DCT slow down the issue rate of the VLIW processors. Then, several new instruction architecture sets are proposed to address this issue. These new instructions can be used to speedup the issue rate, and reduce the total energy consumption. Finally, experimental results of the proposed instruction-level power-efficient strategies on the TI C6416 processor are reported and discussed.

Intelligent video pre-processing and authoring techniques that facilitate people to create MTV-style music video clips are investigated in this research. First, we present an automatic approach to detect and remove bad shots often occurring in home video, such as video with poor lighting or motion blur. Then, we consider the generation of MTV-style video clips by performing video and music tempo analysis and seeking an effective way in matching these two tempos. Experiment results are given to demonstrate the feasibility and efficiency of the proposed techniques for home video editing.

Music genre provides an efficient way to index songs in a music database, and can be used as an effective means to retrieval music of a similar type, i.e. content-based music retrieval. A new two-stage scheme for music genre classification is proposed in this work. At the first stage, we examine a couple of different features, construct their corresponding parametric models (e.g. GMM and HMM) and compute their likelihood functions to yield soft classification results. In particular, the timbre, rhythm and temporal variation features are considered. Then, at the second stage, these soft classification results are integrated to result in a hard decision for final music genre classification. Experimental results are given to demonstrate the performance of the proposed scheme.© (2005) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this work, we consider the problem of assigning OVSF (Orthogonal Variable Spreading Factor) codes to arriving calls for multi-rate code-division multiple access systems, and propose a sequence of algorithms to solve this problem from different angles. First, we introduce two new policies, called FCA (Fixed Code Assignment) with fixed set partitioning and DCA (Dynamic Code Assignment) with call admission control under an objective to maximize the average data throughput of the system. Numerical simulation confirms that optimized FCA and DCA perform better than DCA with a greedy policy as the traffic load increases and high-rate calls become dominant. Second, a suboptimal DCA with call admission control is examined. The objective is to generate an average data throughput of the system close to that of the optimal scheme while demanding much lower design and implementation complexity than the optimal scheme. By means of capacity or class partitioning and partial resource sharing, we can significantly reduce the computational complexity, thus achieving good design and implementation scalability. Numerical evaluation shows the superior performance of the proposed schemes with low complexity.

Techniques for semantic weighting and decomposition of XML schemas are investigated in this work for their efficient management. Two approaches are proposed to calculate the weights of XML elements. The first one is based on the analysis of links and their attributes while the other one is based on the information propagated from all reachable nodes. We analyze the influence of different types of links on the weights of XML elements. The weights of XML elements are then used to decompose an XML schema and to choose representatives of decomposed clusters. These two methods provide consistent results. The decomposition of an XML schema can be conducted via two methods: the repetition-based decomposition and the weight-based decomposition. It is shown that the weight-based solution can achieve a multi-resolution decomposition result.

Music genre provides an efficient way to index songs in the music database, and can be used as an effective means to retrieval music of a similar type, i.e. content-based music retrieval. In addition to other features, the temporal domain features of a music signal are exploited so as to increase the classification rate in this research. Three temporal techniques are examined in depth. First, the hidden Markov model (HMM) is used to emulate the time-varying properties of music signals. Second, to further increase the classification rate, we propose another feature set that focuses on the residual part of music signals. Third, the overall classification rate is enhanced by classifying smaller segments from a test material individually and making decision via majority voting. Experimental results are given to demonstrate the performance of the proposed techniques.

The effects of frequency offsets and aperiodic random spreading sequences on the performance of asynchronous multicarrier code division multiple access (CDMA) systems with correlated fading are investigated in this research. Random parameters including asynchronous delays, correlated Rayleigh fading and spreading sequences are averaged to determine the covariance matrix of the interference-plus-noise vector. An analytic expression for the average signal-to-interference-plus-noise ratio (SINR) is obtained. Furthermore, average bit error probability (BEP) based on the Gaussian approximation is also derived. We show that the system degrades significantly if the frequency offset of the desired user is present, while the system is insensitive to interferers’ frequency offsets. In consequence, it is crucial to estimate and compensate the desired user’s synchronization imperfection. Finally, design tradeoff among the number of sub-carriers, fading correlations, and inter-carrier interference (ICI) are presented in simulation results.

A symbol-by-symbol maximum likelihood (ML) detection scheme for multicarrier (MC) systems is proposed in this work. When the number of subchannels is sufficiently large, the received symbols across all subchannels are approximately uncorrelated. Then, the proposed symbol-by-symbol ML detection, which is obtained by minimizing the symbol error probability, is nearly optimal. Furthermore, we will show how to reduce the complexity of the symbol-by-symbol ML detection when the constellation size is large. Simulation results show that the proposed symbol-by-symbol ML detection scheme outperforms the symbol-by-symbol minimum distance (MD) detection scheme by up to 2 dB in a noisy environment with crosstalk such as the DMT-ADSL system.

A scheme that uses the hidden Markov model (HMM) is proposed in this work to detect unauthorized nuisance packets in IP networks, which waste network resources and may result in the denial of service (DoS) attack. The proposed HMM is designed to differentiate the attack traffic from the normal traffic systematically. The design of the basic HMM model is first introduced, and the operations of the detector are then described in detail. Finally, we show that the detector using HMM is not sensitive to various attack types and able to detect the attack at an earlier stage.

A new scheme to control the transmission power, allocate subcarriers, and choose modulation schemes for each mobile terminal in a multiuser OFDM system is investigated in this work. This proposed scheme attempts to make OFDM systems more flexible and robust to channel variations along time. The problem is decomposed to two stages. In the first stage, we maximize the minimum signal-to-noise ratio (SNR) of subchannels subject to constraints. The total system performance should be maintained at an acceptable level in this stage. Based on the result of the first stage, we choose modulation schemes for subchannels of each mobile terminal with an objective to maximize the total transmission rate while satisfying the performance requirement of each mobile terminal in the second stage. Simulations are conducted under a varying number of mobile terminals with a range of symbol error rates (SERs). Simulation results show that the proposed scheme works well under certain operating environments.

A robust wireless video transmission scheme using MPEG video Markup Language (MPML) with adaptive error correction codes (ECCs) is proposed in this work. MPML can provide selective description for various MPEG video components to meet the special application requirements. We present an efficient MPML compression algorithm to reduce the size of MPML description and apply the adaptive ECCs to protect the MPML-coded description bitstream unequally against channel noises. It is demonstrated by experimental results that the proposed MPML protection can achieve good error resilient performance comparable with the well-known error resilient techniques.

Efficient multimedia encryption algorithms are fundamental to multimedia data security because of the massive data size and the need of real-time processing. In this research, we present an efficient encryption scheme, called the RCI-QM coder, which achieves the objective of encryption with a regular QM coder by applying different coding conventions to encode individual symbols according to a statistically random sequence. One main advantage of our scheme is the negligible computation cost associated with encryption. We also demonstrate with cryptanalysis that the security level of our scheme is high enough to thwart most common attacks. The proposed RCI-QM coder is easy to implement in both software and hardware, while providing backward compatibility to the standard QM coder.

Story units are extracted by general tempo analysis including tempos analysis including tempos of audio and visual information in this research. Although many schemes have been proposed to successfully segment video data into shots using basic low-level features, how to group shots into meaningful units called story units is still a challenging problem. By focusing on a certain type of video such as sport or news, we can explore models with the specific application domain knowledge. For movie contents, many heuristic rules based on audiovisual clues have been proposed with limited success. We propose a method to extract story units using general tempo analysis. Experimental results are given to demonstrate the feasibility and efficiency of the proposed technique.© (2003) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A scalable video summarization and navigation system is proposed in this work. Particularly, given the desired number of keyframes for a video sequence, we first distribute it among underlying video scenes and sinks based on their respective importance ranks. Then, we select the most important shot of each sink as its R-shot and further assign each sink's designated number of keyframes to its R-shot. Finally, a time-constrained keyframe extraction scheme is developed to locate all keyframes. Consequently, we can achieve a scalable video summary from the initial keyframe set by exploiting such a video structure-based ranking scheme. In addition, a content navigation tool is also developed which could help users freely access or locate specific video scenes or shots. Sophisticated user studies have shown that this summarization and navigation system can not only help users quickly browse video content, but also assist them in searching for particular video segments.

We investigate the encoding speed improvement for H.264 with a special focus on fast intra-prediction mode selection in this work. It is possible to adopt the rate-distortion (RD) optimized mode in H.264 to maximize the coding gain at the cost of a very high computational complexity. To reduce the complexity associated with the intra-prediction mode selection, we propose a two-step fast algorithm. In the first step, we make a course-level decision to split all possible candidate modes into two groups: the group to be examined further and the group to be ignored. The sizes of these two groups are adaptively determined based on the block activities. Then, in the second step, we focus on the group of interest, and consider an RD model for final decision-making. It is demonstrated by experiment results that the proposed scheme performs 5 to 7 time faster than the current H.264 encoder (JM5.0c) with little degradation in the coding gain.

The emerging H.264 video coding Standard can achieve a substantial coding gain as compared with existing coding standards. One major contribution of its gain comes from a very rich syntax for motion compensated prediction at the expense of a higher computational complexity. To be more specific, seven modes of different block sizes and shapes (i.e. 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4) are supported. To do full search over all modes requires an extremely large amount of computation. We propose a fast search algorithm for the variable block size motion estimation. The proposed algorithm includes three stages. First, an initial estimate of the motion vector is obtained by a multi-resolution motion search. Then, based on the estimated motion vector and its corresponding distortion measure, a rate-distortion model is used to select the initial mode for motion search. Finally, some early-termination rules are adopted to reject impossible block sizes and motion locatios sooner. By avoiding search through all block sizes, the amount of computation involved in the motion search can be substantially reduced. The proposed algorithm can achieve a speed-up factor up to 120 times when compared to the fastest full-search algorithm.

MPML is an XML-based MPEG video coding method that was proposed in our previous work. A comprehensive study on the tradeoff between the coding bit rates and the PSNR performance with MPML under the noisy channel environment is conducted in this research. The original MPML compression algorithm has been modified for partial protection to reduce the XML overhead. A scheme to realize partial MPML protection is proposed. Simulation results demonstrate that the proposed MPML-based error resilient technique can achieve good performance for wireless video transmission.

Multi-hypothesis motion compensated prediction (MHMCP) predicts a block from a weighted sum of multiple reference blocks in the frame buffer. By efficiently combining these reference blocks, MHMCP can provide less prediction errors so as to reduce the coding bit rates. Although MHMCP was originally proposed to achieve high coding efficiency, it has been observed recently that MHMCP can also enhance the error resilient property of compressed video. In this work, we investigate the error propagation effect in the MHMCP coder. More specifically, we study how the multi-hypothesis number as well as hypothesis coefficients influence the strength of propagating errors. Simulation results are given to confirm our analysis. Finally, several design principles for the MHMCP coder are derived based on our analysis and simulation results.