Search

Lossless image and video compression is required in many professional applications. However, lossless coding results in a high data rate, which leads to a long wait for the user when the channel capacity is limited. To overcome this problem, scalable lossless coding is an elegant solution. It provides a fast accessible preview by a lossy compressed base layer, which can be refined to a lossless output when the enhancement layer is received. Therefore, this paper presents a lossy to lossless scalable coding system where the enhancement layer is coded by means of intra prediction and entropy coding. Several algorithms are evaluated for the prediction step in this paper. It turned out that Sample-based Weighted Prediction is a reasonable choice for usual consumer video sequences and the Median Edge Detection algorithm is better suited for medical content from computed tomography. For both types of sequences the efficiency may be further improved by the much more complex Edge-Directed Prediction algorithm. In the best case, in total only about 2.7% additional data rate has to be invested for scalable coding compared to single-layer JPEG-LS compression for usual consumer video sequences. For the case of the medical sequences scalable coding is even more efficient than JPEG-LS compression for certain values of QP.

Temporal bracketing can create images with higher dynamic range than the underlying sensor. Unfortunately, moving objects cause disturbing artifacts. Moreover, the combination with high frame rates is almost unachiev­ able since a single video frame requires multiple sensor readouts. The combination of multiple synchronized side-by-side cameras equipped with different attenuation filters promises a remedy, since all exposures can be performed at the same time with the same duration using the playout video frame rate. However, a disparity correction is needed to compensate the spatial displacement of the cameras. Unfortunately, the requirements for a high quality disparity correction contradict the goal to increase dynamic range. When using two cameras, disparity correction needs objects to be properly exposed in both cameras. In contrast, a dynamic range in­crease needs the cameras to capture different luminance ranges. As this contradiction has not been addressed in literature so far, this paper proposes a novel solution based on a three camera setup. It enables accurate de­ termination of the disparities and an increase of the dynamic range by nearly a factor of two while still limiting costs. Compared to a two camera solution, the mean opinion score (MOS) is improved by 13.47 units in average for the Middleburry images.

This paper presents a method for iterative minimization of combined residual and prediction error for near-lossless compression of medical computed tomography acquisitions using pixel-wise least-squares prediction. While most other lossy state-of-the-art image compression systems like JPEG 2000 make use of transform-based coding, in lossless coding higher compression ratios can be achieved with plain predictive algorithms like JPEG-LS because of their non-linear data adaptive energy reduction. Yet, applying these algorithms in lossy coding, simple quantization usually leads to error propagation and therefore serious quality loss or rate increase, as prediction accuracy of a pixel value and thus data rate depends on the previously reconstructed image region. The proposed minimization approach modifies the original image to be coded in a way such that the edge-directed prediction method from literature may achieve better predictions while introducing only a minimum amount of distortion. Compared to transform-based coding methods, the distortion introduced by the proposed scheme mostly consists in noise reduction instead of blurring or the introduction of artificial structures. The method also prevents error propagation due to the consideration of all pixel dependencies of the prediction. It is shown that, combined with a context-adaptive arithmetic coder, in high-fidelity coding (i. e., PSNR higher than 55 dB) the proposed method can achieve higher compression ratios than the transform-based approaches JPEG 2000, H.264/AVC, and HEVC intra coding.

Although we observe a steady progress in the development of High Dynamic Range Video (HDRV) technology, current image sensors are still lacking achievable dynamic range for high image quality applications. We propose a new imaging principle that is based on a spatial variation of optical Neutral Density (ND) filters on top of some pixels. In existing work, this method has been used to trade spatial resolution for an increase in dynamic range. We improve this approach by a non-regular placement of these filters. The non-regular sampling is an important step as any sub-sampling with regular patterns leads to aliasing. The non-regular patterns however preserve just a single dominant spatial frequency and enable an image reconstruction without aliasing. In combination with a new image reconstruction approach, we are able to recover image details at high resolution. The iterative reconstruction is based on the assumption that natural images can be represented with few coefficients in the Fourier domain. As typical natural images can be classified as near-sparse, the method enables the reconstruction of images of high objective and visual quality. In extension to theory and simulation of this method we want to present details on a practical implementation of our method. While building a demonstration system we encountered many challenges. This includes effects like crosstalk, aspects like sensor selection and mask fabrication as well as mounting of the masks.

We present a new approach for efficient estimation and storage of tissue deformation in dynamic medical image data like 3-D+t computed tomography reconstructions of human heart acquisitions. Tissue deformation between two points in time can be described by means of a displacement vector field indicating for each voxel of a slice, from which position in the previous slice at a fixed position in the third dimension it has moved to this position. Our deformation model represents the motion in a compact manner using a down-sampled potential function of the displacement vector field. This function is obtained by a Gauss-Newton minimization of the estimation error image, i. e., the difference between the current and the deformed previous slice. For lossless or lossy compression of volume slices, the potential function and the error image can afterwards be coded separately. By assuming deformations instead of translational motion, a subsequent coding algorithm using this method will achieve better compression ratios for medical volume data than with conventional block-based motion compensation known from video coding. Due to the smooth prediction without block artifacts, particularly whole-image transforms like wavelet decomposition as well as intra-slice prediction methods can benefit from this approach. We show that with discrete cosine as well as with Karhunen-Lo`eve transform the method can achieve a better energy compaction of the error image than block-based motion compensation while reaching approximately the same prediction error energy.

Synthesizing novel views from originally available camera perspectives via depth maps is a key issue in the 3D video domain. Up to now, several high-resolution cameras are needed to obtain high-quality intermediate synthesized views. One possibility to reduce costs with regard to the used camera array is to replace some cameras by low-resolution cameras, which are cheaper on the one hand, but provide a much poorer image quality on the other hand. Unfortunately, some of the information inside the desired intermediate view may only be available in the low-resolution reference. Thus, the image quality of the low-resolution reference has a big influence on the visual quality of the synthesized view. This paper proposes a postprocessing step for the synthesized view, based on the non-local means algorithm. Thereby, all areas inserted from the low-resolution reference get efficiently adapted to their high-resolution environment. It is shown, that the non-local means refined image merging leads to a PSNR gain of up to 0.90 dB compared to an unrefined mixed-resolution setup. The approach can be easily extended to a hole-filling algorithm and yields a PSNR gain of up to 0.81 dB for hole areas compared to a reference hole-filling algorithm. The subjective image quality also increases convincingly in both applications.

In-car navigation systems have grown in complexity over the recent years, most notably in terms of route calculation, usability and graphical rendering. In order to guarantee correct system behavior, navigation systems need to be tested under real operating conditions, i.e. with field-tests on the road. In this paper, we will focus on a fast compression solution for 2D navigation renderings, so that field-tests can be archived and handed over to software engineers for subsequent evaluation. No parameters from the rendering procedure are available since access to the system is limited to the raw display signal. Rotation is a dominant factor throughout all navigation sequences, so we show how to reconstruct rotational motion parameters with high accuracy and develop a Global Motion Estimation (GME) method as support for a subsequent H.264/AVC video encoder. By integrating ratedistortion optimization concepts into our scheme, we can efficiently omit the segmentation of static and non-static areas. The runtime of the compression solution, which achieves bitrate savings of up to 19.5%, is evaluated both on a laptop CPU and an embedded OMAP4430 system on chip.

The image processing pipeline of a traditional digital camera is often limited by processing power. A better image quality could be generated only if more complexity was allowed. In a raw data workflow most algorithms are executed off-camera. This allows the use of more sophisticated algorithms for increasing image quality while reducing camera complexity. However, this requires a major change in the processing pipeline: a lossy compression of raw camera images might be used early in the pipeline. Subsequent off-camera algorithms then need to work on modified data. We analyzed this problem for the interpolation of defect pixels. We found that a lossy raw compression spreads the error from uncompensated defects over many pixels. This leads to a problem as this larger error cannot be compensated after compression. The use of high quality, high complexity algorithms in the camera is also not an option. We propose a solution to this problem: Inside the camera only a simple and low complexity defect pixel interpolation is used. This significantly reduces the compression error for neighbors of defects. We then perform a lossy raw compression and compensate for defects afterwards. The high complexity defect pixel interpolation can be used off-camera. This leads to a high image quality while keeping the camera complexity low.

Typical image sensors in digital cameras have a fixed sensitivity, and the amount of captured light energy is often controlled by adjusting exposure time and lens aperture. For high end motion imaging these settings are not available as they are used to set motion blur and depth of field, respectively. In many cases a proper exposure is achieved with additional optical filtering, using so called "neutral density" (ND) filters. We propose a digital equivalent of a neutral density filter, which can replace the handling of optical filters for camera systems. It consists of an adjusted sensor readout and in-camera processing of images. Instead of a single long exposure we capture N short exposures. These images are then combined by averaging. The short exposures reduce the sensitivity by a factor of N, while averaging reconstructs motion blur. In addition we also achieve a reduction of both dynamic and fixed pattern noise which leads to an overall increase in dynamic range. The digital ND filter can be used with regular image sensors and does not require hardware modifications.

In recent years, a new technique called High Dynamic Range (HDR) has gained attention in the image processing field. By representing pixel values with floating point numbers, recorded images can hold significantly more luminance information than ordinary integer images. This paper focuses on the realization of a lossy compression scheme for HDR images. The JPEG2000 standard is used as a basic component and is efficiently integrated into the compression chain. Based on a detailed analysis of the floating point format and the human visual system, a concept for lossy compression is worked out and thoroughly optimized. Our scheme outperforms all other existing lossy HDR compression schemes and shows superior performance both at low and high bitrates.

A 4-D wavelet-based transform is used for efficient and scalable compression of multi-view video data. It is composed of a 1-D temporal wavelet transform, namely Motion Compensated Temporal Filtering (MCTF), a 1-D view-directional wavelet transform, namely Disparity Compensated View Filtering (DCVF), and a 2-D spatial transform. The latter is the subject of study in this paper. Whereas usually fixed isotropic wavelet or wavelet packet transforms have been used in the past for the spatial decomposition of the temporal-view-directional highpass bands, we now introduce the usage of adaptive anisotropic wavelet packet transforms as a generalization of wavelet and wavelet packet transforms. An efficient algorithm to adaptively find the rate-distortion optimal joint anisotropic basis for temporal-view-directional multi-view video subbands is derived. It is shown that the adaptive anisotropic transform performs best, compared with conventional wavelet or wavelet packet transforms.

Image-based rendering techniques require capturing an object or scene from many viewpoints. Often, depth maps are used in addition to the original images for rendering new iews for novel viewpoints not coinciding with one of the original camera positions. Due to the high amount of data, efficient compression is necessary. When the data is stored or transmitted, it is not only desirable to compress the image data but also the depth information. In this paper, the case of sequences recorded with a single, hand-held camera is investigated. These sequences are used for unstructured lumigraph rendering. In addition, the case of multi-view video sequences is analyzed. For both cases, depth maps are compressed using H.264/AVC, and the achievable data rates are studied. For the case of unstructured lumigraph rendering, the effect of depth map compression on the quality of rendered images is analyzed in a second step.

The additive attack public mutual information game is explicitly solved for one of the simplest quantization based watermarking schemes, the scalar Costa scheme (SCS). It is a zero-sum game played between the embedder and the attacker, and the payoff function is the mutual information. The solution of the game, a subgame perfect nash equilibrium, is found by backward induction. Therefore, the Blahut-Arimoto algorithm is employed for numerically optimizing the mutual information over noise distributions. Although the worst case distribution is in general strongly non-Gaussian, the capacity degradation compared to a suboptimal Gaussian noise attack is quite small. The loss, if the embedder optimizes SCS for a Gaussian attack but the worst case attack is employed, is negligible.

Blind spread-spectrum watermarking schemes with correlation based detection algorithms suffer significantly from host signal interference. Improvements are possible when considering the host signal as side-information to the watermark encoder. Costa showed that, in a specific case, host signal interference can be avoided completely. The scalar Costa scheme, which operates on structured codebooks based on scalar quantizers, performs relatively close to the ideal Costa scheme in terms of maximum rate of reliable communication (capacity) over a channel with AWGN attack. The concept of amplitude limited channels in conjunction with repetition of the signal constellation is also known in the communications community by the name modulo channel, which has been studied into great detail. Starting from the modulo channel, which implies a constraint on the amplitude in the receiver, we model the watermarking scenario based on scalar quantizers. The capacity of this model is presented and compared to the theoretical results for SCS watermarking in the case of AWGN attacks. A final analysis of SCS in terms of optimality, based on the results with the modulo channel, shows that even binary SCS is near optimum for one-dimensional (symbol-by-symbol) embedding in terms of capacity for an i.i.d. Gaussian host signal and an appropriate AWGN attack.

Motion estimation constitutes a critical part in video coding since it is computationally expensive and has a substantial impact on the resulting image quality. A very efficient method to estimate motion is through adding a rate constraint for coding of the motion vectors into the estimation process. In this paper we analyze the performance of this approach in the emerging MPEG-4 standard, considering frame- as well as object-based video sequences. The investigations show that rate constrained motion estimation generally offers significant gains in coding efficiency which, however, depend on the shape of the object coded. Complexity measurements further show that a rate constraint imposes only a marginal computational overhead, thus making this approach a useful amendment for an MPEG-4 coder.

This paper presents the future multimedia standard MPEG-4 and discusses its potential use for broadband applications. After an introduction into MPEG-4, scenarios for broadband MPEG-4 sessions are described and some examples are presented. At the same time, the structure of a decoder terminal is explained and the relevant parts from the broadband key issues of the standard are presented in more detail.

A complexity and visual quality analysis of several fast motion estimation (ME) algorithms for the emerging MPEG-4 standard was performed as a basis for HW/SW partitioning for VLSI implementation of a portable multimedia terminal. While the computational complexity for the ME of previously standardized video coding schemes was predictable over time, the support of arbitrarily shaped visual objects (VO), various coding options within MPEG-4 as well as content dependent complexity (caused e.g. by summation truncation for SAD) introduce now content (and therefore time) dependent computational requirements, which can't be determined analytically. Therefore a new time dependent complexity analysis method, based on statistical analysis of memory access bandwidth, arithmetic and control instruction counts utilized by a real processor, was developed and applied. Fast ME algorithms can be classified into search area subsampling, pel decimation, feature matching, adaptive hierarchical ME and simplified distance criteria. Several specific implementations of algorithms belonging to these classes are compared in terms of complexity and PSNR to ME algorithms for arbitrarily and rectangular shaped VOs. It is shown that the average macroblock (MB) computational complexity per arbitrary shaped P-VOP (video object plane) depicts a significant variation over time for the different motion estimation algorithms. These results indicate that theoretical estimations and the number of MBs per VOP are of limited applicability as approximation for computational complexity over time, which is required e.g. for average system load specification (in contrast to worst case specification), for real-time processor task scheduling, and for Quality of Service guarantees of several VOs.

An ultimate goal for future telecommunication applications is giving the viewer the feeling of being present in the scene, or short 3-D telepresence. One way to achieve a natural 3-D impression is to encode image sequences using 3-D model objects and animate them again by computer graphic means regarding the observers' eye positions. A crucial task within such a system is the estimation of 3-D shape parameters based on range information derived from disparity estimation between two or more cameras. The contribution describes a new approach to create a triangulated irregular net over a given region in a depth map. It attaches more importance to rendering aspects regarding texture mapping than to spatial accuracy. The shape estimation starts with the polygonalization of the region boundary, which describes a curve in 3-D space. After projecting the polygon into the image plane a constrained, quality conforming Delaunay triangulation is executed in the image plane. This forces the triangulation to insert all segments of the projected boundary polygon into the mesh and not to create any triangle exceeding a certain area. Finally the triangles of the net are subdivided recursively further, until an approximation criteria depending on the local curvature of the depth map is reached.

In this paper a technique for improving the image quality of block-based video coders is presented which combines the following pre- and postprocessing steps: Before the coding process, the image sharpness is enhanced by applying a special prefilter to the original image. By this means a possible degradation of the image sharpness by the coder is compensated. The decoder output is smoothed by a lowpass filter which reduces artifacts such as block discontinuities. Secondly, the quantization error of the frequency DCT coefficients is reduced by predicting AC coefficients based on the mean values of the neighboring image blocks. This step improves the luminance function within the blocks. While the original sequences show coarse blocking patterns, the images processed by this technique look smooth on the overall with well-preserved local edge structures. The tool supports all standardized DCT codecs, such as MPEG-1/2 or H.261/3.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper we outline a new approach to transform coding of four-color printed images employing variable blocksizes in connection with cpiadtree data structures. A new statistical model based decision criterion for guidance of the quadtree decomposition is proposed. This decision criterion avoids drawbacks of simple deviation tests or detail measures and can easily be extended towards multi-color segmentation. To account for the greatly varying local image content found in printed images, a highly adaptive threshold coding concept using an activity/entropy block classification scheme is introduced. Segmentation results as well as average compression ratios for the proposed system will be given.