Search

VLSI realization of video compression is the key to many real-time multimedia communications systems. Among the video compression algorithms, the newly established MPEG-4 and, in particular, H.264 standards have become increasingly popular. However, the high coding efficiency of such video coding algorithms comes at the cost of a dramatic increase in complexity. Effective and efficient hardware solution to this problem are necesary IN this article we present an overview of the hardware design issues of MPEG-4 and H.264. Both mudule and system architectures of these two coding of encoding the D1 resolotion (720 X 48 ) video at 30 Hz. is presented as an example. In addition, the system integration issues of video compression engines with multimedia communication systems and a general hardware platform for various application are discussed. [ABSTRACT FROM AUTHOR]

The emerging MPEG-4 standard supports the transmission and composition of facial animation with natural video. The new standard will include a facial animation parameter (FAP) set that is defined based on the study of minimal facial actions and is closely related to muscle actions. The FAP set enables model-based representation of natural or synthetic talking-head sequences and allows intelligible visual reproduction of facial expressions, emotions, and speech pronunciations at the receiver. This paper addresses the data-compression issue of talking heads and presents three methods for bit-rate reduction of FAPs. Compression efficiency is achieved by way of transform coding, principal component analysis, and FAP interpolation. These methods are independent of each other in nature and thus can be applied in combination to lower the bit-rate demand of FAPs, making possible the transmission of multiple talking heads over band-limited channels. The basic methods described here have been adopted into the MPEG-4 Visual Committee Draft and are readily applicable to other articulation data such as body animation parameters. The efficacy of the methods is demonstrated by both subjective and objective results [ABSTRACT FROM PUBLISHER]

Recent mobile imaging seeks to expedite the autofocus process by embedding a phase detector in the image sensor to provide information for controlling both the magnitude and direction of lens movement. Compared to conventional contrast-detection autofocus, phase-detection autofocus (PDAF) is able to quickly bring the lens toward the in-focus position. However, the presence of sensor noise, the lack of image contrast, and the spatial offset between the left and right phase detectors can easily affect the performance of phase detection. We present a statistical approach to address this issue by characterizing the distribution of phase shift for a given distance of the lens to the in-focus position. We model the phase shift as a skew-normal distribution and verify it empirically. The results show that the skew-normal distribution is indeed a proper model for the phase shift data. We also propose a method based on Bayes' theorem to determine the lens movement. Experimental results show that the proposed method is able to improve the reliability of PDAF. [ABSTRACT FROM AUTHOR]

Conventional stereoscopic displays suffer from vergence-accommodation conflict and cause visual fatigue. Integral-imaging-based displays resolve the problem by directly projecting the sub-aperture views of a light field into the eyes using a microlens array or a similar structure. However, such displays have an inherent trade-off between angular and spatial resolutions. In this paper, we propose a novel coded time-division multiplexing technique that projects encoded sub-aperture views to the eyes of a viewer with correct cues for vergence-accommodation reflex. Given sparse light field sub-aperture views, our pipeline can provide a perception of high-resolution refocused images with minimal aliasing by jointly optimizing the sub-aperture views for display and the coded aperture pattern. This is achieved via deep learning in an end-to-end fashion by simulating light transport and image formation with Fourier optics. To our knowledge, this work is among the first that optimize the light field display pipeline with deep learning. We verify our idea with objective image quality metrics (PSNR, SSIM, and LPIPS) and perform an extensive study on various customizable design variables in our display pipeline. Experimental results show that light fields displayed using the proposed technique indeed have higher quality than that of baseline display designs.

Significant progress has been made for face detection from normal images in recent years; however, accurate and fast face detection from fisheye images remains a challenging issue because of serious fisheye distortion in the peripheral region of the image. To improve face detection accuracy, we propose a light-weight location-aware network to distinguish the peripheral region from the central region in the feature learning stage. To match the face detector, the shape and scale of the anchor (bounding box) is made location dependent. The overall face detection system performs directly in the fisheye image domain without rectification and calibration and hence is agnostic of the fisheye projection parameters. Experiments on Wider-360 and real-world fisheye images using a single CPU core indeed show that our method is superior to the state-of-the-art real-time face detector RFB Net.

Full-field optical coherence tomography (FF-OCT) has been developed to obtain three-dimensional (3D) OCT data of human skin for early diagnosis of skin cancer. Detection of dermal epidermal junction (DEJ), where melanomas and basal cell carcinomas originate, is an essential step for skin cancer diagnosis. However, most existing DEJ detection methods consider each cross-sectional frame of the 3D OCT data independently, leaving the relationship between neighboring frames unexplored. In this paper, we exploit the continuity of 3D OCT data to enhance DEJ detection. In particular, we propose a method for noise reduction of the training data and a multi-directional convolutional neural network to predict the probability of epidermal pixels in the 3D OCT data, which is more stable than one-directional convolutional neural network for DEJ detection. Our crosscheck refinement method also exploits the domain knowledge to generate a smooth DEJ surface. The average mean error of the entire DEJ detection system is approximately 6 μm., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

For a procam to preserve the color appearance of an image projected on a color surface, the photometric distortion introduced by the color surface has to be properly compensated. The performance of such photometric compensation relies on an accurate estimation of the projector nonlinearity. In this paper, we improve the accuracy of projector nonlinearity estimation by taking inter-pixel coupling into consideration. In addition, to respond quickly to the change of projection area due to projector movement, we reduce the number of calibration patterns from six to one and use the projected image as the calibration pattern. This greatly improves the computational efficiency of re-calibration that needs to be performed on the fly during a multimedia presentation without breaking its continuity. Both objective and subjective results are provided to illustrate the effectiveness of the proposed method for color compensation.

The standard medical practice for cancer diagnosis requires histopathology, which is an invasive and time-consuming procedure. Optical coherence tomography (OCT) is an alternative that is relatively fast, noninvasive, and able to capture three-dimensional structures of epithelial tissue. Unlike most previous OCT systems, which cannot capture crucial cellular-level information for squamous cell carcinoma (SCC) diagnosis, the full-field OCT (FF-OCT) technology used in this paper is able to produce images at sub-micron resolution and thereby facilitates the development of a deep learning algorithm for SCC detection. Experimental results show that the SCC detection algorithm can achieve a classification accuracy of 80% for mouse skin. Using the sub-micron FF-OCT imaging system, the proposed SCC detection algorithm has the potential for in-vivo applications., (© 2020 Wiley-VCH GmbH.)

Light field imaging, which captures spatial-angular information of light incident on image sensors, enables many interesting applications such as image refocusing and augmented reality. However, due to the limited sensor resolution, a trade-off exists between the spatial and angular resolutions. To increase the angular resolution, view synthesis techniques have been adopted to generate new views from existing views. However, traditional learning-based view synthesis mainly considers the image quality of each view of the light field and neglects the quality of the refocused images. In this paper, we propose a new loss function called refocused image error (RIE) to address the issue. The main idea is that the image quality of the synthesized light field should be optimized in the refocused image domain because it is where the light field is viewed. We analyze the behavior of RIE in the spectral domain and test the performance of our approach against previous approaches on both real (INRIA) and software-rendered (HCI) light field datasets using objective assessment metrics such as MSE, MAE, PSNR, SSIM, and GMSD. Experimental results show that the light field generated by our method results in better refocused images than previous methods.

Recent advances in optical coherence tomography (OCT) lead to the development of OCT angiography to provide additional helpful information for diagnosis of diseases like basal cell carcinoma. In this paper, we investigate how to extract blood vessels of human skin from full-field OCT (FF-OCT) data using the robust principal component analysis (RPCA) technique. Specifically, we propose a short-time RPCA method that divides the FF-OCT data into segments and decomposes each segment into a low-rank structure representing the relatively static tissues of human skin and a sparse matrix representing the blood vessels. The method mitigates the problem associated with the slow-varying background and is free of the detection error that RPCA may have when dealing with FF-OCT data. Both short-time RPCA and RPCA methods can extract blood vessels from FF-OCT data with heavy speckle noise, but the former takes only half the computation time of the latter. We evaluate the performance of the proposed method by comparing the extracted blood vessels with the ground truth vessels labeled by a dermatologist and show that the proposed method works equally well for FF-OCT volumes of different quality. The average F-measure improvements over the correlation-mapping OCT method, the modified amplitude-decorrelation OCT angiography method, and the RPCA method, respectively, are 0.1835, 0.1032, and 0.0458.

As more and more stereo cameras are installed on electronic devices, we are motivated to investigate how to leverage disparity information for autofocus. The main challenge is that stereo images captured for disparity estimation are subject to defocus blur unless the lenses of the stereo cameras are at the in-focus position. Therefore, it is important to investigate how the presence of defocus blur would affect stereo matching and, in turn, the performance of disparity estimation. In this paper, we give an analytical treatment of this fundamental issue of disparity-based autofocus by examining the relation between image sharpness and disparity error. A statistical approach that treats the disparity estimate as a random variable is developed. Our analysis provides a theoretical backbone for the empirical observation that, regardless of the initial lens position, disparity-based autofocus can bring the lens to the hill zone of the focus profile in one movement. The insight gained from the analysis is useful for the implementation of an autofocus system.

Flat surfaces in our living environment to be used as replacements of a projection screen are not necessarily white. We propose a perceptual radiometric compensation method to counteract the effect of color projection surfaces on image appearance. It reduces color clipping while preserving the hue and brightness of images based on the anchoring property of human visual system. In addition, it considers the effect of chromatic adaptation on perceptual image quality and fixes the color distortion caused by non-white projection surfaces by properly shifting the color of the image pixels toward the complementary color of the projection surface. User ratings show that our method outperforms the existing methods in 974 out of 1020 subjective tests.

A focus profile depicts the image sharpness (or focus value) as the lens sweeps along the optical axis of a camera. Accurate modeling of the focus profile is important to many imaging tasks. In this paper, we present an approach to focus profile modeling that makes the search of in-focus lens position a mathematically tractable problem, and hereby improves the efficiency and accuracy of image acquisition. The proposed approach entails a transformation that converts the representation of a focus profile to quadratic form. An important feature of the approach is that no prior knowledge of the focus measurement technique is required. Experimental results are provided to demonstrate the effectiveness of the approach.

Switching the liquid crystal display (LCD) backlight of a portable multimedia device to a low power level saves energy but results in poor image quality especially for the low-luminance image areas. In this paper, we propose an image enhancement algorithm that overcomes such effects of dim LCD backlight by taking the human visual property into consideration. It boosts the luminance of image areas below the perceptual threshold while preserving the contrast of the other image areas. We apply the just noticeable difference theory and decompose an image into an HVS response layer and a background luminance layer. The boosting and compression processes, which enhance the visibility of the low-luminance image areas, are carried out in the background luminance layer to avoid luminance gradient reversal and over-compensation. The contrast of the processed image is further enhanced by exploiting the Craik-O'Brein-Cornsweet visual illusion. Experimental results are provided to show the performance of the proposed algorithm.

In this paper, we present a system for estimating the shadow field from a single natural image. Unlike previous works that require extensive user assistance, our system only needs the user to roughly specify the shadow boundary with a broad brush. As the user finishes drawing a stroke, the system starts to estimate the shadow field around the stroke and generates pretty accurate result even if the underlying surface is highly textured. We also propose an optimization scheme to propagate the estimated shadow field to the entire image, achieving a further reduction of user effort required for the system. The shadow field estimated by our system can be used to seamlessly remove the shadow from the image. It is also useful for many shadow editing tasks such as pasting an object's shadow from one image to another. Experimental results on a variety of photos are provided to show the effectiveness of the proposed system.

Due to the sequential-readout structure of complementary metal-oxide semiconductor image sensor array, each scanline of the acquired image is exposed at a different time, resulting in the so-called electronic rolling shutter that induces geometric image distortion when the object or the video camera moves during image capture. In this paper, we propose an image processing technique using a planar motion model to address the problem. Unlike previous methods that involve complex 3-D feature correspondences, a simple approach to the analysis of inter- and intraframe distortions is presented. The high-resolution velocity estimates used for restoring the image are obtained by global motion estimation, BEzier curve fitting, and local motion estimation without resort to correspondence identification. Experimental results demonstrate the effectiveness of the algorithm.