Search

The classical chroma subsampling involves downsampling chrominance components (U and V) while maintaining the luminance component (Y). Recently, a study has attempted to change Y for the chroma-subsampling process, and improved results in the reconstructed image quality are obtained. However, computational complexity remains an issue because the study examines all the candidates in the determined range for minimal pixel distortion for the modified Y; the complexity can be high if the range is large. In this study, we reduced the candidate number to seven at most, which is mathematically optimized. Boundary points for in-the-range red, green, and blue (RGB) values are first decided, followed by the determination of the intervals that concatenate the entire curve (at most seven intervals). Each interval belongs to one of the seven sub-cases of different linear combinations of individual curves. The optimal solution (modified Y) for each sub-case is derived, which is in highly efficient form. Compared with the existing method, the proposed fast method ensures that the image quality is preserved, while the number of search candidates is reduced by 61.30%–69.19% on average, and the computational time of the process is reduced by 31.39%–58.97%, thereby demonstrating an efficient performance.

Dynamic point clouds (DPC) are new media storage formats that allow end-users to watch objects/scenes in a three-dimensional (3D) sense. It can be displayed from different angles throughout time. However, the raw size of a point cloud is huge because there can be millions of points (each containing color triplet and location triplet information) in a point cloud, and there can be multiple point clouds in a DPC. Video-based point cloud compression (V-PCC) is developed to project a 3D point cloud to 2D images: attribute, geometry, and occupancy images. After padding, the 2D images are compressed using the well-established high-efficiency video coding (HEVC). In this study, we first employ an occupancy image to propose a blocky occupancy flag (BOF), to denote the occupancy information on “a block basis”. For coding attribute and geometry images, we use a BOF to develop a fast coding unit (CU) algorithm for early termination of the CU search recursion. We also utilize the geometry images to calculate the 2D and 3D information of each pixel, for 2D/3D spatial homogeneity of the pixels to design fast CU decision. In addition, we proposed a modified rate-distortion optimization for different color components considering the picture order count (POC) structure in HEVC/V-PCC. Finally, we propose an HEVC input pixel modification method based on a BOF to reduce the unnecessary information to be coded for attribute images. Compared with the state-of-the-art fast V-PCC encoding method, the proposed work outperforms by up to 2.31% in Bjøntegaard delta bit rates (BDBR) (with very slight loss by only up to 0.38%), and improves the time saving performances by up to 7.84% for two different testing datasets.

Point cloud become a popular multimedia format. However, due to the massive amount of data, it is necessary to compress the point cloud. After compression, the number of points in the reconstructed point cloud will be reduced due to the quantization or downsampling in the compression process, which may introduce holes in the three-dimensional image. In this paper, we utilize the mathematical morphology to fill the holes in the reconstructed image. The results show that holes are filled with better subjective and objective quality metrics.

This paper presents a multiple edge detection by computing uncorrelatedness of the neighboring pixels in four locations: up, down, left, and right. To recover missing G in R and missing G in B, we use the derived weightings and the pixel contributions from the neighboring pixel information. When demosaicking the R and B arrays, an interpolation is used for the recovery of the missing R in B and missing B in R. The experimental results show that the proposed overall system outperforms the existing works. The proposed work is especially better than the state-of-the-art method by 1.23 dB in composite peak signal-to-noise ratio (CPSNR) and by −0.15 in a Spatial Extension to the CIE L*a*b* DeltaE Color Difference Metric (S-CIELAB), the lower the better, averaged over a standard image dataset.

This article presents a vertical-location approach based on characteristics of barometric pressure sensors (BPSs), where the altitude information of mobile devices is measured and extracted from BPSs based on microelectromechanical systems. On account of environmental factors about atmospheric pressure changes and quantization noises, the calibration approaches are proposed to compute the height information about z-axis and to compensate for the offset difference between BPSs. The experimental results demonstrate that the vertical-location accuracy of the proposed approach can be used for the applications of location-based services.

In this paper, a hardware-oriented lossless EEG compression algorithm including a two-stage prediction, voting prediction and tri-entropy coding is proposed. In two stages prediction, 27 conditions and 6 functions are used to decide how to predict the current data from previous data. Then, voting prediction finds optimal function according to 27 conditions for best function to produce best Error (the difference of predicted data and current data). Moreover, a tri-entropy coding technique is developed based on normal distribution. The two-stage Huffman coding and Golomb-Rice coding was used to generate the binary code of Error value. In CHB-MIT Scalp EEG Database, the novel EEG compression algorithm achieves average compression rate to 2.37. The proposed hardware-oriented algorithm is suitable for VLSI implementation due to its low complexity.

With the rapid development of wireless communication systems and various emerging applications, software implementations could not satisfy the real-time applications of location-based services (LBS). This article presents a three-dimensional (3D) localization algorithms based on positioning and tracking approaches. The proposed real-time positioning and tracking approaches are implemented with field programmable gate arrays (FPGAs). The hardware implementation of the FPGA-based algorithms can improve the traditional matrix operation approaches, and it has advantages of high portability, low cost, and low executing time for LBS applications.

In this paper, we propose a method to predict subsampling scheme by using the machine learning for the chroma reconstruction of screen content images (SCIs). We create a feature matrix with thirty features, and use the classification learner, error-correcting output codes (ECOC) classifier for multiclass learning, to train the model. After testing through the model, we finally get the experimental data that shows us the correlation between the luma and chroma and the accuracy of the model. The accuracy of the model is up to 92%, which provides the decoder with an accurate subsampling scheme. Therefore, with the correct subsampling scheme, it allows the subsampled chroma to be reconstructed accurately.

This paper presents a hardware-oriented image stitching algorithm with characteristic of low complexity. The proposed algorithm is developed by using the feature from accelerated segment test (FAST) algorithm to extract feature point, and combines scale invariant feature transform (SIFT) descriptor to perform feature matching. In order to be suitable for hardware implementation, the proposed algorithm was simplified by using the optimal seam algorithm widely used to achieve seamless image stitching and eliminate ghosting. Moreover, the obvious lines at the stitching seam due to the color difference are reduced by the low-pass filter. Experiment results show that the proposed algorithm has benefits of seamless images with high quality and calculation with low complexity.

The next-generation video compression standard H.266/Future Video Coding (FVC) provides high compression efficiency in terms of the cost of computing the optimal intra mode from 67 modes. We propose an intra mode prediction method based on a convolutional neural network (CNN). An input image set of 20 × 20 blocks is used to train the CNN; the CNN is used to predict the best classes of intra mode direction. The CNN architecture comprises two convolutional layers and a fully connected layer. Compared with the default fast search method in FVC, the proposed method can achieve a 0.033% decrease in Bjontegaard delta bit rate (BDBR) with only a slight increase in time.

Wireless multimedia sensor network (WMSN) is important for environmental monitoring. When the sensors are used as cameras, the network can be regarded as a multiview video system. The Packet loss may occur when the multiview videos are transmitted wirelessly. When the video frames are lost during transmission, a frame reconstruction method is needed in the decoder to estimate the missing pixels. In the proposed work, a reconstruction algorithm for lost frame of multiview videos in the WMSN based on deep learning methods is presented. A novel pixel estimation algorithm is developed using multilayer perceptron regression (MPR) with the deep learning method. Furthermore, a modified inpainting method is proposed with the use of the information from the optical flow algorithm with the neighboring available frames. Compared with the state-of-the-art method, the proposed MPR method with the traditional inpainting method increased the average peak signal-to-noise ratio up to 5.62 dB. The combination of the proposed MPR method with the proposed inpainting method outperformed previous proposed combination up to 8.32 dB on average, showing the significance of the proposed inpainting method.

In this paper, we consider an integrated error concealment system for lost color frames and lost depth frames in multiview videos with depths. We first proposed a pixel-based color error-concealment method with the use of depth information. Instead of assuming that the same moving object in consecutive frames has minimal depth difference, as is done in a state-of-the-art method, a more realistic situation in which the same moving object in consecutive frames can be in different depths is considered. In the derived motion vector candidate set, we consider all the candidate motion vectors in the set, and weight the reference pixels by the depth differences to obtain the final recovered pixel. Compared with the two state-of-the-art methods, the proposed method has average peak signal-to-noise ratio gains of up to 8.73 and 3.98 dB, respectively. Second, we proposed an iterative depth frame error-concealment method. The initial recovered depth frame is obtained by depth-image-based rendering from another available view. The holes in the recovered depth frame are then filled in the proposed priority order. Preprocessing methods (depth difference compensation and inconsistent pixel removal) are performed to improve the performance. Compared with a method that uses the available motion vector in a color frame to recover the lost depth pixels, the hybrid motion vector extrapolation method, the inpainting method and the proposed method have gains of up to 4.31, 10.29, and 6.04 dB, respectively. Finally, for the situation in which the color and the depth frames are lost at the same time, our two methods jointly perform better with a gain of up to 7.79 dB.

With the development of technology, the requirements of hardware equipment and user expectations of visual enjoyment are increasingly gradually. The Joint Video Exploration Team (JVET) has established the latest video compression standard, Future Video Coding (FVC). FVC adopts QuadTree plus Binary Tree (QTBT) based Coding Unit (CU) structure, which not only removes the complex hierarchical structure of the CU, Prediction Unit (PU), and Transform Unit (TU) but also supports square and rectangular coding blocks based on the texture of the video content. Although the QTBT structure can provide superior coding performance, it significantly increases the encoding time, particularly in intra coding. Therefore, developing a fast intra CU partition decision algorithm is essential. In this paper, a fast CU partition decision algorithm in FVC intra coding based on spatial features is proposed. Different spatial features in the pixel domain are proposed in the binary tree and quadtree decision processes. Spatial features for the binary tree are employed for early skipping of the encoding process of CUs with binary tree depth and for early determination of binary tree split mode. Spatial features for the quadtree are employed for early splitting or termination of CUs with quadtree depth. Compared with JEM 5.0, the proposed method can save 23% encoding time on average with a slight increase of 0.62% in the Bjontegaard delta bitrate (BDBR).

As compressed videos are transmitted in the communication networks, video packet loss inevitably occurs. This problem can be solved by error concealment method. We used the motion vector of the available neighboring blocks to estimate the lost motion vector for the lost block. These estimates propagate to predict all other missing motion vectors. We further improved the work by using the idea of the motion vector disparities between neighboring available blocks to modify the motion vector weightings. Furthermore, the differences between the compensated pixels and the decoded pixels in the neighboring blocks are computed for another weighting for improvement. These two novelties are combined as a final indicator to prediction weightings. By comparison against the state-of-the-art method, the four proposed algorithms increase the average peak signal-to-noise ratio (PSNR) by up to 1.86, 1.93, 1.94, and 2.04 dB on average, showing the gradual improvement of our design systems. For other video quality measurements, the average gains of the proposed work against the state-of-the-art work can be up to 0.0575 in structural similarity index metric (SSIM), -0.0278 in video quality metric (VQM) (the lower the better), -0.0008 in motion-based video integrity evaluation (MOVIE) (the lower the better), and 2.77 in subjective evaluation. The proposed work performs slightly worse than a pixel-based state-of-the-art method in PSNR and SSIM but performs better in VQM and MOVIE (both correlate better with human perception) and subjective experiments, with much lower computational complexity.

With the rapid development of portable electronics, wearable devices have become widely used to monitor body signals for long-term health care and home care applications. They detect vital signals through physiological sensors and then transmit them to a cloud database for evaluation and monitoring purposes through wireless communication systems. In this paper, a smart analog-to-digital converter (ADC) was realized by a mixed-signal application-specific integrated circuit (ASIC) based on adaptive resolution and lossless compression techniques for electrocardiogram (ECG) signal monitoring. The sampling clock for the ADC can be adaptively selected according to the characteristic of the signals. The lossless encoder consists of trend forecasting and entropy coding modules. The transmission data rate was decreased efficiently by adaptive resolution and lossless compression techniques. The chip aims to meet the low power consumption for the design, because it reduced the signal transmission rate and maintained high-quality ECG signal detection. The proposed mixed-signal ASIC design was realized using a 0.18- $\mu \text{m}$ CMOS process with a total power consumption of $78.8~\mu \text{W}$ when operating at 1 kHz and a total chip area of 850 $\times 850~\mu \text{m}^{2}$ .

In this paper, we present a novel 3D structure-awareness RGB-D video compression scheme, which applies the proposed 3D super-point clustering to partition the super-points in a colored point cloud, generated from an RGB-D image, into a centroid and a non-centroid super-point datasets. A super-point is a set of 3D points which are characterized with similar feature vectors. Input an RGB-D frame to the proposed scheme, the camera parameters are first used to generate a colored point cloud, which is segmented into multiple super-points using our multiple principal plane analysis (MPPA). These super-points are then grouped into multiple clusters, each of them characterized by a centroid super-point. Next, the median feature vectors of super-points are represented by the K singular value decomposition (K-SVD) based sparse codes. Given a super-point cluster, the sparse codes of the median feature vectors are very similar and thus the redundant information among them are easy to remove by the successive entropy coding. For each super-point, the residual super-point is computed by subtracting the feature vectors inside from the reconstructed median feature vector. These residual feature vectors are also collected and coded using the K-SVD based sparse coding to enhance the quality of the compressed point cloud. This process results in a multiple description coding scheme for 3D point cloud compression. Finally, the compressed point cloud is projected to the 2D image space to obtain the compressed RGB-D image. Experiments demonstrate the effectiveness of our approach which attains better performance than the current state-of-the-art point cloud compression methods.

Due to the rapid development of wireless communication applications, software implementations cannot fulfill real-time computing to satisfy the in-time requirement of location-based services (LBS) applications. This article presents a location-estimation technique based on positioning and tracking algorithms, and the high-performance algorithms are implemented in field programmable gate array (FPGA) approaches, which have the features of pipeline structure and real-time processing. In terms of the FPGA approaches, our results demonstrate that the performance of location accuracy of the proposed approaches is almost the same as those of the software implementations, and it has the advantage of processing time.

In recent years, wireless communication applications have been widely used in various industries, such as the popularity of smart phones, large-scale cloud computing platforms, and the Internet of Things (IoT) in smart cities. Positioning techniques have made great progress in terms of positioning accuracy and usability. These approaches gradually infiltrate all aspects of social life from the areas of navigation, aerospace, aviation, military, and natural disaster prevention. Namely, these techniques become an indispensable and important applications in daily life, such as personnel search, location search, traffic management, vehicle navigation, and route planning. Therefore, an adaptive low-complexity location-estimation approach combined alpha-beta (α-β) filtering algorithm with the Kalman filtering (KF) algorithm is proposed and implemented in this article. The proposed tracking approach is based on the α-β filtering algorithm extracted the coefficients of the Kalman gain from the KF algorithm. In addition, in order to enhance the performance and calculating speed, we propose the hardware implementation for KF algorithm and α-β filtering algorithm. The proposed concept is performed by field programmable gate array (FPGA), which has the features of pipeline structure and real-time processing. Under a stationary environment, as compared with the software implementation for location tracking, the proposed FPGA training, decision, and tracking approach not only can achieve the location accuracy close to the KF tracking approach but has much lower computational complexity and provides better performance than that of non-tracking approaches.

For existing methods on YUV420 subsampling from YUV444 image format, the Y is constrained to change the UV values, or the UVs are constrained to change the Y value. We proposed a method of chroma subsampling with changing YUV values at the same time for YUV420 images. We proposed an assumption (observation) to reduce the algorithm complexity. The experimental results show that the proposed work is better than existing state-of-the-art methods by 1.64 dB and 0.61 dB on average in CPSNR (Composite Peak Signal-to-noise Ratio).

In this paper, a new color image compression algorithm based on Absolute Moment Block Truncation Coding (AMBTC) and entropy coding is proposed. The AMBTC improves the compression performance of Block Truncation Coding (BTC) obviously. A novel technique by using eight different types of bitmap tables is developed for the proposed algorithm. In addition, an entropy coding technique combining prediction and Huffman Coding approach is included in the proposed algorithm and decreased the data amount of luminance images. The experimental results show that more than 82% data amount can be compressed by using the eight different types of novel bitmap tables. Compared with previous studies, this work has better compression performance than previous 4*4 BTC, JPEG, and JPEG-LS algorithms.

In this paper, an optimized backlight power saving algorithm using joint power-peak signal-to-noise ratio (PSNR) characteristics among multiple frames is proposed. In existing state-of-the-art methods, the optimal decision for clipping point is made only with consideration of current frame, and the target frame PSNR are set the same for all frames. Therefore they have the same power-PSNR performance. In the proposed work, we show that better power-PSNR performance can be achieved by assigning different target frame PSNR for different frames. In the proposed algorithm, the decisions for clipping points associated with all frames are made at once. The inefficiency ratio of each clipping point is evaluated and put into the algorithm design. An optimization problem is formulated, and a solution-search procedure is proposed. Compared with state-of-the-art methods, the proposed work performs better in both aspects of power and PSNR; the BD-PSNR and the BD-Power of the proposed work against all other methods are better for every tested video; the BD-PSNR and the BD-Power can be as high as 19.80 dB on average, and the BD-Power can be as low as – 26.25% on average.

This paper presents a novel motion vector recovery and error concealment algorithm with the utilization of encoding partition information for H.264/AVC. The motion vectors for each missing pixel location are derived using available neighboring pixel motion vectors, which contribute to the generation of the missing motion vectors inversely proportional to the distance between them. The motion extrapolation method is used to project the encoding partition information from the reference frame into the current frame with different levels of overlapping of lost pixels. The different levels of overlapping can help determine the estimated encoding partition information in the lost macroblock (MB). Finally, the pixels that are determined to be of the same estimated partition share the same motion vector in order to maintain the integrity of the estimated moving objects in the lost MB. This proposed pixel-based motion vector with partition (PMVP) method compares with the state-of-the-art Zhou's method, Lin's method, and Lie's method. For total average in packet loss rates of 3%, 7%, 16%, and 20%, PMVP is better than Zhou by 0.88, 1.02, 1.05, and 1.01 dB, respectively; Lin by 0.22, 0.32, 0.35, and 0.33 dB, respectively; and Lie by 4.12, 4.98, 4.15, and 3.88 dB, respectively. Therefore, the proposed PMVP performs the best on average among all the methods.

In this paper, we proposed a backlight dimming algorithm for videos, aiming to achieve better video quality measured in video quality metric (VQM) with controlled power consumption. In order to avoid the complex computation of VQM in the testing process, a training procedure is performed to find prediction model. For each training video, a VQM result curve corresponding to different clipping points is first built, to facilitate the consequent search for optimal clipping point. To build a prediction model for optimal clipping point to be used in testing stage, the optimal clipping point of each video is associated with the spatial information (SI) and temporal information (TI) of the video; the association is modeled by a 2-D LOWESS (LOcally WEighted Scatterplot Smoothing) surface. Therefore in the testing phase, for every video, the SI and TI are first computed, and mapped into the LOWESS model to find the predicted optimal clipping point, which is applied to all frames in the video. A generalized version of the proposed method is designed to have even better power reduction performance. The experiment results show that the generalized proposed work can achieve the best power reduction (17.9%), compared with state-of-the-art methods I2GEC (10.5%), MGEC4 (11.2%), MGEC16 (8.7%), and SPBD (8.7%). And the video quality of the generalized proposed method can be maintained, showing almost no visual difference with the original video. Thus, it is demonstrated that the proposed work can achieve good balance between video quality and power consumption.

We propose a fast algorithm with both CU size and PU mode decision method for HEVC.All of the 13 neighboring CTUs are analyzed to predict the current CU depth.k-means method is used to classify the neighboring CTUs into different groups.PU mode candidates are selected according to the CU depth.The proposed method gets 56.71% and 59.76% time reduction with LP and RA profiles. High Efficiency Video Coding (HEVC) adopts complex quadtree-structured CU (coding unit) and PU (prediction unit) modes. Thus, the search process for optimal modes causes high computational complexity in HEVC. To solve this problem, a fast coding algorithm is proposed. Thirteen neighboring CTUs (Coding Tree Unit) are divided into three classes with the k-means method, and are selectively used as the reference CTUs to calculate the predicted CU depth levels for the current CTU. Additionally, for every CU depth, we skip some rarely used PU modes to reduce the complexity of the PU selection algorithm. Experimental results show that compared with the HEVC reference software, our algorithm can achieve 56.71% time saving with Low Delay Configuration profile and 59.76% with Random Access Configuration profile, whereas values of BDBR (Bjontegaard Delta Bit Rate) are only 1.0517% and 0.9918%, respectively. Compared with five state-of-the-art algorithms, the proposed algorithm has significant encoding time reduction with good bitrate performances.

High-efficiency video coding (HEVC) utilizes a quadtree-structured coding unit (CU) for more possible sizes of basic blocks for later prediction. The exhaustive search for the optimal CU depth requires high complexity, which causes difficulties for real-time streaming. In the proposed method, we make use of the residual statistics of prediction units (PUs) in the current encoded CU to make a decision on CU early termination. The residual histogram is discussed, and the optimal early termination (ET) zone is designed; if the PU residual statistics fall in the ET zone, the early termination is triggered. The optimal parameters for the ET zone are solved and analyzed in the training stage, and can be estimated with table lookup and simple computations in the testing stage. Another early termination method using the neighboring SKIP information is also proposed. We use SKIP information from more meaningful CU neighbors defined and used in HEVC. We combine the proposed residual-based and SKIP-based algorithms with a state-of-the-art method. The experimental results show that compared to two state-of-the-art methods in both low delay (LP) and random access (RA) configurations, our preferred proposed combination can achieve the best BDBR results (0.737 % in LP and 0.471 % in RA) with a good corresponding TS performances (49.87 % in LP and 54.96 % in RA), making HEVC more suitable for real-time applications.