Search

Object detection is a fundamental task in computer vision with wide application prospect. And recent years, many novel methods are proposed to tackle this task. However, most algorithms suffer from high computation cost and long inference time, which makes them impossible to be deployed on embedded devices in real industrial application scenarios. In this paper, we propose the Mobile CenterNet to solve this problem. Our method is based on CenterNet but with some key improvements. To enhance detection performance, we adopt HRNet as a powerful backbone and introduce a categorybalanced focal loss to deal with category imbalance problem. Moreover, to compress the model size as well as reduce inference time, knowledge distillation is employed to transfer knowledge from cumbersome model to a compact one. We conduct experiments on a large traffic detection dataset BDD100K and validate the effectiveness of all the modifications. Finally, our method achieves the 1st place in the Embedded Deep Learning Object Detection Model Compression Competition held in ICME 2020.

The learning of Transformation-Equivariant Representations (TERs), which is introduced by Hinton et al. \cite{hinton2011transforming}, has been considered as a principle to reveal visual structures under various transformations. It contains the celebrated Convolutional Neural Networks (CNNs) as a special case that only equivary to the translations. In contrast, we seek to train TERs for a generic class of transformations and train them in an {\em unsupervised} fashion. To this end, we present a novel principled method by Autoencoding Variational Transformations (AVT), compared with the conventional approach to autoencoding data. Formally, given transformed images, the AVT seeks to train the networks by maximizing the mutual information between the transformations and representations. This ensures the resultant TERs of individual images contain the {\em intrinsic} information about their visual structures that would equivary {\em extricably} under various transformations in a generalized {\em nonlinear} case. Technically, we show that the resultant optimization problem can be efficiently solved by maximizing a variational lower-bound of the mutual information. This variational approach introduces a transformation decoder to approximate the intractable posterior of transformations, resulting in an autoencoding architecture with a pair of the representation encoder and the transformation decoder. Experiments demonstrate the proposed AVT model sets a new record for the performances on unsupervised tasks, greatly closing the performance gap to the supervised models., http://maple-lab.net/projects/AVT.htm. arXiv admin note: text overlap with arXiv:1901.04596

The variety of multimedia big data has promoted emerging applications of multiple sensorial media (mulsemedia) types, in which haptic information attracts increasing attentions. Until now, the interaction between haptic signal and conventional audio-visual signals have not been fully investigated. In this work, we make an exploration on the cross-modal interactivity in task-driven scenarios. We first explore the correlation between visual attention and haptic control in three designed tasks: random-trajectory, fixed-trajectory and obstacle-avoidance. Then, we propose a visual-haptic interaction model that estimates kinesthetic position of haptic control with the information of gaze only. By incorporating a Long Short-Term Memory (LSTM) neural network, the proposed model provides effective prediction in the scenarios of fixed-trajectory and obstacle-avoidance, with its performance superior to other selected machine learning-based models. To further examine our model, we execute it in a haptic control task using visual guidance. Implementation results show a high task achievement rate.

Wireless body area network (WBAN) has attracted more and more attention to automatically and intelligently sense the health data of one person for supporting various health applications in smart cities. In the energy-constrained and heterogeneous WBAN system, there are three main issues: 1) the dynamic link characteristics due to the time-varying postures and environments; 2) the high energy efficiency requirement with considering the limited sensor battery; and 3) the high quality-of-service (QoS) requirement due to the importance of health data. To provide long service with high quality, the resource allocation scheme becomes indispensable with considering all these issues. In this paper, a mix-cost parameter is designed to evaluate the energy efficiency and QoS effectiveness, and a resource allocation problem is formulated to minimize the total mix-cost with optimizing the transmission rate, the transmission power, and the allocated time slots for each sensor. Then, a buffer-aware sensor evaluation method with low complexity is introduced to the resource allocation scheme to evaluate the sensor state in real time and then decide when applying for the resource re-allocation by the hub for further improving both the short-term and the long-term QoS performance. Finally, a greedy sub-optimal resource allocation scheme is designed to reduce the time complexity of the resource allocation scheme. Simulation results are presented to demonstrate the effectiveness of the proposed optimal buffer-aware resource allocation scheme as well as the greedy sub-optimal resource allocation scheme with low complexity.

Unsupervised image translation, which aims in translating two independent sets of images, is challenging in discovering the correct correspondences without paired data. Existing works build upon Generative Adversarial Networks (GANs) such that the distribution of the translated images are indistinguishable from the distribution of the target set. However, such set-level constraints cannot learn the instance-level correspondences (e.g. aligned semantic parts in object transfiguration task). This limitation often results in false positives (e.g. geometric or semantic artifacts), and further leads to mode collapse problem. To address the above issues, we propose a novel framework for instance-level image translation by Deep Attention GAN (DA-GAN). Such a design enables DA-GAN to decompose the task of translating samples from two sets into translating instances in a highly-structured latent space. Specifically, we jointly learn a deep attention encoder, and the instance-level correspondences could be consequently discovered through attending on the learned instances. Therefore, the constraints could be exploited on both set-level and instance-level. Comparisons against several state-of-the-arts demonstrate the superiority of our approach, and the broad application capability, e.g, pose morphing, data augmentation, etc., pushes the margin of domain translation problem.1

Recently, non-orthogonal multiple access (NOMA) has been proposed to achieve higher spectral efficiency over conventional orthogonal multiple access. Although it has the potential to meet increasing demands of video services, it is still challenging to provide high performance video streaming. In this research, we investigate, for the first time, a multi-user NOMA system design for video transmission. Various NOMA systems have been proposed for data transmission in terms of throughput or reliability. However, the perceived quality, or the quality-of-experience of users, is more critical for video transmission. Based on this observation, we design a quality-driven scalable video transmission framework with cross-layer support for multi-user NOMA. To enable low complexity multi-user NOMA operations, a novel user grouping strategy is proposed. The key features in the proposed framework include the integration of the quality model for encoded video with the physical layer model for NOMA transmission, and the formulation of multi-user NOMA-based video transmission as a quality-driven power allocation problem. As the problem is non-concave, a global optimal algorithm based on the hidden monotonic property and a suboptimal algorithm with polynomial time complexity are developed. Simulation results show that the proposed multi-user NOMA system outperforms existing schemes in various video delivery scenarios., Comment: 9 pages, 6 figures. This paper has already been accepted by IEEE INFOCOM 2018

Random forest is effective and efficient for detecting facial landmark from visual images, and has achieved the state-of-the-art performance, both in accuracy and speed, by regressing local binary features (LBF). This paper aims to increase the detection accuracy of random forest for facial landmarks and extends it to facial action estimation. First, probabilistic features are designed to overcome the weaknesses of LBF, e.g., feature sparseness and tracking jitter. Second, a deep architecture is introduced to random forest for enhancing the capacity of representation learning. Third, the initial detected facial landmarks are refined and 3D facial actions are estimated jointly by registering a deformable facial model to images based on an optimized iterative closest point framework. Experiments show that the proposed methods significantly outperform the state-of-the-art ones in terms of accuracy, as well as achieve the excellent tracking stability and real-time ability at about 60 fps on an ordinary PC.

Test time and test data volume required to test modern integrated circuits grow rapidly with circuit complexity. Test compression is now widely used in industry to reduce test cost. In this paper, a simple yet highly efficient test data compression technique with small area overhead is presented. Efficient algorithms are developed to determine configurations of the test decompressor and corresponding test patterns in multiple test sessions. These algorithms result in higher test compression ratio and lower test application time with less CPU runtime compared to the latest previous work. Experimental results on IWLS'05 benchmark circuits show that on average we can increase the compression factor by 17.26%, decrease the test application time by 12.28% and cut down the CPU time by 42.92%, with only slight increase of area overhead. More importantly, up to 1500x test compression factor is achieved for a design containing about 2M gates, with only 0.1% area overhead.

In this paper, we address a haze removal problem from a single nighttime image, even in the presence of varicolored and non-uniform illumination. The core idea lies in a novel maximum reflectance prior. We first introduce the nighttime hazy imaging model, which includes a local ambient illumination item in both direct attenuation term and scattering term. Then, we propose a simple but effective image prior, maximum reflectance prior, to estimate the varying ambient illumination. The maximum reflectance prior is based on a key observation: for most daytime haze-free image patches, each color channel has very high intensity at some pixels. For the nighttime haze image, the local maximum intensities at each color channel are mainly contributed by the ambient illumination. Therefore, we can directly estimate the ambient illumination and transmission map, and consequently restore a high quality haze-free image. Experimental results on various nighttime hazy images demonstrate the effectiveness of the proposed approach. In particular, our approach has the advantage of computational efficiency, which is 10-100 times faster than state-of-the-art methods.

Deep convolutional neural networks (CNN) have recently been shown to generate promising results for aesthetics assessment. However, the performance of these deep CNN methods is often compromised by the constraint that the neural network only takes the fixed-size input. To accommodate this requirement, input images need to be transformed via cropping, warping, or padding, which often alter image composition, reduce image resolution, or cause image distortion. Thus the aesthetics of the original images is impaired because of potential loss of fine grained details and holistic image layout. However, such fine grained details and holistic image layout is critical for evaluating an image's aesthetics. In this paper, we present an Adaptive Layout-Aware Multi-Patch Convolutional Neural Network (A-Lamp CNN) architecture for photo aesthetic assessment. This novel scheme is able to accept arbitrary sized images, and learn from both fined grained details and holistic image layout simultaneously. To enable training on these hybrid inputs, we extend the method by developing a dedicated double-subnet neural network structure, i.e. a Multi-Patch subnet and a Layout-Aware subnet. We further construct an aggregation layer to effectively combine the hybrid features from these two subnets. Extensive experiments on the large-scale aesthetics assessment benchmark (AVA) demonstrate significant performance improvement over the state-of-the-art in photo aesthetic assessment.

Node authentication is essential in wireless body area network (WBAN), as the data collected in the WBAN is important and closely related to the personal privacy especially in the health applications. Traditional authentication solutions usually require priori knowledge, advanced hardware or out-of-band channels, which are impractical for the WBAN, and the data-based authentication scheme has been proved to be a promising solution. In this paper, we propose a novel authentication scheme which uses acceleration information collected in the WBAN to determine whether a node is legal. The solution is achieved by exploiting the similarity in the acceleration data between the legal node and the node under authentication. Compared to other data-based authentication schemes, the proposed scheme is more practical, more accurate and with lower false positive rate (FPR).

This paper proposes a 3D virtual animating head system, which can not only talk but also sing. With a reconstructed head mesh model, including external/internal articulators, from multi-source images, biology information are first used to visualize each phoneme with a musical note. The synchronicity between songs and articulatory movements is then modeled by a deep neural network trained on an audio/articulatory corpus. Finally, the visualization results of phonemes are blended by the synchronicity model to produce the song synchronized articulatory animations. Quantitative and qualitative improvements of singing ability on human computer interaction are demonstrated by comparing with other state-of-the-art talking head systems.

Display devices at bit-depth of 10 or higher have been mature but the mainstream media source is still at bit-depth as low as 8. To accommodate the gap, the most economic solution is to render source at low bit-depth for high bit-depth display, which is essentially the procedure of de-quantization. Traditional methods, like zero-padding or bit replication, introduce annoying false contour artifacts. To better estimate the least-significant bits, later works use filtering or interpolation approaches, which exploit only limited neighbor information, can not thoroughly remove the false contours. In this paper, we propose a novel intensity potential field to model the complicated relationships among pixels. Then, an adaptive de-quantization algorithm is proposed to convert low bit-depth images to high bit-depth ones. To the best of our knowledge, this is the first attempt to apply potential field for natural images. The proposed potential field preserves local consistency and models the complicated contexts very well. Extensive experiments on natural image datasets validate the efficiency of the proposed intensity potential field. Significant improvements have been achieved over the state-of-the-art methods on both PSNR and SSIM.

Effective capacity analysis is a powerful tool to investigate the impact of physical layer designs on the link layer delay-sensitive QoS performance, which is important for real-time multimedia applications. In this paper, we rigorously analyze the effective capacities of downlink large-scale antenna systems. The main focus is to establish the fundamental effective capacity in a very-large MIMO system, and to characterize the performance difference between co-located and distributed antenna layouts. To that end, we first analytically derive the closed-form effective capacities for two widely used linear precoding schemes, conjugate beamforming and zero-forcing beamforming. We then analyze the asymptotic average effective capacities when the number of BS antennas and the number of users grow unboundedly with a fixed ratio. The effective capacity gain of the distributed antenna layout over the co-located layout is established via theoretical analysis.

Smart home not only makes people's lives more convenient, but also saves energy and daily expenses for households. Both WiFi and ZigBee are widely deployed in the smart home and operated in the same 2.4GHz ISM band, which results in the coexistence interference. Since the transmitting power of WiFi devices is much higher than that of ZigBee devices, ZigBee is more susceptible to coexisting interference. In this paper, we propose an analytical model to evaluate the performance of ZigBee under WiFi interference in the practical smart home scenario. By considering both channel access behavior and path loss behavior of the WiFi and ZigBee devices, the proposed model is built by involving both the Markov chain model and the indoor path loss model, and the ZigBee performance under WiFi interference is theoretically derived. The simulation results demonstrate the effectiveness of the proposed model for evaluating the ZigBee performance under WiFi interference.

Autism spectrum disorder (ASD) is one of the most common childhood developmental disorders. Early detection and intervention for ASD are critical for increasing child success. In the past decade, utilizing the abnormal eye gaze characteristics of children with autism in regard to certain visual stimuli is emerging as a screening approach due to its cost-efficiency and promising accuracy. However, the effect of visual stimulus on children with ASD has not been considered as a diagnostic consideration in the past. In this paper, we first create a visual stimuli database based on an extensive literature review, then we examine the impact of picture stimuli and exposure time on the quantitative accuracy of screenings for ASD. This is done by extracting gaze distribution in a 2D space and comparing children with ASD to typical peers using the 1st Wasserstein distance. A group of 32 participants with ASD and typical development (TD) were recruited for the study. The f-score accuracy results demonstrate the impact of implementing visual stimulus on screening for ASD. Our study demonstrates that the parsing of “social scene” stimulus with 5-second exposure time has the best performance at 98.24%.

Robust image alignment is a necessary and challenging step for numerous computational photography applications. In particular, large camera motion poses significant challenge to Mobile High Dynamic Range (HDR) Imaging due to hand-held capture of input images and limited computational resources. Aligning images only by detecting and matching image features is computationally expensive and can also be erratic. We present a robust multi-sensory method for aligning exposure bracketed images on mobile cameras. We use inertial sensor based camera pose estimate to pre-warp images and iteratively align them by minimizing alignment error. We also simultaneously estimate local motion masks which can be used to eliminate ghosting artifacts in the final HDR image. We collected HDR image dataset with diverse scenes along with inertial sensor data, which is a novel contribution and have evaluated our performance with existing mobile HDR image alignment techniques in literature.

Everyday, people choose their preferred clothing to wear before leaving home for various activities. It has been recognized that each person may consciously or unconsciously express individual personality through their wearing styles. It will be interesting to ask: Are you what you look like? This paper presents a novel scheme developed to infer personality type from their wearing. The proposed research is justifiably rooted in the psychological findings that reveal intrinsic correlations between clothing style and wearer's inner self in self-image, mood, and social aspirations. First, we build a relatively large dataset with more than 300 persons and over 10,000 portraits, each is labeled with personality type. Then, personality-related clothing features are explored through statistical analysis based on psychological theories. To extract the clothing features from the images, a suite of algorithms, including body detection, GrabCut algorithm and saliency detection have been developed. Binary logistic regression is then applied to verify the Significance Level of the extracted features for predicting personality types. Experimental results demonstrate that the proposed scheme is able to predict several types of personality combination or type pairs with relatively high precisions.

Wireless Body Area Network (WBAN) represents one of the most promising networks to provide health applications for improving the quality of life, such as ubiquitous e-Health services and real-time health monitoring. The resource allocation of an energy-constrained, heterogeneous WBAN is a critical issue that should consider both energy efficiency and Quality of Service (QoS) requirements with the dynamic link characteristics, especially when the limited resource cannot satisfy the expected QoS requirements. In this paper, a buffer aware Energy-efficient and QoS-effective resource allocation scheme is proposed in which the sensor queue buffer states, constraints of QoS metrics and the characteristics of dynamic links are considered. Specifically, a buffer aware sensor evaluation method is designed to dynamically evaluate the sensor state with considering the sensor buffer states for improving the system performance. We then formulate the resource allocation problem for optimizing the transmission power, the transmission rate and the allocated time slots for each sensor to minimize the sum mix-cost, which is defined to characterize the energy cost and QoS cost between attainable QoS support and QoS requirements. Simulation results demonstrate the effectiveness of the buffer aware sensor evaluation method and the proposed energy-efficient and QoS-effective resource allocation scheme.

Mobile display has been considered as the major contributor to the energy consumption of the ever-increasing mobile video services. Current practices in display energy reduction (DER) utilize local computing resources to analyze the video content before DER strategies can be applied in a per-device fashion. For a given video, same analytical computations are repeated in millions of individual devices. In this paper, we demonstrate that a paradigm shifting framework can be designed to systematically move the common DER local processing to the streaming server with the emergence of cloud-based video services. This framework has the potential to replace the massive per-device DER computation by a one-time global video processing in the cloud. To accomplish this ultimate goal in DER, a new family of video rate-distortion (R-D) profile with embedded DER strategies shall be properly generated. This family of rate-distortion-display energy (R-D-DE) profiles contains a set of common DER parameters to be directly extracted and employed by individual mobile devices to achieve desired display energy saving without repeated local computation. Performance evaluations of the proposed design are carried out to show that this family of R-D-DE profiles is indeed able to command the systematic DER design based on the intrinsic relationships among bitrate, video quality and display energy saving.

User-generated videos (UGVs) have dominated contemporary social networking sites (SNSs). Forecasting their popularity is of great relevance to a broad range of online services. All existing studies forecast popularity of UGVs using their popularity statistics that are accumulated for a period of time after they are uploaded. Hence, there is always a substantial time lag (days to weeks) before popularity forecast can take effects. However, such a time lag is undesirable for timely popularity forecast as forecasting initial popularity during UGVs' lifetime is vitally important. In fact, we have found in our measurement that the most popular UGVs usually precede others starting from the beginning days and UGVs generally receive the highest attentions during the first few days. In this paper, we present the first exploration on forecasting initial popularity for UGVs at their uploading moment without accumulating their popularity statistics. Specifically, we first design an effective crawler framework to collect the publicly observable features of videos at their uploading moment. We then collect a representative and large YouTube video data set with 318,627 videos. Based on the data set, we select the most relevant features as predictors and design a neural network-based learning model to forecast initial popularity of just-uploaded UGVs. Experimental results validate the effectiveness of the proposed forecasting model and demonstrate the model's benefits for online services such as in-video advertising and video caching.

With the rapid development of mobile applications in recent years, there is a strong desire on light weight algorithm for view synthesis using uncalibrated images and limited geometry information. To address this challenge, we propose a Bayesian based view synthesis framework to support the rendering of complex scene with multiple planar structures. In this framework, we integrate image segmentation, reference plane selection and hole filling with a Bayesian formulation to perform view synthesis without using 3D geometry information. More specifically, we partition every reference image into multiple planes, estimate geometric and photometric parameters for each plane, synthesize the novel view by Bayesian modeling using selected reference planes, and refine the rendered image by a hole filling scheme. The entire view synthesis process is executed in an iterative manner to pursue high quality visual results. The experimental results show that the proposed method is able to achieve desired performance with less distortion and higher resolution.

Today, mobile users are struggling with accessing overloading and unstructured social media feeds on the severely constrained mobile display. To overcome the challenges associated with browsing social media feeds on mobile devices, we are developing an innovative scheme to automatically create and synthesize the mixed social media digest (pictures, texts and videos) into a magazine-page-like social media visual summary. Given a set of personalized social media digest, a multi-objective optimization is formulated to organize the digest into visual summary in a 9-block-partition fashion with consideration of informative delivery, aesthetic rules and visual perception principles. Each block will be optimized interactively in terms of size, position and color to best represent the overall social media digest. Extensive evaluation and analysis based on user studies demonstrate that the proposed approach is effective in presenting social media content in a visually appealing and compact way. It is expected that this visual summary will lead to much enhanced user experiences for browsing social media digest on mobile devices.

Digital storytelling applications are playing an increasingly important role in people's daily life. In contemporary storytelling applications such as PowerPoint presentation and macro/micro blogs, good presentation images are always highly desired by content creators to boost their presentation in an intuitive and attractive way. Existing studies, however, have not yet addressed the challenging problem of how to select the most appropriate presentation images for storytelling. In this paper, we formulate this problem of presentation image suggestion (given a textual query) as selecting images by maximizing visual and semantic diversity from web image search results of suggested queries. The proposed framework consists of two novel components: 1) click-through-based query suggestion, which is designed to suggest textual queries by searching relevant queries in a constructed query graph that can reflect diverse aspects of a given query, and 2) query-based image selection, which selects the most appropriate presentation images by keeping semantic relevance while maximizing visual diversity and quality, using a novel model based on Conditional Random Field (CRF) by individual and correlation characters. We evaluate the proposed approach by comparing with several baselines and a thorough subjective survey. The evaluations show inspiring results using the proposed approach for automatic suggestion of images for storytelling.

Social media sharing is one of the most popular social interactions in online social networks (OSNs). Due to the diverse networking conditions and various privacy requirements of OSN users, scalable media sharing has become a promising paradigm. It allows a media data distributor to share a media content of different qualities with different data consumers. To guarantee user privacy in scalable media sharing, it is essential to design an effective scalable media access control (SMAC) mechanism. However, all existing schemes cannot support scalable media streams with more than two dimensions, which significantly limits the flexibility of OSN services. In this paper, we present the first multi-dimension SMAC (MD-SMAC) system for social media sharing, based on the proposed scalable ciphertext policy attribute-based encryption (SCP-ABE) algorithm. In the MD-SMAC system, secure and reliable access control can be performed on multidimension scalable media streams according to data consumers' attributes. Through the security analysis, we prove the security and reliability of the MD-SMAC system. We also conduct experiments on mobile devices to demonstrate the computation efficiency of the proposed system.

For object classification, soft assignment (SA) is capable of improving the bag-of-visual-words (BoVW) model and has the advantages in conceptual simplicity. However, the performance of soft assignment is inferior to those recently developed encoding schemes. In this paper, we propose a novel scheme called barycentric coordinates based soft assignment (BCSA) for the classification of object images. While maintaining conceptual simplicity, this scheme will be shown to outperform most of the existing encoding schemes, including sparse and local coding schemes. Furthermore, with only single-scale features, it is able to achieve comparable or even better performance to current state-of-the-art Fisher kernel (FK) encoding scheme. In particular, the proposed BCSA scheme enjoys the following properties: 1) preservation of linear order precision for encoding which makes BCSA robust to linear transform distortions; 2) inheriting naturally the visual word uncertainty which leads to a more expressive model; 3) generating linear classifiable codes that can be learned with significant less computational cost and storage. Extensive experiments based on widely used Caltech-101 and Caltech-256 datasets have been carried out to show its effectiveness of the proposed BCSA scheme in both performance and simplicity.

Wireless Body Area Networks (WBANs) represent one of the most promising networks to provide health applications for improving the quality of life, such as ubiquitous e-Health services and real-time health monitoring. The resource allocation of an energy-constrained, heterogeneous WBAN is a critical issue that should consider both energy efficiency and Quality of Service (QoS) requirements with the dynamic link characteristics, especially when the limited resource cannot satisfy the expected QoS requirements. In this paper, we propose an Energy-efficient and QoS-effective resource allocation that considers a mix-cost parameter characterizing both energy cost and QoS cost between attainable QoS support and QoS requirements. Based on the mix-cost parameter, we first formulate the resource allocation problem as a mixed integer nonlinear programming (MINP) for optimizing the transmission power, the transmission rate and allocated time slots for each sensor to minimize total mix-cost of the system. Then we propose a sub-optimal greedy resource allocation algorithm, which has a much lower complexity compared to exhaustive search. Simulation results demonstrate the advantage of the mix-cost parameter to evaluate energy efficiency and attainable QoS support, as well as verifying the effectiveness of the proposed resource allocation algorithm.

Maintaining color consistency across panoramic video captured by multi-camera array is a challenging problem. In an uncalibrated multi-camera array with automatic camera settings, each camera individually adjusts its parameters in accordance with the region of scene captured, independent of adjacent cameras. This leads to inconsistency in intensity and color in the stitched panoramic image. Selection of one of the images as reference for color correction may yield poor results because no individual camera image may represent the color palette of entire scene. We address these issues by capturing a separate low resolution reference color image with a field of view that encompasses entire scene. The color statistics of the reference image are used to bring each camera array image into a uniform radiance and color palette. We then estimate optimal color correction parameters using a joint pairwise optimization that minimizes overall error in stitched panorama, thus achieving a fast and robust color correction scheme for multi-camera panoramic high resolution imaging.

In this paper, a random access with priority protocol named irregular repetition slotted ALOHA with priority (PIRSA) is introduced. By optimizing the packets repetition and using successive interference cancellation (SIC), irregular repetition slotted ALOHA (IRSA) has achieved a high throughput. Contention resolution diversity slotted ALOHA with access control (AC-CDRSA) enhances both throughput and efficiency under high traffic load. In P-IRSA, priority is considered in order to handle traffics from the users who demands enhanced access and throughput. Access control is added before the traffic, and users with high priority gain higher probability to access the channel under high traffic load. Further more, performance of the proposed P- IRSA in terms of the packet loss rate, throughput and channel efficiency are verified by simulations and are compared with IRSA and AC-CDRSA protocols. Simulation results show that the throughput of users with high priority can be enhanced under high traffic load with the proposed protocol.

In this paper, a novel patch-wise visual saliency model based on Minimum Length Description (MDL) principle is presented. Visual saliency is measured as the unpredicted information of image patch through an order-adaptive predictor under MDL principle. Specifically, each image patch is estimated with a linear combination of several neighboring patches. The number and location of candidate patches are automatically tuned to local contexts based on MDL. Then the entropy of prediction residuals of center patch, which represents the surprise to the visual system, is used to measure the saliency. Furthermore, a structural redundancy operator is also involved to improve the saliency detection performance. Experimental results demonstrate that the predictor under MDL principle along with the structural redundancy operator can improve the accuracy of human fixations prediction. We show that the proposed model outperforms the mainstream algorithms in predicting human fixations.

The popular Structural SIMilarity (SSIM) index has shown to be a good perceptual criterion for testing and optimizing video encoders such as the MPEG-H/H.265 High Efficiency Video Coding (HEVC). However, it is still unclear how to compare two HEVC encoders with a number of bit rates and SSIM values. In this work, we study the video quality comparison of HEVC encoders based on the Bit Rate-SSIM (R-S) curves. Our contributions are three-fold. First, we exploit how to interpolate an R-S curve when only a small number of R-S samples are available. We also develop to minimize the weighted sum of squared errors during the interpolation process, where the weights are pre-defined by users to satisfy the diversified requirements. Second, we propose an approach to evaluate the perceptual coding gain in terms of the R-S improvement between two individual video encoders. Third, as an application, we utilize the proposed approach to compare the R-S performances of HEVC encoders with different configurations.

Wireless Body Area Networks (WBANs) are usually designed for pervasive healthcare applications. Since the primary traffic in WBAN is vital physiological signals, guaranteeing the Quality of Service (QoS) is crucial while designing WBAN. However, QoS of WBAN will be degraded in strong inter-WBAN interference environment such as hospitals and senior communities, where WBANs are densely deployed. In this paper, by focusing on a more practical WBAN model, we propose a non- cooperative power control game to mitigate inter- WBAN interference, in which the cost function is well designed by considering both QoS requirement and energy constraint. The existence of at least one Nash equilibrium (NE) point for the game is proved and a sufficient condition for the uniqueness of the NE is derived. To guarantee non-cooperative among WBANs, an interference segmentation estimate (ISE) algorithm is proposed to obtain an approximation of the NE point. Simulation results demonstrate the effectiveness of the proposed ISE algorithm.

Adaptive HTTP streaming with centralized consideration of multiple streams has gained increasing interest. It poses a special challenge that the interests of both content provider and network operator need to be deliberately balanced. More importantly, the adaptation strategy is required to be flexible enough to be ported to various systems that work under different network environments, QoE levels, and economic objectives. To address these challenges, we propose a Markov Decision Process (MDP) based network-assisted adaptation framework, wherein cost of buffering, significant playback variation, bandwidth management and income of playback are jointly investigated. We then demonstrate its promising service provisioning and maximal profit for a mobile network in which fair or differentiated service is required., Comment: ICIP 2015 submission

An important issue in video target tracking is to deal with occlusion problem. In this paper, a hierarchical anti-occlusion tracking algorithm based on Dual Mode Particle Filter (DMPF) and Oriented FAST and Rotated BRIEF (ORB) is proposed to improve the location accuracy under different occlusion conditions in video target tracking. In the first layer, DMPF is used to track target and preliminarily locate its position under various occlusion status. By using corner matching in the second layer, the target is precisely located according to ORB similarity and corner coordinate, thus a more accurate position of target is obtained. A modularized occlusion judgment scheme is also presented to switch tracking mode timely and accurately in DMPF and a Quantified Matrix based on Weighted RGB color space (WQM) is introduced for both color feature creation and corner detection to save operation time. Simulation results show that the proposed algorithm could provide high tracking accuracy in a real-time manner for different occlusion status.

Resizing video frames is one of the key techniques for multimedia processing. In particular, it is more desirable to generate high-resolution video frames from low-resolution sources to meet the needs of customers with larger screen displays and higher display requirement. The original hybrid DCT-Wiener-based interpolation scheme provides a powerful framework to interpolate video frames by combining the information in both spatial and DCT domain seamlessly. It can provide both better objective and visual qualities with low complexity than existing interpolation methods. This paper presents an enhancement over the original hybrid scheme to further improve its performance. We first analyze the bottleneck of the original hybrid interpolation scheme, and then propose a novel spatial interpolation scheme to further improve its performance. The spatial interpolation scheme designed in this research is a contour-enhanced Wiener filter with unsharp masking post-processing. It is learnt using training video frame pairs through the minimum mean square error estimation, which is able to resolve the quarter-pixel shift issue and achieve much better performance over the original 1-D 6-tap pixel-based Wiener filter. Experiments have been carried out to demonstrate that the proposed scheme achieves noticeable gains over the original hybrid interpolation scheme as well as other existing schemes in both objective and visual qualities.

Wireless Body Area Networks (WBAN) is a promising type of networks that mainly targets at applications in ubiquitous communication and e-Health services. To support long-term pervasive services, both reliability and energy efficiency are required in WBAN communications. In this paper, by adopting compressive sensing (CS) technology, a reliable transmission strategy for WBANs is proposed to alleviate the effect of packet loss and improve the data transmission reliability. Considering the transmission model with both direct link and relay link, two relay policies are present and the corresponding fusion rules are designed to merge the data from both links for better reconstruciton. A reconstruction algorithm, which exploits both block sparsity and temporal correlation, is proposed to reconstructed the merged data. The simulation results demonstrate the effectiveness of the proposed transmission strategy.

Image stitching has been developed to enhance users' immersive experiences by constructing new images of higher spatial resolution and broader field of view from multiple input images. Most image stitching schemes assume 2D transformation of the whole frame between images. There usually exist local misalignments of contents that do not match with the transformation and may cause artifacts such as ghosting. Several schemes including seam cutting, image blending and other image composition algorithms, have been developed to relieve such artifacts. However, these schemes are essentially post-stitching processing that do not address the alignment problem and would fail when the parallax is large. In this paper, we propose a multi-objective content preserving warping (CPW) for local adjustment of image alignment which is optimized considering feature matching, line preserving, line matching, shape smoothness and other factors to significantly reduce the visual artifacts. In particular, we adjust the image warping from their pre-warping results using the homography estimated during the image registration step. Experimental results show that the proposed CPW scheme for image stitching is able to generate much smoother stitching results comparing with the state-of-the-art methods reported in the literature.

Video stitching requires proper seam cutting technique to decide the boundary of the sub video volume cropped from source videos. In theory, approaches such as 3D graph-cuts that search the entire spatiotemporal volume for a cutting surface should provide the best results. However, given the tremendous data size of the camera array video source, the 3D graph-cuts algorithm is extremely resource-demanding and impractical. In this paper, we propose a sequential seam cutting scheme, which is a dynamic programming algorithm that scans the source videos frame-by-frame, updates the pixels' spatiotemporal constraints, and gradually builds the cutting surface in low space complexity. The proposed scheme features flexible seam finding conditions based on temporal and spatial coherence as well as salience. Experimental results show that by relaxing the seam continuity constraint, the proposed video stitching can better handle abrupt motions or sharp edges in the source, reduce stitching artifacts, and render enhanced visual quality.

No single retargeting approach performs well on all images and all target sizes; therefore, hybrid algorithms are often considered as promising alternatives. However, most hybrid schemes are time consuming. In this paper, we propose a fast hybrid framework in which the Fast Content-Aware Image Distance (FCAID) is used to connect fast seam carving with neighboring probability constraints (FSc_Neip) and scaling. FCAID is used to measure the image distance between the resized image given by FSc_Neip and the original image. This fast technique is embedded within the FSc_Neip framework. Our hybrid scheme is locally applied in strip regions. This makes the retargeting scheme globally non-homogeneous. Experimental results demonstrate that our approach comprehensively outperforms other state-of-the-art techniques in terms of image quality and computational complexity.

The amount and availability of user-contributed image data have been dramatically increased during the past ten years. Popular multimedia social networks, e.g. Flicker, commonly utilize user image data to construct user behavior models, social preferences, etc., for the purpose of effective advertisement, better user retention and attraction, and many others. Existing practices of data utilization, however, seriously deteriorate users' personal privacy and have led to increasing criticisms and legislation pressures. In this paper, we aim to construct a privacy-preserving feature detection scheme over encrypted image data. The proposed system enables an interested party to perform a variety of image feature detection tasks, including visual descriptors in MPEG-7 standard, while protecting user privacy relating to image contents. We implement a prototype system based on somewhat homomorphic encryption scheme and the benchmark Caltech256 database. The experimental results show that our system can guarantee effective image feature detection without sacrificing user privacy.

Wireless Body Area Network (WBAN) has become a promising type of networks to provide applications such as real-time health monitoring and ubiquitous e-Health services. One challenge in the design of WBAN is that energy efficiency needs to be ensured to increase the network lifetime in such a resourceconstrained network. Another critical challenge for WBAN is that quality of service (QoS) requirements, including packet loss rate (PLR), throughput and delay, should be guaranteed even under the highly dynamic environment due to changing of body postures. In this paper, we design a unified framework of energy efficient resource allocation scheme for WBAN, in which both constraints of QoS metrics and the characteristics of dynamic links are considered. A transmission rate allocation policy (TRAP) is proposed to carefully adjust the transmission rate at each sensor such that more strict PLR requirement could be achieved even when the link quality is very poor. A QoS optimization problem is then formulated to optimize the transmission power and allocated time slots for each sensor, which minimizes energy consumption subject to the QoS constraints. Numerical results demonstrate the effectiveness of the proposed transmission rate allocation policy and the resource allocation scheme.

Wireless body area network (WBAN) is an emerging technology that provides socialized health monitoring service. However, the quality of service can be severely degraded by concomitant inter-WBAN interference in some specific environments where multiple WBANs are densely deployed, e.g., hospitals and senior citizen communities. In this work, we propose a Bayesian game based power control scheme to mitigate the impact of inter-WBAN interference. By modeling WBANs as players and active links as types of players in the Bayesian game model, the proposed power control scheme tries to maximize each player's expected payoff involving both throughput and energy efficiency. We prove the existence of Bayesian equilibrium (BE) for the proposed power control game and also derive a practical sufficient condition for the uniqueness of BE. A harmonic mean based algorithm is then proposed to obtain an approximation of BE point without the need to pass message among WBANs, which satisfies the non-cooperative manner for inter-WBAN interference mitigation. Simulation results show that the proposed algorithm can converge to the BE point effectively.

Capturing a professional like photo is always a challenging task, especially for novice users. This paper proposes a photography suggestion approach to assist users to take high visual quality photos with human in the scene. In this research, we first investigate a set of aesthetic composition rules and visual perception principles to construct an aesthetic score prediction model in order to measure visual quality in terms of photo composition. Then we conduct a study on professional photos to define a proper size for the enclosure of human into the picture of a given scene. The proposed approach is able to leverage saliency and geometric detection to represent composition features. Finally, we utilize an efficient hierarchical search to obtain the optimal enclosure for human in the scene. Extensive experiments have been performed for hot spot landmark locations. Through subjective evaluation, the results show that the proposed approach can effectively provide appealing composition recommendation to help users take high quality photos with human in the scene.

The authors present in this paper a novel surveillance system using a calibrated stereo camera pair. The system adopts a 2D-3D hybrid approach where vehicle detection and tracking are first performed in the 2D space. Then, both appearance and depth cues are incorporated into the tracking module based on a hybrid 2D-3D approach. After change detection is carried out in the original image domain, moving vehicles can be detected and tracked over time. Vehicle positions and speeds can be estimated very accurately by first retrieving the 3D coordinates of pixels associated with the tracked vehicles resulting in a 3D vehicle point cloud and fitting a cuboid to it. The proposed stereo vehicle surveillance system is capable of extracting several important driving parameters, including vehicle trajectories, speeds, and orientations. Experimental results have confirmed the efficiency and robustness of the system and its applicability to road safety applications, including speeding and drunk driving detection.

Long scene panorama (LSP) can provide the immersive experience for an extended view in one single photograph. The state-of-the-art image mosaicing based LSP techniques are significantly constrained by their high computational complexity, limiting their applications to platforms with rich computing resource. Slit-scanning based schemes, although implementation friendly, heavily rely on the consistency of the camera's speed and suffer from pushbroom distortion. In this paper, we propose a novel scheme which can generate the LSP for the indoor environment via consumer-grade cameras. A novel adaptive resampling scheme is introduced to generate more natural looking panoramas from videos captured by cameras with non-constant moving speed. A multiple slits replacement scheme is also developed to reduce the zigzag distortion and to handle the depth change. Experimental results show that the proposed scheme can achieve satisfactory performance with handheld cameras.

Different from traditional HTTP adaptive streaming (HAS) in which only one client is considered, HAS over multi-client wireless networks faces new challenges. The Quality of Experience (QoE) of users becomes unstable due to users' competition for shared bandwidth. It is thus important to accurately estimate the perceived experience of users and then adapt the streaming process accordingly. Furthermore, the QoE fairness among multiple clients subscribing to the same services shall also be addressed. In this research, we propose a QoE continuum driven HAS adaptation algorithm to address these challenges. We model the QoE continuum as an integrated consideration of cumulative playback quality and playback smoothness. Based on this model, we jointly optimize the quality adaptation of multiple users by considering both QoE history and channel status. Moreover, we propose to use quantization parameter and segment size to represent the video files in a fine-grained fashion, in order to more effectively capture the bandwidth fluctuation. The results from extensive simulations show that the proposed scheme can provide balanced and satisfactory QoE among multiple clients.

Media sharing in cloud environment, which supports sharing media content at any time and from anywhere, is a promising paradigm of social interaction. However, it also brings forth security issues in terms of data confidentiality and access control on media data consumers with different access privileges. One promising solution is scalable media access control, which is capable of providing data confidentiality by encrypting the media data and issuing the key to only authorized data consumers. More importantly, it can empower the data distributor to provide the same media content with various quality levels to the consumers with different privileges. Traditional schemes without utilizing the cloud resources achieve scalable media access control by generating access keys using hash chains. Despite of their computational efficiency, such schemes suffer from various problems including vulnerability to user collusion attack in two-dimensional case, inflexible key distribution, and ambiguous key revocation strategy. In this paper, we propose a novel two-dimensional-scalable access control by generating access keys based on Attribute-Based Encryption (ABE) algorithm. Moreover, the proposed scheme can efficiently achieve comprehensive key management including key distribution and key revocation by fully exploiting the cloud. Security analysis shows that the proposed scheme is able to provide collusion resistance, as well as forward and backward secrecy. We have also evaluated the efficiency of the scheme through numerical analysis and initial implementation.

In this paper, we present a novel subjective quality model for online adaptive movie streaming service. The proposed model considers the Quality of Experience (QoE) of streaming video viewing as a cumulative evaluation process of consecutive segments that compose a story line. Under bandwidth constraint, streaming client may select lower-quality segment, pause playback for re-buffering, or both. The momentary QoE loss at these events are weighted by the content suspension level. Meanwhile such experience penalty remains to influence user's opinion for the rest of the service. If the picture becomes too noisy or the interruptions occur consistently, the user may stop watching. The proposed scheme includes two parts. First, a parametric model estimates the quality loss of a interfered segment based on its network-level packet characteristics. Second, a cumulative function integrates the impact of streaming events. Both steps demand minimum computation and can be updated in real time. A subjective test to train and validate the proposed parametric model is designed and performed. This model is fundamentally different from all existing QoE assessment schemes in that temporally cumulative viewing experience of the users, instead of simple global statistics, is evaluated.