Search

For adaptive streaming applications, low-complexity and accurate video complexity features are necessary to analyze the video content in real time, which ensures fast and compression-efficient video streaming without disruptions. State-of-the-art video complexity features are Spatial Information (SI) and Temporal Information (TI) features which do not correlate well with the encoding parameters in adaptive streaming applications. To this light, Video Complexity Analyzer (VCA) was introduced, determining the features based on Discrete Cosine Transform (DCT)-energy. This paper presents optimizations on VCA for faster and energy-efficient video complexity analysis. Experimental results show that VCA v2.0, using eight CPU threads, Single Instruction Multiple Data (SIMD), and low-pass DCT optimization, determines seven complexity features of Ultra High Definition 8-bit videos with better accuracy at a speed of up to 292.68 fps and an energy consumption of 97.06% lower than the reference SITI implementation., Comment: First International ACM Green Multimedia Systems Workshop (GMSys 2023)

In live streaming applications, a fixed set of bitrate-resolution pairs (known as bitrate ladder) is generally used to avoid additional pre-processing run-time to analyze the complexity of every video content and determine the optimized bitrate ladder. Furthermore, live encoders use the fastest available preset for encoding to ensure the minimum possible latency in streaming. For live encoders, it is expected that the encoding speed is equal to the video framerate. An optimized encoding preset may result in (i) increased Quality of Experience (QoE) and (ii) improved CPU utilization while encoding. In this light, this paper introduces a Content-Adaptive encoder Preset prediction Scheme (CAPS) for adaptive live video streaming applications. In this scheme, the encoder preset is determined using Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for every video segment, the number of CPU threads allocated for each encoding instance, and the target encoding speed. Experimental results show that CAPS yields an overall quality improvement of 0.83 dB PSNR and 3.81 VMAF with the same bitrate, compared to the fastest preset encoding of the HTTP Live Streaming (HLS) bitrate ladder using x265 HEVC open-source encoder. This is achieved by maintaining the desired encoding speed and reducing CPU idle time., Comment: Accepted in PCS2022

The performance of applications that require frame rendering time estimation or dynamic frequency scaling, rely on the accuracy of the workload model that is utilized within these applications. Existing models lack sufficient accuracy in their core model. Hence, they require changes to the target application or the hardware to produce accurate results. This paper introduces a mathematical workload model for a rasterization-based graphics Application Programming Interface (API) pipeline, named GAMORRA, which works based on the load and complexity of each stage of the pipeline. Firstly, GAMORRA models each stage of the pipeline based on their operation complexity and the input data size. Then, the calculated workloads of the stages are fed to a Multiple Linear Regression (MLR) model as explanatory variables. A hybrid offline/online training scheme is proposed as well to train the model. A suite of benchmarks is also designed to tune the model parameters based on the performance of the target system. The experiments were performed on Direct3D 11 and on two different rendering platforms comparing GAMORRA to an AutoRegressive (AR) model, a Frame Complexity Model (FCM) and a frequency-based (FRQ) model. The experiments show an average of 1.27 ms frame rendering time estimation error (9.45%) compared to an average of 1.87 ms error (13.23%) for FCM which is the best method among the three chosen methods. However, this comes at the cost of 0.54 ms (4.58%) increase in time complexity compared to FCM. Furthermore, GAMMORA improves frametime underestimations by 1.1% compared to FCM.

Millimeter-scale implants using ultrasound for power and communication have been proposed for a range of deep-tissue applications, including neural recording and stimulation. However, published implementations have shown high sensitivity to misalignment with the external ultrasound transducer. Ultrasonic beamforming using a phased array to these implants can improve tolerance to misalignment, reduce implant volume, and allow multiple implants to be operated simultaneously in different locations. This paper details the design of a custom planar phased array ultrasound system, which is capable of steering and focusing ultrasound power within a 3D volume. Analysis and simulation is performed to determine the choice of array element pitch, with special attention given to maximizing the power available at the implant while meeting FDA limits for diagnostic ultrasound. Time reversal is proposed as a computationally simple approach to beamforming that is robust despite scattering and inhomogeneity of the acoustic medium. This technique is demonstrated both in active drive and pulse-echo modes, and it is experimentally compared with other beamforming techniques by measuring energy transfer efficiency. Simultaneous power delivery to multiple implants is also demonstrated., Comment: 10 pages, 13 figures, to be published in the IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control. Replacement made after additions/edits suggested during review

High quality (HQ) video services occupy large portions of the total bandwidth and are among the main causes of congestion at network bottlenecks. Since video is resilient to data loss, throwing away less important video packets can ease network congestion with minimal damage to video quality and free up bandwidth for other data flows. Frame type is one of the features that can be used to determine the importance of video packets, but this information is stored in the packet payload. Due to limited processing power of devices in high throughput/speed networks, data encryption and user credibility issues, it is costly for the network to find the frame type of each packet. Therefore, a fast and reliable standalone method to recognize video packet types at network level is desired. This paper proposes a method to model the structure of live video streams in a network node which results in determining the frame type of each packet. It enables the network nodes to mark and if need be to discard less important video packets ahead of congestion, and therefore preserve video quality and free up bandwidth for more important packet types. The method does not need to read the IP layer payload and uses only the packet header data for decisions. Experimental results indicate while dropping packets under packet type prediction degrades video quality with respect to its true type by 0.5-3 dB, it has 7-20 dB improvement over when packets are dropped randomly., Comment: 9, 2 Figs, Journal

Although subjective tests are most accurate image/video quality assessment tools, they are extremely time demanding. In the past two decades, a variety of objective tools, such as SSIM, IW-SSIM, SPSIM, FSIM, etc., have been devised, that well correlate with the subjective tests results. However, the main problem with these methods is that, they do not discriminate the measured quality well enough, especially at high quality range. In this article we show how the accuracy/precision of these Image Quality Assessment (IQA) meters can be increased by mapping them into a Logistic Function (LF). The precisions are tested over a variety of image/video databases. Our experimental tests indicate while the used high-quality images can be discriminated by 23% resolution on the MOS subjective scores, discrimination resolution by the widely used IQAs are only 2%, but their mapped IQAs to Logistic Function at this quality range can be improved to 9.4%. Moreover, their precision at low to mid quality range can also be improved. At this quality range, while the discrimination resolution of MOS of the tested images is 23.2%, those of raw IQAs is nearly 8.9%, but their adapted logistic functions can lead to 17.7%, very close to that of MOS. Moreover, with the used image databases the Pearson correlation of MOS with the logistic function can be improved by 2%-20.2% as well., Comment: 14 pages, single line, Journal paper

High-Efficiency Video Coding (HEVC) surpasses its predecessors in encoding efficiency by introducing new coding tools at the cost of an increased encoding time-complexity. The Coding Tree Unit (CTU) is the main building block used in HEVC. In the HEVC standard, frames are divided into CTUs with the predetermined size of up to 64x64 pixels. Each CTU is then divided recursively into a number of equally sized square areas, known as Coding Units (CUs). Although this diversity of frame partitioning increases encoding efficiency, it also causes an increase in the time complexity due to the increased number of ways to find the optimal partitioning. To address this complexity, numerous algorithms have been proposed to eliminate unnecessary searches during partitioning CTUs by exploiting the correlation in the video. In this paper, existing CTU depth decision algorithms for HEVC are surveyed. These algorithms are categorized into two groups, namely statistics and machine learning approaches. Statistics approaches are further subdivided into neighboring and inherent approaches. Neighboring approaches exploit the similarity between adjacent CTUs to limit the depth range of the current CTU, while inherent approaches use only the available information within the current CTU. Machine learning approaches try to extract and exploit similarities implicitly. Traditional methods like support vector machines or random forests use manually selected features, while recently proposed deep learning methods extract features during training. Finally, this paper discusses extending these methods to more recent video coding formats such as Versatile Video Coding (VVC) and AOMedia Video 1(AV1)., Comment: 27 pages, 12 figures, 5 tables

Ultrasonic backscatter communication has gained popularity in recent years with the advent of deep-tissue sub-mm scale biosensing implants in which piezoceramic (piezo) resonators are used as acoustic antennas. Miniaturization is a key design goal for such implants to reduce tissue displacement and enable minimally invasive implantation techniques. Here, we provide a systematic design approach for the implant piezo geometry and operation frequency to minimize the overall volume of the implant. Moreover, a critical design aspect of an ultrasonic backscatter communication link is the response of the piezo acoustic reflection coefficient $\Gamma$ with respect to the variable shunt impedance, $Z_E$, of the implant uplink modulator. Due to the complexity of the piezo governing equations and multi-domain, electro-acoustical nature of the piezo, $\Gamma(Z_E)$ has often been characterized numerically and the implant uplink modulator has been designed empirically resulting in sub-optimal performance in terms of data rate and linearity. Here, we present a SPICE friendly end-to-end equivalent circuit model of the channel as a piezo-IC co-simulation tool that incorporates inherent path losses present in a typical ultrasonic backscatter channel. The circuit model is then used to simulate the channel transient response in a common CAD tool. To provide further insight into the channel response, we present experimentally validated closed form expressions for $\Gamma(Z_E)$ under various boundary conditions. These expressions couple $\Gamma$ to the commonly used Thevenin equivalent circuit model of the piezo, facilitating systematic design and synthesis of ultrasonic backscatter uplink modulators

A 0.8 mm$^3$ wireless, ultrasonically powered, free-floating neural recording implant is presented. The device is comprised only of a 0.25 mm$^2$ recording IC and a single piezoceramic resonator that is used for both power harvesting and data transmission. Uplink data transmission is performed by analog amplitude modulation of the ultrasound echo. Using a 1.78 MHz main carrier, >35 kbps/mote equivalent uplink data rate is achieved. A technique to linearize the echo amplitude modulation is introduced, resulting in <1.2\% static nonlinearity of the received signal over a $\pm$10 mV input range. The IC dissipates 37.7 $\mu$W, while the neural recording front-end consumes 4 $\mu$W and achieves a noise floor of 5.3 $\mu$V$_{rms}$ in a 5 kHz bandwidth. This work improves sub-mm recording mote depth by >2.5x, resulting in the highest measured depth/volume ratio by $\sim$3x. Orthogonal subcarrier modulation enables simultaneous operation of multiple implants, using a single-element ultrasound external transducer. Dual-mote simultaneous power up and data transmission is demonstrated at a rate of 7 kS/s at the depth of 50 mm., Comment: 11 pages, 22 figures, Submitted to Journal of Solid-State Circuits

Light field imaging is characterized by capturing brightness, color, and directional information of light rays in a scene. This leads to image representations with huge amount of data that require efficient coding schemes. In this paper, lenslet images are rendered into sub-aperture images. These images are organized as a pseudo-sequence input for the HEVC video codec. To better exploit redundancy among the neighboring sub-aperture images and consequently decrease the distances between a sub-aperture image and its references used for prediction, sub-aperture images are divided into four smaller groups that are scanned in a serpentine order. The most central sub-aperture image, which has the highest similarity to all the other images, is used as the initial reference image for each of the four regions. Furthermore, a structure is defined that selects spatially adjacent sub-aperture images as prediction references with the highest similarity to the current image. In this way, encoding efficiency increases, and furthermore it leads to a higher similarity among the co-located Coding Three Units (CTUs). The similarities among the co-located CTUs are exploited to predict Coding Unit depths.Moreover, independent encoding of each group division enables parallel processing, that along with the proposed coding unit depth prediction decrease the encoding execution time by almost 80% on average. Simulation results show that Rate-Distortion performance of the proposed method has higher compression gain than the other state-of-the-art lenslet compression methods with lower computational complexity.