Search

In order to provide improved multimedia services to the end users, developing objective models efficiently predicting 3 Dimensional (3D) video Quality of Experience (QoE) can currently be considered as one of the most significant research areas. Nevertheless, there is currently no model standardized and widely utilized by the researchers due to its efficient and reliable assessment of the 3D video quality. Therefore, highly exploited 2 Dimensional (2D) video quality assessment models such as Structural SIMilarity Index (SSIM) are preferred for the 3D video quality evaluation. However, providing efficiency and reliability for the 3D video quality assessment using the 2D video quality assessment models can only be ensured if they include 3D video related features effecting Human Visual System (HVS). Under the light of these information, the SSIM model is improved for the 3D video quality assessment using perceptually significant feature, contrast and motion characteristics having impact on the HVS in this study. The results obtained by utilizing the improved SSIM model clearly present that the model is quite competent to provide enhanced multimedia services to the end users.

In this paper, we present a Unet architecture made of octave convolution for dental image segmentation problem. In this architecture, the requirements for memory and accuracy are significantly improved compared to previous works in the literature. Compare to state-of-art models on this topic the classification accuracy in dental image segmentation is increased by %2, and the memory usage is decreased by %70. Suggested architecture showed a performance of success on ISBI2015 dataset.

Hyperspectral data provides rich information about a scene in terms of spectral details since it encapsulates measurements/observations from a wide large range of spectrum. To this end, it has been used in different problems mostly related to identification and detection processes. However, the main limitation arises for the accessibility of data. More precisely, there is no sufficient amount of hyperspectral data available compared to visible range data for trainable models. In this paper, we tackle an inverse problem to estimate the relative lidar depth from hyperspectral data. To solve its limitation, we integrate semantic information existed in data with supervised labels to decrease the possibility of parameter overfitting. Moreover, details of the output responses are enhanced with Laplacian pyramids and attention layers in which the model makes predictions from each subsequent scale instead of a single shot prediction from the top of the model. In our experiments, we use the 2018 IEEE GRSS Data Fusion Challenge dataset. From the experimental results, we prove that use of hyperspectral data instead of visible range data improves the performance. Moreover, we show that results are significantly improved if a sparse set of depth measurements is used along with hyperspectral data. Lastly, the integration of semantic information to the solution yields more stable and better results compared to the baselines.

In this study, the effect of visual context information to the performance of learning-based techniques for the super resolution problem is analyzed. Beside the interpretation of the experimental results in detail, its theoretical reasoning is also achieved in the paper. For the experiments, two different visual datasets composed of natural and remote sensing scenes are utilized. From the experimental results, we observe that keeping visual context information in the course of parameter learning for convolutional neural networks yields better performance compared to the baselines. Moreover, we summarize that finetuning pre-trained parameters with the related context yet fewer samples improves the results.

Hyperspectral sensors have high spectral resolution by capturing images in hundreds of bands. Despite the high spectral resolution, low spatial resolution of these sensors restricts the performance of the hyperspectral imaging applications such as target tracking and image classification. Fusing the hyper-spectral image (HSI) with higher spatial resolution RGB or multispectral image (MSI) data is a commonly used method in the resolution enhancement of the HSIs. In this paper, we propose a new fusion technique for the HSI super-resolution. The main contribution of this study is formulating the fusion problem in a quadratic manner and also regularizing the solution quadratically using smoothness prior. Moreover, another contribution of the proposed method is converting the fusion problem from spectral domain to the abundance map domain which gives more robust and spectrally consistent results. In the proposed method, first, abundance maps are obtained using linear spectral unmixing and then a quadratic energy function is obtained using these maps and high resolution (HR) RGB image. In addition, quadratic function is regularized using additional constraints. Solving the regularized quadratic function gives the HR abundance maps and these maps are used to reconstruct HR HSI. Experiments show that proposed method yields better performance as compared to state of the art methods in different performance metrics.

There are numerous criteria which are being used to measure camera performance and for determining such criteria, different tests are applied in different test environments. Within this framework, color accuracy testing at camera performance is one of foremost of such tests. In the scope of this paper, a method has been proposed to reduce user interaction in the color accuracy tests in the literature. At the same time, with the color constancy concept, it has been shown that color variation between the different test setups should also be considered as an important criterion on the camera performance.

Inpainting applications include object removal on images and videos, crack filling, error concealment, texture synthesis, where in this paper, its usage for image coherence and perspective emphasis on video frames in 2D image-to-video conversion system is analysed. Besides, the performance of different techniques in object removal and image reconstruction is compared using visual experiments and quality metrics.

Frame-level visual features are generally aggregated in time with the techniques such as LSTM, Fisher Vectors, NetVLAD etc. to produce a robust video-level representation. We here introduce a learnable aggregation technique whose primary objective is to retain short-time temporal structure between frame-level features and their spatial interdependencies in the representation. Also, it can be easily adapted to the cases where there have very scarce training samples. We evaluate the method on a real-fake expression prediction dataset to demonstrate its superiority. Our method obtains 65% score on the test dataset in the official MAP evaluation and there is only one misclassified decision with the best reported result in the Chalearn Challenge (i.e. 66:7%) . Lastly, we believe that this method can be extended to different problems such as action/event recognition in future., Comment: Submitted to International Conference on Computer Vision Workshops

To perform an accurate vegetation classification in hyperspectral data, feature extraction process prior to classification is very important. Success rates of classifiers in vegetation are rather limited compared to classification of other types of materials. Therefore, it is required to perform an effective feature extraction before classification. Principle Component Analysis(PCA) is a common and easily applicable method for this purpose. However, PCA is not an optimal method for distinguishing between different plant species. In this study, the reasons for PCA not being an adequate method for this purpose are discussed and alternative useful feature extraction methods in classification of plant species are examined. Tests were performed for Spectrally Segmented PCA(SSPCA), Discrete Wavelet Transform(DWT) and Genetic Algorithm(GA) feature extraction methods, their effects on classifier performances were compared and it was observed that all of the mentioned alternatives had noticable improvements in classification performances.

In this paper, an application of short video synthesis from single frame images is realized and a comparative analysis of different methods on image inpainting, which is a computationally costly part of the whole procedure, is provided. Our work is fortified with experiments in order to measure the computational performances and efficiencies of the proposed method and the ones existing in literature.

Super-resolution Reconstruction (SRR) is technique to increase the spatial resolution of images. It is especially useful for hyperspectral images (HSI), which have good spectral resolution but low spatial resolution. In this study, we propose an improvement to our previous work and present a novel MAP-MRF (maximum a posteriori-Markov random Fields) based approach for the SRR of HSI. The key point of our approach is to find the abundance maps of an HSI and perform SRR on the abundance maps using MRF based energy minimization, without needing any other additional source of information. In order to do so, first, PCA is used to determine the endmembers. Second, SISAL and fully constraint least squares (FCLS) are used to estimate the abundance maps. Third, in order to find the high resolution abundance maps, the ill-posed inverse SRR problem for abundances is regularized with a MAP-MRF based approach. The MAP-MRF formulation is restricted with the constraints which are specific to the abundances. Using the non-linear programming (NLP) techniques, the convex MAP formulation is minimized and High Resolution (HR) abundance maps are obtained. Then, these maps are used to construct the HR HSI. This improved SRR method is verified on real data sets, and quantitative performance comparison is achieved using PSNR, SSIM and PSNR metrics. Our results indicate that this improved method gives very close results to the original high resolution images, keeps the spectral consistency, and performs better than the compared algorithms.

This paper describes a novel pre-screener algorithm for landmine detection with a ground penetrating radar (GPR). The pre-screener algorithms are used for finding anomalies that are potential locations of interest. Thus, their processing time is as important as their true detection rate and false alarm rate. The proposed approach is based on Kernel Least Mean Square algorithm. Although Least Mean Square (LMS) based approach has already been used in the literature, KLMS based approach is a novel application for landmine detection with GPR. In this study, KLMS approach is compared with LMS approach in terms of processing time, false alarm rate, and true detection rate.

In this study, we propose a method for panoramic background subtraction by using Pan-Tilt cameras in real-time. The proposed method is based on parallelization of image registration, panorama generation and background subtraction operations to run on Graphics Processing Unit (GPU). Experiments results showed that GPU usage increases speed of the algorithm 33 times without considerable performance loss and makes working real-time possible.

The Sign Language Recognition (SLR) Problem is a highly important research topic, because of its ability to increase the interaction between the people who are hearing-impaired or impediment in speech. We propose a simple but robust system. The proposed system consists of three main steps. First we apply segmentation to the face and hand region by using Fuzzy C-Means Clustering (FCM) and Thresholding. FCM is a clustering technique which employs fuzzy partitioning, in an iterative algorithm. After the face and hands are segmented, the feature vectors are extracted. The feature vectors are chosen among the low level features such as the bounding ellipse, bounding box, and center of mass coordinates, since they are known to be more robust to segmentation errors due to low resolution images. In total there are 23 features for each hand. After the feature vectors are extracted, they are used for recognition with discrete Hidden Markov Model (HMM). Recognition stage is composed of two stages, namely training and classification. The Baum Welch algorithm is used for HMM training. In classification part the likelihood of each HMM is calculated and the HMM with the highest likelihood is chosen. In order to measure the success rate of the system, the eNTERFACE dataset is used. In this dataset 8 different American Sign Language example classified and in user independent case, is shown to be working with 94.19% accuracy.

Despite their high spectral resolution, hyperspectral images have low spatial resolution which adversely affects the applications that use hyperspectral images. In this study, instead of the traditional way of using spectral images, abundances of the endmembers are used in resolution enhancement. In the proposed method, first, endmembers are extracted with the SISAL algorithm. Then, the abundance maps are estimated using FCLS. From the low resolution abundance maps, high resolution abundance maps are obtained with a total variation based minimization. Finally, high resolution hyperspectral images are constructed from high resolution abundance maps. The proposed method is tested on real hyperspectral images. The experimental results and comparative analysis show the effectiveness of the proposed method.

Aggressive driving behavior is among the important causes of traffic accidents. Hence, detection of driver aggressiveness has an importance in terms of decreasing the number of traffic accidents. Collected driving data while the vehicle is in traffic can be used to make inferences about the aggressiveness of the driver. In this study, a multimodal method is proposed in order to detect driver aggressiveness. The proposed method is based on utilizing the visual data taken from the on vehicle camera and sensor data taken from the controller area network bus (CAN-bus) in order to decide whether the driving session involves aggressive driving behavior. Lane following pattern and vehicle following distance information is obtained from the data collected by camera while vehicle speed and engine speed information is obtained from CAN-bus. These information is fused to conceive feature vectors that represent the driving session and aggressiveness decision is made according to the classification of these feature vectors.

8th International Conference on Quality of Multimedia Experience (QoMEX) -- JUN 06-08, 2016 -- Lisbon, PORTUGAL WOS: 000391251500046 In this study a novel, spatio-temporal characteristics based video content analysis method is presented. The proposed method has been evaluated on different video quality assessment databases, which include videos with different characteristics and distortion types. Test results obtained on different databases demonstrate the robustness and accuracy of the proposed content analysis method. Moreover, this analysis method is employed in order to examine the performance improvement in audiovisual quality assessment when the video content is taken into consideration. IEEE, IEEE Signal Proc Soc, NOS, YouTube, EURASIP, Qualinet, QoENet, Inst telecomunicacoes, Tecnico Lisboa

The Harmonized Tariff Schedule for the classification of goods is a major determinant of customs duties and taxes. The basic HS Code is 6 digits long but can be extended according to the needs of the countries such as application of custom duties based on details of the product. Finding the correct, consistent, legally defensible HS Code is at the heart of Import Compliance. However finding the best code can be complicated, especially in the case of specialized products. In this paper, we propose an automatic HS code detection system based on visual properties of the product together with textual analysis of its labels/explanations. The proposed system first uses morphological parsing in order to extract roots of the words occurring in the textual phrases. Processed text information is further processed by the topic modeling module of the system to find the best matching HS Code definitions within the system. The result of the topic modeling is used to trigger visual search based on quantized local features. The proposed algorithm is evaluated using a database of 4494 Binding Tariffs published in 2014 by the European Union. The results show that accuracy rate above 80 % can be achieved for 4-digit HS Codes.

Among the human related factors, aggressive driving behavior is one of the major causes of traffic accidents [17]. On the other hand, detection and characterization of driver aggressiveness is a challenging task since there exist different psychological causes behind it. However, information about the driver behavior could be extracted from the data that is collected via different sensing devices. This paper presents a method to detect driver aggressiveness using only visual information provided by forward camera. The proposed method is based on detection of the road lines and the vehicles on the road and extracts information related with road lane departure rate, speed of the vehicle and possible forward collision time. Using these extracted features, a classifier is utilized in order to detect if driver shows an aggressive driving behavior. The proposed method is tested by a subjective testing method using 76 different driving sessions and achieved 90.4% success.

Hyperspectral imaging is widely used in many fields such as geology, medicine, meteorology, and so on. Despite the high spectral resolution, the spatial resolution of the hyperspectral sensors is severely limited. In this paper, we propose a novel maximum a posteriori (MAP)-based approach based on the joint superresolution of the abundance maps, to enhance the resolution of hyperspectral images. In the proposed approach, first, the endmembers and their abundance maps are estimated using Vertex Component Analysis (VCA) and Fully Constrained Least Squares (FCLS), respectively. Second, a high resolution (HR) abundance map is reconstructed for each low resolution (LR) abundance map using a MAP-based approach. In the MAP-formulation data, smoothness and edge preservation constraints are extended to include a unity constraint term specific to abundances. Finally, HR hyperspectral images are reconstructed using the HR abundance maps. The proposed algorithm is tested on both synthetic images and real image sequences. The experimental results and comparative analysis verify the effectiveness of the proposed algorithm.

In this paper, scene recognition problem is investigated in detail by exploiting scene representation, dimension reduction and classification stages. Unlike the other studies, the proposed algorithm has preferred to model the overall structure of the scene instead of an object-based proposal. For that purpose, some of the visual representations like MPEG-7, Gist, BoW, Vlad and Fisher, are classified singly or jointly with Support Vector Machine and Random Forest. The evaluation tests are conducted on MIT indoor dataset [2] and from the results, %31 average precision has been attained by combining Scalable Color, Homogeneous Texture and Vlad with Support Vector Machine.

Most of the traffic accidents are caused by human related factors. One of the most important human related factor in terms of traffic accident risk is aggressive driving behavior. Although driver aggressiveness is related with psychological reasons, it can be detected by observation of driving behavior using different sensing devices. This work presents a driver aggressiveness detection method exploiting the visual data obtained by on vehicle camera. The proposed method uses this visual data in order to extract features such as lane departure rate and possible collision time by detection of road lines and vehicles on the road. These features are used to train a classifier which decides on whether an aggressive behavior is performed in related driving session. According to conducted tests the proposed method is observed to achieve 90% correct detection rate of aggressive driving behavior.

23nd Signal Processing and Communications Applications Conference (SIU) -- MAY 16-19, 2015 -- Inonu Univ, Malatya, TURKEY NUR YILMAZ, Gokce/0000-0002-0015-9519; B. Akar, Gozde/0000-0002-4227-5606 WOS: 000380500900222 In this study, a novel, content aware audiovisual quality assessment (AVQA) method using a spatio-temporal characteristics based video classification method has been proposed and evaluated on AVQA database created by University of Plymouth. The proposed AVQA method is evaluated using subjective audio mean opinion score (MOS) and subjective video MOS. Results indicate that both classification method and the proposed content dependent AVQA method are quite satisfactory Dept Comp Engn & Elect & Elect Engn, Elect & Elect Engn, Bilkent Univ

IEEE International Conference on Image Processing (ICIP) -- OCT 27-30, 2014 -- Paris, FRANCE B. Akar, Gozde/0000-0002-4227-5606; NUR YILMAZ, Gokce/0000-0002-0015-9519 WOS: 000370063600121 The increasing demand for streaming video raises the need for flexible and easily implemented Video Quality Assessment (VQA) metrics. Although there are different VQA metrics, most of these are either Full-Reference (FR) or Reduced-Reference (RR). Both FR and RR metrics bring challenges for on-the-fly multimedia systems due to the necessity of additional network traffic for reference data. No-eference (NR) video metrics, on the other hand, as the name suggests, are much more flexible for user-end applications. This introduces a need for robust and efficient NR VQA metrics. In this paper, an NR VQA metric considering spatiotemporal information, bit rate, and packet loss rate characteristics of a video content is proposed. The proposed metric is evaluated on EPFL-PoliMI dataset, which includes different video content characteristics. The experimental results show that the proposed metric is a robust and accurate NR VQA metric towards diverse video content characteristics. IEEE

The recent improvements on internet technologies and video coding techniques cause an increase in copyright infringements especially for video. Frequently, image-based approaches appear as an essential solution due to the fact that joint usage of quantization-based indexing and weak geometric consistency stages give a capability to compare duplicate videos quickly. However, exploiting purely spatial content ignores the temporal variation of video. In this work, we propose a system that combines the state-of-the-art quantization-based indexing scheme with a novel trajectory-based geometric consistency on spatio-temporal features. This combination improves duplicate video matching task significantly. Briefly, spatial mean and variance of the trajectories are incorporated to establish a weak geometric consistency among pair of frames. To show the success of the proposed method, content-based video copy detection field is selected and TRECVID 2009 dataset is utilized. The experimental results show that constituting trajectory-based consistency on corresponding feature pairs outperforms the performances of merely utilizing spatiotemporal signature and visual signature with enhanced weak geometric consistency.

Need for automatic video copy detection is increased with the recent technical developments in the internet technologies and video recording. Even though image-based techniques with bag-of-word kind of representations are accepted as the best solution because of robustness and speed; they discard the convenient geometric relation which exists among interest points. In this work, we propose a novel geometric relation which computes a binary signature leveraging existence and non-existence of interest points in the neighborhood area. The experimental results on TRECVID 2009 content-based video copy detection dataset show that combination of our method with recently proposed quantization-based indexing and weak geometric consistency schemes outperforms classical representations.

Technological developments and increases of the copyright infringements have led a demand on developing automatic video copy detection systems. Generally, for this task, even if visual local descriptors are preferred, descriptors should be indexed according to make an effective search. In this work, performance of three indexing methods which are defined in literature is compared for video copy detection task. Additionally estimating geometric relation between video frames has positive effect on performance is shown. Evaluation of proposed methods are tested on TRECVID 2009 content based video copy detection dataset and from obtained results, product quantization method yields much more accurate result with respect to classical bag-of-word method is observed.

Remote sensing is used in a spreading manner by many governmental and industrial institutions worldwide in recent years. Target detection has an important place among the applications developed using satellite imagery. In this paper, an original circular target detection algorithm has been proposed based on a radial transformation. The algorithm consists of three stages such as pre-processing, target detection, and postprocessing. In the pre-processing stage, bilateral noise reduction filtering and vegetation detection operations are completed which they are required by target detection step. The target detection stage finds the circular target by a radial transformation algorithm and variables obtained from the training, and postprocessing stage carries out the elimination of falsely detected targets by utilizing the vegetation information. The Petroleum Oil Lubricants (POL) depots in the industrial areas and harbors have been chosen as an application area of the proposed algorithm. The algorithm has been trained and tested on a data set which includes 4-band images with Near-Infrared band. Proposed algorithm is able to detect many circular targets with different types and sizes as a consequence of using a full radial transformation search as well as it gives rewarding results on industrial areas and harbors in the experiments conducted.

22nd IEEE Signal Processing and Communications Applications Conference (SIU) -- APR 23-25, 2014 -- Karadeniz Teknik Univ, Trabzon, TURKEY B. Akar, Gozde/0000-0002-4227-5606 WOS: 000356351400423 In the last two decades the Internet technology has boosted and the connection speeds have been incrased from kilobits to hundred megabits scale. With the rising coverage of the Internet and the usage of mobile devices such as tablets and smart phones, the usage of social media and especially multimedia elements has been increased rapidly. This increment in streaming multimedia created a need for the assessment of the user experience on multimedia and especially video. Even though there are different Video Quality Assessment (VQA) methods for that purpose, most of them are Full-Reference (FR) or Reduced-Reference (RR). In today's world with many mobile devices, the application of these methods are not possible since they need the reference data. The No-Reference (NR) video metrics are much more suitable for the case. In this paper, the main objective is to evaluate a previously proposed NR VQA metric with a new dataset and to compare the results to other high-performance NR metrics such as G.1070 and G.1070E which do not utilize spatial and temporal characteristics of a given video sequence. Evaluation and comparison results show the accuracy and robustness of the proposed metric. IEEE, Karadeniz Tech Univ, Dept Comp Engn & Elect & Elect Engn

20th IEEE International Conference on Image Processing (ICIP) -- SEP 15-18, 2013 -- Melbourne, AUSTRALIA B. Akar, Gozde/0000-0002-4227-5606; NUR YILMAZ, Gokce/0000-0002-0015-9519 WOS: 000351597600012 Many researchers have been developing objective video quality assessment methods due to increasing demand for perceived video quality measurement results by end users to speed-up advancements of multimedia services. However, most of these methods are either Full-Reference (FR) metrics, which require the original video or Reduced-Reference (RR) metrics, which need some features extracted from the original video. No-Reference (NR) metrics, on the other hand, do not require any information about the original video; hence, are much more suitable for applications like video streaming. This paper presents a novel, objective, NR video quality assessment algorithm. The proposed algorithm is based on utilization of spatial extent of video, temporal extent of video using motion vectors, bit rate, and packet loss ratio. Test results obtained using LIVE video quality database demonstrate the accuracy and robustness of the proposed metric. Inst Elect & Elect Engineers, IEEE Signal Proc Soc

In this study, 3D video streaming is taken from a different perspective: how 3D streaming will adapt if both the streamer and the receiver are mobile? The full system chain is completed on mobile devices. Video is captured from a stereo camera pair, compressed by using H.264/AVC compression standard and streamed on TCP on a low power embedded system with OMAP3530 SoC ARM processor. A mobile device with autostereoscopic display is used on the receiver side. Being based on mobile devices, this system permits the possibility of 3D video communication and 3D streaming between mobile recipients on wireless network. For the system competence, the performance is tested and the relevant results are presented.

In this paper a novel fast circle detection algorithm is proposed which depends on the spatial properties of the connected components on the image. Two 1-D transforms of each connected component is obtained by taking the Radon Transform of the image for two different directions, which are in fact the integrations of the image through horizontal and vertical directions. Circles are detected using the similarities of detected peaks on the transformed functions and the characteristics of the values in between those peaks. The success of the method is analyzed using synthetic images and the performance of the method is presented and compared with Modified Hough Transform (MHT) using synthetic images.

In recent years intelligent transportation systems has been an active research area in computer vision. This work aims to classify vehicles as car, van or truck by using videos which are taken from an uncalibrated camera which sees any highway from above. In this work, moving blobs are detected by a background-foreground extraction algorithm, classification is done by extracting the blob features and histogram of oriented gradients features of detected blobs and by giving those features to support vector machine classifier. Suggested method has been tried on 2 different traffic video and results have been presented.

This paper proposes a new maximum a posteriori (MAP) based super-resolution (SR) image reconstruction method targeting edges and textures in images. Unlike conventional MAP based SR image reconstruction methods a spatially varying image prior is employed which is updated according to the frequency content of the reconstructed image at each iteration at different locations. Two alternative methods based on discrete cosine transforms (DCT) and Gabor filters are proposed for determining the image prior. The proposed method is validated through simulations and real experiments which clearly demonstrates significant visual improvements especially on edges and textures compared to state-of-the-art SR methods.

This study presents the stereo video capture and compression on embedded platforms in real time and streaming of the video to mobile devices over a wireless network, as an end-to-end system. The major components of the system are the state-of-the art compression standard H.264/AVC used with side-by-side representation of stereo video on the OMAP3530 SoC (SystemonChip) ARM processor and the mobile device with autostereoscopic display at the receiver side. With the presented architecture, 3D video communication between the mobile users connected to a wireless network is made possible. Furthermore, performance tests are conducted for the system and results are provided in this study.

Breast cancer detection with infrared imaging is a technique that is hazardous radiation free, increases the early detection chance and has a high detection probability. Asymmetry analysis is used for the diagnosis and the analysis is done using the breast segments obtained from the infrared image. In this work, a breast segmentation system based on parabolic Hough transform is presented. Besides, a filtering method using connected components is proposed to increase both the computation speed and the segmentation performance.

19th IEEE International Conference on Image Processing (ICIP) -- SEP 30-OCT 03, 2012 -- Lake Buena Vista, FL NUR YILMAZ, Gokce/0000-0002-0015-9519; B. Akar, Gozde/0000-0002-4227-5606 WOS: 000319334900152 In order to speed up the wide-spread proliferation of the 3D video technologies (e.g., coding, transmission, display, etc), the effect of these technologies on 3D perception should be efficiently and reliably investigated. Using Full-Reference (FR) objective metrics for this investigation is not practical especially for "on the fly" 3D perception evaluation. Thus, a Reduced Reference (RR) metric is proposed to predict the depth perception of 3D video in this paper. The color-plus-depth 3D video representation is exploited for the proposed metric. Since the significant depth levels of the depth map sequences have great influence on the depth perception of users, they are considered as side information in the proposed RR metric. To determine the significant depth levels, the depth map sequences are abstracted using bilateral filter. Video Quality Metric (VQM) is utilized to predict the depth perception ensured by the significant depth levels due to its well correlation with the Human Visual System (HVS). The performance assessment results present that the proposed RR metric can be utilized in place of a FR metric to reliably measure the depth perception of 3D video with a low overhead. Inst Elect & Elect Engineers (IEEE), IEEE Signal Proc Soc

In this paper, we present a complete framework of an end-to-end error resilient transmission of 3D video over DVB-H and provide an analysis of transmission parameters. We perform the analysis for various layering, protection strategy and prediction structure using different contents and different channel conditions.

This paper presents a new quality evaluation model for stereoscopic videos using depth map segmentation. This study includes both objective and subjective evaluation. The goal of this study is to understand the effect of different depth levels on the overall 3D quality. Test sequences with different coding schemes are used. The results show that overall quality has a strong correlation with the quality of the background, where disparity is smaller relative to the foreground. The results also showed that content type is an important factor in determining the visual quality. While depth segmentation gives information related to 3D perception, additional work is required to develop an objective metric.

In this study, we propose a heuristic methodology for modeling the end-to-end distortion characteristics of an error resilient broadcast system for 3D video over Digital Video Broadcasting - Handheld (DVB-H).We also use this model to optimally select the parameters of the video encoder and the error correction scheme, namely, Multi Protocol Encapsulation - Forward Error Correction (MPE-FEC), minimizing the overall distortion. The proposed method models the RQ curve of video encoder and performance of channel codec to jointly derive the optimal encoder bit rates and unequal error protection (UEP) rates specific to the 3D video broadcast. Moreover, the distortion on the 3D video quality caused by packet losses and the loss rate of the channel is estimated. Finally, with the use of analytical models and estimated single packet loss distortions, end-to-end distortions are minimized and optimal encoder bit rates and UEP rates are obtained.

This paper proposes a new super-resolution (SR) image reconstruction method targeting edges and textured regions in thermal images. Proposed two stage method runs two Bayesian SR estimators at the first stage. These estimators are; maximum likelihood method and maximum a-posteriori method with Tikhonov type regularization. The piksel-to-piksel difference image of these two estimates is an high frequency (HF) image including observation noise, process noise, edges and textures (that are smoothed out by regularizers). In the second stage of the proposed method, the difference image is post-processed to extract edge and texture information while eliminating noise. The proposed method uses a Gabor filter family to analyze this difference image at various frequencies and directions. The strong frequency components are restored and added to the MAP estimate to obtain the final image. The proposed methods are validated through simulations on several textured surfaces from Brodatz data base. Peak-signal-to-noise ratio (PSNR) measures and illustrations clearly shows the success of the proposed method. The Real experiments on uncooled thermal cameras are also conducted to compare the methods to classical SR methods known to literature.

In this paper, a new color super-resolution (SR) image reconstruction method for HSV domain is presented. Applying classical SR algorithms to each channel of the HSV video causes an artifact due to the cyclic nature of the hue channel. Proposed method solves this problem without converting the HSV video to RGB video. The proposed method reconstructs the saturation and value channels using maximum aposteriori based SR algorithm. For the hue channel the problematic pixels causing the artifacts are masked and a nearest neighbor interpolation is applied. The proposed method is compared to the classical approach where HSV video is converted to RGB domain and each channel is processed by a MAP based SR algorithm. Peak-signal-to-noise-ratio (PSNR) measures show that the proposed approach does not lower the PSNR values. In this paper, it has also been shown that increasing the resolution of the hue channel does not affect the detail perception. Hence applying SR methods to only S and V channels is sufficient to increase the real-time performance of the image processing systems.

In this paper, we deal with layered predictive compression of animated meshes represented by series of 3D static meshes with same connectivity. We propose two schemes to improve the prediction. First improvement is using weighted spatial prediction rather than averaging neighbor vertices. The second improvement is a novel predictor based on rotation angle of incident triangles in current and previous frames. The experimental results show that around 6–10 % bitrate reduction can be achieved by replacing the spatial prediction in the reference coder with the proposed weighted spatial prediction and 9–18 % bitrate reduction is possible with the proposed angle based predictor using weighted spatial prediction, depending on the content and quantization level.

Mobile TV has recently received a lot of attention worldwide with the advances in technologies such as Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting - Handheld (DVB-H) and MediaFLO. On the other hand 3DTV is a new approach to watching TV, introducing the third dimension for a more realistic and interactive experience. With the merge of these two technologies it will be possible to have 3DTV products and services based on portable platforms with switchable 2D/3D autostereoscopic displays. The paper presents the European Mobile3DTV project approach toward achieving such a merge. The project specifically addresses the mobile 3DTV delivery over DVB-H system. It develops a technology demonstration system comprising suitable stereo-video content-creation techniques; efficient, scalable and flexible stereo-video encoders with error resilience and error-concealment capabilities, tailored for robust transmission over DVB-H; and also the corresponding stereo-video decoders and players working on a portable terminal device equipped with an autostereoscopic display.

In this paper, we propose a system for user directed real time view synthesis for hand-held devices. Stored image frames with corresponding depth maps are used as input to the system. Users view point choice is captured using a GYRO based system. OMAP3530 microprocessor is used as the main processor which processes suggested view synthesis algorithm with occlusion handling and frame enhancement techniques. Proposed algorithms are implemented on DSP core and ARM core of OMAP3530 separately and their performances are evaluated through experiments. In addition, two daughter cards are designed for user view point determination. First daughter card handles communication process with EVM board and calculates view point according to the input from the second daughter card with single axis response GYRO sensor (ADIS16060).

In this paper, we study the effect of different slice modes and protection methods on the error performance of video + depth based 3D video broadcast over DVB-H. We provide an end-to-end transmission framework and analyze the effect of slice interleaving by means of 5 different slice modes and protection method by 5 different MPE-FEC rate for video and depth.

Date of Conference: 26-29 September 2010 Conference Name: 17th International Conference on Image Processing, IEEE 2010 In this paper, we study the effect of limited amplitude resolution (pixel depth) in super-resolution problem. The problem we address differs from the standard super-resolution problem in that amplitude resolution is considered as important as spatial resolution. We study the trade-off between the pixel depth and spatial resolution of low resolution (LR) images in order to obtain the best visual quality in the reconstructed high resolution (HR) image. The proposed framework reveals great flexibility in terms of pixel depth and number of LR images in super-resolution problem, and demonstrates that it is possible to obtain target visual qualities with different measurement scenarios including images with different amplitude and spatial resolutions.

In this paper a context based super-resolution (SR) image reconstruction method is proposed. The proposed maximum a-posteriori (MAP) based estimator identifies local gradients and textures for selecting the optimal SR method for the region of interest. Texture segmentation and gradient map estimation are done prior to the reconstruction stage. Gradient direction is used for optimal noise reduction along the edges for non-textured regions. On the other hand, regularization term is cancelled for textured regions so that the resultant method reduces to maximum likelihood (ML) solution. It is demonstrated on Brodatz Texture Database that ML solution gives the best PSNR values on textures compared to the regularized SR methods in the literature. Experimental results show that the proposed hybrid method has superior performance in terms of Peak Signal-to-Noise-Ratio (PSNR), Structural Similarity Index Measure (SSIM) compared the SR methods in the literature.