Search

In order to provide improved multimedia services to the end users, developing objective models efficiently predicting 3 Dimensional (3D) video Quality of Experience (QoE) can currently be considered as one of the most significant research areas. Nevertheless, there is currently no model standardized and widely utilized by the researchers due to its efficient and reliable assessment of the 3D video quality. Therefore, highly exploited 2 Dimensional (2D) video quality assessment models such as Structural SIMilarity Index (SSIM) are preferred for the 3D video quality evaluation. However, providing efficiency and reliability for the 3D video quality assessment using the 2D video quality assessment models can only be ensured if they include 3D video related features effecting Human Visual System (HVS). Under the light of these information, the SSIM model is improved for the 3D video quality assessment using perceptually significant feature, contrast and motion characteristics having impact on the HVS in this study. The results obtained by utilizing the improved SSIM model clearly present that the model is quite competent to provide enhanced multimedia services to the end users.

Segmentation of abdominal organs has been a comprehensive, yet unresolved, research field for many years. In the last decade, intensive developments in deep learning (DL) introduced new state-of-the-art segmentation systems. Despite outperforming the overall accuracy of existing systems, the effects of DL model properties and parameters on the performance are hard to interpret. This makes comparative analysis a necessary tool towards interpretable studies and systems. Moreover, the performance of DL for emerging learning approaches such as cross-modality and multi-modal semantic segmentation tasks has been rarely discussed. In order to expand the knowledge on these topics, the CHAOS – Combined (CT-MR) Healthy Abdominal Organ Segmentation challenge was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI), 2019, in Venice, Italy. Abdominal organ segmentation from routine acquisitions plays an important role in several clinical applications, such as pre-surgical planning or morphological and volumetric follow-ups for various diseases. These applications require a certain level of performance on a diverse set of metrics such as maximum symmetric surface distance (MSSD) to determine surgical error-margin or overlap errors for tracking size and shape differences. Previous abdomen related challenges are mainly focused on tumor/lesion detection and/or classification with a single modality. Conversely, CHAOS provides both abdominal CT and MR data from healthy subjects for single and multiple abdominal organ segmentation. Five different but complementary tasks were designed to analyze the capabilities of participating approaches from multiple perspectives. The results were investigated thoroughly, compared with manual annotations and interactive methods. The analysis shows that the performance of DL models for single modality (CT / MR) can show reliable volumetric analysis performance (DICE: 0.98 ± 0.00 / 0.95 ± 0.01), but the best MSSD performance remains limited (21.89 ± 13.94 / 20.85 ± 10.63 mm). The performances of participating models decrease dramatically for cross-modality tasks both for the liver (DICE: 0.88 ± 0.15 MSSD: 36.33 ± 21.97 mm). Despite contrary examples on different applications, multi-tasking DL models designed to segment all organs are observed to perform worse compared to organ-specific ones (performance drop around 5%). Nevertheless, some of the successful models show better performance with their multi-organ versions. We conclude that the exploration of those pros and cons in both single vs multi-organ and cross-modality segmentations is poised to have an impact on further research for developing effective algorithms that would support real-world clinical applications. Finally, having more than 1500 participants and receiving more than 550 submissions, another important contribution of this study is the analysis on shortcomings of challenge organizations such as the effects of multiple submissions and peeking phenomenon. © 2020 Elsevier B.V., 116E133, BIDEB-2214 College of Environmental Science and Forestry, State University of New York, ESF: 1059B191701102, BIDEB-2219, ZS/2016/08/80646 Türkiye Bilimsel ve Teknolojik Araştirma Kurumu, TÜBITAK, The organizers would like to thank Ivana Isgum and Tom Vercauteren in the challenge committee of ISBI 2019 for their guidance and support. We express our gratitude to supporting organizations of the grand-challenge.org platform. We thank Esranur Kazaz, Umut Baran Ekinci, Ece K?se, Fabian Isensee, David V?lgyes, and Javier Coronel for their contributions. Last but not least, our special thanks go to Ludmila I. Kuncheva for her valuable contributions. This work is supported by Scientific and Technological Research Council of Turkey (TUBITAK) ARDEB-EEEAG under grant number 116E133 and TUBITAK BIDEB-2214 International Doctoral Research Fellowship Programme. The work of P. Ernst, S. Chatterjee, O. Speck and, A. N?rnberger was conducted within the context of the International Graduate School MEMoRIAL at OvGU Magdeburg, Germany, supported by ESF (project no. ZS/2016/08/80646). The work of S. Aslan within the context of Ca? Foscari University of Venice is supported by under TUBITAK BIDEB-2219 grant no 1059B191701102., This work is supported by Scientific and Technological Research Council of Turkey (TUBITAK) ARDEB-EEEAG under grant number 116E133 and TUBITAK BIDEB-2214 International Doctoral Research Fellowship Programme. The work of P. Ernst, S. Chatterjee, O. Speck and, A. Nürnberger was conducted within the context of the International Graduate School MEMoRIAL at OvGU Magdeburg, Germany, supported by ESF (project no. ZS/2016/08/80646). The work of S. Aslan within the context of Ca’ Foscari University of Venice is supported by under TUBITAK BIDEB-2219 grant no 1059B191701102.

In this paper, we present a Unet architecture made of octave convolution for dental image segmentation problem. In this architecture, the requirements for memory and accuracy are significantly improved compared to previous works in the literature. Compare to state-of-art models on this topic the classification accuracy in dental image segmentation is increased by %2, and the memory usage is decreased by %70. Suggested architecture showed a performance of success on ISBI2015 dataset.

Purpose To compare the accuracy and repeatability of emerging machine learning based (i.e. deep) automatic segmentation algorithms with those of well-established semi-automatic (interactive) methods for determining liver volume in living liver transplant donors at computerized tomography (CT) imaging. Methods A total of 12 (6 semi-, 6 full-automatic) methods are evaluated. The semi-automatic segmentation algorithms are based on both traditional iterative models including watershed, fast marching, region growing, active contours and modern techniques including robust statistical segmenter and super-pixels. These methods entail some sort of interaction mechanism such as placing initialization seeds on images or determining a parameter range. The automatic methods are based on deep learning and they include three framework templates (DeepMedic, NiftyNet and U-Net) the first two of which are applied with default parameter sets and the last two involve adapted novel model designs. For 20 living donors (6 training and 12 test datasets), a group of imaging scientists and radiologists created ground truths by performing manual segmentations on contrast material-enhanced CT images. Each segmentation is evaluated using five metrics (i.e. volume overlap and relative volume errors, average/RMS/maximum symmetrical surface distances). The results are mapped to a scoring system and a final grade is calculated by taking their average. Accuracy and repeatability were evaluated using slice by slice comparisons and volumetric analysis. Diversity and complementarity are observed through heatmaps. Majority voting and Simultaneous Truth and Performance Level Estimation (STAPLE) algorithms are utilized to obtain the fusion of the individual results. Results The top four methods are determined to be automatic deep models having 79.63, 79.46 and 77.15 and 74.50 scores. Intra-user score is determined as 95.14. Overall, deep automatic segmentation outperformed interactive techniques on all metrics. The mean volume of liver of ground truth is found to be 1409.93 mL ± 271.28 mL, while it is calculated as 1342.21 mL ± 231.24 mL using automatic and 1201.26 mL ± 258.13 mL using interactive methods, showing higher accuracy and less variation on behalf of automatic methods. The qualitative analysis of segmentation results showed significant diversity and complementarity enabling the idea of using ensembles to obtain superior results. The fusion of automatic methods reached 83.87 with majority voting and 86.20 using STAPLE that are only slightly less than fusion of all methods that achieved 86.70 (majority voting) and 88.74 (STAPLE). Conclusion Use of the new deep learning based automatic segmentation algorithms substantially increases the accuracy and repeatability for segmentation and volumetric measurements of liver. Fusion of automatic methods based on ensemble approaches exhibits best results almost without any additional time cost due to potential parallel execution of multiple models.

Hyperspectral data provides rich information about a scene in terms of spectral details since it encapsulates measurements/observations from a wide large range of spectrum. To this end, it has been used in different problems mostly related to identification and detection processes. However, the main limitation arises for the accessibility of data. More precisely, there is no sufficient amount of hyperspectral data available compared to visible range data for trainable models. In this paper, we tackle an inverse problem to estimate the relative lidar depth from hyperspectral data. To solve its limitation, we integrate semantic information existed in data with supervised labels to decrease the possibility of parameter overfitting. Moreover, details of the output responses are enhanced with Laplacian pyramids and attention layers in which the model makes predictions from each subsequent scale instead of a single shot prediction from the top of the model. In our experiments, we use the 2018 IEEE GRSS Data Fusion Challenge dataset. From the experimental results, we prove that use of hyperspectral data instead of visible range data improves the performance. Moreover, we show that results are significantly improved if a sparse set of depth measurements is used along with hyperspectral data. Lastly, the integration of semantic information to the solution yields more stable and better results compared to the baselines.

In this study, the effect of visual context information to the performance of learning-based techniques for the super resolution problem is analyzed. Beside the interpretation of the experimental results in detail, its theoretical reasoning is also achieved in the paper. For the experiments, two different visual datasets composed of natural and remote sensing scenes are utilized. From the experimental results, we observe that keeping visual context information in the course of parameter learning for convolutional neural networks yields better performance compared to the baselines. Moreover, we summarize that finetuning pre-trained parameters with the related context yet fewer samples improves the results.

Hyperspectral sensors have high spectral resolution by capturing images in hundreds of bands. Despite the high spectral resolution, low spatial resolution of these sensors restricts the performance of the hyperspectral imaging applications such as target tracking and image classification. Fusing the hyper-spectral image (HSI) with higher spatial resolution RGB or multispectral image (MSI) data is a commonly used method in the resolution enhancement of the HSIs. In this paper, we propose a new fusion technique for the HSI super-resolution. The main contribution of this study is formulating the fusion problem in a quadratic manner and also regularizing the solution quadratically using smoothness prior. Moreover, another contribution of the proposed method is converting the fusion problem from spectral domain to the abundance map domain which gives more robust and spectrally consistent results. In the proposed method, first, abundance maps are obtained using linear spectral unmixing and then a quadratic energy function is obtained using these maps and high resolution (HR) RGB image. In addition, quadratic function is regularized using additional constraints. Solving the regularized quadratic function gives the HR abundance maps and these maps are used to reconstruct HR HSI. Experiments show that proposed method yields better performance as compared to state of the art methods in different performance metrics.

There are numerous criteria which are being used to measure camera performance and for determining such criteria, different tests are applied in different test environments. Within this framework, color accuracy testing at camera performance is one of foremost of such tests. In the scope of this paper, a method has been proposed to reduce user interaction in the color accuracy tests in the literature. At the same time, with the color constancy concept, it has been shown that color variation between the different test setups should also be considered as an important criterion on the camera performance.

Inpainting applications include object removal on images and videos, crack filling, error concealment, texture synthesis, where in this paper, its usage for image coherence and perspective emphasis on video frames in 2D image-to-video conversion system is analysed. Besides, the performance of different techniques in object removal and image reconstruction is compared using visual experiments and quality metrics.

This study presents a novel landmine identification method that estimates intrinsic parameters of buried objects from their primary and secondary GPR reflections to reduce false alarm rates of GPR-based landmine detection algorithms. To achieve this, two different features are extracted from A-scan GPR data of buried objects. The first feature identifies significant GPR signal length. The second feature estimates intrinsic impedance of the object. These two features are classified with support vector machine (SVM) classifier. The experimental results show that the proposed features have very high discrimination power which reduces false alarm rates to a great extent.

In this paper, we propose a novel hyperspectral unmixing technique based on deep spectral convolution networks (DSCN). Particularly, three important contributions are presented throughout this paper. First, fully-connected linear operation is replaced with spectral convolutions to extract local spectral characteristics from hyperspectral signatures with a deeper network architecture. Second, instead of batch normalization, we propose a spectral normalization layer which improves the selectivity of filters by normalizing their spectral responses. Third, we introduce two fusion configurations that produce ideal abundance maps by using the abstract representations computed from previous layers. In experiments, we use two real datasets to evaluate the performance of our method with other baseline techniques. The experimental results validate that the proposed method outperforms baselines based on Root Mean Square Error (RMSE)., Comment: Accepted to ICIP 2018

The cost-effective visual representation and fast query-by-example search are two challenging goals that should be maintained for web-scale visual retrieval tasks on moderate hardware. This paper introduces a fast and robust method that ensures both of these goals by obtaining state-of-the-art performance for an image-to-video search scenario. Hence, we present critical enhancements to well-known indexing and visual representation techniques by promoting faster, better and moderate retrieval performance. We also boost the superiority of our method for some visual challenges by exploiting individual decisions of local and global descriptors at query time. For instance, local content descriptors represent copied/duplicated scenes with large geometric deformations such as scale, orientation and affine transformation. In contrast, the use of global content descriptors is more practical for near-duplicate and semantic searches. Experiments are conducted on a large-scale Stanford I2V dataset. The experimental results show that our method is useful in terms of complexity and query processing time for large-scale visual retrieval scenarios, even if local and global representations are used together. The proposed method is superior and achieves state-of-the-art performance based on the mean average precision (MAP) score of this dataset. Lastly, we report additional MAP scores after updating the ground annotations unveiled by retrieval results of the proposed method, and it shows that the actual performance., Comment: ICPR 2020

Accurate correction of atmospheric effects on data captured by an infrared (IR) camera is crucial for several applications such as vegetation monitoring, temperature monitoring, satellite images, hyperspectral imaging, numerical model simulations, surface properties characterization, and IR measurement interpretation. Atmospheric effects depend on the temporal changes, i. e., year, season, day, hour, etc., and on the geometry between the camera and the measured scene, i. e., line of sight. The orientation and the optical depth of the camera significantly affect the variation of the geometry across the pixels. In this paper, we propose a method to estimate the range and zenith angle of each pixel using only the Global Positioning System (GPS) coordinates of the camera and a point of interest in the scene. The estimated geometry and measured meteorological data are used to obtain the spectral atmospheric transmittance and path radiance. Furthermore, we propose an atmospheric effects removal, i. e., atmospheric correction, method that considers the spectral characteristics of the detector, lens, and filter. The proposed atmospheric correction process is analyzed in detail with the simultaneous measurements of two IR cameras. In this process, an enhanced temperature calibration method is developed and it is shown that the temperature accuracy for the dynamic range of the IR camera is very close to the noise equivalent temperature difference (NETD) value of the camera.

Optical cameras give detailed images in clear waters. However, in dark or turbid waters, information coming from electro-optical sensors is insufficient for accurate scene perception. Imaging sonars, also known as acoustic cameras, can provide enhanced target details in these scenarios. In this paper, a computationally efficient 2D high-frequency, forward-looking sonar image simulator is presented. Stages and requirements of the image formation process are explained in detail. For the postprocessing of the returned sonar signals, a novel computation engine is proposed based on the geometric structures of the simulated surfaces. By treating all the continuous surfaces separately, the simulator is able to exactly calculate bright and shadowed zones in the 2D sonar image.

Frame-level visual features are generally aggregated in time with the techniques such as LSTM, Fisher Vectors, NetVLAD etc. to produce a robust video-level representation. We here introduce a learnable aggregation technique whose primary objective is to retain short-time temporal structure between frame-level features and their spatial interdependencies in the representation. Also, it can be easily adapted to the cases where there have very scarce training samples. We evaluate the method on a real-fake expression prediction dataset to demonstrate its superiority. Our method obtains 65% score on the test dataset in the official MAP evaluation and there is only one misclassified decision with the best reported result in the Chalearn Challenge (i.e. 66:7%) . Lastly, we believe that this method can be extended to different problems such as action/event recognition in future., Comment: Submitted to International Conference on Computer Vision Workshops

Data acquired from multi-channel sensors is a highly valuable asset to interpret the environment for a variety of remote sensing applications. However, low spatial resolution is a critical limitation for previous sensors and the constituent materials of a scene can be mixed in different fractions due to their spatial interactions. Spectral unmixing is a technique that allows us to obtain the material spectral signatures and their fractions from hyperspectral data. In this paper, we propose a novel endmember extraction and hyperspectral unmixing scheme, so called \textit{EndNet}, that is based on a two-staged autoencoder network. This well-known structure is completely enhanced and restructured by introducing additional layers and a projection metric (i.e., spectral angle distance (SAD) instead of inner product) to achieve an optimum solution. Moreover, we present a novel loss function that is composed of a Kullback-Leibler divergence term with SAD similarity and additional penalty terms to improve the sparsity of the estimates. These modifications enable us to set the common properties of endmembers such as non-linearity and sparsity for autoencoder networks. Lastly, due to the stochastic-gradient based approach, the method is scalable for large-scale data and it can be accelerated on Graphical Processing Units (GPUs). To demonstrate the superiority of our proposed method, we conduct extensive experiments on several well-known datasets. The results confirm that the proposed method considerably improves the performance compared to the state-of-the-art techniques in literature., To appear in IEEE Transaction on Geoscience and Remote Sensing

To perform an accurate vegetation classification in hyperspectral data, feature extraction process prior to classification is very important. Success rates of classifiers in vegetation are rather limited compared to classification of other types of materials. Therefore, it is required to perform an effective feature extraction before classification. Principle Component Analysis(PCA) is a common and easily applicable method for this purpose. However, PCA is not an optimal method for distinguishing between different plant species. In this study, the reasons for PCA not being an adequate method for this purpose are discussed and alternative useful feature extraction methods in classification of plant species are examined. Tests were performed for Spectrally Segmented PCA(SSPCA), Discrete Wavelet Transform(DWT) and Genetic Algorithm(GA) feature extraction methods, their effects on classifier performances were compared and it was observed that all of the mentioned alternatives had noticable improvements in classification performances.

A vehicle classification system, which uses features based on radiometry, is developed for single band infrared (IR) image sequences. In this context, the process is divided into three components. These are moving vehicle detection, radiance estimation, and classification. The major contribution of this paper lies in the usage of the radiance values as features, other than the raw output of IR camera output, to improve the classification performance of the detected objects. The motivation behind this is that each vehicle class has a discriminating radiance value that originates from the source temperature of the object modified by the intrinsic characteristics of the radiating surface and the environment. As a consequence, significant performance gains are achieved due to the use of radiance values in classification for the utilized measurement system.

In this paper, an application of short video synthesis from single frame images is realized and a comparative analysis of different methods on image inpainting, which is a computationally costly part of the whole procedure, is provided. Our work is fortified with experiments in order to measure the computational performances and efficiencies of the proposed method and the ones existing in literature.

Super-resolution (SR) image reconstruction refers to methods where a higher resolution image is reconstructed using a set of overlapping aliased low-resolution observations of the same scene. Although edge preservation has been a widely explored topic in SR literature, texture-specific regularisation has recently gained interest. In this study, texture-specific regularisation is handled as a post-processing step. A two stage method is proposed, comprising multiple SR reconstructions with different regularisation parameters followed by a restoration step for preserving edges and textures. In the first stage, two maximum-a-posteriori estimators with two different amounts of regularisation are employed. In the second stage, pixel-to-pixel difference between these two estimates is post-processed to restore edges and textures. Frequency selective characteristics of discrete cosine transform and Gabor filters are utilised in the post-processing step. Experiments on synthetically generated images and real experiments demonstrate that the proposed methods give better results compared with the state-of-the-art SR methods especially on textures and edges.

Ground penetrating radar (GPR) is a powerful technology for detection and identification of buried explosives, especially with little or no metal content. However, subsurface clutter and soil distortions increase false alarm rates of current GPR-based landmine detection and identification methods. Most existing algorithms use shape-based, image-based, and physics-based techniques. Analysis of these techniques indicates that each type of algorithm has a different perspective to solve the landmine detection and identification problem. Therefore, one type of method has stronger and weaker points with respect to the other types of algorithms. To reduce false alarm rates of the current GPR-based landmine detection and identification methods, we propose a combined feature utilizing both physics-based and image-based techniques. Combined features are classified with a support vector machine classifier. The proposed algorithm is tested on a simulated data set that contained more than 500 innocuous object signatures and 400 landmine signatures, over half of which are completely nonmetal. The results presented indicate that the proposed method has significant performance benefits for landmine detection and identification in GPR data.

Super-resolution Reconstruction (SRR) is technique to increase the spatial resolution of images. It is especially useful for hyperspectral images (HSI), which have good spectral resolution but low spatial resolution. In this study, we propose an improvement to our previous work and present a novel MAP-MRF (maximum a posteriori-Markov random Fields) based approach for the SRR of HSI. The key point of our approach is to find the abundance maps of an HSI and perform SRR on the abundance maps using MRF based energy minimization, without needing any other additional source of information. In order to do so, first, PCA is used to determine the endmembers. Second, SISAL and fully constraint least squares (FCLS) are used to estimate the abundance maps. Third, in order to find the high resolution abundance maps, the ill-posed inverse SRR problem for abundances is regularized with a MAP-MRF based approach. The MAP-MRF formulation is restricted with the constraints which are specific to the abundances. Using the non-linear programming (NLP) techniques, the convex MAP formulation is minimized and High Resolution (HR) abundance maps are obtained. Then, these maps are used to construct the HR HSI. This improved SRR method is verified on real data sets, and quantitative performance comparison is achieved using PSNR, SSIM and PSNR metrics. Our results indicate that this improved method gives very close results to the original high resolution images, keeps the spectral consistency, and performs better than the compared algorithms.

In this paper, a decision level fusion using multiple pre-screener algorithms is proposed for the detection of buried landmines from Ground Penetrating Radar (GPR) data. The Kernel Least Mean Square (KLMS) and the Blob Filter pre-screeners are fused together to work in real time with less false alarms and higher true detection rates. The effect of the kernel variance is investigated for the KLMS algorithm. Also, the results of the KLMS and KLMS+Blob filter algorithms are compared to the LMS method in terms of processing time and false alarm rates. Proposed algorithm is tested on both simulated data and real data collected at the field of IPA Defence at METU, Ankara, Turkey.

This paper describes a novel pre-screener algorithm for landmine detection with a ground penetrating radar (GPR). The pre-screener algorithms are used for finding anomalies that are potential locations of interest. Thus, their processing time is as important as their true detection rate and false alarm rate. The proposed approach is based on Kernel Least Mean Square algorithm. Although Least Mean Square (LMS) based approach has already been used in the literature, KLMS based approach is a novel application for landmine detection with GPR. In this study, KLMS approach is compared with LMS approach in terms of processing time, false alarm rate, and true detection rate.

In this study, we propose a method for panoramic background subtraction by using Pan-Tilt cameras in real-time. The proposed method is based on parallelization of image registration, panorama generation and background subtraction operations to run on Graphics Processing Unit (GPU). Experiments results showed that GPU usage increases speed of the algorithm 33 times without considerable performance loss and makes working real-time possible.

The Sign Language Recognition (SLR) Problem is a highly important research topic, because of its ability to increase the interaction between the people who are hearing-impaired or impediment in speech. We propose a simple but robust system. The proposed system consists of three main steps. First we apply segmentation to the face and hand region by using Fuzzy C-Means Clustering (FCM) and Thresholding. FCM is a clustering technique which employs fuzzy partitioning, in an iterative algorithm. After the face and hands are segmented, the feature vectors are extracted. The feature vectors are chosen among the low level features such as the bounding ellipse, bounding box, and center of mass coordinates, since they are known to be more robust to segmentation errors due to low resolution images. In total there are 23 features for each hand. After the feature vectors are extracted, they are used for recognition with discrete Hidden Markov Model (HMM). Recognition stage is composed of two stages, namely training and classification. The Baum Welch algorithm is used for HMM training. In classification part the likelihood of each HMM is calculated and the HMM with the highest likelihood is chosen. In order to measure the success rate of the system, the eNTERFACE dataset is used. In this dataset 8 different American Sign Language example classified and in user independent case, is shown to be working with 94.19% accuracy.

Despite their high spectral resolution, hyperspectral images have low spatial resolution which adversely affects the applications that use hyperspectral images. In this study, instead of the traditional way of using spectral images, abundances of the endmembers are used in resolution enhancement. In the proposed method, first, endmembers are extracted with the SISAL algorithm. Then, the abundance maps are estimated using FCLS. From the low resolution abundance maps, high resolution abundance maps are obtained with a total variation based minimization. Finally, high resolution hyperspectral images are constructed from high resolution abundance maps. The proposed method is tested on real hyperspectral images. The experimental results and comparative analysis show the effectiveness of the proposed method.

In this paper, we present an approach that can detect scene nudity level with high precision using different deep net configurations. For this purpose, a recent approach [1] which has intense and very deep convolution layers is used. During net modelling, we strive to obtain most successful net configuration by comparing different Dropout models and image sizes -64 × 64, 128 × 128-. Additionally, leveraging the generalization capability of Support Vector Machine (SVM), improvement on success rate is demonstrated by retraining the features obtained at different output levels of the nets with SVM. At test and training stages, scene is investigated under three nudity levels: regular, semi-nudity and full-nudity. In order to evaluate false alarm rates of the net models, tests are conducted on different datasets which are SUN2012, LEAR Human and a dataset contains only semi-nudity samples besides validation set determined for each class. The results indicate that high precision rates can be achieved with low false alarm rate exploiting deep net models.

8th International Conference on Quality of Multimedia Experience (QoMEX) -- JUN 06-08, 2016 -- Lisbon, PORTUGAL WOS: 000391251500046 In this study a novel, spatio-temporal characteristics based video content analysis method is presented. The proposed method has been evaluated on different video quality assessment databases, which include videos with different characteristics and distortion types. Test results obtained on different databases demonstrate the robustness and accuracy of the proposed content analysis method. Moreover, this analysis method is employed in order to examine the performance improvement in audiovisual quality assessment when the video content is taken into consideration. IEEE, IEEE Signal Proc Soc, NOS, YouTube, EURASIP, Qualinet, QoENet, Inst telecomunicacoes, Tecnico Lisboa

A desirable characteristic for an e-learning system is to provide the learner the most appropriate information based on his requirements and preferences. This can be achieved by capturing and utilizing the learner model. Learner models can be extracted based on personality factors like learning styles, behavioral factors like user's browsing history and knowledge factors like user's prior knowledge. In this paper, we address the problem of extracting the learner model based on Felder-Silverman learning style model. The target learners in this problem are the ones studying basic science. Using NBTree classification algorithm in conjunction with Binary Relevance classifier, the learners are classified based on their interests. Then, learners' learning styles are detected using these classification results. Experimental results are also conducted to evaluate the performance of the proposed automated learner modeling approach. The results show that the match ratio between the obtained learner's learning style using the proposed learner model and those obtained by the questionnaires traditionally used for learning style assessment is consistent for most of the dimensions of Felder-Silverman learning style.

Research efforts on 3DTV technology have been strengthened worldwide recently, covering the whole media processing chain from capture to display. Different 3DTV systems rely on different 3D scene representations that integrate various types of data. Efficient coding of these data is crucial for the success of 3DTV. Compression of pixel-type data including stereo video, multiview video, and associated depth or disparity maps extends available principles of classical video coding. Powerful algorithms and open international standards for multiview video coding and coding of video plus depth data are available and under development, which will provide the basis for introduction of various 3DTV systems and services in the near future. Compression of 3D mesh models has also reached a high level of maturity. For static geometry, a variety of powerful algorithms are available to efficiently compress vertices and connectivity. Compression of dynamic 3D geometry is currently a more active field of research. Temporal prediction is an important mechanism to remove redundancy from animated 3D mesh sequences. Error resilience is important for transmission of data over error prone channels, and multiple description coding (MDC) is a suitable way to protect data. MDC of still images and 2D video has already been widely studied, whereas multiview video and 3D meshes have been addressed only recently. Intellectual property protection of 3D data by watermarking is a pioneering research area as well. The 3D watermarking methods in the literature are classified into three groups, considering the dimensions of the main components of scene representations and the resulting components after applying the algorithm. In general, 3DTV coding technology is maturating. Systems and services may enter the market in the near future. However, the research area is relatively young compared to coding of other types of media. Therefore, there is still a lot of room for improvement and new development of algorithms.

We present a survey of transport methods for 3-D video ranging from early analog 3DTV systems to most recent digital technologies that show promise in designing 3DTV systems of tomorrow. Potential digital transport architectures for 3DTV include the DVB architecture for broadcast and the Internet Protocol (IP) architecture for wired or wireless streaming. There are different multiview representation/compression methods for delivering the 3-D experience, which provide a tradeoff between compression efficiency, random access to views, and ease of rate adaptation, including the "video-plus-depth" compressed representation and various multiview video coding (MVC) options. Commercial activities using these representations in broadcast and IP streaming have emerged, and successful transport of such data has been reported. Motivated by the growing impact of the Internet protocol based media transport technologies, we focus on the ubiquitous Internet as the network infrastructure of choice for future 3DTV systems. Current research issues in unicast and multicast mode multiview video streaming include network protocols such as DCCP and peer-to-peer protocols, effective congestion control, packet loss protection and concealment, video rate adaptation, and network/service scalability. Examples of end-to-end systems for multiview video streaming have been provided.

We address efficient compression and real-time streaming of stereoscopic video over the current Internet. We first propose content-adaptive stereo video coding (CA-SC), where additional coding gain, over that can be achieved by exploiting only inter-view correlations, is targeted by down-sampling one of the views spatially or temporally depending on the content, based on the well-known theory that the human visual system can perceive high frequencies in three-dimensional (3D) from the higher quality view. We also developed stereoscopic 3D video streaming server and clients by modifying available open source platforms, where each client can view the video in mono or stereo mode depending on its display capabilities. The performance of the end-to-end stereoscopic streaming system is demonstrated using subjective quality tests.

Among the human related factors, aggressive driving behavior is one of the major causes of traffic accidents [17]. On the other hand, detection and characterization of driver aggressiveness is a challenging task since there exist different psychological causes behind it. However, information about the driver behavior could be extracted from the data that is collected via different sensing devices. This paper presents a method to detect driver aggressiveness using only visual information provided by forward camera. The proposed method is based on detection of the road lines and the vehicles on the road and extracts information related with road lane departure rate, speed of the vehicle and possible forward collision time. Using these extracted features, a classifier is utilized in order to detect if driver shows an aggressive driving behavior. The proposed method is tested by a subjective testing method using 76 different driving sessions and achieved 90.4% success.

In this paper, scene recognition problem is investigated in detail by exploiting scene representation, dimension reduction and classification stages. Unlike the other studies, the proposed algorithm has preferred to model the overall structure of the scene instead of an object-based proposal. For that purpose, some of the visual representations like MPEG-7, Gist, BoW, Vlad and Fisher, are classified singly or jointly with Support Vector Machine and Random Forest. The evaluation tests are conducted on MIT indoor dataset [2] and from the results, %31 average precision has been attained by combining Scalable Color, Homogeneous Texture and Vlad with Support Vector Machine.

The recent improvements on internet technologies and video coding techniques cause an increase in copyright infringements especially for video. Frequently, image-based approaches appear as an essential solution due to the fact that joint usage of quantization-based indexing and weak geometric consistency stages give a capability to compare duplicate videos quickly. However, exploiting purely spatial content ignores the temporal variation of video. In this work, we propose a system that combines the state-of-the-art quantization-based indexing scheme with a novel trajectory-based geometric consistency on spatio-temporal features. This combination improves duplicate video matching task significantly. Briefly, spatial mean and variance of the trajectories are incorporated to establish a weak geometric consistency among pair of frames. To show the success of the proposed method, content-based video copy detection field is selected and TRECVID 2009 dataset is utilized. The experimental results show that constituting trajectory-based consistency on corresponding feature pairs outperforms the performances of merely utilizing spatiotemporal signature and visual signature with enhanced weak geometric consistency.

Need for automatic video copy detection is increased with the recent technical developments in the internet technologies and video recording. Even though image-based techniques with bag-of-word kind of representations are accepted as the best solution because of robustness and speed; they discard the convenient geometric relation which exists among interest points. In this work, we propose a novel geometric relation which computes a binary signature leveraging existence and non-existence of interest points in the neighborhood area. The experimental results on TRECVID 2009 content-based video copy detection dataset show that combination of our method with recently proposed quantization-based indexing and weak geometric consistency schemes outperforms classical representations.

Technological developments and increases of the copyright infringements have led a demand on developing automatic video copy detection systems. Generally, for this task, even if visual local descriptors are preferred, descriptors should be indexed according to make an effective search. In this work, performance of three indexing methods which are defined in literature is compared for video copy detection task. Additionally estimating geometric relation between video frames has positive effect on performance is shown. Evaluation of proposed methods are tested on TRECVID 2009 content based video copy detection dataset and from obtained results, product quantization method yields much more accurate result with respect to classical bag-of-word method is observed.

Remote sensing is used in a spreading manner by many governmental and industrial institutions worldwide in recent years. Target detection has an important place among the applications developed using satellite imagery. In this paper, an original circular target detection algorithm has been proposed based on a radial transformation. The algorithm consists of three stages such as pre-processing, target detection, and postprocessing. In the pre-processing stage, bilateral noise reduction filtering and vegetation detection operations are completed which they are required by target detection step. The target detection stage finds the circular target by a radial transformation algorithm and variables obtained from the training, and postprocessing stage carries out the elimination of falsely detected targets by utilizing the vegetation information. The Petroleum Oil Lubricants (POL) depots in the industrial areas and harbors have been chosen as an application area of the proposed algorithm. The algorithm has been trained and tested on a data set which includes 4-band images with Near-Infrared band. Proposed algorithm is able to detect many circular targets with different types and sizes as a consequence of using a full radial transformation search as well as it gives rewarding results on industrial areas and harbors in the experiments conducted.

22nd IEEE Signal Processing and Communications Applications Conference (SIU) -- APR 23-25, 2014 -- Karadeniz Teknik Univ, Trabzon, TURKEY B. Akar, Gozde/0000-0002-4227-5606 WOS: 000356351400423 In the last two decades the Internet technology has boosted and the connection speeds have been incrased from kilobits to hundred megabits scale. With the rising coverage of the Internet and the usage of mobile devices such as tablets and smart phones, the usage of social media and especially multimedia elements has been increased rapidly. This increment in streaming multimedia created a need for the assessment of the user experience on multimedia and especially video. Even though there are different Video Quality Assessment (VQA) methods for that purpose, most of them are Full-Reference (FR) or Reduced-Reference (RR). In today's world with many mobile devices, the application of these methods are not possible since they need the reference data. The No-Reference (NR) video metrics are much more suitable for the case. In this paper, the main objective is to evaluate a previously proposed NR VQA metric with a new dataset and to compare the results to other high-performance NR metrics such as G.1070 and G.1070E which do not utilize spatial and temporal characteristics of a given video sequence. Evaluation and comparison results show the accuracy and robustness of the proposed metric. IEEE, Karadeniz Tech Univ, Dept Comp Engn & Elect & Elect Engn

20th IEEE International Conference on Image Processing (ICIP) -- SEP 15-18, 2013 -- Melbourne, AUSTRALIA B. Akar, Gozde/0000-0002-4227-5606; NUR YILMAZ, Gokce/0000-0002-0015-9519 WOS: 000351597600012 Many researchers have been developing objective video quality assessment methods due to increasing demand for perceived video quality measurement results by end users to speed-up advancements of multimedia services. However, most of these methods are either Full-Reference (FR) metrics, which require the original video or Reduced-Reference (RR) metrics, which need some features extracted from the original video. No-Reference (NR) metrics, on the other hand, do not require any information about the original video; hence, are much more suitable for applications like video streaming. This paper presents a novel, objective, NR video quality assessment algorithm. The proposed algorithm is based on utilization of spatial extent of video, temporal extent of video using motion vectors, bit rate, and packet loss ratio. Test results obtained using LIVE video quality database demonstrate the accuracy and robustness of the proposed metric. Inst Elect & Elect Engineers, IEEE Signal Proc Soc

In this paper a novel fast circle detection algorithm is proposed which depends on the spatial properties of the connected components on the image. Two 1-D transforms of each connected component is obtained by taking the Radon Transform of the image for two different directions, which are in fact the integrations of the image through horizontal and vertical directions. Circles are detected using the similarities of detected peaks on the transformed functions and the characteristics of the values in between those peaks. The success of the method is analyzed using synthetic images and the performance of the method is presented and compared with Modified Hough Transform (MHT) using synthetic images.

In recent years intelligent transportation systems has been an active research area in computer vision. This work aims to classify vehicles as car, van or truck by using videos which are taken from an uncalibrated camera which sees any highway from above. In this work, moving blobs are detected by a background-foreground extraction algorithm, classification is done by extracting the blob features and histogram of oriented gradients features of detected blobs and by giving those features to support vector machine classifier. Suggested method has been tried on 2 different traffic video and results have been presented.

In this work, BoW, VLAD, and Hamming Embedding(HE) methods are examined that are used in image retrieval systems. Also shows that results can be improved using geometric consistency with binary signature obtained from Hamming Embedding. Oxford and Paris dataset are used for tests and obtained test results are comparatively examined over average precision rate. From results, observed that by adding geometric consistency information using binary signatures makes improvement on performance.

This paper proposes a new maximum a posteriori (MAP) based super-resolution (SR) image reconstruction method targeting edges and textures in images. Unlike conventional MAP based SR image reconstruction methods a spatially varying image prior is employed which is updated according to the frequency content of the reconstructed image at each iteration at different locations. Two alternative methods based on discrete cosine transforms (DCT) and Gabor filters are proposed for determining the image prior. The proposed method is validated through simulations and real experiments which clearly demonstrates significant visual improvements especially on edges and textures compared to state-of-the-art SR methods.

This study presents the stereo video capture and compression on embedded platforms in real time and streaming of the video to mobile devices over a wireless network, as an end-to-end system. The major components of the system are the state-of-the art compression standard H.264/AVC used with side-by-side representation of stereo video on the OMAP3530 SoC (SystemonChip) ARM processor and the mobile device with autostereoscopic display at the receiver side. With the presented architecture, 3D video communication between the mobile users connected to a wireless network is made possible. Furthermore, performance tests are conducted for the system and results are provided in this study.

Breast cancer detection with infrared imaging is a technique that is hazardous radiation free, increases the early detection chance and has a high detection probability. Asymmetry analysis is used for the diagnosis and the analysis is done using the breast segments obtained from the infrared image. In this work, a breast segmentation system based on parabolic Hough transform is presented. Besides, a filtering method using connected components is proposed to increase both the computation speed and the segmentation performance.

19th IEEE International Conference on Image Processing (ICIP) -- SEP 30-OCT 03, 2012 -- Lake Buena Vista, FL NUR YILMAZ, Gokce/0000-0002-0015-9519; B. Akar, Gozde/0000-0002-4227-5606 WOS: 000319334900152 In order to speed up the wide-spread proliferation of the 3D video technologies (e.g., coding, transmission, display, etc), the effect of these technologies on 3D perception should be efficiently and reliably investigated. Using Full-Reference (FR) objective metrics for this investigation is not practical especially for "on the fly" 3D perception evaluation. Thus, a Reduced Reference (RR) metric is proposed to predict the depth perception of 3D video in this paper. The color-plus-depth 3D video representation is exploited for the proposed metric. Since the significant depth levels of the depth map sequences have great influence on the depth perception of users, they are considered as side information in the proposed RR metric. To determine the significant depth levels, the depth map sequences are abstracted using bilateral filter. Video Quality Metric (VQM) is utilized to predict the depth perception ensured by the significant depth levels due to its well correlation with the Human Visual System (HVS). The performance assessment results present that the proposed RR metric can be utilized in place of a FR metric to reliably measure the depth perception of 3D video with a low overhead. Inst Elect & Elect Engineers (IEEE), IEEE Signal Proc Soc

In this paper, we present a complete framework of an end-to-end error resilient transmission of 3D video over DVB-H and provide an analysis of transmission parameters. We perform the analysis for various layering, protection strategy and prediction structure using different contents and different channel conditions.

This paper presents a new quality evaluation model for stereoscopic videos using depth map segmentation. This study includes both objective and subjective evaluation. The goal of this study is to understand the effect of different depth levels on the overall 3D quality. Test sequences with different coding schemes are used. The results show that overall quality has a strong correlation with the quality of the background, where disparity is smaller relative to the foreground. The results also showed that content type is an important factor in determining the visual quality. While depth segmentation gives information related to 3D perception, additional work is required to develop an objective metric.