Search

Günümüzdeki en revaçta araştırma alanlarından birisi kullanıcılara gelişmiş çoklu-ortam servisleri sağlayabilmek adına 3 Boyutlu (3B) video Kalite Deneyimini (KD) etkin olarak tahmin eden objektif metriklerin geliştirilmesidir. Fakat, literatürde standartlaşmış ve yaygın kullanılan bir metrik henüz bulunmamaktadır. Bu yüzden, Structural SIMilarity Index (SSIM) gibi 2 Boyutlu (2B) video kalite ölçümünde sıklıkla kullanılan metrikler 3B video kalite ölçümünde de kullanılmaktadır. Ancak bu metrikler İnsan Görme Sitemini (İGS) etkileyen 3B video bağlantılı özellikleri içermedikleri için güvenilir 3B video kalite ölçümü sağlamaktan oldukça uzaktırlar. Bunları göz önüne alarak, bu çalışmada, SSIM, zıtlık, hareket ve yapısal bilgi karakteristikleri gibi İGS’yi etkileyen 3B video özellikleri ile geliştirilmiştir. Geliştirilen SSIM metriği kullanılarak elde edilen sonuçlar, bu metriğin gelişmiş çoklu-ortam servisleri sağlayabilmek açısından etkinliğini kanıtlamaktadır.

In order to provide improved multimedia services to the end users, developing objective models efficiently predicting 3 Dimensional (3D) video Quality of Experience (QoE) can currently be considered as one of the most significant research areas. Nevertheless, there is currently no model standardized and widely utilized by the researchers due to its efficient and reliable assessment of the 3D video quality. Therefore, highly exploited 2 Dimensional (2D) video quality assessment models such as Structural SIMilarity Index (SSIM) are preferred for the 3D video quality evaluation. However, providing efficiency and reliability for the 3D video quality assessment using the 2D video quality assessment models can only be ensured if they include 3D video related features effecting Human Visual System (HVS). Under the light of these information, the SSIM model is improved for the 3D video quality assessment using perceptually significant feature, contrast and motion characteristics having impact on the HVS in this study. The results obtained by utilizing the improved SSIM model clearly present that the model is quite competent to provide enhanced multimedia services to the end users.

Segmentation of abdominal organs has been a comprehensive, yet unresolved, research field for many years. In the last decade, intensive developments in deep learning (DL) introduced new state-of-the-art segmentation systems. Despite outperforming the overall accuracy of existing systems, the effects of DL model properties and parameters on the performance are hard to interpret. This makes comparative analysis a necessary tool towards interpretable studies and systems. Moreover, the performance of DL for emerging learning approaches such as cross-modality and multi-modal semantic segmentation tasks has been rarely discussed. In order to expand the knowledge on these topics, the CHAOS – Combined (CT-MR) Healthy Abdominal Organ Segmentation challenge was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI), 2019, in Venice, Italy. Abdominal organ segmentation from routine acquisitions plays an important role in several clinical applications, such as pre-surgical planning or morphological and volumetric follow-ups for various diseases. These applications require a certain level of performance on a diverse set of metrics such as maximum symmetric surface distance (MSSD) to determine surgical error-margin or overlap errors for tracking size and shape differences. Previous abdomen related challenges are mainly focused on tumor/lesion detection and/or classification with a single modality. Conversely, CHAOS provides both abdominal CT and MR data from healthy subjects for single and multiple abdominal organ segmentation. Five different but complementary tasks were designed to analyze the capabilities of participating approaches from multiple perspectives. The results were investigated thoroughly, compared with manual annotations and interactive methods. The analysis shows that the performance of DL models for single modality (CT / MR) can show reliable volumetric analysis performance (DICE: 0.98 ± 0.00 / 0.95 ± 0.01), but the best MSSD performance remains limited (21.89 ± 13.94 / 20.85 ± 10.63 mm). The performances of participating models decrease dramatically for cross-modality tasks both for the liver (DICE: 0.88 ± 0.15 MSSD: 36.33 ± 21.97 mm). Despite contrary examples on different applications, multi-tasking DL models designed to segment all organs are observed to perform worse compared to organ-specific ones (performance drop around 5%). Nevertheless, some of the successful models show better performance with their multi-organ versions. We conclude that the exploration of those pros and cons in both single vs multi-organ and cross-modality segmentations is poised to have an impact on further research for developing effective algorithms that would support real-world clinical applications. Finally, having more than 1500 participants and receiving more than 550 submissions, another important contribution of this study is the analysis on shortcomings of challenge organizations such as the effects of multiple submissions and peeking phenomenon. © 2020 Elsevier B.V., 116E133, BIDEB-2214 College of Environmental Science and Forestry, State University of New York, ESF: 1059B191701102, BIDEB-2219, ZS/2016/08/80646 Türkiye Bilimsel ve Teknolojik Araştirma Kurumu, TÜBITAK, The organizers would like to thank Ivana Isgum and Tom Vercauteren in the challenge committee of ISBI 2019 for their guidance and support. We express our gratitude to supporting organizations of the grand-challenge.org platform. We thank Esranur Kazaz, Umut Baran Ekinci, Ece K?se, Fabian Isensee, David V?lgyes, and Javier Coronel for their contributions. Last but not least, our special thanks go to Ludmila I. Kuncheva for her valuable contributions. This work is supported by Scientific and Technological Research Council of Turkey (TUBITAK) ARDEB-EEEAG under grant number 116E133 and TUBITAK BIDEB-2214 International Doctoral Research Fellowship Programme. The work of P. Ernst, S. Chatterjee, O. Speck and, A. N?rnberger was conducted within the context of the International Graduate School MEMoRIAL at OvGU Magdeburg, Germany, supported by ESF (project no. ZS/2016/08/80646). The work of S. Aslan within the context of Ca? Foscari University of Venice is supported by under TUBITAK BIDEB-2219 grant no 1059B191701102., This work is supported by Scientific and Technological Research Council of Turkey (TUBITAK) ARDEB-EEEAG under grant number 116E133 and TUBITAK BIDEB-2214 International Doctoral Research Fellowship Programme. The work of P. Ernst, S. Chatterjee, O. Speck and, A. Nürnberger was conducted within the context of the International Graduate School MEMoRIAL at OvGU Magdeburg, Germany, supported by ESF (project no. ZS/2016/08/80646). The work of S. Aslan within the context of Ca’ Foscari University of Venice is supported by under TUBITAK BIDEB-2219 grant no 1059B191701102.

In this paper, we present a Unet architecture made of octave convolution for dental image segmentation problem. In this architecture, the requirements for memory and accuracy are significantly improved compared to previous works in the literature. Compare to state-of-art models on this topic the classification accuracy in dental image segmentation is increased by %2, and the memory usage is decreased by %70. Suggested architecture showed a performance of success on ISBI2015 dataset.

Purpose To compare the accuracy and repeatability of emerging machine learning based (i.e. deep) automatic segmentation algorithms with those of well-established semi-automatic (interactive) methods for determining liver volume in living liver transplant donors at computerized tomography (CT) imaging. Methods A total of 12 (6 semi-, 6 full-automatic) methods are evaluated. The semi-automatic segmentation algorithms are based on both traditional iterative models including watershed, fast marching, region growing, active contours and modern techniques including robust statistical segmenter and super-pixels. These methods entail some sort of interaction mechanism such as placing initialization seeds on images or determining a parameter range. The automatic methods are based on deep learning and they include three framework templates (DeepMedic, NiftyNet and U-Net) the first two of which are applied with default parameter sets and the last two involve adapted novel model designs. For 20 living donors (6 training and 12 test datasets), a group of imaging scientists and radiologists created ground truths by performing manual segmentations on contrast material-enhanced CT images. Each segmentation is evaluated using five metrics (i.e. volume overlap and relative volume errors, average/RMS/maximum symmetrical surface distances). The results are mapped to a scoring system and a final grade is calculated by taking their average. Accuracy and repeatability were evaluated using slice by slice comparisons and volumetric analysis. Diversity and complementarity are observed through heatmaps. Majority voting and Simultaneous Truth and Performance Level Estimation (STAPLE) algorithms are utilized to obtain the fusion of the individual results. Results The top four methods are determined to be automatic deep models having 79.63, 79.46 and 77.15 and 74.50 scores. Intra-user score is determined as 95.14. Overall, deep automatic segmentation outperformed interactive techniques on all metrics. The mean volume of liver of ground truth is found to be 1409.93 mL ± 271.28 mL, while it is calculated as 1342.21 mL ± 231.24 mL using automatic and 1201.26 mL ± 258.13 mL using interactive methods, showing higher accuracy and less variation on behalf of automatic methods. The qualitative analysis of segmentation results showed significant diversity and complementarity enabling the idea of using ensembles to obtain superior results. The fusion of automatic methods reached 83.87 with majority voting and 86.20 using STAPLE that are only slightly less than fusion of all methods that achieved 86.70 (majority voting) and 88.74 (STAPLE). Conclusion Use of the new deep learning based automatic segmentation algorithms substantially increases the accuracy and repeatability for segmentation and volumetric measurements of liver. Fusion of automatic methods based on ensemble approaches exhibits best results almost without any additional time cost due to potential parallel execution of multiple models.

In this study, the effect of visual context information to the performance of learning-based techniques for the super resolution problem is analyzed. Beside the interpretation of the experimental results in detail, its theoretical reasoning is also achieved in the paper. For the experiments, two different visual datasets composed of natural and remote sensing scenes are utilized. From the experimental results, we observe that keeping visual context information in the course of parameter learning for convolutional neural networks yields better performance compared to the baselines. Moreover, we summarize that finetuning pre-trained parameters with the related context yet fewer samples improves the results.

Inverse problems in imaging such as denoising, deblurring, superresolution (SR) have been addressed for many decades. In recent years, convolutional neural networks (CNNs) have been widely used for many inverse problem areas. Although their indisputable success, CNNs are not mathematically validated as to how and what they learn. In this paper, we prove that during training, CNN elements solve for inverse problems which are optimum solutions stored as CNN neuron filters. We discuss the necessity of mutual coherence between CNN layer elements in order for a network to converge to the optimum solution. We prove that required mutual coherence can be provided by the usage of residual learning and skip connections. We have set rules over training sets and depth of networks for better convergence, i.e. performance., PostPrint IET Signal Processing Journal

In this study, we propose a novel single image Bayesian super-resolution (SR) algorithm where the hyperspectral image (HSI) is the only source of information. The main contribution of the proposed approach is to convert the ill-posed SR reconstruction (SRR) problem in the spectral domain to a quadratic optimization problem in the abundance map domain. In order to do so, Markov Random Field (MRF) based energy minimization approach is proposed and proved that the solution is quadratic. The proposed approach consists of five main steps. First, the number of endmembers in the scene is determined using virtual dimensionality. Second, the endmembers and their low resolution abundance maps are computed using simplex identification via the splitted augmented Lagrangian (SISAL) and fully constrained least squares (FCLS) algorithms. Third, high resolution (HR) abundance maps are obtained using our proposed maximum a posteriori (MAP) based energy function. This energy function is minimized subject to smoothness, unity and boundary constraints. Fourth, the HR abundance maps are further enhanced with texture preserving methods. Finally, HR HSI is reconstructed using the extracted endmembers and the enhanced abundance maps. The proposed method is tested on three real HSI datasets; namely the Cave, Harvard and Hyperspectral Remote Sensing Scenes (HRSS) and compared to state-of-the-art alternative methods using peak signal to noise ratio, structural similarity, spectral angle mapper and relative dimensionless global error in synthesis metrics. It is shown that the proposed method outperforms the state of the art methods in terms of quality while preserving the spectral consistency.

There are numerous criteria which are being used to measure camera performance and for determining such criteria, different tests are applied in different test environments. Within this framework, color accuracy testing at camera performance is one of foremost of such tests. In the scope of this paper, a method has been proposed to reduce user interaction in the color accuracy tests in the literature. At the same time, with the color constancy concept, it has been shown that color variation between the different test setups should also be considered as an important criterion on the camera performance.

Inpainting applications include object removal on images and videos, crack filling, error concealment, texture synthesis, where in this paper, its usage for image coherence and perspective emphasis on video frames in 2D image-to-video conversion system is analysed. Besides, the performance of different techniques in object removal and image reconstruction is compared using visual experiments and quality metrics.

This study presents a novel landmine identification method that estimates intrinsic parameters of buried objects from their primary and secondary GPR reflections to reduce false alarm rates of GPR-based landmine detection algorithms. To achieve this, two different features are extracted from A-scan GPR data of buried objects. The first feature identifies significant GPR signal length. The second feature estimates intrinsic impedance of the object. These two features are classified with support vector machine (SVM) classifier. The experimental results show that the proposed features have very high discrimination power which reduces false alarm rates to a great extent.

The cost-effective visual representation and fast query-by-example search are two challenging goals that should be maintained for web-scale visual retrieval tasks on moderate hardware. This paper introduces a fast and robust method that ensures both of these goals by obtaining state-of-the-art performance for an image-to-video search scenario. Hence, we present critical enhancements to well-known indexing and visual representation techniques by promoting faster, better and moderate retrieval performance. We also boost the superiority of our method for some visual challenges by exploiting individual decisions of local and global descriptors at query time. For instance, local content descriptors represent copied/duplicated scenes with large geometric deformations such as scale, orientation and affine transformation. In contrast, the use of global content descriptors is more practical for near-duplicate and semantic searches. Experiments are conducted on a large-scale Stanford I2V dataset. The experimental results show that our method is useful in terms of complexity and query processing time for large-scale visual retrieval scenarios, even if local and global representations are used together. The proposed method is superior and achieves state-of-the-art performance based on the mean average precision (MAP) score of this dataset. Lastly, we report additional MAP scores after updating the ground annotations unveiled by retrieval results of the proposed method, and it shows that the actual performance., Comment: ICPR 2020

Accurate correction of atmospheric effects on data captured by an infrared (IR) camera is crucial for several applications such as vegetation monitoring, temperature monitoring, satellite images, hyperspectral imaging, numerical model simulations, surface properties characterization, and IR measurement interpretation. Atmospheric effects depend on the temporal changes, i. e., year, season, day, hour, etc., and on the geometry between the camera and the measured scene, i. e., line of sight. The orientation and the optical depth of the camera significantly affect the variation of the geometry across the pixels. In this paper, we propose a method to estimate the range and zenith angle of each pixel using only the Global Positioning System (GPS) coordinates of the camera and a point of interest in the scene. The estimated geometry and measured meteorological data are used to obtain the spectral atmospheric transmittance and path radiance. Furthermore, we propose an atmospheric effects removal, i. e., atmospheric correction, method that considers the spectral characteristics of the detector, lens, and filter. The proposed atmospheric correction process is analyzed in detail with the simultaneous measurements of two IR cameras. In this process, an enhanced temperature calibration method is developed and it is shown that the temperature accuracy for the dynamic range of the IR camera is very close to the noise equivalent temperature difference (NETD) value of the camera.

Optical cameras give detailed images in clear waters. However, in dark or turbid waters, information coming from electro-optical sensors is insufficient for accurate scene perception. Imaging sonars, also known as acoustic cameras, can provide enhanced target details in these scenarios. In this paper, a computationally efficient 2D high-frequency, forward-looking sonar image simulator is presented. Stages and requirements of the image formation process are explained in detail. For the postprocessing of the returned sonar signals, a novel computation engine is proposed based on the geometric structures of the simulated surfaces. By treating all the continuous surfaces separately, the simulator is able to exactly calculate bright and shadowed zones in the 2D sonar image.

Data acquired from multi-channel sensors is a highly valuable asset to interpret the environment for a variety of remote sensing applications. However, low spatial resolution is a critical limitation for previous sensors and the constituent materials of a scene can be mixed in different fractions due to their spatial interactions. Spectral unmixing is a technique that allows us to obtain the material spectral signatures and their fractions from hyperspectral data. In this paper, we propose a novel endmember extraction and hyperspectral unmixing scheme, so called \textit{EndNet}, that is based on a two-staged autoencoder network. This well-known structure is completely enhanced and restructured by introducing additional layers and a projection metric (i.e., spectral angle distance (SAD) instead of inner product) to achieve an optimum solution. Moreover, we present a novel loss function that is composed of a Kullback-Leibler divergence term with SAD similarity and additional penalty terms to improve the sparsity of the estimates. These modifications enable us to set the common properties of endmembers such as non-linearity and sparsity for autoencoder networks. Lastly, due to the stochastic-gradient based approach, the method is scalable for large-scale data and it can be accelerated on Graphical Processing Units (GPUs). To demonstrate the superiority of our proposed method, we conduct extensive experiments on several well-known datasets. The results confirm that the proposed method considerably improves the performance compared to the state-of-the-art techniques in literature., To appear in IEEE Transaction on Geoscience and Remote Sensing

To perform an accurate vegetation classification in hyperspectral data, feature extraction process prior to classification is very important. Success rates of classifiers in vegetation are rather limited compared to classification of other types of materials. Therefore, it is required to perform an effective feature extraction before classification. Principle Component Analysis(PCA) is a common and easily applicable method for this purpose. However, PCA is not an optimal method for distinguishing between different plant species. In this study, the reasons for PCA not being an adequate method for this purpose are discussed and alternative useful feature extraction methods in classification of plant species are examined. Tests were performed for Spectrally Segmented PCA(SSPCA), Discrete Wavelet Transform(DWT) and Genetic Algorithm(GA) feature extraction methods, their effects on classifier performances were compared and it was observed that all of the mentioned alternatives had noticable improvements in classification performances.

A vehicle classification system, which uses features based on radiometry, is developed for single band infrared (IR) image sequences. In this context, the process is divided into three components. These are moving vehicle detection, radiance estimation, and classification. The major contribution of this paper lies in the usage of the radiance values as features, other than the raw output of IR camera output, to improve the classification performance of the detected objects. The motivation behind this is that each vehicle class has a discriminating radiance value that originates from the source temperature of the object modified by the intrinsic characteristics of the radiating surface and the environment. As a consequence, significant performance gains are achieved due to the use of radiance values in classification for the utilized measurement system.

In this paper, an application of short video synthesis from single frame images is realized and a comparative analysis of different methods on image inpainting, which is a computationally costly part of the whole procedure, is provided. Our work is fortified with experiments in order to measure the computational performances and efficiencies of the proposed method and the ones existing in literature.

Infrared imagery is widely used in many applications in both civilian and military areas. In landmine detection, the goal is to detect the anomalies between mine surface and soil from variation of reflected/emitted thermal radiation. In this study, various types of anomaly detection techniques of IR are investigated and the feasibility of these techniques for use in landmine detection is analyzed. Additionally, effects of parameters for algorithms are compared and the parameters are optimized for increasing detection accuracy. Furthermore, fusion of the algorithms is performed to reduce False Alarm Rate (FAR). We also prepare an experimental setup to reflect the effects of environmental changes on FLIR imagery recording. Soil and various types of landmine mock-ups are also examined in this setup. Finally, all anomalies are mapped into local coordinate space for indicating possible landmines locations.

Ground penetrating radar (GPR) is a powerful technology for detection and identification of buried explosives, especially with little or no metal content. However, subsurface clutter and soil distortions increase false alarm rates of current GPR-based landmine detection and identification methods. Most existing algorithms use shape-based, image-based, and physics-based techniques. Analysis of these techniques indicates that each type of algorithm has a different perspective to solve the landmine detection and identification problem. Therefore, one type of method has stronger and weaker points with respect to the other types of algorithms. To reduce false alarm rates of the current GPR-based landmine detection and identification methods, we propose a combined feature utilizing both physics-based and image-based techniques. Combined features are classified with a support vector machine classifier. The proposed algorithm is tested on a simulated data set that contained more than 500 innocuous object signatures and 400 landmine signatures, over half of which are completely nonmetal. The results presented indicate that the proposed method has significant performance benefits for landmine detection and identification in GPR data.

Super-resolution Reconstruction (SRR) is technique to increase the spatial resolution of images. It is especially useful for hyperspectral images (HSI), which have good spectral resolution but low spatial resolution. In this study, we propose an improvement to our previous work and present a novel MAP-MRF (maximum a posteriori-Markov random Fields) based approach for the SRR of HSI. The key point of our approach is to find the abundance maps of an HSI and perform SRR on the abundance maps using MRF based energy minimization, without needing any other additional source of information. In order to do so, first, PCA is used to determine the endmembers. Second, SISAL and fully constraint least squares (FCLS) are used to estimate the abundance maps. Third, in order to find the high resolution abundance maps, the ill-posed inverse SRR problem for abundances is regularized with a MAP-MRF based approach. The MAP-MRF formulation is restricted with the constraints which are specific to the abundances. Using the non-linear programming (NLP) techniques, the convex MAP formulation is minimized and High Resolution (HR) abundance maps are obtained. Then, these maps are used to construct the HR HSI. This improved SRR method is verified on real data sets, and quantitative performance comparison is achieved using PSNR, SSIM and PSNR metrics. Our results indicate that this improved method gives very close results to the original high resolution images, keeps the spectral consistency, and performs better than the compared algorithms.

In this paper, a decision level fusion using multiple pre-screener algorithms is proposed for the detection of buried landmines from Ground Penetrating Radar (GPR) data. The Kernel Least Mean Square (KLMS) and the Blob Filter pre-screeners are fused together to work in real time with less false alarms and higher true detection rates. The effect of the kernel variance is investigated for the KLMS algorithm. Also, the results of the KLMS and KLMS+Blob filter algorithms are compared to the LMS method in terms of processing time and false alarm rates. Proposed algorithm is tested on both simulated data and real data collected at the field of IPA Defence at METU, Ankara, Turkey.

In this study, we propose a method for panoramic background subtraction by using Pan-Tilt cameras in real-time. The proposed method is based on parallelization of image registration, panorama generation and background subtraction operations to run on Graphics Processing Unit (GPU). Experiments results showed that GPU usage increases speed of the algorithm 33 times without considerable performance loss and makes working real-time possible.

Despite their high spectral resolution, hyperspectral images have low spatial resolution which adversely affects the applications that use hyperspectral images. In this study, instead of the traditional way of using spectral images, abundances of the endmembers are used in resolution enhancement. In the proposed method, first, endmembers are extracted with the SISAL algorithm. Then, the abundance maps are estimated using FCLS. From the low resolution abundance maps, high resolution abundance maps are obtained with a total variation based minimization. Finally, high resolution hyperspectral images are constructed from high resolution abundance maps. The proposed method is tested on real hyperspectral images. The experimental results and comparative analysis show the effectiveness of the proposed method.