Search

Recent advancements in high dynamic range (HDR) display technology have significantly enhanced the contrast ratios and peak brightness of modern displays. In the coming years, it is expected that HDR televisions capable of delivering significantly higher brightness and, therefore, contrast levels than today’s models will become increasingly accessible and affordable to consumers. While HDR technology has gained prominence over the past few years, low dynamic range (LDR) content is still consumed due to a substantial volume of historical multimedia content being recorded and preserved in LDR. Although the amount of HDR content will continue to increase as HDR becomes more prevalent, a large portion of multimedia content currently remains in LDR. In addition, it is worth noting that although the HDR standard supports multimedia content with luminance levels up to 10,000 cd/m2 (a standard measure of brightness), most HDR content is typically limited to a maximum brightness of around 1000 cd/m2. This limitation aligns with the current capabilities of consumer HDR TVs but is a factor approximately five times brighter than current LDR TVs. To accurately present LDR content on a HDR display, it is processed through a dynamic range expansion process known as inverse tone mapping (iTM). This LDR to HDR conversion faces many challenges, including the inducement of noise artifacts, false contours, loss of details, desaturated colors, and temporal inconsistencies. This paper introduces complete inverse tone mapping, artifact suppression, and a highlight enhancement pipeline for video sequences designed to address these challenges. Our LDR-to-HDR technique is capable of adapting to the peak brightness of different displays, creating HDR video sequences with a peak luminance of up to 6000 cd/m2. Furthermore, this paper presents the results of comprehensive objective and subjective experiments to evaluate the effectiveness of the proposed pipeline, focusing on two primary aspects: real-time operation capability and the quality of the HDR video output. Our findings indicate that our pipeline enables real-time processing of Full HD (FHD) video (1920 × 1080 pixels), even on hardware that has not been optimized for this task. Furthermore, we found that when applied to existing HDR content, typically capped at a brightness of 1000 cd/m2, our pipeline notably enhances its perceived quality when displayed on a screen that can reach higher peak luminances.

Hyperspectral image (HSI) processing tasks frequently rely on spatial–spectral total variation (SSTV) to quantify the local smoothness of image structures. However, conventional SSTV only considers a sparse structure of gradient maps computed along the spatial and spectral dimensions while neglecting other correlations. To address this limitation, we introduce low-rank guided SSTV (LRSTV), which characterizes the sparsity and low-rank priors of the gradient map simultaneously. First, we verify through numerical tests and theoretical analyses that the gradient tensors are not only sparse but also low-rank. Subsequently, to model the low rankness of the gradient map, we use the tensor average rank to represent the low Tucker rank of gradient tensors. The convex envelope of the tensor average rank is then employed to penalize the rank on the gradient map after the Fourier transform along the spectral dimension. By naturally encoding the sparsity and low-rank priors of the gradient map, LRSTV results in a more accurate representation of the original image. Finally, we demonstrate the effectiveness of LRSTV by integrating it into the HSI processing model, replacing conventional SSTV, and testing it on two public datasets with nine cases of mixed noise and two datasets with realistic noise. The results show that LRSTV outperforms conventional SSTV in terms of accuracy and robustness.

Over the last two decades, UAVs have become an indispensable acquisition platform in the remote sensing community. Meanwhile, advanced lightweight sensors have been introduced in the market, including LiDAR scanners with multiple beams and hyperspectral cameras measuring reflectance using many different narrow-banded filters. To date, however, few fully fledged drone systems exist that combine different sensing modalities in a way that complements the strengths and weaknesses of each. In this paper, we present our multimodal drone payload and sensor fusion pipeline, which allows multispectral point clouds to be generated at subcentimeter accuracy. To that end, we combine high-frequency navigation outputs from a professional-grade GNSS with photogrammetric bundle adjustment and a dedicated point cloud registration algorithm that takes full advantage of LiDAR’s specifications. We demonstrate that the latter significantly improves the quality of the reconstructed point cloud in terms of fewer ghosting effects and less noise. Finally, we thoroughly discuss the impact of the quality of the GNSS/INS system on the structure from the motion and LiDAR SLAM reconstruction process.

We propose a novel and efficient algorithm for detection of specular reflections and light sources (highlights) in cinematic content. The detection of highlights is important for reconstructing them properly in the conversion of the low dynamic range (LDR) to high dynamic range (HDR) content. Highlights are often difficult to be distinguished from bright diffuse surfaces, due to their brightness being reduced in the conventional LDR content production. Moreover, the cinematic LDR content is subject to the artistic use of effects that change the apparent brightness of certain image regions (e.g. limiting depth of field, grading, complex multi-lighting setup, etc.). To ensure the robustness of highlights detection to these effects, the proposed algorithm goes beyond considering only absolute brightness and considers five different features. These features are: the size of the highlight relative to the size of the surrounding image structures, the relative contrast in the surrounding of the highlight, its absolute brightness expressed through the luminance (luma feature), through the saturation in the color space (maxRGB feature) and through the saturation in white (minRGB feature). We evaluate the algorithm on two different image data-sets. The first one is a publicly available LDR image data-set without cinematic content, which allows comparison to the broader State of the art. Additionally, for the evaluation on cinematic content, we create an image data-set consisted of manually annotated cinematic frames and real-world images. For the purpose of demonstrating the proposed highlights detection algorithm in a complete LDR-to-HDR conversion pipeline, we additionally propose a simple inverse-tone-mapping algorithm. The experimental analysis shows that the proposed approach outperforms conventional highlights detection algorithms on both image data-sets, achieves high quality reconstruction of the HDR content and is suited for use in LDR-to-HDR conversion.

Today, solar energy is taking an increasing share of the total energy mix. Unfortunately, many operational photovoltaic plants suffer from a plenitude of defects resulting in non-negligible power loss. The latter highly impacts the overall performance of the PV site; therefore, operators need to regularly inspect their solar parks for anomalies in order to prevent severe performance drops. As this operation is naturally labor-intensive and costly, we present in this paper a novel system for improved PV diagnostics using drone-based imagery. Our solution consists of three main steps. The first step locates the solar panels within the image. The second step detects the anomalies within the solar panels. The final step identifies the root cause of the anomaly. In this paper, we mainly focus on the second step comprising the detection of anomalies within solar panels, which is done using a region-based convolutional neural network (CNN). Experiments on six different PV sites with different specifications and a variety of defects demonstrate that our anomaly detector achieves a true positive rate or recall of more than 90% for a false positive rate of around 2% to 3% tested on a dataset containing nearly 9000 solar panels. Compared to the best state-of-the-art methods, the experiments revealed that we achieve a slightly higher true positive rate for a substantially lower false positive rate, while tested on a more realistic dataset.

Abstract  In this paper, we present a new approach for reconstructing low-resolution document images. Unlike other conventional reconstruction methods, the unknown pixel values are not estimated based on their local surrounding neighbourhood, but on the whole image. In particular, we exploit the multiple occurrence of characters in the scanned document. In order to take advantage of this repetitive behaviour, we divide the image into character segments and match similar character segments to filter relevant information before the reconstruction. A great advantage of our proposed approach over conventional approaches is that we have more information at our disposal, which leads to a better reconstruction of the high-resolution (HR) image. Experimental results confirm the effectiveness of our proposed method, which is expressed in a better optical character recognition (OCR) accuracy and visual superiority to other traditional interpolation and restoration methods. [ABSTRACT FROM AUTHOR]

The installation of solar plants everywhere in the world increases year by year. Automated diagnostic methods are needed to inspect the solar plants and to identify anomalies within these photovoltaic panels. The inspection is usually carried out by unmanned aerial vehicles (UAVs) using thermal imaging sensors. The first step in the whole process is to detect the solar panels in those images. However, standard image processing techniques fail in case of low-contrast images or images with complex backgrounds. Moreover, the shades of power lines or structures similar to solar panels impede the automated detection process. In this research, two self-developed methods are compared for the detection of panels in this context, one based on classical techniques and another one based on deep learning, both with a common post-processing step. The first method is based on edge detection and classification, in contrast to the second method is based on training a region based convolutional neural networks to identify a panel. The first method corrects for the low contrast of the thermal image using several preprocessing techniques. Subsequently, edge detection, segmentation and segment classification are applied. The latter is done using a support vector machine trained with an optimized texture descriptor vector. The second method is based on deep learning trained with images that have been subjected to three different pre-processing operations. The postprocessing use the detected panels to infer the location of panels that were not detected. This step selects contours from detected panels based on the panel area and the angle of rotation. Then new panels are determined by the extrapolation of these contours. The panels in 100 random images taken from eleven UAV flights over three solar plants are labeled and used to evaluate the detection methods. The metrics for the new method based on classical techniques reaches a precision of 0.997, a recall of 0.970 and a F1 score of 0.983. The metrics for the method of deep learning reaches a precision of 0.996, a recall of 0.981 and a F1 score of 0.989. The two panel detection methods are highly effective in the presence of complex backgrounds.

Interpolation from a Color Filter Array (CFA) is the most common method for obtaining full color image data. Its success relies on the smart combination of a CFA and a demosaicing algorithm. Demosaicing on the one hand has been extensively studied. Algorithmic development in the past 20 years ranges from simple linear interpolation to modern neural-network-based (NN) approaches that encode the prior knowledge of millions of training images to fill in missing data in an inconspicious way. CFA design, on the other hand, is less well studied, although still recognized to strongly impact demosaicing performance. This is because demosaicing algorithms are typically limited to one particular CFA pattern, impeding straightforward CFA comparison. This is starting to change with newer classes of demosaicing that may be considered generic or CFA-agnostic. In this study, by comparing performance of two state-of-the-art generic algorithms, we evaluate the potential of modern CFA-demosaicing. We test the hypothesis that, with the increasing power of NN-based demosaicing, the influence of optimal CFA design on system performance decreases. This hypothesis is supported with the experimental results. Such a finding would herald the possibility of relaxing CFA requirements, providing more freedom in the CFA design choice and producing high-quality cameras.

In this paper, we present a complete loop detection and correction system developed for data originating from lidar scanners. Regarding detection, we propose a combination of a global point cloud matcher with a novel registration algorithm to determine loop candidates in a highly effective way. The registration method can deal with point clouds that are largely deviating in orientation while improving the efficiency over existing techniques. In addition, we accelerated the computation of the global point cloud matcher by a factor of 2–4, exploiting the GPU to its maximum. Experiments demonstrated that our combined approach more reliably detects loops in lidar data compared to other point cloud matchers as it leads to better precision–recall trade-offs: for nearly 100% recall, we gain up to 7% in precision. Finally, we present a novel loop correction algorithm that leads to an improvement by a factor of 2 on the average and median pose error, while at the same time only requires a handful of seconds to complete.

Today, many MRI reconstruction techniques exist for undersampled MRI data. Regularization-based techniques inspired by compressed sensing allow for the reconstruction of undersampled data that would lead to an ill-posed reconstruction problem. Parallel imaging enables the reconstruction of MRI images from undersampled multi-coil data that leads to a well-posed reconstruction problem. Autocalibrating pMRI techniques encompass pMRI techniques where no explicit knowledge of the coil sensivities is required. A first purpose of this paper is to derive a novel autocalibration approach for pMRI that allows for the estimation and use of smooth, but high-bandwidth coil profiles instead of a compactly supported kernel. These high-bandwidth models adhere more accurately to the physics of an antenna system. The second purpose of this paper is to demonstrate the feasibility of a parameter-free reconstruction algorithm that combines autocalibrating pMRI and compressed sensing. Therefore, we present several techniques for automatic parameter estimation in MRI reconstruction. Experiments show that a higher reconstruction accuracy can be had using high-bandwidth coil models and that the automatic parameter choices yield an acceptable result.

In this paper, we propose a novel approach to obtain accurate 3D reconstructions of large-scale environments by means of a mobile acquisition platform. The system incorporates a Velodyne LiDAR scanner, as well as a Point Grey Ladybug panoramic camera system. It was designed with genericity in mind, and hence, it does not make any assumption about the scene or about the sensor set-up. The main novelty of this work is that the proposed LiDAR mapping approach deals explicitly with the inhomogeneous density of point clouds produced by LiDAR scanners. To this end, we keep track of a global 3D map of the environment, which is continuously improved and refined by means of a surface reconstruction technique. Moreover, we perform surface analysis on consecutive generated point clouds in order to assure a perfect alignment with the global 3D map. In order to cope with drift, the system incorporates loop closure by determining the pose error and propagating it back in the pose graph. Our algorithm was exhaustively tested on data captured at a conference building, a university campus and an industrial site of a chemical company. Experiments demonstrate that it is capable of generating highly accurate 3D maps in very challenging environments. We can state that the average distance of corresponding point pairs between the ground truth and estimated point cloud approximates one centimeter for an area covering approximately 4000 m 2 . To prove the genericity of the system, it was tested on the well-known Kitti vision benchmark. The results show that our approach competes with state of the art methods without making any additional assumptions.