Search

Multi-layer displays are a type of 3D display composed of multiple LCD panels arranged in front of a backlight. For generating the layers, iterative methods are commonly employed, which lead to long computational times that are not suitable for real-time 3D imaging systems. This study addresses the challenge of real-time computation of multi-layer images using a depth camera and demonstrates an end-to-end capture to display process. Unlike iterative methods, our approach directly computes the layers. Firstly, we pre-process the depth input. Secondly, we recreate a focal stack by slicing views based on depth and applying blur. Then, we calculate the multi-layer compensation to diminish artifacts from object blur across layers. The entire pipeline, from Kinect to layered 3D display, achieves a frame rate between 20 and 28 frames per second, at the cost of reduced quality in the reproduced views, info:eu-repo/semantics/inPress

Tensor displays are multiview glasses-free 3D displays composed of a stack of LCD layers. The images displayed on each layer are usually generated from a dense set of viewpoints (light field) or a set of focused images (focal stack). This paper presents an alternative using a single color view with depth for the generation of the images' layers. To do so various single-RGBD pipelines using intermediate data formats to generate the layers' images, including a direct method, are compared. Experiments show that by a single-color view with depth as input, one can provide a quality close to the light field or focal stack pipelines, hence reducing memory storage costs., info:eu-repo/semantics/published

Interactive view synthesis is a fundamental task in computer vision, aiming at recreating a natural scene from any viewpoint using a sparse set of images. The primary focus of this Ph.D. thesis is to explore the acquisition processes and the ability to render high-quality novel views dynamically to a user. Furthermore, this research targets real-time rendering as the second objective. In this thesis, we explore two different ways a scene can be reconstructed. The first option is to estimate the three-dimensional (3D) structure of the scene, by means of a set of points in space called a point cloud (PC). Such PC can be captured by a variety of devices and algorithms such as Time of Flight (ToF) cameras, stereo matching, or structure-from-motion (SfM). Alternatively, the scene can be represented by a set of input views with their associated depth maps, which can be used in depth image-based rendering (DIBR) to synthesize new images.We explore depth image-based rendering algorithms, using pictures of a scene and their associated depth maps. These algorithms project the color values at the novel view position using the depth information. However, the quality of the depth map highly impacts the accuracy of the synthesized views. Therefore, we started by improving the Depth Estimation Reference Software (DERS) of the Moving Picture Experts Group (MPEG), a worldwide standardization committee for video compression. Unlike DERS, our Reference Depth Estimation (RDE) software can take any number of input views, leading to more robust results. It is currently used to generate novel depth maps for standardized datasets. The depth estimation did not reach real-time generation; it takes minutes to hours to create a depth map depending on the input views. We therefore explored active depth sensing devices, such as Microsoft Kinect, to acquire color data and depth maps simultaneously. With the availability of these depth maps, we address the DIBR problem by providing a novel algorit, Doctorat en Sciences de l'ingénieur et technologie, info:eu-repo/semantics/nonPublished

info:eu-repo/semantics/published

info:eu-repo/semantics/published

Capturing and preprocessing 3D data is an increasingly important field, with advances in 3D displays, virtual reality, and digital fabrication driving the need for high-quality 3D data. Systems capable of acquiring 3D content can be separated into two classes: active and passive. Time-of-Flight (ToF) cameras are popular active depth sensors that suffer from significant noise when reconstructing geometry. In the first half of this thesis, we show a novel two-stage noise removal strategy for multi-view ToF systems in which the depth maps are first processed by a Convolutional Neural Network (CNN) to remove most of the depth map noise, and then further processed by a special 3D neural network to resolve inter-camera inconsistencies. Results on real data show a systematic improvement over noise removal tools from the literature. We then present a calibration method for accurately estimating the pose of multiple ToF cameras by using a 3D calibration object, ensuring proper alignment of the reprojected point clouds over multiple positions within the scene's 3D volume. In the second half of this thesis, Light Field (LF) systems are presented as passive depth sensors. A new geometric model and calibration method for plenoptic 1.0 devices is showcased, describing distortion effects at the microlens level for the first time. Improved quality in 3D reconstruction shows the validity of the model. A 3D Convolutional Neural Network is then shown for depth estimation from an LF image array, allowing for faster and lighter training than regular CNN from the literature. Finally, a novel spherical LF acquisition system is presented, capable of placing a camera at arbitrary positions around an object with promising results in photogrammetry and view synthesis., Doctorat en Sciences de l'ingénieur et technologie, info:eu-repo/semantics/nonPublished

Who has never immersed themselves in the memories of a photo album or the universe rendered by a movie? Cameras have been invented to record a part of the world at a precise moment for recollection, information or entertainment purposes. However, beside the ever-increasing quality of the captured images, only one viewpoint, holding in a flat image, is captured. In order to increase the immersion, new methods have emerged to reconstruct and render the three dimensions of a scene, allowing the user to better understand, measure or simply live the captured moment.Due to the public and industry's interest for immersive representations of the real-world (3D cinema, metaverse, virtual reality,), the MPEG-I group (subgroup of MPEG for Immersive video coding) launched a vast work for standardization of immersive video coding. Research on view synthesis, the technique to create new viewpoints from a given set of input images, has been tackled by companies such as Google and NVIDIA, obtaining incredibly photo-realistic results. However, these methods are not yet accessible at large, due to the acquisition setups, the reconstruction process, the hardware cost or the processing time needed to render new viewpoints.In this manuscript, we present a view synthesis method relying on the principle of depth image-based rendering: given a set of real-world images representing a scene, along with some geometry information represented by depth maps, we reconstruct new images taken from any location in the scene. We aim to reach photo-realistic results, wide navigation range, real-time processing on midrange material and multi-input modality.We demonstrate the quality of the image synthesized by this method compared to other state-of-the-art view synthesis approaches. Thanks to a GPU implementation, we reach real-time results and high visual quality with less than ten input images. Thus, we prove that our method is suitable not only for real-time virtual reality applications, but also fo, Doctorat en Sciences de l'ingénieur et technologie, info:eu-repo/semantics/nonPublished

info:eu-repo/semantics/nonPublished

Virtual Reality and Free Viewpoint navigation require high-quality rendered images to be realistic. Current hardware assisted raytracing methods cannot reach the expected quality in real-time and are also limited by the 3D mesh quality. An alternative is depth image-based rendering (DIBR) where the input only consists of images and their associated depth maps for synthesizing virtual views to the Head Mounted Display (HMD). The MPEG Immersive Video (MIV) standard uses such a DIBR algorithm called the Reference View Synthesizer (RVS). We have first implemented a GPU version, called the Real-time Accelerated View Synthesizer (RaViS), that synthesizes two virtual views in real-time for the HMD. In the present paper, we explore the differences between desktop and embedded GPU platforms, porting RaViS to an embedded HMD without the need for a separate, discrete desktop GPU. The proposed solution gives a first insight into DIBR View Synthesis techniques in embedded HMDs using OpenGL and Vulkan, a cross-platform 3D rendering library with support for embedded devices., SCOPUS: cp.p, info:eu-repo/semantics/published

In this research, we went through the main stages to render a 3D scene in 6 Degrees of Freedom: data acquisition, point cloud or depth map estimation, surface reconstruction and view synthesis rendering.We generated datasets, some of them computer-generated using Blender, and some of them real scenery for which we built two acquisition rigs. On the first one we could put any type of camera – RGB or Time-of-Flight – which could then be moved in space in X, Y or Z direction thus sampling the scene anywhere within a cube meter; the second one, more rigid, holding an array of 3-by-3 Light Field cameras. The acquired data was used to devise some depth estimation algorithms based on multi stereo matching algorithms improved with deep learning techniques.We also worked on the triangulation of a fast laser dot moving through the scene. We proved that the resulting depth map from one light field camera can be improved by using neighboring cameras while increasing the parallax. As each of the generated depth maps can be reprojected in space to form a point cloud, we implemented an improved a registration algorithm to put multiple point clouds together, enforcingthe 3D structure of the scene. Finally, we rendered scenes on two different devices: a head mounted display and an holographic display. To render a point cloud in real time within an oculus, we stored it using an efficient data structure and optimizedthe rendering time using sub samples of the point cloud selected using level-of-detail based on the distance between the user and the various parts of the scene, and frustum culling. The visual comfort was brought to the user by the means of splatting techniques to simulate the surfaces of the scene. To render a scene on an holographic screen, we first captured it from different view points with light field cameras, then estimated the depth to be able to synthesize views in between, and use them as input to display on the screen.Our work achieved state-of-the-art results, Doctorat en Sciences de l'ingénieur et technologie, info:eu-repo/semantics/nonPublished

Tensor displays are screens able to render a light field with correct depth perception without wearing glasses. Such devices have already been shown to be able to accurately render a scene composed of Lambertian objects. This paper presents the model and prototyping of a tensor display with three layers, using re-purposed computer monitors, and extends the light field factorization method to non-Lambertian objects. Furthermore, we examine the relation and limitations between the depth-of-field and the depth range with Lambertian and non-Lambertian scenes. Non-Lambertian scenes contain out-of-range disparities that can not be properly rendered with the usual optimization method. We propose to artificially compress the disparity range of the scene by using two light fields focused on different depths, effectively solving the problem and allowing to render the scene clearly on both simulated and prototyped tensor display., info:eu-repo/semantics/published

Exploiting alternative visions, such as thermography, and acoustic imaging could play a fundamental role in computer vision by enhancing the accuracy of our current standard digital vision. A machine vision built on all these modalities can see in the visible spectrum, the IR spectrum, and from visualizing sound can capture physical object details that are not visible to the human eye. With all these sensor modalities implemented in one camera, a huge amount of data is produced. Thus, it is imperative to find ways to use the information from all the sensors and fuse them to get the most benefits.Despite all the advances that have been made in the development of multi-modal spectral sensors, some obstacles hinder the fusion of their output details in order to use them as integral images. The major drawback of thermal and acoustic imaging is the lack of potential benefits by their cost effect on sensor resolution, the price of which increases exponentially with increased resolution. Their resolutions are much lower than for standard digital cameras, due to the more expensive sensor technology involved in their production.In this thesis, we aim to improve the beyond visible spectrum machine vision by integrating multi-modal spectral sensors. The emphasis is on transforming the produced images to enhance their resolutions to match the expected human perception, and to improve machine vision accuracy in its applications. We started with an extensive review of the state of the art in the image Super-resolution problem and introduced two datasets, one for thermal images and one for acoustic images. We thus analyzed, as a novel experiment, the applicability of super-resolution models developed for natural images on the thermal image super-resolution problem. The reported results outperformed the traditional up-sampling methods.Image quality is essential, as it refers to the amount of relevant information that an image conveys. Therefore, enhanced image quality can be obtain, Doctorat en Sciences de l'ingénieur et technologie, info:eu-repo/semantics/nonPublished

info:eu-repo/semantics/published

info:eu-repo/semantics/published

info:eu-repo/semantics/published

info:eu-repo/semantics/published

info:eu-repo/semantics/published