Search

We consider the problem of reconstructing superimposed temperature and wind flow fields from acoustic measurements. A new technique based solely on acoustic wave propagation is presented. In contrast to the usual straight ray assumption, a bent ray model is considered in order to achieve higher accuracy. We also develop a lab size experiment for temperature estimation.

We study a new image device called gigavision camera or the gigapixel digital film camera. The main differences between a conventional and a gigavision camera are that the pixels of the latter are binary and orders of magnitude smaller. A gigavision camera can be built using standard memory chip technology, where each memory bit is designed to be light sensitive. A conventional gray level image can be obtained from the binary gigavision image by low-pass filtering and sampling. The main advantage of the gigavision camera is that its response is non-linear and similar to a logarithmic function, which makes it suitable for acquiring high dynamic range scenes. The larger the number of binary pixels considered, the higher the dynamic range of the gigavision camera will be.

Starting from a sample of a given size, texture synthesis algorithms are used to create larger texture images. A good algorithm produces synthesized textures that are pixelwise different but perceptually indistinguishable from the original image. The sample image should be chosen ensuring that it contains a number of pattern repetitions sufficient to produce valuable synthesis results. Since textures can be characterized by patterns of different dimensions, this must be done in an adaptive way. In this article, we propose a method that automatically adapts the sample size for natural textures synthesis, according to the different patterns dimensions. The method is based on the measure of the spatial dependence between the texture pixel values. This measure is used to estimate the size of the smallest texture window that is still perceived as texture by human observers. The sample size is determined from this measure by applying a multiplicative factor that depends on the algorithm used for synthesis. We perform a simple subjective experiment to estimate this factor for three different synthesis algorithms. We show that the measure of spatial dependence based on the correlation between pixels performs well when it is used to adapt the sample size.

Dynamic textures are sequences of images showing temporal regularity; they can be found in videos representing smoke, flames, flowing water, or moving grass, for instance. Recently, one method based on linear dynamic system theory was proposed to synthesize dynamic textures. Textures are represented as the output of a linear dynamic system and synthesis is reduced to matrix multiplication operations. In this report, we study the problem of implementing this method using fixed-point arithmetic, as required in many portable devices, such as palms or mobile phones. This is done by jointly evaluating the effect of model coefficient quantization and of fixed-point precision arithmetic, which are both source of errors with respect to the floating-point implementation. Our analysis shows that using model coefficients quantized using less bit permits to obtain visual synthesis results comparable to the more expensive floating-point implementation, with the advantage of requiring far less buffer memory space and permitting to perform faster synthesis.

Dynamic textures are sequences of images showing temporal regularity. Examples can be found in videos representing smoke, flames, ocean waves, wind-shaken forests, etc. Dynamic texture modelling and synthesis has usually been done considering RGB color images. In this paper, we analyze the use of different color encodings, which permit to model luminance and chrominance information separately. We find that this separation is more appropriate, since it takes advantage of the spatial and temporal characteristics of the color channels and leads to more flexible and compact representations. We show that compared to RGB, similar synthesis performance can be achieved using YCbCr or Lab color encodings, using half of the model coefficients and less computational power.

In many applications, the sampling frequency is limited by the physical characteristics of the components: the pixel pitch, the rate of the A/D converter, etc. A low-pass filter is then often applied before the sampling operation to avoid aliasing. However, when multiple copies are available, it is possible to use the information that is inherently present in the aliasing to reconstruct a higher resolution signal. If the different copies have unknown relative offsets, this is a non-linear problem in the offsets and the signal coefficients. They are not easily separable in the set of equations describing the super-resolution problem. Thus, we perform joint registration and reconstruction from multiple unregistered sets of samples. We give a mathematical formulation for the problem when there are M sets of N samples of a signal that is described by L expansion coefficients. We prove that the solution of the registration and reconstruction problem is generically unique if MN >= L+M-1. We describe two subspace-based methods to compute this solution. Their complexity is analyzed, and some heuristic methods are proposed. Finally, some numerical simulation results on one and two-dimensional signals are given to show the performance of these methods.

In previously known imaging devices as in still and motion cameras, for example, image sensor signal response typically is linear as a function of intensity of incident light. Desirably, however, akin to the response of the human eye, response is sought to be nonlinear and, more particularly, essentially logarithmic. Preferred nonlinearity is realized in image sensor devices of the invention upon severely limiting the number of pixel states, combined with clustering of pixels into what may be termed as super-pixels.

We have studied a camera with a very large number of binary pixels referred to as the gigavision camera or the gigapixel digital film camera. Potential advantages of this new camera design include improved dynamic range, thanks to its logarithmic sensor response curve, and reduced exposure time in low light conditions, due to its highly sensitive photon detection mechanism. We use maximum likelihood estimator (MLE) to reconstruct a high quality conventional image from the binary sensor measurements of the gigavision camera. We prove that when the threshold T is "1", the negative log-likelihood function is a convex function. Therefore, optimal solution can be achieved using convex optimization. Base on filter bank techniques, fast algorithms are given for computing the gradient and the multiplication of a vector and Hessian matrix of the negative log-likelihood function. We show that with a minor change, our algorithm also works for estimating conventional images from multiple binary images. Numerical experiments with synthetic 1-D signals and images verify the effectiveness and quality of the proposed algorithm. Experimental results also show that estimation performance can be improved by increasing the oversampling factor or the number of binary images.

The sampling of the soundfield using circular microphone arrays is studied. We give a new representation of the soundfield sampled on such an array using the two-dimensional Fourier transform. We then show how we can reconstruct the soundfield at all possible po- sitions on the array from a finite number of measurements. We give the necessary azimuthal angular sampling frequency necessary to reconstruct the data up to some temporal frequency. This new tech- nique is applied to interpolate Head Related Transfer Functions (HRTFs) where we also derive an angular sampling theorem.

A system for automated annotation of images and videos points a mobile device towards an object of interest, such as a building or landscape scenery, for the device to display an image of the scene with an annotation for the object. An annotation can include names, historical information, and links to databases of images, videos, and audio files. Different techniques can be used for determining positional placement of annotations, and, by using multiple techniques, positioning can be made more precise and reliable. The level of detail of annotation information can be adjusted according to the precision of the techniques used. Required computations can be distributed in an annotation system including mobile devices, servers and an inter-connecting network, allowing for tailoring of annotated images to mobile devices of different levels of complexity. A trade-off can be taken into account between precision of annotation and communication cost, delay and/or power consumption. An annotation database can be updated in a self-organizing way. Public information as available on the web can be converted to annotation data.

The sampling and interpolation of a sound field in two and three dimensions along a circle is discussed. The Fourier domain representation of the sound field is used, and an angular sampling theorem is developed for the sampling of the sound field along a circle. Based on these results, HRTF sampling and interpolation are discussed. This method achieves very precise interpolation in terms of mean square error. However, these results are only possible if very finely spaced HRTF measurements are available. A method is proposed to improve interpolation results when the HRTF measurements are more coarsely spaced than dictated by the angular Nyquist theorem. The proposed method interpolates the HRTFs in a subband domain. In subbands where small angular aliasing occurs, the previous method is applied. In the other subbands, interpolation is carried out in a complex temporal envelope domain to avoid aliasing. Simulations with models and measured data show that the proposed algorithm performs significantly better than previous methods in a mean square error sense.

The trends in the design of image sensors are to build sensors with low noise, high sensitivity, high dynamic range, and small pixel size. How can we benefit from pixels with small size and high sensitivity? In this dissertation, we study a new image sensor that is reminiscent of traditional photographic film. Each pixel in the sensor has a binary response, giving only a one-bit quantized measurement of the local light intensity. The response function of the image sensor is non-linear and similar to a logarithmic function, which makes the sensor suitable for high dynamic range imaging. We first formulate the oversampled binary sensing scheme as a parameter estimation problem based on quantized Poisson statistics. We show that, with a single-photon quantization threshold and large oversampling factors, the Cramér-Rao lower bound (CRLB) of the estimation variance approaches that of an ideal unquantized sensor, that is, as if there were no quantization in the sensor measurements. Furthermore, the CRLB is shown to be asymptotically achievable by the maximum likelihood estimator (MLE). By showing that the log-likelihood function is concave, we guarantee the global optimality of iterative algorithms in finding the MLE. We study the performance of the oversampled binary sensing scheme in presence of dark current noise. The noise model is an additive Bernoulli noise with a known parameter, and the noise only flips the binary output from "0" to "1". We show that the binary sensor is quite robust with respect to noise and its dynamic range is only slightly reduced. The binary sensor first benefits from the increasing of the oversampling factor and then suffers in term of dynamic range. We again use the MLE to estimate the light intensity. When the threshold is a single photon, we show that the log-likelihood function is still concave. Thus, the global optimality can be achieved. But for thresholds larger than "1", this property does not hold true. By proving that when the light intensity is piecewise-constant, the likelihood function is a strictly pseudoconcave function, we guarantee to find the optimal solution of the MLE using iterative algorithms for arbitrary thresholds. For the general linear light field model, the log-likelihood function is not even quasiconcave when thresholds are larger than "1". In this circumstance, we find an initial solution by approximating the light intensity field with a piecewise-constant model, and then we use Newton's method to refine the estimation using the exact model. We then examine one of the most important parameters in the binary sensor, i.e., the threshold used to generate binary values. We prove the intuitive result that large thresholds achieve better estimation performance for strong light intensities, while small thresholds work better for low light intensities. To make a binary sensor that works in a larger range of light intensities, we propose to design a threshold array containing multiple thresholds instead of a single threshold for the binary sensing. The criterion is to minimize the average CRLB which is a good approximation of the mean squared error (MSE). The performance analysis on the new binary sensor verifies the effectiveness of our design. Again, the MLE is used for reconstructing the light intensity field from the binary measurements. By showing that the log-likelihood function is concave for arbitrary threshold arrays, we ensure that the iterative algorithms can find the optimal solution of the MLE. Finally, we study the reconstruction problem for the binary image sensor under a generalized piecewise-constant light intensity field model, which is quite useful when parameters like oversampling factors are unknown. We directly estimate light exposure values, i.e., the number of photons hitting on each pixel. We assume that the light exposure values are piecewise-constant and we use the MLE for the reconstruction. This optimization problem is solved by iteratively working out two subproblems. The first one is to find the optimal light exposure value for each segment, given the optimal segmentation of the binary measurements. The second one is to find the optimal segmentation of the binary measurements given the optimal light exposure values for each segment. Several algorithms are provided for solving this optimization problem. Dynamic programming can obtain the optimal solution for 1-D signals, but the computation is quite heavy. To reduce the burden of computation, we propose a greedy algorithm and a method based on pruning of binary trees or quadtrees.

In nature, textures can be found everywhere. Texture, together with color and shape, represents the fundamental characteristics of objects. Texture conveys an idea of repetition of a certain structure, which is not limited only to the visual domain. We can refer to the texture of a sound, a fabric, or a certain wine. In image processing, a static texture is defined as an image showing spatial stationarity, while a dynamic texture is a sequence of images showing temporal stationarity. Texture synthesis is the process of producing artificial textures starting from a given texture sample. In this thesis, we consider texture synthesis starting from compact representations. For static, this reduces to define the smallest portion of a texture that, when used for synthesis, produces a synthetic texture indistinguishable from the original image. We designed a psychophysical experiment to quantify this size for a collection of textures and two synthesis algorithms: the parametric algorithm by Portilla and Simoncelli and the procedural algorithm by Nealen and Alexa. The results of the experiment show that for pre-attentive vision condition the size of the compact texture does not depend on the algorithm used for synthesis but is a characteristic of the texture. We evaluated if an objective function, based on spatial correlation between pixels, is able to predict the texture size necessary for synthesis. We found that a statistical measure based on random walks correlated well with the results of the subjective experiment. In the case of dynamic textures, the term "compact" applies to the model size that is used for synthesis. We propose a dynamic texture analysis that is able to obtain a more compact model starting from the linear model of Soatto and Doretto. Current methods perform a dimension reduction of the data by applying the SVD to the video frames unfolded into column vectors. This permits only to exploit the temporal correlation. We avoid the unfolding operations and decompose the signal directly using a multidimensional decomposition known as Higher-Order SVD (HOSVD). Chromatic components are exploited more efficiently by combining the HOSVD decomposition with the YCbCr color encoding for the input data. Tests show that the combined model has far fewer parameters than models derived with other algorithms, for the same visual quality and approximately the same computational synthesis cost. The HOSVD-based model was used in two applications for portable devices where memory and computational power are limited. We created a video effect for webcams and smartphones where a dynamic texture is synthesized and blended to the video in real time.

This thesis is a study of the spatial evolution of the sound field. We first present an analysis of the sound field along different geometries. In the case of the sound field studied along a line in a room, we describe a two-dimensional function characterizing the sound field along space and time. Calculating the Fourier transform of this function leads to a spectrum having a butterfly shape. The spectrum is shown to be almost bandlimited along the spatial frequency dimension, which allows the interpolation of the sound field at any position along the line when a sufficient number of microphones is present. Using this Fourier representation of the sound field, we develop a spatial sampling theorem trading off quality of reconstruction with spatial sampling frequency. The study is generalized for planes of microphones and microphones located in three dimensions. The presented theory is compared to simulations and real measurements of room impulse responses. We describe a similar theory for circular arrays of microphones or loudspeakers. Application of this theory is presented for the study of the angular sampling of head-related transfer functions (HRTFs). As a result, we show that to reconstruct HRTFs at any possible angle in the horizontal plane, an angular spacing of 5 degrees is necessary for HRTFs sampled at 44.1 kHz. Because recording that many HRTFs is not easy, we develop interpolation techniques to achieve acceptable results for databases containing two or four times fewer HRTFs. The technique is based on the decomposition of the HRTFs in their carrier and complex envelopes. With the Fourier representation of the sound field, it is then shown how one can correctly obtain all room impulse responses measured along a trajectory when using a moving loudspeaker or microphone. The presented method permits the reconstruction of the room impulse responses at any position along the trajectory, provided that the speed satisfies a given relation. The maximal speed is shown to be dependent on the maximal frequency emitted and the radius of the circle. This method takes into account the Doppler effect present when one element is moving in the scenario. It is then shown that the measurement of HRTFs in the horizontal plane can be achieved in less than one second. In the last part, we model spatio-temporal channel impulse responses between a fixed source and a moving receiver. The trajectory followed by the moving element is modeled as a continuous autoregressive process. The presented model is simple and versatile. It allows the generation of random trajectories with a controlled smoothness. Application of this study can be found in the modeling of acoustic channels for acoustic echo cancellation or of time-varying multipath electromagnetic channels used in mobile wireless communications.

Acoustic tomography aims at recovering the unknown parameters that describe a field of interest by studying the physical characteristics of sound propagating through the considered field. The tomographic approach is appealing in that it is non-invasive and allows to obtain a significantly larger amount of data compared to the classical one-sensor one-measurement setup. It has, however, two major drawbacks which may limit its applicability in a practical setting: the methods by which the tomographic data are acquired and then converted to the field values are computationally intensive and often ill-conditioned. This thesis specifically addresses these two shortcomings by proposing novel acoustic tomography algorithms for signal acquisition and field reconstruction. The first part of our exposition deals with some theoretical aspects of the tomographic sampling problems and associated reconstruction schemes for scalar and vector tomography. We show that the classical time-of-flight measurements are not sufficient for full vector field reconstruction. As a solution, an additional set of measurements is proposed. The main advantage of the proposed set is that it can be directly computed from acoustic measurements. It thus avoids the need for extra measuring devices. We then describe three novel reconstruction methods that are conceptually quite different. The first one is based on quadratic optimization and does not require any a priori information. The second method builds upon the notion of sparsity in order to increase the reconstruction accuracy when little data is available. The third approach views tomographic reconstruction as a parametric estimation problem and solves it using recent sampling results on non-bandlimited signals. The proposed methods are compared and their respective advantages are outlined. The second part of our work is dedicated to the application of the proposed algorithms to three practical problems: breast cancer detection, thermal therapy monitoring, and temperature monitoring in the atmosphere. We address the problem of breast cancer detection by computing a map of sound speed in breast tissue. A noteworthy contribution of this thesis is the development of a signal processing technique that significantly reduces the artifacts that arise in very inhomogeneous and absorbent tissue. Temperature monitoring during thermal therapies is then considered. We show how some of our algorithms allow for an increased spatial resolution and propose ways to reduce the computational complexity. Finally, we demonstrate the feasibility of tomographic temperature monitoring in the atmosphere using a custom-built laboratory-scale experiment. In particular, we discuss various practical aspects of time-of-flight measurement using cheap, off-the-shelf sensing devices.