Search

Huge advancements have been made over the years in terms of modern image-sensing hardware and visual computing algorithms (e.g., computer vision, image processing, and computational photography). However, to this day, there still exists a current gap between the hardware and software design in an imaging system, which silos one research domain from another. Bridging this gap is the key to unlocking new visual computing capabilities for end applications in commercial photography, industrial inspection, and robotics. In this survey, we explore existing works in the literature that can be leveraged to replace conventional hardware components in an imaging system with software for enhanced reconfigurability. As a result, the user can program the image sensor in a way best suited to the end application. We refer to this as software-defined imaging (SDI), where image sensor behavior can be altered by the system software depending on the user’s needs. The scope of our survey covers imaging systems for single-image capture, multi-image, and burst photography, as well as video. We review works related to the sensor primitives, image signal processor (ISP) pipeline, computer architecture, and operating system elements of the SDI stack. Finally, we outline the infrastructure and resources for SDI systems, and we also discuss possible future research directions for the field.

Ion Channel sensors have several applications including DNA sequencing, biothreat detection, and medical applications. Ion-channel sensors mimic the selective transport mechanism of cell membranes and can detect a wide range of analytes at the molecule level. Analytes are sensed through changes in signal patterns. Papers in the literature have described different methods for ion channel signal analysis. In this paper, we describe a series of new graphical tools for ion channel signal analysis which can be used for research and education. The paper focuses on the utility of this tools in biosensor classes. Teaching signal processing and machine learning for ion channel sensors is challenging because of the multidisciplinary content and student backgrounds which include physics, chemistry, biology and engineering. The paper describes graphical ion channel analysis tools developed for an on-line simulation environment called J-DSP. The tools are integrated and assessed in a graduate bio-sensor course through computer laboratory exercises.

Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning (ML), and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling, and hyperparameter optimization. Existing solutions attempt to adaptively trade off between global exploration and local exploitation, in which the initial exploratory sample is critical to their success. While discrepancy-based samples have become the de facto approach for exploration, results from computer graphics suggest that coverage-based designs, e.g., Poisson disk sampling, can be a superior alternative. In order to successfully adopt coverage-based sample designs to ML applications, which were originally developed for 2-D image analysis, we propose fundamental advances by constructing a parameterized family of designs with provably improved coverage characteristics and developing algorithms for effective sample synthesis. Using experiments in sample mining and hyperparameter optimization for supervised learning, we show that our approach consistently outperforms the existing exploratory sampling methods in both blind exploration and sequential search with Bayesian optimization.

Modern data analysis pipelines are becoming increasingly complex due to the presence of multiview information sources. While graphs are effective in modeling complex relationships, in many scenarios, a single graph is rarely sufficient to succinctly represent all interactions, and hence, multilayered graphs have become popular. Though this leads to richer representations, extending solutions from the single-graph case is not straightforward. Consequently, there is a strong need for novel solutions to solve classical problems, such as node classification, in the multilayered case. In this article, we consider the problem of semisupervised learning with multilayered graphs. Though deep network embeddings, e.g., DeepWalk, are widely adopted for community discovery, we argue that feature learning with random node attributes, using graph neural networks, can be more effective. To this end, we propose to use attention models for effective feature learning and develop two novel architectures, GrAMME-SG and GrAMME-Fusion, that exploit the interlayer dependences for building multilayered graph embeddings. Using empirical studies on several benchmark data sets, we evaluate the proposed approaches and demonstrate significant performance improvements in comparison with the state-of-the-art network embedding strategies. The results also show that using simple random features is an effective choice, even in cases where explicit node attributes are not available.

Building highly nonlinear and nonparametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task-specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this paper, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the deep kernel machine optimization framework, that creates an ensemble of dense embeddings using Nyström kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pretrained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques.

A cyber physical system approach for a utility-scale photovoltaic (PV) array monitoring and control is presented in this article. This system consists of sensors that capture voltage, current, temperature, and irradiance parameters for each solar panel which are then used to detect, predict and control the performance of the array. More specifically the article describes a customized machine-learning method for remote fault detection and a computer vision framework for cloud movement prediction. In addition, a consensus-based distributed approach is proposed for resource optimization, and a secure authentication protocol that can detect intrusions and cyber threats is presented. The proposed system leverages video analysis of skyline imagery that is used along with other measured parameters to reconfigure the solar panel connection topology and optimize power output. Additional benefits of this cyber physical approach are associated with the control of inverter transients. Preliminary results demonstrate improved efficiency and robustness in renewable energy systems using advanced cyber enabled sensory analysis and fusion devices and algorithms.

Sparse representations using learned dictionaries are being increasingly used with success in several data processing and machine learning applications. The increasing need for learning sparse models in large-scale applications motivates the development of efficient, robust, and provably good dictionary learning algorithms. Algorithmic stability and generalizability are desirable characteristics for dictionary learning algorithms that aim to build global dictionaries, which can efficiently model any test data similar to the training samples. In this paper, we propose an algorithm to learn dictionaries for sparse representations from large scale data, and prove that the proposed learning algorithm is stable and generalizable asymptotically. The algorithm employs a 1-D subspace clustering procedure, the $K$ -hyperline clustering, to learn a hierarchical dictionary with multiple levels. We also propose an information-theoretic scheme to estimate the number of atoms needed in each level of learning and develop an ensemble approach to learn robust dictionaries. Using the proposed dictionaries, the sparse code for novel test data can be computed using a low-complexity pursuit procedure. We demonstrate the stability and generalization characteristics of the proposed algorithm using simulations. We also evaluate the utility of the multilevel dictionaries in compressed recovery and subspace learning applications.

ABSTRACTModern data collection in the Earth Sciences has propelled the need for understanding signal processing and time-series analysis techniques. However, there is an educational disconnect in the lack of instruction of time-series analysis techniques in many Earth Science academic departments. Furthermore, there are no platform-independent freeware tools available for teaching Earth signals analysis. In order to address these issues, we developed the Java-Digital Signal Processing/Earth Systems Edition (J-DSP/ESE), a platform-independent software tool that can be integrated with the Earth Science university curriculum for signal processing and analysis instruction. This tool has an intuitive block-based programming environment, and students do not need be familiar with any programming language to use it. In order to demonstrate the utility of this software in an instructional environment, we developed three tutorials related to basic signal processing, and signal analysis of modern and past climate change. The tutorials use published data to examine the relationship between 20th century atmospheric CO2and global temperature, and the relationship between ocean temperature and solar radiation over the past 300,000 y. The tutorials were administered in two workshops with different communities of students in Earth Science and electrical engineering. Our technical assessments show that the students were able to comprehend basic signal processing and analysis of climate signals using J-DSP/ESE. In the subjective assessments, a vast majority of students stated that the software was easy to learn and use, and that it significantly improved their understanding of climate change.

Single-walled carbon nanotubes (SWNTs) have been used extensively for sensor fabrication due to their high surface-to-volume ratio, nanosized structure, and interesting electronic property. Lack of selectivity is a major limitation for SWNT-based sensors. However, surface modification of SWNTs with a suitable molecular recognition system can enhance the sensitivity. On the other hand, porphyrins have been widely investigated as functional materials for chemical sensor fabrication due to their several unique and interesting physicochemical properties. Structural differences between free-base and metal-substituted porphyrins make them suitable for improving the selectivity of sensors. However, their poor conductivity is an impediment in the fabrication of prophyrin-based chemiresistor sensors. The present attempt is to resolve these issues by combining free-base and metallo-porphyrins with SWNTs to fabricate SWNT–porphyrin hybrid chemiresistor sensor arrays for monitoring volatile organic compounds in the air. Differences in sensing performance were noticed for porphyrins with different functional groups and with different central metal atoms. The mechanistic study for acetone sensing was done using field-effect transistor measurements and revealed that the sensing mechanism of the ruthenium octaethyl porphyrin hybrid device was governed by the electrostatic gating effect, whereas the iron tetraphenyl porphyrin hybrid device was governed by electrostatic gating and Schottky barrier modulation in combination. Further, the recorded electronic responses for all hybrid sensors were analyzed using a pattern-recognition analysis tool. The pattern-recognition analysis confirmed a definite pattern in response for different hybrid materials and could efficiently differentiate analytes from one another. This discriminating capability of the hybrid nanosensor devices opens up the possibilities for further development of highly dense nanosensor arrays with suitable porphyrins for E-nose applications.

Many modern speech bandwidth extension techniques predict the high-frequency band based on features extracted from the lower band. While this method works for certain types of speech, problems arise when the correlation between the low and the high bands is not sufficient for adequate prediction. These situations require that additional high-band information is sent to the decoder. This overhead information, however, can be cleverly quantized using human auditory system models. In this paper, we propose a novel speech compression method that relies on bandwidth extension. The novelty of the technique lies in an elaborate perceptual model that determines a quantization scheme for wideband recovery and synthesis. Furthermore, a source/filter bandwidth extension algorithm based on spectral spline fitting is proposed. Results reveal that the proposed system improves the quality of narrowband speech while performing at a lower bitrate. When compared to other wideband speech coding schemes, the proposed algorithms provide comparable speech quality at a lower bitrate.

In this paper, it is shown how an automatic recognition algorithm, based on hidden Markov models (HMM), can benefit by properly utilizing findings from perceptual experiments on nasals. Perceptual studies on nasal consonants have shown that both nasal murmurs and formant transitions are important in the identification of place of articulation. Thus both acoustic segments bordering the nasal release were incorporated into this HMM‐based system. A 7% improvement in alveolar recognition was obtained by explicitly modeling the vowel‐nasal transition segments. Further overall improvement (6%) was realized by making the HMM recognizer ‘‘focus’’ more on the vowel‐nasal transition segments bordering the nasal release, and less on the nasal murmur and vowel portion of the /&Vegr; m/ and /&Vegr; n/ syllables. An overall average [m]–[n] recognition of 95% was obtained when testing this technique on 60 speakers outside the training set.

There have been many reports on the recognition of vowels by cochlear implant patients fitted with various devices. Most of these reports, however, used a single male and/or female talker to test vowel recognition. This paper presents results on the recognition of vowels produced by a large number of men, women, and children [Hillenbrand etal., J. Acoust. Soc. Am. 97, 3099–3111 (1995)]. The aim of this study was to investigate why certain groups of speakers, e.g., children, and certain vowels are harder for the cochlear implant patients to identify. A total of 5 Ineraid patients who had been fit with continuous interleaved sampling (CIS) signal processors were used in this study. Patients’ performance on vowel recognition ranged from 54% to 84%. The sex and age of the talker (that is, the talker’s vocal tract size) seemed to have a significant effect on the performance for some of the patients, and in particular the ‘‘poor’’ patients. Individual patient’s results on vowel recognition, and a detailed analysis of the vowel confusions will be presented.

This paper presents a new method for the selection of sinusoidal components for use in compact representations of narrowband audio. The method consists of ranking and selecting the most perceptually relevant sinusoids. The idea behind the method is to maximize the matching between the auditory excitation pattern associated with the original signal and the corresponding auditory excitation pattern associated with the modeled signal that is being represented by a small set of sinusoidal parameters. The proposed component-selection methodology is shown to outperform the maximum signal-to-mask ratio selection strategy in terms of subjective quality.

The automatic recognition of stop and nasal consonants is known to be a difficult recognition task. This paper presents various techniques that can be used to improve the discrimination of stop and nasal consonants. An improved spectral representation for stop consonants is proposed, which unlike other feature representations, emphasizes the mid‐to‐high regions of the spectrum. A subspace projection approach, which is used as a preprocessing step in a hidden Markov model based system, is also proposed for improved nasal discrimination. This approach finds a transformation matrix which maps the original nasal observation onto a subspace such that the ‘‘distance’’ between the nasals is maximized on the subspace. Two statistical distance measures are investigated for finding the transformation matrix, namely the divergence and the Bhattacharyya measures. Results on stop and nasal consonant recognition will be presented using the subspace approach and the improved spectral stop representation.