Search

We focus on the problem of producing well-calibrated out-of-distribution (OOD) detectors, in order to enable safe deployment of medical image classifiers. Motivated by the difficulty of curating suitable calibration datasets, synthetic augmentations have become highly prevalent for inlier/outlier specification. While there have been rapid advances in data augmentation techniques, this paper makes a striking finding that the space in which the inliers and outliers are synthesized, in addition to the type of augmentation, plays a critical role in calibrating OOD detectors. Using the popular energy-based OOD detection framework, we find that the optimal protocol is to synthesize latent-space inliers along with diverse pixel-space outliers. Based on empirical studies with multiple medical imaging benchmarks, we demonstrate that our approach consistently leads to superior OOD detection ($15\% - 35\%$ in AUROC) over the state-of-the-art in a variety of open-set recognition settings.

Explanation techniques that synthesize small, interpretable changes to a given image while producing desired changes in the model prediction have become popular for introspecting black-box models. Commonly referred to as counterfactuals, the synthesized explanations are required to contain discernible changes (for easy interpretability) while also being realistic (consistency to the data manifold). In this paper, we focus on the case where we have access only to the trained deep classifier and not the actual training data. While the problem of inverting deep models to synthesize images from the training distribution has been explored, our goal is to develop a deep inversion approach to generate counterfactual explanations for a given query image. Despite their effectiveness in conditional image synthesis, we show that existing deep inversion methods are insufficient for producing meaningful counterfactuals. We propose DISC (Deep Inversion for Synthesizing Counterfactuals) that improves upon deep inversion by utilizing (a) stronger image priors, (b) incorporating a novel manifold consistency objective and (c) adopting a progressive optimization strategy. We find that, in addition to producing visually meaningful explanations, the counterfactuals from DISC are effective at learning classifier decision boundaries and are robust to unknown test-time corruptions., Comment: Neurips 2021

Modern audio source separation techniques rely on optimizing sequence model architectures such as, 1D-CNNs, on mixture recordings to generalize well to unseen mixtures. Specifically, recent focus is on time-domain based architectures such as Wave-U-Net which exploit temporal context by extracting multi-scale features. However, the optimality of the feature extraction process in these architectures has not been well investigated. In this paper, we examine and recommend critical architectural changes that forge an optimal multi-scale feature extraction process. To this end, we replace regular $1-$D convolutions with adaptive dilated convolutions that have innate capability of capturing increased context by using large temporal receptive fields. We also investigate the impact of dense connections on the extraction process that encourage feature reuse and better gradient flow. The dense connections between the downsampling and upsampling paths of a U-Net architecture capture multi-resolution information leading to improved temporal modelling. We evaluate the proposed approaches on the MUSDB test dataset. In addition to providing an improved performance over the state-of-the-art, we also provide insights on the impact of different architectural choices on complex data-driven solutions for source separation.

With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art results in several clinical prediction tasks. Despite the success of RNNs, its sequential nature prohibits parallelized computing, thus making it inefficient particularly when processing long sequences. Recently, architectures which are based solely on attention mechanisms have shown remarkable success in transduction tasks in NLP, while being computationally superior. In this paper, for the first time, we utilize attention models for clinical time-series modeling, thereby dispensing recurrence entirely. We develop the \textit{SAnD} (Simply Attend and Diagnose) architecture, which employs a masked, self-attention mechanism, and uses positional encoding and dense interpolation strategies for incorporating temporal order. Furthermore, we develop a multi-task variant of \textit{SAnD} to jointly infer models with multiple diagnosis tasks. Using the recent MIMIC-III benchmark datasets, we demonstrate that the proposed approach achieves state-of-the-art performance in all tasks, outperforming LSTM models and classical baselines with hand-engineered features., Comment: AAAI 2018

Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning, and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling and hyper-parameter optimization. Existing solutions attempt to adaptively trade-off between global exploration and local exploitation, wherein the initial exploratory sample is critical to their success. While discrepancy-based samples have become the de facto approach for exploration, results from computer graphics suggest that coverage-based designs, e.g. Poisson disk sampling, can be a superior alternative. In order to successfully adopt coverage-based sample designs to ML applications, which were originally developed for 2-d image analysis, we propose fundamental advances by constructing a parameterized family of designs with provably improved coverage characteristics, and by developing algorithms for effective sample synthesis. Using experiments in sample mining and hyper-parameter optimization for supervised learning, we show that our approach consistently outperforms existing exploratory sampling methods in both blind exploration, and sequential search with Bayesian optimization.

A quantile is defined as a value below which random draws from a given distribution falls with a given probability. In a centralized setting where the cumulative distribution function (CDF) is unknown, the empirical CDF (ECDF) can be used to estimate such quantiles after aggregating the data. In a fully distributed sensor network, however, it is challenging to estimate quantiles. This is because each sensor node observes local measurement data with limited storage and data transmission power which make it difficult to obtain the global ECDF. This paper proposes consensus-based quantile estimation for such a distributed network. The states of the proposed algorithm are recursively updated with two-steps at each iteration: one is a local update based on the measurement data and the current state, and the other is averaging the updated states with neighboring nodes. We consider the realistic case of communication links between nodes being corrupted by independent random noise. It is shown that the estimated state sequence is asymptotically unbiased and converges toward the sample quantile in the mean-square sense. The two step-size sequences corresponding to the averaging and local update steps result in a mixed-time scale algorithm with proper decay rates in order to achieve convergence. We also provide applications to distributed estimation of trimmed mean, computation of median, maximum, or minimum values and identification of outliers through simulation.

Integrating sensing and machine learning is important in elevating precision in several Internet of Things (IoT) and mobile applications. In our Electrical Engineering classes, we have begun developing self-contained modules to train students in this area. We focus specifically in developing modules in machine learning including pre-processing, feature extraction and classification. We have also embedded in these modules software to provide hands-on training. In this paper, we describe our efforts to develop an online simulation environment that will support web-based laboratories for training undergraduate students from Electrical Engineering and other disciplines in sensors and machine learning. We also present our efforts to enable students to visualize and understand the inner workings of various machine learning algorithms along with descriptions of their performance with several types of synthetic and sensor data.

This study proposes a unifying framework for m-Health video communication systems that provides for the joint optimization of video quality, bitrate demands, and encoding time. The framework is video modality and infrastructure independent and facilitates adaptation to the best available encoding mode that satisfies underlying technology and application imposed constraints. The scalability of the proposed algorithm is demonstrated using different HEVC encoding configurations and realistic modelling of 802.11× wireless infrastructure for emergency scenery and response videos. Extensive experimentation shows that a jointly optimal solution in the encoding time, bitrate, and video quality space is feasible. © 2015 IEEE. 2015-November 173 176 Sponsors: Conference code: 116805 Cited By :4

Sparse representations with learned dictionaries have been successful in several image analysis applications. In this paper, we propose and analyze the framework of ensemble sparse models, and demonstrate their utility in image restoration and unsupervised clustering. The proposed ensemble model approximates the data as a linear combination of approximations from multiple \textit{weak} sparse models. Theoretical analysis of the ensemble model reveals that even in the worst-case, the ensemble can perform better than any of its constituent individual models. The dictionaries corresponding to the individual sparse models are obtained using either random example selection or boosted approaches. Boosted approaches learn one dictionary per round such that the dictionary learned in a particular round is optimized for the training examples having high reconstruction error in the previous round. Results with compressed recovery show that the ensemble representations lead to a better performance compared to using a single dictionary obtained with the conventional alternating minimization approach. The proposed ensemble models are also used for single image superresolution, and we show that they perform comparably to the recent approaches. In unsupervised clustering, experiments show that the proposed model performs better than baseline approaches in several standard datasets.

Java-DSP is a freely accessible web-based software, primarily used in signal processing education and research. In this paper, we present Java-DSP modules that have been developed for the study and analysis of the MPEG-1 Layer III algorithm. We have embedded JLayer1.0, an open source MP3 library, to Java-DSP and developed an intuitive interface to expose undergraduate and graduate students to the several modules in the encoding/decoding process. The Java-DSP MP3 decoder block is an interactive function which can be used to examine the characteristics and visualize outputs of different modules in the algorithm. Some of the important functions incorporated in the proposed interface include the analysis of the hybrid filter bank, polyphase filters and the window switching based on perceptual criteria. The MP3 algorithm represents a compelling framework for teaching certain aspects in DSP. We are using this module to introduce filter banks and windowing to undergraduate students. © 2011 IEEE. 168 173 Sponsors: Inst. Electr. Electron. Eng. (IEEE) Signal Process. Soc. Conference code: 84554 Cited By :1

Advances in video compression, network technologies, and computer technologies have contributed to the rapid growth of mobile health (m-health) systems and services. Wide deployment of such systems and services is expected in the near future, and it's foreseen that they will soon be incorporated in daily clinical practice. This study focuses in describing the basic components of an end-to-end wireless medical video telemedicine system, providing a brief overview of the recent advances in the field, while it also highlights future trends in the design of telemedicine systems that are diagnostically drivenv. © 2010 IEEE. 1045 1048 Conference code: 83008 Cited By :12

A relationship between the Fisher information and the characteristic function is established with the help of two inequalities. A necessary and sufficient condition for equality is found. These results are used to determine the asymptotic efficiency of a distributed estimation algorithm that uses constant modulus transmissions over Gaussian multiple access channels. The loss in efficiency of the distributed estimation scheme relative to the centralized approach is quantified for different sensing noise distributions. It is shown that the distributed estimator does not incur an efficiency loss if and only if the sensing noise distribution is Gaussian., Comment: Submitted to IEEE Transactions on Information Theory; 8 pages; 3 figures

This NSF Phase 3 CCLI multidisciplinary project consists of a collaborative implementation and national dissemination effort that involves Arizona State University (ASU), Johns Hopkins University (JHU), Prairie View A&M University, University of Washington-Bothell (UWB), Rose-Hulman Institute of Technology, University of New Mexico (UNM), and the University of Cyprus (UCY). The project involves significant educational technology innovations and software extensions that will enable the online software J-DSP to be used in multiple disciplines including digital signal processing, earth systems, renewable energy, biologically-inspired sensors, and arts and media. Problems addressed include the delivery of technologyenhanced laboratory experiences to undergraduate students using novel Java tools and the requirement of a broad assessment of these practices at several universities. © 2009 IEEE. Sponsors: American Society for Engineering Society, ASEE IEEE IEEE Computer Society ATM San Antonio College, SAC Conference code: 79825

Sponsors: BOEING Conference code: 77079

The intima-media thickness (IMT) of the common carotid artery (CCA) is widely used as an early indicator of cardiovascular disease (CVD). It was proposed but not thoroughly investigated that the media layer (ML), its composition and texture, may be indicative for identifying the risk of stroke and differentiating between patients of high and low risk. In this study we investigate the usefulness of texture analysis of the ML of the CCA. The study was performed on 100 longitudinal ultrasound images acquired from asymptomatic subjects at risk of atherosclerosis. The images were separated into three different age groups, namely below 50, 50 to 60, and above 60 years old. A total of 61 different texture features were extracted from the intima-media complex (IMC), ML and the intima layer (IL). The IMC and ML were segmented manually by a neurovascular expert and automatically by a snakes segmentation system. It was shown that texture features extracted from the IL, ML and IMC are significantly different (mean, gray scale median (GSM), standard deviation, contrast, difference variance, periodicity) and that some of them can be associated with the increase (difference variance, entropy) or decrease (GSM) of patient's age. It was also shown that the GSM of the ML falls linearly with increasing ML thickness (MLT) and with increasing age. Further research on more subjects is required for estimating other features that may provide information for patients at risk of stroke. Conference code: 77628