Search

Transfer learning (TL) is an increasingly popular approach to training deep learning (DL) models that leverages the knowledge gained by training a foundation model on diverse, large-scale datasets for use on downstream tasks where less domain- or task-specific data is available. The literature is rich with TL techniques and applications; however, the bulk of the research makes use of deterministic DL models which are often uncalibrated and lack the ability to communicate a measure of epistemic (model) uncertainty in prediction. Unlike their deterministic counterparts, Bayesian DL (BDL) models are often well-calibrated, provide access to epistemic uncertainty for a prediction, and are capable of achieving competitive predictive performance. In this study, we propose variational inference pre-trained audio neural networks (VI-PANNs). VI-PANNs are a variational inference variant of the popular ResNet-54 architecture which are pre-trained on AudioSet, a large-scale audio event detection dataset. We evaluate the quality of the resulting uncertainty when transferring knowledge from VI-PANNs to other downstream acoustic classification tasks using the ESC-50, UrbanSound8K, and DCASE2013 datasets. We demonstrate, for the first time, that it is possible to transfer calibrated uncertainty information along with knowledge from upstream tasks to enhance a model's capability to perform downstream tasks., Comment: Published in IEEE Access

Labeling errors in datasets are common, if not systematic, in practice. They naturally arise in a variety of contexts-human labeling, noisy labeling, and weak labeling (i.e., image classification), for example. This presents a persistent and pervasive stress on machine learning practice. In particular, neural network (NN) architectures can withstand minor amounts of dataset imperfection with traditional countermeasures such as regularization, data augmentation, and batch normalization. However, major dataset imperfections often prove insurmountable. We propose and study the implementation of Rockafellian Relaxation (RR), a new loss reweighting, architecture-independent methodology, for neural network training. Experiments indicate RR can enhance standard neural network methods to achieve robust performance across classification tasks in computer vision and natural language processing (sentiment analysis). We find that RR can mitigate the effects of dataset corruption due to both (heavy) labeling error and/or adversarial perturbation, demonstrating effectiveness across a variety of data domains and machine learning tasks.

In practice, optimization models are often prone to unavoidable inaccuracies due to dubious assumptions and corrupted data. Traditionally, this placed special emphasis on risk-based and robust formulations, and their focus on ``conservative" decisions. We develop, in contrast, an ``optimistic" framework based on Rockafellian relaxations in which optimization is conducted not only over the original decision space but also jointly with a choice of model perturbation. The framework enables us to address challenging problems with ambiguous probability distributions from the areas of two-stage stochastic optimization without relatively complete recourse, probability functions lacking continuity properties, expectation constraints, and outlier analysis. We are also able to circumvent the fundamental difficulty in stochastic optimization that convergence of distributions fails to guarantee convergence of expectations. The framework centers on the novel concepts of exact and limit-exact Rockafellians, with interpretations of ``negative'' regularization emerging in certain settings. We illustrate the role of Phi-divergence, examine rates of convergence under changing distributions, and explore extensions to first-order optimality conditions. The main development is free of assumptions about convexity, smoothness, and even continuity of objective functions. Numerical results in the setting of computer vision and text analytics with label noise illustrate the framework.

The explosive growth of user-generated video presents opportunities and challenges. The videos may possess valuable information that was once unavailable. On the other hand, the information may be buried or difficult to access with traditional methods. Automatic keyframe video summarization technologies exist that attempt to address this problem. A keyframe summary can often be viewed quicker than the underlying video. However, a theoretical framework for objectively assessing keyframe summary quality has been absent. The work presented here bridges this gap by presenting a semantically high-level, stakeholder-centered evaluation framework. The framework can capture common stakeholder concerns and quantitatively measure the extent to which they are satisfied by keyframe summaries.With this framework, keyframe summary stakeholders and algorithm developers can now identify when success has been achieved. This work also develops a number of novel keyframe summarization algorithms and shows, using the evaluation framework, that they outperform baseline methods.

The explosive growth of user-generated video presents opportunities and challenges. The videos may possess valuable information that was once unavailable. On the other hand, the information may be buried or difficult to access with traditional methods. Automatic keyframe video summarization technologies exist that attempt to address this problem. A keyframe summary can often be viewed quicker than the underlying video. However, a theoretical framework for objectively assessing keyframe summary quality has been absent. The work presented here bridges this gap by presenting a semantically high-level, stakeholder-centered evaluation frame-work. The framework can capture common stakeholder concerns and quantitatively measure the extent to which they are satisfied by keyframe summaries.With this framework, keyframe summary stakeholders and algorithm developers can now identify when success has been achieved. This work also develops a number of novel keyframe summarization algorithms and shows, using the evaluation framework, that they outperform baseline methods.

The explosive growth of user-generated video presents opportunities and challenges. The videos may possess valuable information that was once unavailable. On the other hand, the information may be buried or difficult to access with traditional methods. Automatic keyframe video summarization technologies exist that attempt to address this problem. A keyframe summary can often be viewed quicker than the underlying video. However, a theoretical framework for objectively assessing keyframe summary quality has been absent. The work presented here bridges this gap by presenting a semantically high-level, stakeholder-centered evaluation frame-work. The framework can capture common stakeholder concerns and quantitatively measure the extent to which they are satisfied by keyframe summaries.With this framework, keyframe summary stakeholders and algorithm developers can now identify when success has been achieved. This work also develops a number of novel keyframe summarization algorithms and shows, using the evaluation framework, that they outperform baseline methods., http://archive.org/details/automatickeyfram1094542614, Lieutenant, United States Navy, Approved for public release; distribution is unlimited.