Search

We combine concept-based neural networks with generative, flow-based classifiers into a novel, intrinsically explainable, exactly invertible approach to supervised learning. Prototypical neural networks, a type of concept-based neural network, represent an exciting way forward in realizing human-comprehensible machine learning without concept annotations, but a human-machine semantic gap continues to haunt current approaches. We find that reliance on indirect interpretation functions for prototypical explanations imposes a severe limit on prototypes' informative power. From this, we posit that invertibly learning prototypes as distributions over the latent space provides more robust, expressive, and interpretable modeling. We propose one such model, called ProtoFlow, by composing a normalizing flow with Gaussian mixture models. ProtoFlow (1) sets a new state-of-the-art in joint generative and predictive modeling and (2) achieves predictive performance comparable to existing prototypical neural networks while enabling richer interpretation., Comment: Accepted to ECCV'24. Code available at https://github.com/craymichael/ProtoFlow

Surging interest in deep learning from high-stakes domains has precipitated concern over the inscrutable nature of black box neural networks. Explainable AI (XAI) research has led to an abundance of explanation algorithms for these black boxes. Such post hoc explainers produce human-comprehensible explanations, however, their fidelity with respect to the model is not well understood - explanation evaluation remains one of the most challenging issues in XAI. In this paper, we ask a targeted but important question: can popular feature-additive explainers (e.g., LIME, SHAP, SHAPR, MAPLE, and PDP) explain feature-additive predictors? Herein, we evaluate such explainers on ground truth that is analytically derived from the additive structure of a model. We demonstrate the efficacy of our approach in understanding these explainers applied to symbolic expressions, neural networks, and generalized additive models on thousands of synthetic and several real-world tasks. Our results suggest that all explainers eventually fail to correctly attribute the importance of features, especially when a decision-making process involves feature interactions., Comment: Accepted to NeurIPS Workshop XAI in Action: Past, Present, and Future Applications. arXiv admin note: text overlap with arXiv:2106.08376

Machine learning has achieved remarkable success over the past couple of decades, often attributed to a combination of algorithmic innovations and the availability of high-quality data available at scale. However, a third critical component is the fine-tuning of hyperparameters, which plays a pivotal role in achieving optimal model performance. Despite its significance, hyperparameter optimization (HPO) remains a challenging task for several reasons. Many HPO techniques rely on naive search methods or assume that the loss function is smooth and continuous, which may not always be the case. Traditional methods, like grid search and Bayesian optimization, often struggle to quickly adapt and efficiently search the loss landscape. Grid search is computationally expensive, while Bayesian optimization can be slow to prime. Since the search space for HPO is frequently high-dimensional and non-convex, it is often challenging to efficiently find a global minimum. Moreover, optimal hyperparameters can be sensitive to the specific dataset or task, further complicating the search process. To address these issues, we propose a new hyperparameter optimization method, HomOpt, using a data-driven approach based on a generalized additive model (GAM) surrogate combined with homotopy optimization. This strategy augments established optimization methodologies to boost the performance and effectiveness of any given method with faster convergence to the optimum on continuous, discrete, and categorical domain spaces. We compare the effectiveness of HomOpt applied to multiple optimization techniques (e.g., Random Search, TPE, Bayes, and SMAC) showing improved objective performance on many standardized machine learning benchmarks and challenging open-set recognition tasks.

We implemented, optimized and fully tested over multiple runs a superconducting Josephson junction fabrication process tailored for the integrated digital circuits that are used for control and readout of superconducting qubits operating at millikelvin temperatures. This process was optimized for highly energy efficient single flux quantum (ERSFQ) circuits with the critical currents reduced by factor of ~10 as compared to those operated at 4.2 K. Specifically, it implemented Josephson junctions with 10 uA unit critical current fabricated with a 10 uA/um2 critical current density. In order to circumvent the substantial size increase of the SFQ circuit inductors, we employed a NbN high kinetic inductance layer (HKIL) with a 8.5 pH/sq sheet inductance. Similarly, to maintain the small size of junction resistive shunts, we used a non-superconducting PdAu alloy with a 4.0 ohm/sq sheet resistance. For integration with quantum circuits in a multi-chip module, 5 and 10 um height bump processes were also optimized. To keep the fabrication process in check, we developed and thoroughly tested a comprehensive Process Control Monitor chip set., Comment: 10 pages, 5 figures, 1 table

Image manipulation detection algorithms designed to identify local anomalies often rely on the manipulated regions being ``sufficiently'' different from the rest of the non-tampered regions in the image. However, such anomalies might not be easily identifiable in high-quality manipulations, and their use is often based on the assumption that certain image phenomena are associated with the use of specific editing tools. This makes the task of manipulation detection hard in and of itself, with state-of-the-art detectors only being able to detect a limited number of manipulation types. More importantly, in cases where the anomaly assumption does not hold, the detection of false positives in otherwise non-manipulated images becomes a serious problem. To understand the current state of manipulation detection, we present an in-depth analysis of deep learning-based and learning-free methods, assessing their performance on different benchmark datasets containing tampered and non-tampered samples. We provide a comprehensive study of their suitability for detecting different manipulations as well as their robustness when presented with non-tampered data. Furthermore, we propose a novel deep learning-based pre-processing technique that accentuates the anomalies present in manipulated regions to make them more identifiable by a variety of manipulation detection methods. To this end, we introduce an anomaly enhancement loss that, when used with a residual architecture, improves the performance of different detection algorithms with a minimal introduction of false positives on the non-manipulated data. Lastly, we introduce an open-source manipulation detection toolkit comprising a number of standard detection algorithms.

Augmented reality and other photo editing filters are popular methods used to modify faces online. Considering the important role of facial perception in communication, how do we perceive this increasing number of modified faces? In this paper we present the results of six surveys that measure familiarity with different styles of facial filters, perceived strangeness of faces edited with different filters, and ability to discern whether images are filtered. Our results demonstrate that faces modified with more traditional face filters are perceived similarly to unmodified faces, and faces filtered with augmented reality filters are perceived differently from unmodified faces. We discuss possible explanations for these results, including a societal adjustment to traditional photo editing techniques or the inherent differences in the different types of filters. We conclude with a discussion of how to build online spaces more responsibly based on our results.

For many real-world decision-making problems subject to uncertainty, it may be essential to deal with multiple and often conflicting objectives while taking the decision-makers' risk preferences into account. Conditional value-at-risk (CVaR) is a widely applied risk measure to address risk-averseness of the decision-makers. In this paper, we use the subset-based polyhedral representation of the CVaR to reformulate the bi-objective two-stage stochastic facility location problem presented in Nazemi et al. (2021). We propose an approximate cutting-plane method to deal with this more computationally challenging subset-based formulation. Then, the cutting plane method is embedded into the epsilon-constraint method, the balanced-box method, and a recently developed matheuristic method to address the bi-objective nature of the problem. Our computational results show the effectiveness of the proposed method. Finally, we discuss how incorporating an approximation of the subset-based polyhedral formulation affects the obtained solutions., Comment: arXiv admin note: substantial text overlap with arXiv:2007.07767

Magnesium and its alloys are the most investigated materials for solid-state hydrogen storage in the form of metal hydrides, but there are still unresolved problems with the kinetics and thermodynamics of hydrogenation and dehydrogenation of this group of materials. Severe plastic deformation (SPD) methods, such as equal-channel angular pressing (ECAP), high-pressure torsion (HPT), intensive rolling and fast forging, have been widely used to enhance the activation, air resistance, and hydrogenation/dehydrogenation kinetics of Mg-based hydrogen storage materials by introducing ultrafine/nanoscale grains and crystal lattice defects. These severely deformed materials, particularly in the presence of alloying additives or second-phase nanoparticles, can show not only fast hydrogen absorption/desorption kinetics but also good cycling stability. It was shown that some materials that are apparently inert to hydrogen can absorb hydrogen after SPD processing. Moreover, the SPD methods were effectively used for hydrogen binding-energy engineering and synthesizing new magnesium alloys with low thermodynamic stability for reversible low/room-temperature hydrogen storage, such as nanoglasses, high-entropy alloys, and metastable phases including the high-pressure {\gamma}-MgH2 polymorph. This article reviews recent advances in the development of Mg-based hydrogen storage materials by SPD processing and discusses their potential in future applications.

Managing novelty in perception-based human activity recognition (HAR) is critical in realistic settings to improve task performance over time and ensure solution generalization outside of prior seen samples. Novelty manifests in HAR as unseen samples, activities, objects, environments, and sensor changes, among other ways. Novelty may be task-relevant, such as a new class or new features, or task-irrelevant resulting in nuisance novelty, such as never before seen noise, blur, or distorted video recordings. To perform HAR optimally, algorithmic solutions must be tolerant to nuisance novelty, and learn over time in the face of novelty. This paper 1) formalizes the definition of novelty in HAR building upon the prior definition of novelty in classification tasks, 2) proposes an incremental open world learning (OWL) protocol and applies it to the Kinetics datasets to generate a new benchmark KOWL-718, 3) analyzes the performance of current state-of-the-art HAR models when novelty is introduced over time, 4) provides a containerized and packaged pipeline for reproducing the OWL protocol and for modifying for any future updates to Kinetics. The experimental analysis includes an ablation study of how the different models perform under various conditions as annotated by Kinetics-AVA. The protocol as an algorithm for reproducing experiments using the KOWL-718 benchmark will be publicly released with code and containers at https://github.com/prijatelj/human-activity-recognition-in-an-open-world. The code may be used to analyze different annotations and subsets of the Kinetics datasets in an incremental open world fashion, as well as be extended as further updates to Kinetics are released., Comment: 39 pages, 16 figures, 3 tables, Pre-print submitted to JAIR

Recent trends in the machine learning community show that models with fidelity toward human perceptual measurements perform strongly on vision tasks. Likewise, human behavioral measurements have been used to regularize model performance. But can we transfer latent knowledge gained from this across different learning objectives? In this work, we introduce PERCEP-TL (Perceptual Transfer Learning), a methodology for improving transfer learning with the regularization power of psychophysical labels in models. We demonstrate which models are affected the most by perceptual transfer learning and find that models with high behavioral fidelity -- including vision transformers -- improve the most from this regularization by as much as 1.9\% Top@1 accuracy points. These findings suggest that biologically inspired learning agents can benefit from human behavioral measurements as regularizers and psychophysical learned representations can be transferred to independent evaluation tasks., Comment: 8 pages, 5 figures, student paper

For the last decade, convolutional neural networks (CNNs) have vastly superseded their predecessors in nearly all vision tasks in artificial intelligence, including object recognition. However, despite abundant advancements, they continue to pale in comparison to biological vision. This chasm has prompted the development of biologically-inspired models that have attempted to mimic the human visual system, primarily at a neural level, which is evaluated using standard dataset benchmarks. However, more work is needed to understand how these models perceive the visual world. This article proposes a state-of-the-art procedure that generates a new metric, Psychophysical-Score, which is grounded in visual psychophysics and is capable of reliably estimating perceptual responses across numerous models -- representing a large range in complexity and biological inspiration. We perform the procedure on twelve models that vary in degree of biological inspiration and complexity, we compare the results against the aggregated results of 2,390 Amazon Mechanical Turk workers who together provided ~2.7 million perceptual responses. Each model's Psychophysical-Score is compared against the state-of-the-art neural activity-based metric, Brain-Score. Our study indicates that models with a high correlation to human perceptual behavior also have a high correlation with the corresponding neural activity.