Search

Addressing the critical shortage of mental health resources for effective screening, diagnosis, and treatment remains a significant challenge. This scarcity underscores the need for innovative solutions, particularly in enhancing the accessibility and efficacy of therapeutic support. Embodied agents with advanced interactive capabilities emerge as a promising and cost-effective supplement to traditional caregiving methods. Crucial to these agents' effectiveness is their ability to simulate non-verbal behaviors, like backchannels, that are pivotal in establishing rapport and understanding in therapeutic contexts but remain under-explored. To improve the rapport-building capabilities of embodied agents we annotated backchannel smiles in videos of intimate face-to-face conversations over topics such as mental health, illness, and relationships. We hypothesized that both speaker and listener behaviors affect the duration and intensity of backchannel smiles. Using cues from speech prosody and language along with the demographics of the speaker and listener, we found them to contain significant predictors of the intensity of backchannel smiles. Based on our findings, we introduce backchannel smile production in embodied agents as a generation problem. Our attention-based generative model suggests that listener information offers performance improvements over the baseline speaker-centric generation approach. Conditioned generation using the significant predictors of smile intensity provides statistically significant improvements in empirical measures of generation quality. Our user study by transferring generated smiles to an embodied agent suggests that agent with backchannel smiles is perceived to be more human-like and is an attractive alternative for non-personal conversations over agent without backchannel smiles., Comment: Accepted to the Machine Learning for Cognitive and Mental Health Workshop at AAAI 2024

Quantum subspace methods (QSMs) are a class of quantum computing algorithms where the time-independent Schrodinger equation for a quantum system is projected onto a subspace of the underlying Hilbert space. This projection transforms the Schrodinger equation into an eigenvalue problem determined by measurements carried out on a quantum device. The eigenvalue problem is then solved on a classical computer, yielding approximations to ground- and excited-state energies and wavefunctions. QSMs are examples of hybrid quantum-classical methods, where a quantum device supported by classical computational resources is employed to tackle a problem. QSMs are rapidly gaining traction as a strategy to simulate electronic wavefunctions on quantum computers, and thus their design, development, and application is a key research field at the interface between quantum computation and electronic structure. In this review, we provide a self-contained introduction to QSMs, with emphasis on their application to the electronic structure of molecules. We present the theoretical foundations and applications of QSMs, and we discuss their implementation on quantum hardware, illustrating the impact of noise on their performance., Comment: 34 pages, 11 figures

Personalized prediction is a machine learning approach that predicts a person's future observations based on their past labeled observations and is typically used for sequential tasks, e.g., to predict daily mood ratings. When making personalized predictions, a model can combine two types of trends: (a) trends shared across people, i.e., person-generic trends, such as being happier on weekends, and (b) unique trends for each person, i.e., person-specific trends, such as a stressful weekly meeting. Mixed effect models are popular statistical models to study both trends by combining person-generic and person-specific parameters. Though linear mixed effect models are gaining popularity in machine learning by integrating them with neural networks, these integrations are currently limited to linear person-specific parameters: ruling out nonlinear person-specific trends. In this paper, we propose Neural Mixed Effect (NME) models to optimize nonlinear person-specific parameters anywhere in a neural network in a scalable manner. NME combines the efficiency of neural network optimization with nonlinear mixed effects modeling. Empirically, we observe that NME improves performance across six unimodal and multimodal datasets, including a smartphone dataset to predict daily mood and a mother-adolescent dataset to predict affective state sequences where half the mothers experience at least moderate symptoms of depression. Furthermore, we evaluate NME for two model architectures, including for neural conditional random fields (CRF) to predict affective state sequences where the CRF learns nonlinear person-specific temporal transitions between affective states. Analysis of these person-specific transitions on the mother-adolescent dataset shows interpretable trends related to the mother's depression symptoms., Comment: camera-ready version

Many-body entangled quantum spin systems exhibit emergent phenomena such as topological quantum spin liquids with distinct excitation spectra accessed in inelastic neutron scattering (INS) experiments. Here we simulate the dynamics of a quantum spin dimer, the basic quantum unit of emergent many-body spin systems. While canonical Trotterization methods require deep circuits precluding long time-scale simulations, we demonstrate 'direct' Resource-Efficient Fast-forwarding (REFF) measurements with short-depth circuits that can be used to capture longer time dynamics on quantum hardware. The temporal evolution of the 2-spin correlation coefficients enabled the calculation of the dynamical structure factor $S(\mathbf{Q},\omega)$ - the key component of the neutron scattering cross-section. We simulate the triplet gap and the triplet splitting of the quantum dimer with sufficient fidelity to compare to experimental neutron data. Our results on current circuit hardware pave an important avenue to benchmark, or even predict, the outputs of the costly INS experiments., Comment: 24 pages, 3 tables, 16 figures, main text and supplementary materials

Variational approaches, such as variational Monte Carlo (VMC) or the variational quantum eigensolver (VQE), are powerful techniques to tackle the ground-state many-electron problem. Often, the family of variational states is not invariant under the reparametrization of the Hamiltonian by single-particle basis transformations. As a consequence, the representability of the ground-state wave function by the variational ansatz strongly dependents on the choice of the single-particle basis. In this manuscript we study the joint optimization of the single-particle basis, together with the variational state in the VMC (with neural quantum states) and VQE (with hardware-efficient circuits) approaches. We show that the joint optimization of the single-particle basis with the variational state parameters yields significant improvements in the expressive power and optimization landscape in a variety of chemistry and condensed matter systems. We also realize the first active-space calculation using neural quantum states, where the single-particle basis transformations are applied to all of the orbitals in the basis set.

We propose a class of randomized quantum Krylov diagonalization (rQKD) algorithms capable of solving the eigenstate estimation problem with modest quantum resource requirements. Compared to previous real-time evolution quantum Krylov subspace methods, our approach expresses the time evolution operator, $e^{-i\hat{H} \tau}$, as a linear combination of unitaries and subsequently uses a stochastic sampling procedure to reduce circuit depth requirements. While our methodology applies to any Hamiltonian with fast-forwardable subcomponents, we focus on its application to the explicitly double-factorized electronic-structure Hamiltonian. To demonstrate the potential of the proposed rQKD algorithm, we provide numerical benchmarks for a variety of molecular systems with circuit-based statevector simulators, achieving ground state energy errors of less than 1~kcal~mol$^{-1}$ with circuit depths orders of magnitude shallower than those required for low-rank deterministic Trotter-Suzuki decompositions.

We demonstrate a method that merges the quantum filter diagonalization (QFD) approach for hybrid quantum/classical solution of the time-independent electronic Schr\"odinger equation with a low-rank double factorization (DF) approach for the representation of the electronic Hamiltonian. In particular, we explore the use of sparse "compressed" double factorization (C-DF) truncation of the Hamiltonian within the time-propagation elements of QFD, while retaining a similarly compressed but numerically converged double-factorized representation of the Hamiltonian for the operator expectation values needed in the QFD quantum matrix elements. Together with significant circuit reduction optimizations and number-preserving post-selection/echo-sequencing error mitigation strategies, the method is found to provide accurate predictions for low-lying eigenspectra in a number of representative molecular systems, while requiring reasonably short circuit depths and modest measurement costs. The method is demonstrated by experiments on noise-free simulators, decoherence- and shot-noise including simulators, and real quantum hardware.

Standard machine learning approaches require centralizing the users' data in one computer or a shared database, which raises data privacy and confidentiality concerns. Therefore, limiting central access is important, especially in healthcare settings, where data regulations are strict. A potential approach to tackling this is Federated Learning (FL), which enables multiple parties to collaboratively learn a shared prediction model by using parameters of locally trained models while keeping raw training data locally. In the context of AI-assisted pain-monitoring, we wish to enable confidentiality-preserving and unobtrusive pain estimation for long-term pain-monitoring and reduce the burden on the nursing staff who perform frequent routine check-ups. To this end, we propose a novel Personalized Federated Deep Learning (PFDL) approach for pain estimation from face images. PFDL performs collaborative training of a deep model, implemented using a lightweight CNN architecture, across different clients (i.e., subjects) without sharing their face images. Instead of sharing all parameters of the model, as in standard FL, PFDL retains the last layer locally (used to personalize the pain estimates). This (i) adds another layer of data confidentiality, making it difficult for an adversary to infer pain levels of the target subject, while (ii) personalizing the pain estimation to each subject through local parameter tuning. We show using a publicly available dataset of face videos of pain (UNBC-McMaster Shoulder Pain Database), that PFDL performs comparably or better than the standard centralized and FL algorithms, while further enhancing data privacy. This, has the potential to improve traditional pain monitoring by making it more secure, computationally efficient, and scalable to a large number of individuals (e.g., for in-home pain monitoring), providing timely and unobtrusive pain measurement., Comment: 12 pages, 6 figures

Critical obstacles in training classifiers to detect facial actions are the limited sizes of annotated video databases and the relatively low frequencies of occurrence of many actions. To address these problems, we propose an approach that makes use of facial expression generation. Our approach reconstructs the 3D shape of the face from each video frame, aligns the 3D mesh to a canonical view, and then trains a GAN-based network to synthesize novel images with facial action units of interest. To evaluate this approach, a deep neural network was trained on two separate datasets: One network was trained on video of synthesized facial expressions generated from FERA17; the other network was trained on unaltered video from the same database. Both networks used the same train and validation partitions and were tested on the test partition of actual video from FERA17. The network trained on synthesized facial expressions outperformed the one trained on actual facial expressions and surpassed current state-of-the-art approaches.

Quantum Gibbs state sampling algorithms generally suffer from either scaling exponentially with system size or requiring specific knowledge of spectral properties \textit{a priori}. Also, these algorithms require a large overhead of bath or scratch/ancilla qubits. We propose a method, termed the minimal effective Gibbs ansatz (MEGA), which uses a quantum computer to determine a minimal ensemble of pure states that accurately reproduce thermal averages of typical observables. This technique employs properties of correlation functions that can be split into a lesser and greater part; here, we primarily focus on single-particle Green's functions. When properly measured, these correlation functions provide a simple test to indicate how close a given pure state or ensemble of pure states are to providing accurate thermal expectation values. Further, we show that when properties such as the eigenstate thermalization hypothesis hold, this approach leads to accurate results with a sparse ensemble of pure states; sometimes only one suffices. We illustrate the ansatz using exact diagonalization simulations on small clusters for the Fermi-Hubbard and Hubbard-like models. Even if MEGA becomes as computationally complex as other Gibbs state samplers, it still gains an advantage due to its ease of implementation without any \textit{a priori} information about the Hamiltonian and in the efficient allocation of available qubits by eliminating bath qubits and using a minimal number of ancilla., Comment: 12 pages

The field of Automatic Facial Expression Analysis has grown rapidly in recent years. However, despite progress in new approaches as well as benchmarking efforts, most evaluations still focus on either posed expressions, near-frontal recordings, or both. This makes it hard to tell how existing expression recognition approaches perform under conditions where faces appear in a wide range of poses (or camera views), displaying ecologically valid expressions. The main obstacle for assessing this is the availability of suitable data, and the challenge proposed here addresses this limitation. The FG 2017 Facial Expression Recognition and Analysis challenge (FERA 2017) extends FERA 2015 to the estimation of Action Units occurrence and intensity under different camera views. In this paper we present the third challenge in automatic recognition of facial expressions, to be held in conjunction with the 12th IEEE conference on Face and Gesture Recognition, May 2017, in Washington, United States. Two sub-challenges are defined: the detection of AU occurrence, and the estimation of AU intensity. In this work we outline the evaluation protocol, the data used, and the results of a baseline method for both sub-challenges., Comment: FERA 2017 Baseline Paper

Facial action units (AUs) are essential to decode human facial expressions. Researchers have focused on training AU detectors with a variety of features and classifiers. However, several issues remain. These are spatial representation, temporal modeling, and AU correlation. Unlike most studies that tackle these issues separately, we propose a hybrid network architecture to jointly address them. Specifically, spatial representations are extracted by a Convolutional Neural Network (CNN), which, as analyzed in this paper, is able to reduce person-specific biases caused by hand-crafted features (eg, SIFT and Gabor). To model temporal dependencies, Long Short-Term Memory (LSTMs) are stacked on top of these representations, regardless of the lengths of input videos. The outputs of CNNs and LSTMs are further aggregated into a fusion network to produce per-frame predictions of 12 AUs. Our network naturally addresses the three issues, and leads to superior performance compared to existing methods that consider these issues independently. Extensive experiments were conducted on two large spontaneous datasets, GFT and BP4D, containing more than 400,000 frames coded with 12 AUs. On both datasets, we report significant improvement over a standard multi-label CNN and feature-based state-of-the-art. Finally, we provide visualization of the learned AU models, which, to our best knowledge, reveal how machines see facial AUs for the first time.

Facial expressions are an important way through which humans interact socially. Building a system capable of automatically recognizing facial expressions from images and video has been an intense field of study in recent years. Interpreting such expressions remains challenging and much research is needed about the way they relate to human affect. This paper presents a general overview of automatic RGB, 3D, thermal and multimodal facial expression analysis. We define a new taxonomy for the field, encompassing all steps from face detection to facial expression recognition, and describe and classify the state of the art methods accordingly. We also present the important datasets and the bench-marking of most influential methods. We conclude with a general discussion about trends, important questions and future lines of research.

Estimation of facial expressions, as spatio-temporal processes, can take advantage of kernel methods if one considers facial landmark positions and their motion in 3D space. We applied support vector classification with kernels derived from dynamic time-warping similarity measures. We achieved over 99% accuracy - measured by area under ROC curve - using only the 'motion pattern' of the PCA compressed representation of the marker point vector, the so-called shape parameters. Beyond the classification of full motion patterns, several expressions were recognized with over 90% accuracy in as few as 5-6 frames from their onset, about 200 milliseconds., Comment: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, Portland, Oregon, 28 June 2013 (accepted)