Search

Does talking about loss with a romantic partner have salutary personal and relationship effects? Prior evidence reveals the benefits of emotional disclosure in couple relationships, yet disclosure about loss has been overlooked in research on couple communication. Using a novel communication paradigm with young-adult heterosexual romantic partners (

Sustained high levels of blood glucose in type 2 diabetes (T2DM) can have disastrous long-term health consequences. An essential component of clinical interventions for T2DM is monitoring dietary intake to keep plasma glucose levels within an acceptable range. Yet, current techniques to monitor food intake are time intensive and error prone. To address this issue, we are developing techniques to automatically monitor food intake and the composition of those foods using continuous glucose monitors (CGMs). This article presents the results of a clinical study in which participants consumed nine standardized meals with known macronutrients amounts (carbohydrate, protein, and fat) while wearing a CGM. We built a multitask neural network to estimate the macronutrient composition from the CGM signal, and compared it against a baseline linear regression. The best prediction result comes from our proposed neural network, trained with subject-dependent data, as measured by root mean squared relative error and correlation coefficient. These findings suggest that it is possible to estimate macronutrient composition from CGM signals, opening the possibility to develop automatic techniques to track food intake.

Hospital workers often experience burnout due to the demanding job responsibilities and long work hours. Data yielding from ambulatory monitoring combined with machine learning algorithms can afford us a better understanding of the naturalistic processes that contribute to this burnout. Motivated by the challenges related to the accurate tracking of well-being in real-life, prior work has investigated group-specific machine learning (GS-ML) models that are tailored to groups of participants. We examine a novel GS-ML for estimating well-being from real-life multimodal measures collected in situ from hospital workers. In contrast to the majority of prior work that uses pre-determined clustering criteria, we propose an iterative procedure that refines participant clusters based on the representations learned by the GS-ML models. Motivated by prior work that highlights the differential impact of job demands on well-being, we further explore the participant clusters in terms of demography and job-related attributes. Results indicate that the GS-ML models mostly outperform general models in estimating well-being constructs. The GS-ML models further depict different degrees of predictive power for each participant cluster, as distinguished upon age, education, occupational role, and number of supervisees. The observed discrepancies with respect to the GS-ML model decisions are discussed in association with algorithmic bias.

Interpersonal conflict between couples is a significant source of stress with long-lasting effects on partners’ physical and psychological health. Motivated by findings in psychological science, we study how couples with distinct relationship functioning characteristics experience conflict in real life. We propose sub-population specific machine learning models using hierarchical and adaptive learning frameworks to automatically detect interpersonal conflict through the ambulatory monitoring of couples’ physiological signals, audio samples, and linguistic indices. Results indicate that the proposed models outperform a general model learned for the entire population and separate models independently trained on each sub-population, providing a foundation toward personalized health applications.

Digital images are extensively used to increase the accuracy and timeliness of progress reports, safety training, requests for information (RFIs), productivity monitoring, and claims and litigation. While these images can be sorted using date and time tags, the task of searching an image dataset for specific visual content is not trivial. In pattern recognition, generating metadata tags describing image contents (objects, scenes) or appearance (colors, context) is referred to as multi-label image annotation. Given the large number and diversity of construction images, it is desirable to generate image tags automatically. Previous work has applied pattern matching to synthetic images or images obtained from constrained settings. In this paper, we present deep learning (particularly, transfer learning) algorithms to annotate construction imagery from unconstrained real-world settings with high fidelity. We propose convolutional neural network (CNN)-based algorithms which take RGB values as input and output the labels of detected objects. Particularly, we have investigated two categories of classification tasks: single-label classification, i.e., a single class (among multiple predefined classes) is assigned to an image, and multi-label classification, i.e., a set of (one or more) classes is assigned to an image. For both cases, the VGG-16 model, pre-trained on the ImageNet dataset, is trained on construction images retrieved with web mining techniques and labeled by human annotators. Testing the trained model on previously unseen photos yields an accuracy of ~90% for single-label classification and ~85% for multi-label classification, indicating the high sensitivity and specificity of the designed methodology in reliably identifying the contents of construction imagery.

Data collection using unmanned aerial vehicles (UAVs) in construction and heavy civil projects is subject to compliance with strict operational rules and safety regulations. Both the US Fed...

Burgeoning attention has recently been paid to measuring and assessing the effects of urban built environments on pedestrians’ physiological signals (e.g., gait patterns, blood volume pulse...

The term “aesthetic experience” corresponds to the inner state of\ud a person exposed to form and content of artistic objects.\ud Exploring certain aesthetic values of artistic objects, as well as\ud interpreting the aesthetic experience of people when exposed to\ud art can contribute towards understanding (a) art and (b) people’s\ud affective reactions to artwork. Focusing on different types of\ud artistic content, such as movies, music, urban art and other\ud artwork, the goal of this workshop is to enhance the\ud interdisciplinary collaboration between affective computing and\ud aesthetics researchers.

With the increasing trend of older adults living alone, an efficient and nonintrusive way to monitor these individuals’ mental health status is required for early diagnosis of mental diseas...

The present study investigated whether the presence of a romantic partner in daily life is associated with attenuated sympathetic nervous system responses. Additionally, romantic attachment style was tested as a moderator. For one day, 106 heterosexual young adult dating couples wore ambulatory sensors that monitored electrodermal activity (EDA) - an index of sympathetic arousal. Couples reported whether they were together or apart for every hour of the data collection day. Men and women exhibited lower EDA during hours in which their partner was present compared to hours in which they were absent. Additionally, romantic attachment style moderated this association; those who had low anxious attachment showed a stronger attenuating effect of partner presence compared to those with higher anxious attachment. Similarly, those who had low avoidant attachment showed heightened effects of partner presence compared to those with higher avoidant attachment. Romantic partner presence may facilitate everyday health-promoting physiological processes.

Background Although past longitudinal research demonstrates that romantic partners affect one another’s health outcomes, considerably less is known about how romantic experiences “get under the skin” in everyday life. Purpose The current study investigated whether young couples’ naturally occurring feelings of closeness to and annoyance with each other during waking hours were associated with their overnight cardiovascular activity. Methods Participants were 63 heterosexual young adult dating couples (Mage = 23.07). Using ecological momentary assessments, couples reported their hourly feelings of closeness to and annoyance with their partners across 1 day; subsequent overnight heart rate was captured through wearable electrocardiogram biosensors. Actor–partner interdependence models tested whether individuals’ overnight heart rate varied as a function of (a) their own daytime feelings of closeness and annoyance (actor effects) and (b) their partner’s daytime feelings of closeness and annoyance (partner effects) while controlling for daytime heart rate. Results Although young adults’ feelings of romantic closeness and annoyance were unrelated to their own overnight heart rate (i.e., no actor effects), gender-specific partner effects emerged. Young men’s nocturnal heart rate was uniquely predicted by their female partner’s daytime relationship feelings. When women felt closer to their partners during the day, men exhibited lower overnight heart rate. When women felt more annoyed with their partners during the day, men exhibited heightened overnight heart rate. Conclusions The findings illustrate gender-specific links between couple functioning and physiological arousal in the everyday lives of young dating couples, implicating physiological sensitivity to partner experiences as one potential pathway through which relationships affect health.

Workers in various industries are often subject to challenging physical motions that may lead to work-related musculoskeletal disorders (WMSDs). To prevent WMSDs, health and safety organizations have established rules and guidelines that regulate duration and frequency of labor-intensive activities. In this paper, a methodology is introduced to unobtrusively evaluate the ergonomic risk levels caused by overexertion. This is achieved by collecting time-stamped motion data from body-mounted smartphones (i.e., accelerometer, linear accelerometer, and gyroscope signals), automatically detecting workers’ activities through a classification framework, and estimating activity duration and frequency information. This study also investigates various data acquisition and processing settings (e.g., smartphone’s position, calibration, window size, and feature types) through a leave-one-subject-out cross-validation framework. Results indicate that signals collected from arm-mounted smartphone device, when calibrated, can yield accuracy up to 90.2% in the considered 3-class classification task. Further post-processing the output of activity classification yields very accurate estimation of the corresponding ergonomic risk levels. This work contributes to the body of knowledge by expanding the current state in workplace health assessment by designing and testing ubiquitous wearable technology to improve the timeliness and quality of ergonomic-related data collection and analysis.

By monitoring human behavior unobtrusively, mobile sensing technologies have the potential to improve our daily lives. Initial results from a field study demonstrate that such passive technologies can detect a complex psychological state in an uncontrolled, real-life environment. In the web extra at https://youtu.be/n8Ap3Z44ojQ, guest editor Katarzyna Wac interviews authors Adela Timmons and Theodora Chaspari, quality-of-life technology researchers at the University of Southern California.

The COVID-19 pandemic caused by the novel SARS-Coronavirus-2 (n-SARS-CoV-2) has impacted people's lives in unprecedented ways. During the time of the pandemic, social vloggers have used social media to actively share their opinions or experiences in quarantine. This paper collected videos from YouTube to track emotional responses in conversational vlogs and their potential associations with events related to the pandemic. In particular, vlogs uploaded from locations in New York City were analyzed given that this was one of the first epicenters of the pandemic in the United States. We observed some common patterns in vloggers' acoustic and linguistic features across the time span of the quarantine, which is indicative of changes in emotional reactivity. Additionally, we investigated fluctuations of acoustic and linguistic patterns in relation to COVID-19 events in the New York area (e.g. the number of daily new cases, number of deaths, and extension of stay-at-home order and state of emergency). Our results indicate that acoustic features, such as zero-crossing-rate, jitter, and shimmer, can be valuable for analyzing emotional reactivity in social media videos. Our findings further indicate that some of the peaks of the acoustic and linguistic indices align with COVID-19 events, such as the peak in the number of deaths and emergency declaration., Comment: accepted in ACM Multimedia workshop on Media Analytics for Societal Trends (MAST)

Parametric dictionaries can increase the ability of sparse representations to meaningfully capture and interpret the underlying signal information, such as encountered in biomedical problems. Given a mapping function from the atom parameter space to the actual atoms, we propose a sparse Bayesian framework for learning the atom parameters, because of its ability to provide full posterior estimates, take uncertainty into account and generalize on unseen data. Inference is performed with Markov Chain Monte Carlo, that uses block sampling to generate the variables of the Bayesian problem. Since the parameterization of dictionary atoms results in posteriors that cannot be analytically computed, we use a Metropolis–Hastings–within-Gibbs framework, according to which variables with closed-form posteriors are generated with the Gibbs sampler, while the remaining ones with the Metropolis Hastings from appropriate candidate-generating densities. We further show that the corresponding Markov Chain is uniformly ergodic ensuring its convergence to a stationary distribution independently of the initial state. Results on synthetic data and real biomedical signals indicate that our approach offers advantages in terms of signal reconstruction compared to previously proposed Steepest Descent and Equiangular Tight Frame methods. This paper demonstrates the ability of Bayesian learning to generate parametric dictionaries that can reliably represent the exemplar data and provides the foundation towards inferring the entire variable set of the sparse approximation problem for signal denoising, adaptation, and other applications.

Speech-based machine learning (ML) has been heralded as a promising solution for tracking prosodic and spectrotemporal patterns in real-life that are indicative of emotional changes, providing a valuable window into one's cognitive and mental state. Yet, the scarcity of labelled data in ambulatory studies prevents the reliable training of ML models, which usually rely on "data-hungry" distribution-based learning. Leveraging the abundance of labelled speech data from acted emotions, this paper proposes a few-shot learning approach for automatically recognizing emotion in spontaneous speech from a small number of labelled samples. Few-shot learning is implemented via a metric learning approach through a siamese neural network, which models the relative distance between samples rather than relying on learning absolute patterns of the corresponding distributions of each emotion. Results indicate the feasibility of the proposed metric learning in recognizing emotions from spontaneous speech in four datasets, even with a small amount of labelled samples. They further demonstrate superior performance of the proposed metric learning compared to commonly used adaptation methods, including network fine-tuning and adversarial learning. Findings from this work provide a foundation for the ambulatory tracking of human emotion in spontaneous speech contributing to the real-life assessment of mental health degradation., IEEE Transactions on Affective Computing, early access article. doi: 10.1109/TAFFC.2021.3109485

Individuals exposed to aggression and who perpetrate aggression against others show differences in their physiological activation during stress; the goal of the present study is to investigate physiological stress reactivity as a factor contributing to the intergenerational transmission of aggression. To test associations between family-of-origin aggression (FOA), physiological reactivity in daily life, and dating aggression perpetration, we used ecological momentary assessment to monitor fluctuations in young adult (M(age) = 23.1 years) dating couples’ electrodermal activity (EDA) over an entire day and examined how naturally-occurring bouts of annoyance between partners relate to EDA, FOA, and dating aggression perpetration. Dating perpetration was linked to lower general levels of EDA in both men and women, while FOA was linked to lower general levels of EDA in men only. For women, multi-group, multilevel models showed that FOA and dating aggression perpetration moderated the association between feeling annoyed and EDA, such that those with greater FOA and dating aggression perpetration showed greater EDA reactivity during naturally-occurring relationship stress. Furthermore, this pattern of EDA reactivity mediated the link between FOA and dating aggression perpetration in women. These results provide evidence that FOA and dating aggression perpetration are linked to patterns of physiological responsivity in everyday life and suggest that these patterns could be important factors contributing to the intergenerational transmission of aggression.

The inherent diversity of human behavior limits the capabilities of general large-scale machine learning systems, that usually require ample amounts of data to provide robust descriptors of the outcomes of interest. Motivated by this challenge, personalized and population-specific models comprise a promising line of work for representing human behavior, since they can make decisions for clusters of people with common characteristics, reducing the amount of data needed for training. We propose a multi-task learning (MTL) framework for developing population-specific models of interpersonal conflict between couples using ambulatory sensor and mobile data from real-life interactions. The criteria for population clustering include global indices related to couples' relationship quality and attachment style, person-specific factors of partners' positivity, negativity, and stress levels, as well as fluctuating factors of daily emotional arousal obtained from acoustic and physiological indices. Population-specific information is incorporated through a MTL feed-forward neural network (FF-NN), whose first layers capture the common information across all data samples, while its last layers are specific to the unique characteristics of each population. Our results indicate that the proposed MTL FF-NN trained solely on the sensor-based acoustic, linguistic, and physiological modalities provides unweighted and weighted F1-scores of 0.51 and 0.75, respectively, outperforming the corresponding baselines of a single general FF-NN trained on the entire dataset and separate FF-NNs trained on each population cluster individually. These demonstrate the feasibility of such ambulatory systems for detecting real-life behaviors and possibly intervening upon them, and highlights the importance of taking into account the inherent diversity of different populations from the general pool of data.

Biometric sensors and portable devices are being increasingly embedded into our everyday life, creating the need for robust physiological models that efficiently represent, analyze, and interpret the acquired signals. We propose a knowledge-driven method to represent electrodermal activity (EDA), a psychophysiological signal linked to stress, affect, and cognitive processing. We build EDA-specific dictionaries that accurately model both the slow varying tonic part and the signal fluctuations, called skin conductance responses (SCR), and use greedy sparse representation techniques to decompose the signal into a small number of atoms from the dictionary. Quantitative evaluation of our method considers signal reconstruction, compression rate, and information retrieval measures, that capture the ability of the model to incorporate the main signal characteristics, such as SCR occurrences. Compared to previous studies fitting a predetermined structure to the signal, results indicate that our approach provides benefits across all aforementioned criteria. This paper demonstrates the ability of appropriate dictionaries along with sparse decomposition methods to reliably represent EDA signals and provides a foundation for automatic measurement of SCR characteristics and the extraction of meaningful EDA features.

Wearable technology permeates every aspect of our daily life increasing the need of reliable and interpretable models for processing the large amount of biomedical data. We propose the EDA-Gram, a multidimensional fingerprint of the electrodermal activity (EDA) signal, inspired by the widely-used notion of spectrogram. The EDA-Gram is based on the sparse decomposition of EDA from a knowledge-driven set of dictionary atoms. The time axis reflects the analysis frames, the spectral dimension depicts the width of selected dictionary atoms, while intensity values are computed from the atom coefficients. In this way, EDA-Gram incorporates the amplitude and shape of Skin Conductance Responses (SCR), which comprise an essential part of the signal. EDA-Gram is further used as a foundation for signal-specific feature design. Our results indicate that the proposed representation can accentuate fine-grain signal fluctuations, which might not always be apparent through simple visual inspection. Statistical analysis and classification/regression experiments further suggest that the derived features can differentiate between multiple arousal levels and stress-eliciting environments for two datasets.

Crowdsourcing pedestrians’ physiological responses (e.g., electrodermal activity (EDA), gait patterns, and blood volume pulse) offers a unique opportunity for assessing and maintaining built environments in a neighborhood. However, raw physiological signals acquired from naturalistic ambulatory settings cannot effectively capture prominent local patterns in the data stream, since diverse technical challenges (e.g., electrode contact noise and motion artifacts) and confounding factors (e.g., heightened physiology due to the movement) make it difficult to detect significant fine-grain signal fluctuations. Motivated by this issue, this paper proposes a method to identify physical disorders that cause pedestrians physical discomfort and/or emotional distress, by using saliency detection analysis on physiological responses. A bottom-up segmentation approach was used as an unsupervised way to divide each physiological signal into homogeneous segments. A physiological saliency cue (PSC) is proposed to calculate the distinctiveness of physiological responses over each segment in contrast to the remaining segments, and the collective PSC of a physical point of interest is computed across participants. The results, obtained from physiological signals collected from wearable devices, indicate that the suggested saliency detection analysis is effectual in capturing prominent local patterns. Our statistical analysis further indicates that the proposed PSC features can be indicative of physical disorders. The outcome of this research will provide a foundation towards using physiological signals to evaluate built environments, and towards promoting neighborhood walkability, increasing feelings of safety in the urban space, and augmenting residents’ well-being.

Early intervention in individuals with Autism Spectrum Disorder (ASD) can improve core and associated symptoms and facilitate skills that increase social opportunities. However, determining effective intervention success in this population, and the mechanisms that produce it, is currently restricted to observable behavior. The need of therapy assessment metrics beyond traditional behavioral criteria, led to the use of physiological signals for capturing child-therapist internal dynamics during an intervention session. Internal physiological states were measured through Electrodermal Activity (EDA) and modeled in relation to observed self- and co-regulatory behaviors. A common measure of EDA, Skin Conductance Response (SCR), was the primary signal of interest and assumed to form a non-homogeneous Poisson Process whose rate function is determined by observed regulatory behaviors. Through likelihood and residual goodness of fit analysis, statistical tests and classification tasks, our results indicate that SCR changes and observable behavior in child-therapist dyads are temporally associated and the estimated model parameters can be linked to the types of regulation stimuli.

Speech and spoken language cues offer a valuable means to measure and model human behavior. Computational models of speech behavior have the potential to support health care through assistive technologies, informed intervention, and efficient long-term monitoring. The Interspeech 2013 Autism SubChallenge addresses two developmental disorders that manifest in speech: autism spectrum disorders and specific language impairment. We present classification results with an analysis on the development set including a discussion of potential confounds in the data such as recording condition differences. We hence propose study of features within these domains that may inform realistic separability between groups as well as have the potential to be used for behavioral intervention and monitoring. We investigate template-based prosodic and formant modeling as well as goodness of pronunciation modeling, reporting above chance classification accuracies. Index Terms: autism spectrum disorders, intonation, specific language impairment, goodness of pronunciation

Signal-derived measures can provide effective ways towards quantifying human behavior. Verbal Response Latencies (VRLs) of children with Autism Spectrum Disorders (ASD) during conversational interactions are able to convey valuable information about their cognitive and social skills. Motivated by the inherent gap between the external behavior and inner affective state of children with ASD, we study their VRLs in relation to their explicit but also implicit behavioral cues. Explicit cues include the children's language use, while implicit cues are based on physiological signals. Using these cues, we perform classification and regression tasks to predict the duration type (short/long) and value of VRLs of children with ASD while they interacted with an Embodied Conversational Agent (ECA) and their parents. Since parents are active participants in these triadic interactions, we also take into account their linguistic and physiological behaviors. Our results suggest an association between VRLs and these externalized and internalized signal information streams, providing complementary views of the same problem.

The quality of shared enjoyment in interactions is a key aspect related to Autism Spectrum Disorders (ASD). This paper discusses two types of enjoyment: the first refers to humorous events and is associated with one's positive affective state and the second is used to facilitate social interactions between people. These types of shared enjoyment are objectively specified by their proximity to a voiced and unvoiced laughter instance, respectively. The goal of this work is to study the acoustic differences of areas surrounding the two kinds of shared enjoyment instances, called “social zones”, using data collected from children with autism, and their parents, interacting with an Embodied Conversational Agent (ECA). A classification task was performed to predict whether a “social zone” surrounds a voiced or an unvoiced laughter instance. Our results indicate that humorous events are more easily recognized than events acting as social facilitators and that related speech patterns vary more across children compared to other interlocutors.

Impaired social communication and social reciprocity are the primary phenotypic distinctions between autism spectrum disorders (ASD) and other developmental disorders. We investigate quantitative conversational cues in child-psychologist interactions using acoustic-prosodic, turn-taking, and language features. Results indicate the conversational quality degraded for children with higher ASD severity, as the child exhibited difficulties conversing and the psychologist varied her speech and language strategies to engage the child. When interacting with children with increasing ASD severity, the psychologist exhibited higher prosodic variability, increased pausing, more speech, atypical voice quality, and less use of conventional conversational cue such as assents and non-fluencies. Children with increasing ASD severity spoke less, spoke slower, responded later, had more variable prosody, and used personal pronouns, affect language, and fillers less often. We also investigated the predictive power of features from interaction subtasks with varying social demands placed on the child. We found that acoustic prosodic and turn-taking features were more predictive during higher social demand tasks, and that the most predictive features vary with context of interaction. We also observed that psychologist language features may be robust to the amount of speech in a subtask, showing significance even when the child is participating in minimal-speech, low social-demand tasks. Index Terms: autism spectrum disorders, atypical prosody, social reciprocity, turn-taking, language cues