Search

Over 50 million people undergo surgeries each year in the United States, with over 70% of them filling opioid prescriptions within one week of the surgery. Due to the highly addictive nature of these opiates, a post-surgical window is a crucial time for pain management to ensure accurate prescription of opioids. Drug prescription nowadays relies primarily on self-reported pain levels to determine the frequency and dosage of pain drug. Patient pain self-reports are, however, influenced by subjective pain tolerance, memories of past painful episodes, current context, and the patient's integrity in reporting their pain level. Therefore, objective measures of pain are needed to better inform pain management. This paper explores a wearable system, named Painometry, which objectively quantifies users' pain perception based-on multiple physiological signals and facial expressions of pain. We propose a sensing technique, called sweep impedance profiling (SIP), to capture the movement of the facial muscle corrugator supercilii, one of the important physiological expressions of pain. We deploy SIP together with other biosignals, including electroencephalography (EEG), photoplethysmogram (PPG), and galvanic skin response (GSR) for pain quantification. From the anatomical and physiological correlations of pain with these signals, we designed Painometry, a multimodality sensing system, which can accurately quantify different levels of pain safely. We prototyped Painometry by building a custom hardware, firmware, and associated software. Our evaluations use the prototype on 23 subjects, which corresponds to 8832 data points from 276 minutes of an IRB-approved experimental pain-inducing protocol. Using leave-one-out cross-validation to estimate performance on unseen data shows 89.5% and 76.7% accuracy of quantification under 3 and 4 pain states, respectively.

Background Studies that use ecological momentary assessments (EMAs) or wearable sensors to track numerous attributes, such as physical activity, sleep, and heart rate, can benefit from reductions in missing data. Maximizing compliance is one method of reducing missing data to increase the return on the heavy investment of time and money into large-scale studies. Objective This paper aims to identify the extent to which compliance can be prospectively predicted from individual attributes and initial compliance. Methods We instrumented 757 information workers with fitness trackers for 1 year and conducted EMAs in the first 56 days of study participation as part of an observational study. Their compliance with the EMA and fitness tracker wearing protocols was analyzed. Overall, 31 individual characteristics (eg, demographics and personalities) and behavioral variables (eg, early compliance and study portal use) were considered, and 14 variables were selected to create beta regression models for predicting compliance with EMAs 56 days out and wearable compliance 1 year out. We surveyed study participation and correlated the results with compliance. Results Our modeling indicates that 16% and 25% of the variance in EMA compliance and wearable compliance, respectively, could be explained through a survey of demographics and personality in a held-out sample. The likelihood of higher EMA and wearable compliance was associated with being older (EMA: odds ratio [OR] 1.02, 95% CI 1.00-1.03; wearable: OR 1.02, 95% CI 1.01-1.04), speaking English as a first language (EMA: OR 1.38, 95% CI 1.05-1.80; wearable: OR 1.39, 95% CI 1.05-1.85), having had a wearable before joining the study (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.50, 95% CI 1.23-1.83), and exhibiting conscientiousness (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.34, 95% CI 1.14-1.58). Compliance was negatively associated with exhibiting extraversion (EMA: OR 0.74, 95% CI 0.64-0.85; wearable: OR 0.67, 95% CI 0.57-0.78) and having a supervisory role (EMA: OR 0.65, 95% CI 0.54-0.79; wearable: OR 0.66, 95% CI 0.54-0.81). Furthermore, higher wearable compliance was negatively associated with agreeableness (OR 0.68, 95% CI 0.56-0.83) and neuroticism (OR 0.85, 95% CI 0.73-0.98). Compliance in the second week of the study could help explain more variance; 62% and 66% of the variance in EMA compliance and wearable compliance, respectively, was explained. Finally, compliance correlated with participants’ self-reflection on the ease of participation, usefulness of our compliance portal, timely resolution of issues, and compensation adequacy, suggesting that these are avenues for improving compliance. Conclusions We recommend conducting an initial 2-week pilot to measure trait-like compliance and identify participants at risk of long-term noncompliance, performing oversampling based on participants’ individual characteristics to avoid introducing bias in the sample when excluding data based on noncompliance, using an issue tracking portal, and providing special care in troubleshooting to help participants maintain compliance.

Affect detection in the "real" wild - where people go about their daily routines in their homes and workplaces - is arguably a different problem than affect detection in the lab or in the "quasi" wild (e.g., YouTube videos). How will our affect detection systems hold up when put to the test in the real wild? Some in the Affective Computing community had an opportunity to address this question as part of the MOSAIC (Multimodal Objective Sensing to Assess Individuals with Context [1]) program which ran from 2017 to 2020. Results were sobering, but informative. I'll discuss those efforts with an emphasis on performance achieved, insights gleaned, challenges faced, and lessons learned.

This study answered novel questions about the connection between high school extracurricular dosage (number of activities and participation duration) and the attainment of a bachelor's degree. Using data from the Common Application and the National Student Clearinghouse (N = 311,308), we found that greater extracurricular participation positively predicted bachelor's degree attainment. However, among students who ultimately earned a bachelor's degree, participating in more than a moderate number of high school activities (3 or 4) predicted decreasing odds of earning a bachelor's degree on time (within 4 years). This effect intensified as participation duration increased, such that students who participated in the greatest number of high school activities for the most years were the most likely to delay college graduation.

Several psychologists posit that performance is not only a function of personality but also of situational contexts, such as day-level activities. Yet in practice, since only personality assessments are used to infer job performance, they provide a limited perspective by ignoring activity. However, multi-modal sensing has the potential to characterize these daily activities. This paper illustrates how empirically measured activity data complements traditional effects of personality to explain a worker's performance. We leverage sensors in commodity devices to quantify the activity context of 603 information workers. By applying classical clustering methods on this multisensor data, we take a person-centered approach to describe workers in terms of both personality and activity. We encapsulate both these facets into an analytical framework that we call organizational personas. On interpreting these organizational personas we find empirical evidence to support that, independent of a worker's personality, their activity is associated with job performance. While the effects of personality are consistent with the literature, we find that the activity is equally effective in explaining organizational citizenship behavior and is less but significantly effective for task proficiency and deviant behaviors. Specifically, personas that exhibit a daily-activity pattern with fewer location visits, batched phone-use, shorter desk-sessions and longer sleep duration, tend to perform better on all three performance metrics. Organizational personas are a descriptive framework to identify the testable hypotheses that can disentangle the role of malleable aspects like activity in determining the performance of a worker population.

Virtual learning environments (VLEs) are increasingly used at-scale in educational contexts to facilitate teaching and promote learning, and the data they produce can be used for educationa...

Achieving important goals is widely assumed to require confronting obstacles, failing repeatedly, and persisting in the face of frustration. Yet empirical evidence linking achievement and frustration tolerance is lacking. To facilitate work on this important topic, we developed and validated a novel behavioral measure of frustration tolerance: the Mirror Tracing Frustration Task (MTFT). In this 5-min task, participants allocate time between a difficult tracing task and entertaining games and videos. In two studies of young adults (Study 1: N = 148, Study 2: N = 283), we demonstrated that the MTFT increased frustration more than 18 other emotions, and that MTFT scores were related to self-reported frustration tolerance. Next, we assessed whether frustration tolerance correlated with similar constructs, including self-control and grit, as well as objective measures of real-world achievement. In a prospective longitudinal study of high-school seniors (N = 391), MTFT scores predicted grade-point average and standardized achievement test scores, and-more than 2 years after completing the MTFT-progress toward a college degree. Though small in size (i.e., rs ranging from .10 to .24), frustration tolerance predicted outcomes over and above a rich set of covariates, including IQ, sociodemographics, self-control, and grit. These findings demonstrate the validity of the MTFT and highlight the importance of frustration tolerance for achieving valued goals. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Compared with admissions test scores, why are high school grades better at predicting college graduation? We argue that success in college requires not only cognitive ability but also self-regulatory competencies that are better indexed by high school grades. In a national sample of 47,303 students who applied to college for the 2009/2010 academic year, Study 1 affirmed that high school grades out-predicted test scores for 4-year college graduation. In a convenience sample of 1,622 high school seniors in the Class of 2013, Study 2 revealed that the incremental predictive validity of high school grades for college graduation was explained by composite measures of self-regulation, whereas the incremental predictive validity of test scores was explained by composite measures of cognitive ability.

Emotional regularity is the degree to which a person maintains and returns to a set of emotional states over time. The present investigation examined associations between emotional regularity and extant emotion measures as well as psychologically relevant dimensions of personality, health, and real-world occupational outcomes. Participants included 598 U.S. adults who provided daily experience sampling reports on their emotional states for approximately two months. Results suggest that emotional regularity was related to, but distinct from, well-established measures of emotion including emotional intensity, variability, covariation, inertia, granularity, and emodiversity. Furthermore, emotional regularity significantly predicted measures of personality, psychological health, and occupational outcomes even when accounting for extant emotion measures and sociodemographic covariates. Finally, it explained modest (7.5%) improvement (in terms of cross-validated RSq.) over baseline models containing emotional intensity, variability, and sociodemographic covariates. These findings suggest that emotional regularity may provide an important indicator of healthy emotional functioning and may be a promising area for further scientific discovery.

Frequent monitoring of participant compliance is necessary when conducting large-scale, longitudinal studies to ensure that the collected data is of sufficiently high quality. While the need for achieving high compliance has been underscored and there are discussions on incentives and factors affecting compliance, little is shared about the actual processes and tools used for monitoring compliance in such studies. Monitoring participant compliance with respect to multi-modal data can be a tedious process, especially if there are only a few personnel involved. In this case study, we describe the iterative design of an interactive visualization system we developed for monitoring compliance and refined based on changing requirements in an ongoing study. We find that the visualization system, leveraging the digital medium, both facilitates the exploratory tasks of monitoring participant compliance and supports asynchronous collaboration among non-co-located researchers. Our documented requirements for checking participant compliance as well as the design of the visualization system can help inform the compliance-monitoring process in future studies., 8 pages, 4 figures

Human abnormal physical and psychological behaviors, such as high level of stress, may result in negative impacts on work and life, if not handled efficiently. However, the continuous collection of behavioral data from questionnaires is not feasible, as is often the case for the natural downside of survey data gathering. Thanks to the proliferation of mobile sensors, it brings compelling opportunities for us to more deeply analyze human behavior. In this work, we ask the question of detecting anomalies in human physical and psychological behaviors from multivariate temporal data from multi-modal sensors. In the past decades, many efforts have been made in developing anomaly detection methods, but there remain several challenges in this specific domain problem: 1) data contains missing values at random positions. 2) data from multiple sensors is of multi-resolution and multivariate. 3) human behaviors are correlated to each other, thus it poses a multi-label problem. 4) the available labeled instances are limited, which requires the semi-supervised learning setting. 5) the frequency of anomaly occurrence is much smaller than that of normal instances, leading to imbalance problems. We propose a novel framework MBead to resolve these concerns. MBead consists of three key components: reweighted autoencoder to capture the dependency across temporal domain and multiple modalities, relevance learning module to learn the pairwise relations among labeled instances, and temporal prediction module to detect the anomalies while trained in semi-supervised settings. Extensive experiments show our MBead outperforms seven state-of-art baselines on three tasks of behavior anomaly detection: stress, affect, and work performance.

We conducted a meta-analysis of 33 independent samples (N = 2924) to address whether adding anthropomorphic faces to multimedia graphics and/or adding pleasant colors are effective emotional design approaches. We found significant positive meta-analytic effects for retention (k = 18, d+ = 0.387), comprehension (k = 14, d+ = 0.317), and transfer (k = 27, d+ = 0.327) under a random-effects model. Effects for affective-motivational variables were mixed, with a robust effect for intrinsic motivation (k = 23, d+ = 0.255), a weaker effect for liking/enjoyment (k = 20, d+ = 0.109), and a marginal effect for positive affect (k = 15, d+ = 0.113). The manipulations did not significantly (ps > .227) influence perceptions of learning (k = 11, d+ = 0.097) or effort (k = 20, d+ = 0.051), but reduced perceptions of difficulty (k = 14, d+ = −0.208). Four of the outcome variables (retention, transfer, intrinsic motivation, and perceived effort) were sufficiently heterogeneous. There was no major issue with publication bias, influential cases, or outliers. With one exception, there was no evidence of moderation by experimental contrast, dynamicity of materials, age, language/culture, prior mood, time-on-task, and publication type after adjusting for multiple comparisons. There was provisional evidence that age moderated the effect of the manipulations on intrinsic motivation, such that larger effects were revealed for children compared to older learners. Altogether, anthropomorphisms/colors appear to be useful design principles.

Abstract. Material re-exposure (e.g., re-reading) is a popular mnemonic strategy, however, its utility has been questioned. We extend research on re-reading to re-watching – an emerging mnemonic technique given the increased use of recorded lectures today (e.g., in online courses). Consistent with findings from recent investigations of re-reading, there were no benefits of massed re-watching on memory for lecture material and re-watching increased rates of mind wandering. We discuss implications for understanding the cognitive consequences of re-exposure-based mnemonics.

Item does not contain fulltext Physiological limitations on the visual system require gaze to move from location to location to extract the most relevant information within a scene. Therefore, gaze provides a real-time index of the information-processing priorities of the visual system. We investigated gaze allocation during mind wandering (MW), a state where cognitive priorities shift from processing task-relevant external stimuli (i.e., the visual world) to task-irrelevant internal thoughts. In both a main study and a replication, we recorded the eye movements of college-aged adults who studied images of urban scenes and responded to pseudorandom thought probes on whether they were mind wandering or attentively viewing at the time of the probe. Probe-caught MW was associated with fewer and longer fixations, greater fixation dispersion, and more frequent eyeblinks (only observed in the main study) relative to periods of attentive scene viewing. These findings demonstrate that gaze indices typically considered to represent greater engagement with scene processing (e.g., longer fixations) can also indicate MW. In this way, the current work exhibits a need for empirical investigations and computational models of gaze control to account for MW for a more accurate representation of the moment-to-moment information-processing priorities of the visual system.

Online education provides the opportunity to present lecture material to students in different formats or modalities, however there is debate about which lecture formats are best. Here, we conducted four experiments with 19–68 year old online participants to address the question of whether visuals of the instructor in online video lectures benefit learning. In Experiments 1 (N = 168) and 2 (N = 206) participants were presented with a lecture in one of three modalities (audio, audio with text, or audio with visuals of the instructor). Participants reported on their attentiveness – mind wandering (MW) – throughout the lecture and then completed a comprehension test. We found no evidence of an advantage for video lectures with visuals of the instructor in terms of a reduction in MW or increase in comprehension. In fact, we found evidence of a comprehension cost, suggesting that visuals of instructors in video lectures may act as a distractor. In Experiments 3 (N = 88) and 4 (N = 109) we explored learners' subjective evaluations of lecture formats across 4 different lecture formats (audio, text, audio + text, audio + instructor, audio + text + instructor). The results revealed learners not only find online lectures with visuals of the instructor more enjoyable and interesting, they believe this format most facilitates their learning. Taken together, these results suggest visuals of the instructor potentially impairs comprehension, but learners prefer and believe they learn most effectively with this format. We refer to as the Instructor Presence Effect and discuss implications for multimedia learning and instructional design.

Analyzing the quality of classroom talk is central to educational research and improvement efforts. In particular, the presence of authentic teacher questions, where answers are not predetermined by the teacher, helps constitute and serves as a marker of productive classroom discourse. Further, authentic questions can be cultivated to improve teaching effectiveness and consequently student achievement. Unfortunately, current methods to measure question authenticity do not scale because they rely on human observations or coding of teacher discourse. To address this challenge, we set out to use automatic speech recognition, natural language processing, and machine learning to train computers to detect authentic questions in real-world classrooms automatically. Our methods were iteratively refined using classroom audio and human-coded observational data from two sources: (a) a large archival database of text transcripts of 451 observations from 112 classrooms; and (b) a newly collected sample of 132 high-quality audio recordings from 27 classrooms, obtained under technical constraints that anticipate large-scale automated data collection and analysis. Correlations between human-coded and computer-coded authenticity at the classroom level were sufficiently high ( r = .602 for archival transcripts and .687 for audio recordings) to provide a valuable complement to human coding in research efforts.

Educational video games are hypothesized to be good environments for promoting learning; however, research on conceptual learning from games is mixed. We tested whether embedding a learning support in the form of short animations illustrating physics concepts that can be used to aid gameplay improved learning. Ninety-six 7th to 11th grade students were randomly assigned to play Physics Playground with or without the learning supports over a 4-day period. Results indicate that students who played a version of the game with embedded learning supports showed more improvement on a far- (d = 0.36), but not on a near-transfer physics assessment (d = 0.17) compared to those who played without the supports. The learning supports did not affect students’ enjoyment with the game. We conclude that the game-embedded animations were effective at promoting conceptual learning without sacrificing the fun of game-based learning.

We investigated how affective states influence expository text comprehension and whether text valence moderates the effects (i.e., mood congruency). In Experiment 1 participants were randomly assigned to a happy or sad affective state (elicited via films) before reading a positive or negative version of a scientific text on animal adaptations. Participants (n = 79) in the sad (film) group had higher scores on deep-reasoning (d = .312) but not surface-level questions on a subsequent multiple-choice comprehension assessment; there was also no evidence for mood congruence. Using a neutral version of the same text, in Experiment 2 participants (n = 52) in a fearful condition performed better on surface-level comprehension questions (d = .594) compared with a sad condition, but the groups were on par for deep-reasoning questions. Experiment 3 (n = 595) did not replicate the findings from Experiment 2 (no comprehension differences between the sad and fear groups) and there were no differences between th...

Affective computing (AC) adopts a computational approach to study affect. We highlight the AC approach towards automated affect measures that jointly model machine-readable physiological/behavioral signals with affect estimates as reported by humans or experimentally elicited. We describe the conceptual and computational foundations of the approach followed by two case studies: one on discrimination between genuine and faked expressions of pain in the lab, and the second on measuring nonbasic affect in the wild. We discuss applications of the measures, analyze measurement accuracy and generalizability, and highlight advances afforded by computational tipping points, such as big data, wearable sensing, crowdsourcing, and deep learning. We conclude by advocating for increasing synergies between AC and affective science and offer suggestions toward that direction.

We hypothesize that cognitively engaged readers dynamically adjust their reading times with respect to text complexity (i.e., reading times should increase for difficult sections and decrease for easier ones) and failure to do so should impair comprehension. This hypothesis is consistent with theories of text comprehension but has surprisingly been untested. We tested this hypothesis by analyzing 4 datasets in which participants (N = 484) read expository texts using a self-paced reading paradigm. Participants self-reported mind wandering in response to pseudorandom thought-probes during reading and completed comprehension assessments after reading. We computed two measures of cognitive coupling by regressing each participant's paragraph-level reading times on two measures of text complexity: Flesch-Kincaid Grade Level and Word Concreteness scores. The two coupling measures yielded convergent findings: coupling was a negative predictor of mind wandering and a positive predictor of both text- and inference-level comprehension. Goodness-of-fit, measured with Akaike information criterion, also improved after adding coupling to the reading-time only models. Furthermore, cognitive coupling mediated the relationship between mind wandering and comprehension, supporting the hypothesis that mind wandering engenders a decoupling of attention from external stimuli. (PsycINFO Database Record

Assessment of job performance, personalized health and psychometric measures are domains where data-driven and ubiquitous computing exhibits the potential of a profound impact in the future. Existing techniques use data extracted from questionnaires, sensors (wearable, computer, etc.), or other traits, to assess well-being and cognitive attributes of individuals. However, these techniques can neither predict individual's well-being and psychological traits in a global manner nor consider the challenges associated to processing the data available, that is incomplete and noisy. In this paper, we create a benchmark for predictive analysis of individuals from a perspective that integrates: physical and physiological behavior, psychological states and traits, and job performance. We design data mining techniques as benchmark and uses real noisy and incomplete data derived from wearable sensors to predict 19 constructs based on 12 standardized well-validated tests. The study included 757 participants who were knowledge workers in organizations across the USA with varied work roles. We developed a data mining framework to extract the meaningful predictors for each of the 19 variables under consideration. Our model is the first benchmark that combines these various instrument-derived variables in a single framework to understand people's behavior by leveraging real uncurated data from wearable, mobile, and social media sources. We verify our approach experimentally using the data obtained from our longitudinal study. The results show that our framework is consistently reliable and capable of predicting the variables under study better than the baselines when prediction is restricted to the noisy, incomplete data.

Personality traits such as grit and self-control are important determinants of success in life outcomes. However, most measures of these traits, which rely on self-reports, might be biased when used for the purpose of evaluating education policies or interventions. Recent research has shown the potential of survey effort-in particular, item non-response and careless answering-as a proxy measure of these traits. The current investigation uses a dataset of high school seniors (N = 513) to investigate survey effort measures in relationship with teacher reports, performance task measures, high school academic outcomes, and college attendance. Our results show promise for use of survey effort as proxy measures of grit and self-control.

We adopt a multimodal approach to investigating team interactions in the context of remote collaborative problem solving (CPS). Our goal is to understand multimodal patterns that emerge and their relation with collaborative outcomes. We measured speech rate, body movement, and galvanic skin response from 101 triads (303 participants) who used video conferencing software to collaboratively solve challenging levels in an educational physics game. We use multi-dimensional recurrence quantification analysis (MdRQA) to quantify patterns of team-level regularity, or repeated patterns of activity in these three modalities. We found that teams exhibit significant regularity above chance baselines. Regularity was unaffected by task factors. but had a quadratic relationship with session time in that it initially increased but then decreased as the session progressed. Importantly, teams that produce more varied behavioral patterns (irregularity) reported higher emotional valence and performed better on a subset of the problem solving tasks. Regularity did not predict arousal or subjective perceptions of the collaboration. We discuss implications of our findings for the design of systems that aim to improve collaborative outcomes by monitoring the ongoing collaboration and intervening accordingly.

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level cross-validation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns = 104, 130, 147) answered multiple-choice comprehension questions ~30 min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372, ps