Search

BackgroundThe Harmonized Diagnostic Assessment of Dementia for the Longitudinal Aging Study in India (LASI-DAD) is the first and only nationally representative study on late-life cognition and dementia in India (n=4096). LASI-DAD obtained clinical consensus diagnosis of dementia for a subsample of 2528 respondents. ObjectiveThis study develops a machine learning model that uses data from the clinical consensus diagnosis in LASI-DAD to support the classification of dementia status. MethodsClinicians were presented with the extensive data collected from LASI-DAD, including sociodemographic information and health history of respondents, results from the screening tests of cognitive status, and information obtained from informant interviews. Based on the Clinical Dementia Rating (CDR) and using an online platform, clinicians individually evaluated each case and then reached a consensus diagnosis. A 2-step procedure was implemented to train several candidate machine learning models, which were evaluated using a separate test set for predictive accuracy measurement, including the area under receiver operating curve (AUROC), accuracy, sensitivity, specificity, precision, F1 score, and kappa statistic. The ultimate model was selected based on overall agreement as measured by kappa. We further examined the overall accuracy and agreement with the final consensus diagnoses between the selected machine learning model and individual clinicians who participated in the clinical consensus diagnostic process. Finally, we applied the selected model to a subgroup of LASI-DAD participants for whom the clinical consensus diagnosis was not obtained to predict their dementia status. ResultsAmong the 2528 individuals who received clinical consensus diagnosis, 192 (6.7% after adjusting for sampling weight) were diagnosed with dementia. All candidate machine learning models achieved outstanding discriminative ability, as indicated by AUROC >.90, and had similar accuracy and specificity (both around 0.95). The support vector machine model outperformed other models with the highest sensitivity (0.81), F1 score (0.72), and kappa (.70, indicating substantial agreement) and the second highest precision (0.65). As a result, the support vector machine was selected as the ultimate model. Further examination revealed that overall accuracy and agreement were similar between the selected model and individual clinicians. Application of the prediction model on 1568 individuals without clinical consensus diagnosis classified 127 individuals as living with dementia. After applying sampling weight, we can estimate the prevalence of dementia in the population as 7.4%. ConclusionsThe selected machine learning model has outstanding discriminative ability and substantial agreement with a clinical consensus diagnosis of dementia. The model can serve as a computer model of the clinical knowledge and experience encoded in the clinical consensus diagnostic process and has many potential applications, including predicting missed dementia diagnoses and serving as a clinical decision support tool or virtual rater to assist diagnosis of dementia.

BackgroundSMS text messaging is an inexpensive, private, and scalable technology-mediated assessment mode that can alleviate many barriers faced by the safety net population to receive depression screening. Some existing studies suggest that technology-mediated assessment encourages self-disclosure of sensitive health information such as depressive symptoms while other studies show the opposite effect. ObjectiveThis study aimed to evaluate the validity of using SMS text messaging to screen depression and related conditions, including anxiety and functional disability, in a low-income, culturally diverse safety net primary care population. MethodsThis study used a randomized design with 4 study groups that permuted the order of SMS text messaging and the gold standard interview (INTW) assessment. The participants for this study were recruited from the participants of the prior Diabetes-Depression Care-management Adoption Trial (DCAT). Depression was screened by using the 2-item and 8-item Patient Health Questionnaire (PHQ-2 and PHQ-8, respectively). Anxiety was screened by using the 2-item Generalized Anxiety Disorder scale (GAD-2), and functional disability was assessed by using the Sheehan Disability Scale (SDS). Participants chose to take up the assessment in English or Spanish. Internal consistency and test-retest reliability were evaluated by using Cronbach alpha and intraclass correlation coefficient (ICC), respectively. Concordance was evaluated by using an ICC, a kappa statistic, an area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity. A regression analysis was conducted to examine the association between the participant characteristics and the differences in the scores between the SMS text messaging and INTW assessment modes. ResultsOverall, 206 participants (average age 57.1 [SD 9.18] years; females: 119/206, 57.8%) were enrolled. All measurements except the SMS text messaging–assessed PHQ-2 showed Cronbach alpha values ≥.70, indicating acceptable to good internal consistency. All measurements except the INTW-assessed SDS had ICC values ≥0.75, indicating good to excellent test-retest reliability. For concordance, the PHQ-8 had an ICC of 0.73 and AUROC of 0.93, indicating good concordance. The kappa statistic, sensitivity, and specificity for major depression (PHQ-8 ≥8) were 0.43, 0.60, and 0.86, respectively. The concordance of the shorter PHQ-2, GAD-2, and SDS scales was poor to fair. The regression analysis revealed that a higher level of personal depression stigma was associated with reporting higher SMS text messaging–assessed PHQ-8 and GAD-2 scores than the INTW-assessed scores. The analysis also determined that the differences in the scores were associated with marital status and personality traits. ConclusionsDepression screening conducted using the longer PHQ-8 scale via SMS text messaging demonstrated good internal consistency, test-retest reliability, and concordance with the gold standard INTW assessment mode. However, care must be taken when deploying shorter scales via SMS text messaging. Further regression analysis supported that a technology-mediated assessment, such as SMS text messaging, may create a private space with less pressure from the personal depression stigma and therefore encourage self-disclosure of depressive symptoms. Trial RegistrationClinicalTrials.gov NCT01781013; https://clinicaltrials.gov/ct2/show/NCT01781013 International Registered Report Identifier (IRRID)RR2-10.2196/12392

Researchers have become increasingly interested in response times to survey items as a measure of cognitive effort. We used machine learning to develop a prediction model of response times based on 41 attributes of survey items (e.g., question length, response format, linguistic features) collected in a large, general population sample. The developed algorithm can be used to derive reference values for expected response times for most commonly used survey items.

Objectives Self-reported survey data are essential for monitoring the health and well-being of the population as it ages. For studies of aging to provide precise and unbiased results, it is necessary that the self-reported information meets high psychometric standards. In this study, we examined whether the quality of survey responses in panel studies of aging depends on respondents' cognitive abilities. Methods Over 17 million survey responses from 157,844 participants aged 50 years and older in 10 epidemiological studies of aging were analyzed. We derived 6 common statistical indicators of response quality from each participant's data and estimated the correlations with participants' cognitive test scores at each study wave. Effect sizes (correlations) were synthesized across studies, cognitive tests, and waves using individual participant data meta-analysis methods. Results Respondents with lower cognitive scores showed significantly more missing item responses (overall effect size ρ ^ = −0.144), random measurement error (⁠ ρ ^ = −0.192), Guttman errors (⁠ ρ ^ = −0.233), multivariate outliers (⁠ ρ ^ = −0.254), and acquiescent responses (⁠ ρ ^ = −0.078); the overall effect for extreme responses (⁠ ρ ^ = −0.045) was not significant. Effect sizes were consistent across studies, modes of survey administsration, and different cognitive functioning domains, although some cognitive domain specificity was also observed. Discussion Lower-quality responses among respondents with lower cognitive abilities add random and systematic errors to survey measures, reducing the reliability, validity, and reproducibility of survey study results in aging research. [ABSTRACT FROM AUTHOR]

Associations between various forms of activity engagement (e.g. work, leisure) and the experience of stress in workers have been widely documented. The mechanisms underlying these effects, however, are not fully understood. Our goal was to investigate if perceived whole day workload accounted for the relationships between daily frequencies of activities (i.e. work hours and leisure/rest) and daily stress. We analysed data from 56 workers with type 1 diabetes (T1D) who completed approximately two weeks of intensive longitudinal assessments. Daily whole day workload was measured with an adapted version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX). A variety of occupations were reported, including lawyer, housekeeper and teacher. In multilevel path analyses, day-to-day changes in whole day workload mediated 67% (p <.001), 61% (p <.001), 38% (p <.001), and 55% (p <.001) of the within-person relationships between stress and work hours, rest frequency, active leisure frequency, and day of week, respectively. Our results provided evidence that whole day workload perception may contribute to the processes linking daily activities with daily stress in workers with T1D. Perceived whole day workload may deserve greater attention as a possible stress intervention target, ones that perhaps ergonomists would be especially suited to address. [ABSTRACT FROM AUTHOR]

Researchers have become increasingly interested in response times to survey items as a measure of cognitive effort. We used machine learning to develop a prediction model of response times based on 41 attributes of survey items (e.g., question length, response format, linguistic features) collected in a large, general population sample. The developed algorithm can be used to derive reference values for expected response times for most commonly used survey items. [ABSTRACT FROM AUTHOR]

Monitoring of cognitive abilities in large-scale survey research is receiving increasing attention. Conventional cognitive testing, however, is often impractical on a population level highlighting the need for alternative means of cognitive assessment. We evaluated whether response times (RTs) to online survey items could be useful to infer cognitive abilities. We analyzed >5 million survey item RTs from >6000 individuals administered over 6.5 years in an internet panel together with cognitive tests (numerical reasoning, verbal reasoning, task switching/inhibitory control). We derived measures of mean RT and intraindividual RT variability from a multilevel location-scale model as well as an expanded version that separated intraindividual RT variability into systematic RT adjustments (variation of RTs with item time intensities) and residual intraindividual RT variability (residual error in RTs). RT measures from the location-scale model showed weak associations with cognitive test scores. However, RT measures from the expanded model explained 22–26% of the variance in cognitive scores and had prospective associations with cognitive assessments over lag-periods of at least 6.5 years (mean RTs), 4.5 years (systematic RT adjustments) and 1 year (residual RT variability). Our findings suggest that RTs in online surveys may be useful for gaining information about cognitive abilities in large-scale survey research. [ABSTRACT FROM AUTHOR]

Our objective was to investigate the validity of four-item and six-item versions of the National Aeronautics and Space Administration Task Load Index (NASA-TLX, or TLX for short) for measuring workload over a whole day in the repeated measures context. We analysed data on 51 people with type 1 diabetes from whom we collected ecological momentary assessment and daily diary data over 14 days. The TLX was administered at the last survey of every day. Confirmatory factor analysis fit statistics indicated that neither the TLX-6 nor TLX-4 were a unidimensional representation of whole day workload. In exploratory analyses, another set of TLX items we refer to as TLX-4v2 was sufficiently unidimensional. Raw sum scores from the TLX-6 and TLX-4v2 had plausible relationships with other measures, as evidenced by intra-person correlations and mixed-effects models. TLX-6 appears to capture multiple factors contributing to workload, while TLX-4v2 assesses the single factor of 'mental strain'. Practitioner Summary: Using within-person longitudinal data, we found evidence supporting the validity of a measure evaluating whole-day workload (i.e. workload derived from all sources, not only paid employment) derived from the NASA-TLX. This measure may be useful to assess how day-to-day variations in workload impact quality of life among adults. Abbreviations: NASA-TLX or TLX: National Aeronautics and Space Administration Task Load Index; TLX-6: six item version of the NASA-TLX; TLX-4: four item version of the NASA-TLX, TLX-4v2: four item NASA-TLX version two; NIOSH: National Institute for Occupational Safety and Health; CFA: confirmatory factor analysis; T1D: type 1 diabetes; EMA: ecological momentary assessment; BG: blood glucose; SD: standard deviation; CV: coefficient of variation; RMSEA: root mean square error of approximation; CFI: comparative fit index; TLI: Tucker-Lewis Index; SRMR: standardized root mean square residual; AIC: Akaike information criterion; BIC: Bayesian information criterion; χ2: Chi-square statistic [ABSTRACT FROM AUTHOR]