Back to Search
Start Over
Detecting Recovery Problems �Just in Time�: Application of Automated Linguistic Analysis and Supervised Machine Learning to an Online Substance Abuse Forum
- Source :
- Journal of Medical Internet Research
- Publication Year :
- 2018
- Publisher :
- JMIR Publications Inc., 2018.
-
Abstract
- Background: Online discussion forums allow those in addiction recovery to seek help through text-based messages, including when facing triggers to drink or use drugs. Trained staff (or “moderators”) may participate within these forums to offer guidance and support when participants are struggling but must expend considerable effort to continually review new content. Demands on moderators limit the scalability of evidence-based digital health interventions. Objective: Automated identification of recovery problems could allow moderators to engage in more timely and efficient ways with participants who are struggling. This paper aimed to investigate whether computational linguistics and supervised machine learning can be applied to successfully flag, in real time, those discussion forum messages that moderators find most concerning. Methods: Training data came from a trial of a mobile phone-based health intervention for individuals in recovery from alcohol use disorder, with human coders labeling discussion forum messages according to whether or not authors mentioned problems in their recovery process. Linguistic features of these messages were extracted via several computational techniques: (1) a Bag-of-Words approach, (2) the dictionary-based Linguistic Inquiry and Word Count program, and (3) a hybrid approach combining the most important features from both Bag-of-Words and Linguistic Inquiry and Word Count. These features were applied within binary classifiers leveraging several methods of supervised machine learning: support vector machines, decision trees, and boosted decision trees. Classifiers were evaluated in data from a later deployment of the recovery support intervention. Results: To distinguish recovery problem disclosures, the Bag-of-Words approach relied on domain-specific language, including words explicitly linked to substance use and mental health (“drink,” “relapse,” “depression,” and so on), whereas the Linguistic Inquiry and Word Count approach relied on language characteristics such as tone, affect, insight, and presence of quantifiers and time references, as well as pronouns. A boosted decision tree classifier, utilizing features from both Bag-of-Words and Linguistic Inquiry and Word Count performed best in identifying problems disclosed within the discussion forum, achieving 88% sensitivity and 82% specificity in a separate cohort of patients in recovery. Conclusions: Differences in language use can distinguish messages disclosing recovery problems from other message types. Incorporating machine learning models based on language use allows real-time flagging of concerning content such that trained staff may engage more efficiently and focus their attention on time-sensitive issues.
- Subjects :
- Male
Online discussion
020205 medical informatics
Computer science
Word count
Decision tree
Health Informatics
02 engineering and technology
Machine learning
computer.software_genre
Education, Distance
Machine Learning
03 medical and health sciences
0302 clinical medicine
0202 electrical engineering, electronic engineering, information engineering
health communication
Humans
supervised machine learning
030212 general & internal medicine
Health communication
Original Paper
business.industry
Linguistics
social support
self-help groups
Digital health
substance-related disorders
Identification (information)
Alternating decision tree
Female
Artificial intelligence
Computational linguistics
business
computer
Subjects
Details
- Database :
- OpenAIRE
- Journal :
- Journal of Medical Internet Research
- Accession number :
- edsair.doi.dedup.....15a3c678e092331b1baa60aba4f77491