Journal of Substance Abuse Treatment 2016.
Abstract
Motivational interviewing (MI) is an efficacious treatment for
substance use disorders and other problem behaviors. Studies on MI
fidelity and mechanisms of change typically use human raters to
code therapy sessions, which requires considerable time, training,
and financial costs. Natural language processing techniques have
recently been utilized for coding MI sessions using machine
learning techniques, rather than human coders, and preliminary
results have suggested these methods hold promise. The current
study extends this previous work by introducing two natural
language processing models for automatically coding MI sessions
via computer. The two models differ in the way they semantically
represent session content, utilizing either 1) simple discrete
sentence features (DSF model) and 2) more complex recursive neural
networks (RNN model). Utterance- and session-level predictions
from these models were compared to ratings provided by human
coders using a large sample of MI sessions (N = 341 sessions;
78,977 clinician and client talk turns) from 6 MI studies. Results
show that the DSF model generally had slightly better performance
compared to the RNN model. The DSF model had “good” or higher
utterance-level agreement with human coders (Cohen's kappa > 0.60)
for open and closed questions, affirm, giving information, and
follow/neutral (all therapist codes); considerably higher
agreement was obtained for session-level indices, and many
estimates were competitive with human-to-human agreement. However,
there was poor agreement for client change talk, client sustain
talk, and therapist MI-inconsistent behaviors. Natural language
processing methods provide accurate representations of human
derived behavioral codes and could offer substantial improvements
to the efficiency and scale in which MI mechanisms of change
research and fidelity monitoring are conducted.
Links
- Link to paper
- This paper on Google Scholar
Bib Entry
@article{tanana2016comparison, title={A comparison of natural language processing methods for automated coding of motivational interviewing}, author={Tanana, Michael and Hallgren, Kevin A and Imel, Zac E and Atkins, David C and Srikumar, Vivek}, journal={Journal of substance abuse treatment}, volume={65}, pages={43--50}, year={2016}, publisher={Elsevier} }