Meghan Broadbent, Mattia Medina Grespan, Katherine Axford, Xinyao Zhang, Vivek Srikumar, Brent Kious and Zac Imel
Frontiers in psychiatry, volume 14, 2023.
Abstract
Introduction: With the increasing utilization of text-based suicide crisis counseling, new means of identifying at risk clients must be explored. Natural language processing (NLP) holds promise for evaluating the content of crisis counseling; here we use a data-driven approach to evaluate NLP methods in identifying client suicide risk. Methods: De-identified crisis counseling data from a regional text-based crisis encounter and mobile tipline application were used to evaluate two modeling approaches in classifying client suicide risk levels. A manual evaluation of model errors and system behavior was conducted. Results: The neural model outperformed a term frequency-inverse document frequency (tf-idf) model in the false-negative rate. While 75% of the neural model’s false negative encounters had some discussion of suicidality, 62.5% saw a resolution of the client’s initial concerns. Similarly, the neural model detected signals of suicidality in 60.6% of false-positive encounters. Discussion: The neural model demonstrated greater sensitivity in the detection of client suicide risk. A manual assessment of errors and model performance reflected these same findings, detecting higher levels of risk in many of the false-positive encounters and lower levels of risk in many of the false negatives. NLP-based models can detect the suicide risk of text-based crisis encounters from the encounter’s content.
Links
- Link to paper
- See on Google Scholar
Bib Entry
@article{broadbent2023machine, author = {Broadbent, Meghan and Medina Grespan, Mattia and Axford, Katherine and Zhang, Xinyao and Srikumar, Vivek and Kious, Brent and Imel, Zac}, title = {A machine learning approach to identifying suicide risk among text-based crisis counseling encounters}, journal = {Frontiers in psychiatry}, year = {2023}, volume = {14} }