Download PDF
Appl Clin Inform 2019; 10(05): 952-963
DOI: 10.1055/s-0039-3401814
AMIA CIC 2019
Georg Thieme Verlag KG Stuttgart · New York

Unsupervised Machine Learning of Topics Documented by Nurses about Hospitalized Patients Prior to a Rapid-Response Event

Zfania Tom Korach
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Kenrick D. Cato
2   School of Nursing, Columbia University, New York, New York, United States
,
Sarah A. Collins
2   School of Nursing, Columbia University, New York, New York, United States
3   Department of Biomedical Informatics, Columbia University, New York, New York, United States
,
Min Jeoung Kang
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Christopher Knaplund
2   School of Nursing, Columbia University, New York, New York, United States
,
Patricia C. Dykes
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Liqin Wang
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Kumiko O. Schnock
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Jose P. Garcia
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Haomiao Jia
2   School of Nursing, Columbia University, New York, New York, United States
,
Frank Chang
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Jessica M. Schwartz
2   School of Nursing, Columbia University, New York, New York, United States
,
Li Zhou
1   Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
› Author Affiliations Funding This work was funded by the National Institute of Nursing Research (NINR) award number 1R01NR016941. Jessica Schwartz is a pre-doctoral fellow funded by the National Institute of Nursing Research Reducing Health Disparities Through Informatics (RHeaDI) T32NR007969 and was funded by a grant from CRICO titled “2017-2019: Resilience in Clinical Deterioration Survival: Learning from Different Outcomes in Critical and Acute Care.”
Further Information

Publication History

13 August 2019

06 November 2019

Publication Date:
18 December 2019 (online)

   Permissions and Reprints

Abstract

Background In the hospital setting, it is crucial to identify patients at risk for deterioration before it fully develops, so providers can respond rapidly to reverse the deterioration. Rapid response (RR) activation criteria include a subjective component (“worried about the patient”) that is often documented in nurses' notes and is hard to capture and quantify, hindering active screening for deteriorating patients.

Objectives We used unsupervised machine learning to automatically discover RR event risk/protective factors from unstructured nursing notes.

Methods In this retrospective cohort study, we obtained nursing notes of hospitalized, nonintensive care unit patients, documented from 2015 through 2018 from Partners HealthCare databases. We applied topic modeling to those notes to reveal topics (clusters of associated words) documented by nurses. Two nursing experts named each topic with a representative Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) concept. We used the concepts along with vital signs and demographics in a time-dependent covariates extended Cox model to identify risk/protective factors for RR event risk.

Results From a total of 776,849 notes of 45,299 patients, we generated 95 stable topics, of which 80 were mapped to 72 distinct SNOMED CT concepts. Compared with a model containing only demographics and vital signs, the latent topics improved the model's predictive ability from a concordance index of 0.657 to 0.720. Thirty topics were found significantly associated with RR event risk at a 0.05 level, and 11 remained significant after Bonferroni correction of the significance level to 6.94E-04, including physical examination (hazard ratio [HR] = 1.07, 95% confidence interval [CI], 1.03–1.12), informing doctor (HR = 1.05, 95% CI, 1.03–1.08), and seizure precautions (HR = 1.08, 95% CI, 1.04–1.12).

Conclusion Unsupervised machine learning methods can automatically reveal interpretable and informative signals from free-text and may support early identification of patients at risk for RR events.

Keywords

hospital rapid response team - nursing assessment - electronic health records - survival analysis - nursing notes - natural language processing - machine learning - medicine

Protection of Human and Animal Subjects

The study was approved by the institutional review board of Partners HealthCare System.


 

  • References

  • 1 Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-response systems as a patient safety strategy: a systematic review. Ann Intern Med 2013; 158 (5 Pt 2): 417-425
    CrossrefPubMedGoogle Scholar
  • 2 Solomon RS, Corwin GS, Barclay DC, Quddusi SF, Dannenberg MD. Effectiveness of rapid response teams on rates of in-hospital cardiopulmonary arrest and mortality: a systematic review and meta-analysis. J Hosp Med 2016; 11 (06) 438-445
    CrossrefPubMedGoogle Scholar
  • 3 Huh JW, Lim C-M, Koh Y. , et al. Activation of a medical emergency team using an electronic medical recording-based screening system*. Crit Care Med 2014; 42 (04) 801-808
    CrossrefPubMedGoogle Scholar
  • 4 Kollef MH, Chen Y, Heard K. , et al. A randomized trial of real-time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med 2014; 9 (07) 424-429
    CrossrefPubMedGoogle Scholar
  • 5 Collins SA, Cato K, Albers D. , et al. Relationship between nursing documentation and patients' mortality. Am J Crit Care 2013; 22 (04) 306-313
    CrossrefPubMedGoogle Scholar
  • 6 Bellomo R, Goldsmith D, Uchino S. , et al. Prospective controlled trial of effect of medical emergency team on postoperative morbidity and mortality rates. Crit Care Med 2004; 32 (04) 916-921
    CrossrefPubMedGoogle Scholar
  • 7 Hillman K, Chen J, Cretikos M. , et al; MERIT study investigators. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet 2005; 365 (9477): 2091-2097
    CrossrefPubMedGoogle Scholar
  • 8 Parr MJ, Hadfield JH, Flabouris A, Bishop G, Hillman K. The Medical Emergency Team: 12 month analysis of reasons for activation, immediate outcome and not-for-resuscitation orders. Resuscitation 2001; 50 (01) 39-44
    CrossrefPubMedGoogle Scholar
  • 9 Hodgetts TJ, Kenward G, Vlackonikolis I. , et al. Incidence, location and reasons for avoidable in-hospital cardiac arrest in a district general hospital. Resuscitation 2002; 54 (02) 115-123
    CrossrefPubMedGoogle Scholar
  • 10 Rubin TN, Chambers A, Smyth P, Steyvers M. Statistical topic models for multi-label document classification. Mach Learn 2012; 88 (01) 157-208
    CrossrefPubMedGoogle Scholar
  • 11 Wang L, Sha L, Lakin JR. , et al. Development and validation of a deep learning algorithm for mortality prediction in selecting patients with dementia for earlier palliative care interventions. JAMA Netw Open 2019; 2 (07) e196972
    CrossrefPubMedGoogle Scholar
  • 12 Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res 2003; 3 (Jan): 993-1022
    PubMedGoogle Scholar
  • 13 Perotte A, Ranganath R, Hirsch JS, Blei D, Elhadad N. Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis. J Am Med Inform Assoc 2015; 22 (04) 872-880
    CrossrefPubMedGoogle Scholar
  • 14 Disease Trajectories and End-of-Life Care for Dementias: Latent Topic Modeling and Trend Analysis Using Clinical Notes. ResearchGate. Available at: https://www.researchgate.net/publication/328899169_Disease_Trajectories_and_End-of-Life_Care_for_Dementias_Latent_Topic_Modeling_and_Trend_Analysis_Using_Clinical_Notes . Accessed December 18, 2018
    PubMedGoogle Scholar
  • 15 Shao Y, Mohanty AF, Ahmed A. , et al. Identification and use of frailty indicators from text to examine associations with clinical outcomes among patients with heart failure. AMIA Annu Symp Proc 2017; 2016: 1110-1118
    PubMedGoogle Scholar
  • 16 Manning CD, Surdeanu M, Bauer J, Finkel J, Bethard SJ, McClosky D. The Stanford CoreNLP Natural Language Processing Toolkit. In: Association for Computational Linguistics (ACL) System Demonstrations; 2014: 55-60 . Available at: http://www.aclweb.org/anthology/P/P14/P14-5010 . Accessed November 25, 2019
    PubMedGoogle Scholar
  • 17 gensim: topic modelling for humans. Available at: https://radimrehurek.com/gensim/ . Accessed March 4, 2018
    PubMed
  • 18 Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol 1995; 48 (12) 1495-1501
    CrossrefPubMedGoogle Scholar
  • 19 Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 1995; 48 (12) 1503-1510
    CrossrefPubMedGoogle Scholar
  • 20 Andersen PK. Repeated assessment of risk factors in survival analysis. Stat Methods Med Res 1992; 1 (03) 297-315
    CrossrefPubMedGoogle Scholar