Graphical Presentations of Clinical Data in a Learning Electronic Medical Record

 

 Permissions and Reprints

Abstract

Background Complex electronic medical records (EMRs) presenting large amounts of data create risks of cognitive overload. We are designing a Learning EMR (LEMR) system that utilizes models of intensive care unit (ICU) physicians' data access patterns to identify and then highlight the most relevant data for each patient.

Objectives We used insights from literature and feedback from potential users to inform the design of an EMR display capable of highlighting relevant information.

Methods We used a review of relevant literature to guide the design of preliminary paper prototypes of the LEMR user interface. We observed five ICU physicians using their current EMR systems in preparation for morning rounds. Participants were interviewed and asked to explain their interactions and challenges with the EMR systems. Findings informed the revision of our prototypes. Finally, we conducted a focus group with five ICU physicians to elicit feedback on our designs and to generate ideas for our final prototypes using participatory design methods.

Results Participating physicians expressed support for the LEMR system. Identified design requirements included the display of data essential for every patient together with diagnosis-specific data and new or significantly changed information. Respondents expressed preferences for fishbones to organize labs, mouseovers to access additional details, and unobtrusive alerts minimizing color-coding. To address the concern about possible physician overreliance on highlighting, participants suggested that non-highlighted data should remain accessible. Study findings led to revised prototypes, which will inform the development of a functional user interface.

Conclusion In the feedback we received, physicians supported pursuing the concept of a LEMR system. By introducing novel ways to support physicians' cognitive abilities, such a system has the potential to enhance physician EMR use and lead to better patient outcomes. Future plans include laboratory studies of both the utility of the proposed designs on decision-making, and the possible impact of any automation bias.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the University of Pittsburgh Institutional Review Board (protocols 17030258, 17040082).

Publication History

Received: 17 October 2019

Accepted: 09 March 2020

Article published online:
14 October 2020

© 2020. Thieme. All rights reserved.

Georg Thieme Verlag KG
Stuttgart · New York

• References

• 1 Hall A, Walton G. Information overload within the health care system: a literature review. Health Info Libr J 2004; 21 (02) 102-108
  CrossrefPubMedGoogle Scholar
• 2 King AJ, Cooper GF, Hochheiser H, Clermont G, Visweswaran S. Development and preliminary evaluation of a prototype of a Learning Electronic Medical Record system. AMIA Annu Symp Proc 2015; 2015: 1967-1975
  PubMedGoogle Scholar
• 3 King AJ, Hochheiser H, Visweswaran S, Clermont G, Cooper GF. Eye-tracking for clinical decision support: a method to capture automatically what physicians are viewing in the EMR. AMIA Jt Summits Transl Sci Proc 2017; 2017: 512-521
  PubMedGoogle Scholar
• 4 King AJ, Cooper GF, Hochheiser H, Clermont G, Hauskrecht M, Visweswaran S. Using machine learning to predict the information seeking behavior of clinicians using an electronic medical record system. AMIA Annu Symp Proc 2018; 2018: 673-682
  PubMedGoogle Scholar
• 5 Rind A. Interactive information visualization to explore and query Electronic Health Records. Found Trends Human–Computer Interact. 2013; 5 (03) 207-298
  CrossrefPubMedGoogle Scholar
• 6 Stead WW, Lin HS. Computational Technology for Effective Health Care. Vol 22. Washington, D.C.: National Academies Press; 2009
  Google Scholar
• 7 West VL, Borland D, Hammond WE. Innovative information visualization of Electronic Health Record data: a systematic review. J Am Med Inform Assoc 2015; 22 (02) 330-339
  CrossrefPubMedGoogle Scholar
• 8 Jaspers MWM, Peute LWP, Lauteslager A, Bakker PJM. Pre-post evaluation of physicians' satisfaction with a redesigned electronic medical record system. Stud Health Technol Inform 2008; 136: 303-308
  PubMedGoogle Scholar
• 9 Sittig DF, Murphy DR, Smith MW, Russo E, Wright A, Singh H. Graphical display of diagnostic test results in electronic health records: a comparison of 8 systems. J Am Med Inform Assoc 2015; 22 (04) 900-904
  CrossrefPubMedGoogle Scholar
• 10 Armijo D, McDonnell C, Werner K. Electronic Health Record Usability: Interface Design Considerations. Vol 09. Rockville, MD; 2009
  Google Scholar
• 11 Bawden D. Perspectives on information overload. Aslib Proc 1999; 51 (08) 249-255
  CrossrefPubMedGoogle Scholar
• 12 Gross B. The Managing of Organizations: The Administrative Struggle. New York: Free Press of Glencoe; 1964
  Google Scholar
• 13 Food and Drug Administration (FDA). FDASIA health IT report. Proposed strategy and recommendations for a risk-based framework 2014. Accessed March 26, 2020 at: https://www.fda.gov/about-fda/cdrh-reports/fdasia-health-it-report
  PubMedGoogle Scholar
• 14 Thongprayoon C, Harrison AM, O'Horo JC, Berrios RAS, Pickering BW, Herasevich V. The effect of an electronic checklist on critical care provider workload, errors, and performance. J Intensive Care Med 2016; 31 (03) 205-212
  CrossrefPubMedGoogle Scholar
• 15 Patel VL, Currie LM. Clinical cognition and biomedical informatics: issues of patient safety. Int J Med Inform 2005; 74 (11-12): 869-885
  CrossrefPubMedGoogle Scholar
• 16 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007; 14 (04) 415-423
  CrossrefPubMedGoogle Scholar
• 17 Holden RJ. Cognitive performance-altering effects of electronic medical records: An application of the human factors paradigm for patient safety. Cogn Technol Work 2011; 13 (01) 11-29
  CrossrefPubMedGoogle Scholar
• 18 Pickering BW, Gajic O, Ahmed A, Herasevich V, Keegan MT. Data utilization for medical decision making at the time of patient admission to ICU. Crit Care Med 2013; 41 (06) 1502-1510
  CrossrefPubMedGoogle Scholar
• 19 Wright MC, Dunbar S, Macpherson BC. et al. Toward designing information display to support critical care. a qualitative contextual evaluation and visioning effort. Appl Clin Inform 2016; 7 (04) 912-929
  Article in Thieme ConnectPubMedGoogle Scholar
• 20 Kannampallil TG, Jones LK, Patel VL, Buchman TG, Franklin A. Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings. J Am Med Inform Assoc 2014; 21 (e2): e249-e256
  CrossrefPubMedGoogle Scholar
• 21 Kannampallil TG, Franklin A, Mishra R, Almoosa KF, Cohen T, Patel VL. Understanding the nature of information seeking behavior in critical care: implications for the design of health information technology. Artif Intell Med 2013; 57 (01) 21-29
  CrossrefPubMedGoogle Scholar
• 22 Pollack AH, Tweedy CG, Blondon K, Pratt W. Knowledge crystallization and clinical priorities: evaluating how physicians collect and synthesize patient-related data. AMIA Annu Symp Proc 2014; 2014: 1874-1883
  PubMedGoogle Scholar
• 23 Pickering BW, Herasevich V, Ahmed A, Gajic O. Novel representation of clinical information in the ICU: developing user interfaces which reduce information overload. Appl Clin Inform 2010; 1 (02) 116-131
  Article in Thieme ConnectPubMedGoogle Scholar
• 24 Ware C. Guidelines. In: Information Visualization. Cambridge, MA: Elsevier; 2013: 445-457
  CrossrefGoogle Scholar
• 25 Faiola A, Newlon C. Advancing critical care in the ICU: a human-centered biomedical data visualization system. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol 6779 LNCS. Indianapolis, IN: Indiana University, School of Informatics; 2011: 119-128
  Google Scholar
• 26 Horn W, Popow C, Unterasinger L. Support for fast comprehension of ICU data: visualization using metaphor graphics. Methods Inf Med 2001; 40 (05) 421-424
  Article in Thieme ConnectPubMedGoogle Scholar
• 27 Bade R, Schlechtweg S, Miksch S. Connecting time-oriented data and information to a coherent interactive visualization. In: Proceedings of the 2004 Conference on Human Factors in Computing Systems - CHI '04. Vol 6. New York, New York, USA: ACM Press; 2004: 105-112 . Accessed March 26, 2020 at: https://dl.acm.org/doi/10.1145/985692.985706
  PubMedGoogle Scholar
• 28 Klimov D, Shahar Y, Taieb-Maimon M. Intelligent visualization and exploration of time-oriented data of multiple patients. Artif Intell Med 2010; 49 (01) 11-31
  CrossrefPubMedGoogle Scholar
• 29 Spenke M. Visualization and interactive analysis of blood parameters with InfoZoom. Artif Intell Med 2001; 22 (02) 159-172
  CrossrefPubMedGoogle Scholar
• 30 Pickering BW, Dong Y, Ahmed A. et al. The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial. Int J Med Inform 2015; 84 (05) 299-307
  CrossrefPubMedGoogle Scholar
• 31 Wang TD, Wongsuphasawat K, Plaisant C, Shneiderman B. Visual information seeking in multiple electronic health records. In: Proceedings of the ACM International Conference on Health Informatics - IHI '10. New York, New York, USA: ACM Press; 2010: 46 . Accessed March 26, 2020 at: https://dl.acm.org/doi/10.1145/1882992.1883001
  PubMedGoogle Scholar
• 32 Zeng Q, Cimino JJ. Evaluation of a system to identify relevant patient information and its impact on clinical information retrieval. Proc AMIA Symp 1999; 642-646
  PubMedGoogle Scholar
• 33 Pieczkiewicz DS, Finkelstein SM, Hertz MI. Design and evaluation of a web-based interactive visualization system for lung transplant home monitoring data. AMIA Annu Symp Proc 2007; 598-602
  PubMedGoogle Scholar
• 34 Mamykina L, Vawdrey DK, Stetson PD, Zheng K, Hripcsak G. Clinical documentation: composition or synthesis?. J Am Med Inform Assoc 2012; 19 (06) 1025-1031
  CrossrefPubMedGoogle Scholar
• 35 Shahar Y, Goren-Bar D, Boaz D, Tahan G. Distributed, intelligent, interactive visualization and exploration of time-oriented clinical data and their abstractions. Artif Intell Med 2006; 38 (02) 115-135
  CrossrefPubMedGoogle Scholar
• 36 Senathirajah Y, Kaufman D, Bakken S. Beyond copy and paste: clinician approaches to meeting information needs during note writing. Stud Health Technol Inform 2014; 205: 599-603
  PubMedGoogle Scholar
• 37 van Merriënboer JJG, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ 2010; 44 (01) 85-93
  CrossrefPubMedGoogle Scholar
• 38 Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Crit Care Med 2011; 39 (07) 1626-1634
  CrossrefPubMedGoogle Scholar
• 39 Karsh B-T, Holden RJ, Alper SJ, Or CKLL. A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Qual Saf Health Care 2006; 15 (Suppl. 01) i59-i65
  CrossrefPubMedGoogle Scholar
• 40 Plaisant C, Milash B, Rose A, Widoff S, Shneiderman B. LifeLines: visualizing personal histories. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Common Ground - CHI '96. New York, New York, USA: ACM Press; 1996 :221-ff
  PubMedGoogle Scholar
• 41 National Center for Biotechnology Information (US). PubMed Help. NCBI Bookshelf. Published 2014. Accessed December 12, 2018 at: https://www.ncbi.nlm.nih.gov/books/NBK3827/
  PubMedGoogle Scholar
• 42 Institute of Electrical and Electronics Engineers. About IEEE Xplore® Digital Library. Accessed July 19, 2019 at: https://ieeexplore.ieee.org/xpl/aboutUs.jsp
  PubMed
• 43 Association for Computing Machinery. ACM digital library. Published 2019. Accessed July 19, 2019 at: https://dl.acm.org/
  PubMed
• 44 Tufte ER. The visual display of quantitative information. IEEE Power Eng Rev 1988; 8 (02) 20-20
  CrossrefPubMedGoogle Scholar
• 45 Aigner W, Miksch S, Schumann H, Tominski C. Visualization of time-oriented data. 2011
  PubMedGoogle Scholar
• 46 Kzillen M, Jarrett N. Qualitative data analysis: an introduction. J Adv Nurs 2007; 59 (05) 557-557
  CrossrefPubMedGoogle Scholar
• 47 Evolution Solutions Co. Ltd. Pencil Project. Published 2016. Accessed April 28, 2016 at: http://pencil.evolus.vn/
  PubMed
• 48 McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012; 22 (03) 276-282
  CrossrefPubMedGoogle Scholar
• 49 QSR. NVivo 11: Research software for analysis and insight. Published 2016. Accessed December 31, 2016 at: http://www.qsrinternational.com/
  PubMedGoogle Scholar
• 50 Muller MJ. PICTIVE - an exploration in participatory design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Reaching through Technology - CHI '91. New York, New York, USA: ACM Press; 1991: 223-225
  PubMedGoogle Scholar
• 51 Estiri H, Lovins T, Afzalan N, Stephens KA. Applying a participatory design approach to define objectives and properties of a “data profiling” tool for electronic health data. AMIA Jt Summits Transl Sci Proc 2016; 2016: 60-67
  PubMedGoogle Scholar
• 52 Rose AF, Schnipper JL, Park ER, Poon EG, Li Q, Middleton B. Using qualitative studies to improve the usability of an EMR. J Biomed Inform 2005; 38 (01) 51-60
  CrossrefPubMedGoogle Scholar
• 53 Pollack AH, Miller A, Mishra SR, Pratt W. PD-atricians: leveraging physicians and Participatory Design to develop novel clinical information tools. AMIA. Annu Symp proceedings AMIA Symp. 2016;2016:1030–1039 Accessed March 26, 2020 at: https://www.ncbi.nlm.nih.gov/pubmed/28269900
  PubMedGoogle Scholar
• 54 Lincoln YS, Guba EG. Naturalistic Inquiry. Vol 5. Beverly Hills, CA: Sage Publications; 1985
  Google Scholar
• 55 Findlater L, Moffatt K, McGrenere J, Dawson J. Ephemeral adaptation. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09. New York, New York, USA: ACM Press; 2009: 1655
  PubMedGoogle Scholar
• 56 Asan O, Holden RJ, Flynn KE, Yang Y, Azam L, Scanlon MC. Provider use of a novel EHR display in the pediatric intensive care unit. large customizable interactive monitor (LCIM). Appl Clin Inform 2016; 7 (03) 682-692
  Article in Thieme ConnectPubMedGoogle Scholar
• 57 Zheng K, Hanauer DA, Weibel N, Agha Z, Cognitive Informatics for Biomedicine. In: Patel VL, Kannampallil TG, Kaufman DR. , eds. Cham: Springer International Publishing; 2015
• 58 Youn-Ah Kang, Görg C, Stasko J. How can visual analytics assist investigative analysis? Design implications from an evaluation. IEEE Trans Vis Comput Graph 2011; 17 (05) 570-583
  CrossrefPubMedGoogle Scholar
• 59 Novak LL, Holden RJ, Anders SH, Hong JY, Karsh B-T. Using a sociotechnical framework to understand adaptations in health IT implementation. Int J Med Inform 2013; 82 (12) e331-e344
  CrossrefPubMedGoogle Scholar
• 60 Patterson ES, Cook RI, Render ML. Improving patient safety by identifying side effects from introducing bar coding in medication administration. J Am Med Inform Assoc 2002; 9 (05) 540-553
  CrossrefPubMedGoogle Scholar
• 61 Koppel R, Wetterneck T, Telles JL, Karsh B-T. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008; 15 (04) 408-423
  CrossrefPubMedGoogle Scholar
• 62 Harrison MI, Koppel R. Interactive sociotechnical analysis. In: Handbook of Research on Advances in Health Informatics and Electronic Healthcare Applications. PA, USA: IGI Global; 1AD:33-51
  Google Scholar
• 63 Rivera-Rodriguez AJ, Karsh B-T. Interruptions and distractions in healthcare: review and reappraisal. Qual Saf Health Care 2010; 19 (04) 304-312
  CrossrefPubMedGoogle Scholar
• 64 Mosier KL, Skitka LJ, Parasuraman R, Mouloua M. Human decision makers and automated decision aids: made for each other?. In: Automation and Human Performance: Theory and Applications. Erlbaum; 1996: 201-220
  Google Scholar
• 65 Ancker JS, Kern LM, Edwards A. , et al; HITEC Investigators. How is the electronic health record being used? Use of EHR data to assess physician-level variability in technology use. J Am Med Inform Assoc 2014; 21 (06) 1001-1008
  CrossrefPubMedGoogle Scholar
• 66 Rasmussen R. Electronic whiteboards in emergency medicine: a systematic review. In: IHI'12 - Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium.; 2012: 483-492
  PubMedGoogle Scholar
• 67 Dowding D, Randell R, Gardner P. et al. Dashboards for improving patient care: review of the literature. Int J Med Inform 2015; 84 (02) 87-100
  CrossrefPubMedGoogle Scholar
• 68 HL7 FHIR Foundation. HL7 FHIR. Accessed January 4, 2020 at: https://www.hl7.org/fhir/
  PubMed
• 69 Giordanengo A, Årsand E, Woldaregay AZ. et al. Design and prestudy assessment of a dashboard for presenting self-collected health data of patients with diabetes to clinicians: Iterative approach and qualitative case study. JMIR Diabetes 2019; 4 (03) e14002
  CrossrefPubMedGoogle Scholar
• 70 Epic Systems Corporation. Epic App Orchard. Accessed January 4, 2020 at: https://apporchard.epic.com/
  PubMed
• 71 Cerner Corporation. Leverage the power of the HL7® FHIR® standard in your SMART app. Accessed January 4, 2020 at: https://fhir.cerner.com/
  PubMedGoogle Scholar
• 72 Nielsen J. Usability Engineering. San Francisco, CA: Morgan Kaufmann; 1994
  Google Scholar
• 73 Tajgardoon M, Samayamuthu MJ, Calzoni L, Visweswaran S. Patient-Specific Explanations for Predictions of Clinical Outcomes. ACI Open 2019; 03 (02) e88-e97
  Article in Thieme ConnectPubMedGoogle Scholar
• 74 Dowding D, Merrill JA. The development of heuristics for evaluation of dashboard visualizations. Appl Clin Inform 2018; 9 (03) 511-518
  Article in Thieme ConnectPubMedGoogle Scholar

• Supplementary Material