AMIA 2019 Clinical Informatics Conference Workshops

Tuesday, April 30

8:00 a.m. – 12:30 p.m.

C. Jaffe, HL7; V. Nguyen, Principal Stratametrics; R. Leftwich, Intersystems; M. Tripathi, Massachusetts eHealth Collaborative; W. Kubick, HL7; S. Rab, Rush Medical Center

HL7 FHIR is now 9 years old. It’s emergence as a critical path toward interoperability has been a challenging process. With the creation of the Argonaut Project in the fall of 2014, by EHR vendors and academic medical centers, a private-sector force drove both the development and implementation of FHIR. Over the ensuing years, the development of FHIR was paralleled by a growing international implementation community.

Within a year, the Office of the National Coordinator had established the certification requirement for open APIs (Application Programming Interfaces). The real turning point came in early 2018, when Apple announced that FHIR had been imbedded into the IOS operating system and included 12 pilot academic health systems. Today, that number has grown to more than 500 hospitals and health systems.

At the same time, a dozen large private-sector payers announced the creation of the Da Vinci Project, which would leverage FHIR to enable Value Based Care. Soon thereafter, the Center for Medicare & Medicaid Services (CMS) announced their participation. After a measured start, FHIR implementation guides have been balloted for several critical functions toward their ambitious goals. In 2019, Da Vinci will see the publication of as many as 7 more guides.

In December of 2018, HL7 published the first normative edition of FHIR, now recognized as Release 4 (R4). This release promised the implementation community a new level of stability and, for the first time, introduced true backward compatibility.

Perhaps the most remarkable acceleration of FHIR implementation came this February with the Notice of Proposed Rule Making (NPRM), by both CMS and ONC, which established multiple requirements for FHIR in both clinical care and the Medicare/Medicaid transaction landscape.

This workshop will provide the historical perspective of HL7 FHIR, the technical implementations that have spurred wide-spread adoption, the implications of the NPRM as the comment period closes, and the future roadmaps of Argonaut, Da Vinci, and the FHIR development community. This includes the release of CDS Hooks, for clinical decision support and more, and the publication of Bulk Data on FHIR, which provides for the transmission of large data cohorts for clinical research, population health, translational science, and genomic integration into EHR platforms.

The interactive session will be geared to the rapidly growing communities of FHIR implementers. No technical expertise in FHIR development or implementation will be required.

J. Hollberg, N. Safdar, Emory University; P. Fu, Harbor-UCLA; R. Schreiber, Geisinger Holy Spirit

All informatics leaders function in a complicated world. In order to be successful, they must effectively work with other C-suite executives, especially the CIO, other members of IT, and ancillary teams including nursing, pharmacy, and lab. As CxIOs' responsibilities have increased, so too has the size of their teams, which means that they must also possess effective HR management skills. While CxIOs have the qualities necessary to become successful leaders, many have not had the opportunity for formal training to develop these tactical leadership skills.

This workshop will focus on participants' development of practical skills needed to build and manage an informatics team, grow relationships with other stakeholders, navigate conflict with IT, and negotiate C-suite politics. It will include a combination of didactics, hands-on individual and small group learning exercises, and broad discussion so that the participants have the opportunity to learn from each other and discuss their unique leadership challenges.

8:00 a.m. – 10:00 a.m.

T. Sethi, Stanford University/Indraprastha Institute of Information Technology Delhi/All India Institute of Medical Sciences

Networks are one of the most intuitive representations of complex data. However, most networks rely on pair-wise associations which limits their use for making decisions. Bayesian Decision Networks (BDNs) extend a class of probabilistic graphical models known as Bayesian Networks by using decision theory. They have been used in business settings for decision making. However, BDNs are under-exploited in clinical and public health settings because of the complex nature of datasets which makes it difficult for these networks to be hand-specified. This workshop will teach the participants to learn Bayesian-network models directly from data, assess them rigorously with statistical bootstrap evaluations, draw quantitative inferences, learn optimal decisions and deploy their models as a web-application based upon R/Shiny framework. Participants will learn to use these models both for probabilistic reasoning and causal inference depending upon the study design. Since a BDN is a directed acyclic graph and provides a single joint-multivariate fit on the data, it automatically learns the confounder, mediator and collider effects in the data – hence providing an end-to-end statistical and machine learning framework for knowledge-discovery in addition to decision making. Fitting a joint probabilistic model decreases the chance of false edges because the structure must agree with global and local distributions. Unlike most other forms of Artificial Intelligence and Machine Learning, BDNs are white-box models falling in the class of Explainable AI (XAI) and Fair Accountable Transparent ML (FAT-ML). The workshop will cover an end-to-end walkthrough of the open-source platform wiseR, developed by the instructor and his team in collaboration with computer scientists and clinicians at Stanford and India. The workshop will cover preliminary theory and two case-studies, in a clinical setting for Sepsis and a public health setting (Health Inequality) for learning decisions and policy, both published and available with linked open-data.

J. Manning, Atrium Health's Carolinas Medical Center

As the SMART® app platform continues to mature, there exists a growing need to design and create targeted, health care focused apps that satisfy a specific need or use case. Managing multiple code bases (such as Java/Kotlin for Android or Swift for iOS) can be a challenge, and while multiple options exist to develop an Android/iOS using a single code base, each comes with its own set of tradeoffs.

Thankfully, the app market is rapidly evolving. In 2017, Google announced a public release of an open-source software development kit called Flutter, which uses the Dart programming language. Since its release, the Flutter community has grown substantially and the platform’s “hot reload” feature allows for dynamic changes to be performed in real-time while an app is still running on a device or emulator. Put simply, we can now make rapid design tweaks on-the-fly, and in a manner that harnesses agile development techniques to its full potential.

As health care providers, it may be easy to ideate and dream of ways that a simple app could make our lives better at the bedside. Now, it’s becoming very easy to turn an idea into a reality. The barrier to enter a product on the app store has become marginalized. All skill levels are welcome and encouraged to attend, though I will focus more heavily on the basics and on how to design/layout your app. This will be a collaborative workshop, so feel free to join and help out if you have more development experience or have previously coded with Flutter.

10:30 a.m. – 12:30 p.m.

D. King, S. Richardson, S. Khan, J. Solomon, Northwell Health

This workshop will introduce attendees to the concept of human-centered design and common methods employed in usability testing of health information technology (IT). Human-centered design is a philosophy that assumes that innovation should start by getting close to users and observing their activities. This represents a fundamental paradigm shift in the design of health IT. Much of our technology is not designed to be easy to use or with the needs of the end-user in mind. As a result, many products that address important health needs go unused. This workshop will employ, as a test case, a clinical decision support tool that our group built as a part of a large multi-site NIH funded project. The tool helps providers risk-stratify patients for strep pharyngitis and pneumonia. The goal of the tool is to reduce antibiotic prescribing for upper respiratory infections as well as to standardize care. About 44% of antibiotic prescriptions for upper respiratory infection are unnecessary. As a result of employing the methods of user-centered design and usability testing, our group has been able to increase user adoption of clinical decision support tools from the 10%-20% range to 62%. This workshop will cover key methodologies including focus groups, key informant interviews, think aloud testing, near live testing, live testing, and thematic analysis. After learning about how these techniques are applied, attendees will split into groups and conduct usability testing sessions on sample software wireframes. Attendees will be encouraged to document their usability feedback and will present their findings to the larger group for discussion. The concepts covered by this workshop are helpful for virtually any type of project that involves an end user. Through this workshop, participants will gain an understanding of how these tools can be employed to help with research, practice improvement, and patient care.

M. Basit, V. Kannan, D. Willett, R. Medford, University of Texas Southwestern Health System

Designing effective Clinical Decision Support (CDS) tools in an Electronic Health Record (EHR) can prove challenging, due to complex real-world scenarios and newly-discovered requirements. As such, deploying new CDS EHR tools shares much in common with new product development, where “agile” principles and practices consistently prove more effective than traditional project management. Typical agile principles and practices can thus prove helpful on CDS projects, including time-boxed “sprints” and lightweight requirements gathered with User Stories and acceptance criteria. Modeling CDS behavior removes ambiguity and promotes shared understanding of desired behavior, but risks analysis paralysis. An Agile Modeling approach can foster effective rapid-cycle CDS design and optimization. The agile practice of automated testing for test-driven design and regression testing can be applied to CDS development in EHRs using open-source tools. Ongoing monitoring of CDS behavior once released to production can identify anomalies and prompt rapid-cycle redesign to further enhance CDS effectiveness. The workshop participant will learn about these topics in interactive didactic sessions, with time for practicing the techniques taught.

1:30 p.m. – 6:00 p.m.

L. Heermann, Intermountain Health care; R. Leftwich, Intersystems; V. Nguyen, Stratametrics; J. McClay, University of Nebraska

HL7 created Fast Health care Interoperability Resources (FHIR®) as a next generation standard to address clinical data interoperability. Clinicians on FHIR® evolved in 2014 as an event held at each HL7 Working Group Meeting (3 times yearly) to educate clinicians about HL7 FHIR® and provide feedback to the HL7 FHIR® team regarding the clinical viability and usability of the FHIR® standard. This workshop will bring the Clinicians on FHIR® activity to the broader clinical audience attending the AMIA 2019 Clinical Informatics Conference. The workshop is designed to educate attendees about HL7 FHIR® and tools available to access, review, and provide feedback to the HL7 FHIR® team regarding the evolving HL7 FHIR®® standard. It is also intended to make the audience aware of the potential of FHIR® for innovation in their organizations. The faculty will provide lectures describing HL7 FHIR® history, background, and fundamental principles. Examples of applications using the SMART on FHIR® platform will also be discussed. After this initial overview of FHIR®, the attendees will be guided through using online tools to examine HL7 FHIR® Resources (the basic building blocks of FHIR®) and build FHIR® Profiles (implementation guides for specific use cases). Commonly used resources such as “patient,” “condition,” and “observation” will be reviewed and demonstrated as an initial introduction to the tools. Attendees will then be guided on how they can apply their own use cases to the tools introduced for further exploration into FHIR®.

1:30 p.m. – 3:30 p.m.

S. Papagari Sangareddy, D. Franzke, D. Dcruz, F. Reza, U.S. Centers for Disease Control and Prevention

A well-formulated problem is half-solved. Problem formulation and problem solving are important cognitive as well as practical activities in any given discipline. The CDC’s Public Health Informatics Fellowship Program trains informatics professionals in solving clinical and public health problems through development and application of shared mental models.

This hands-on workshop will share concepts of mental models, and the practical application of sharing them ,using a case-based method. Specifically, this workshop will demonstrate applications of shared mental models for formulating informatics problems by having participants collaboratively engage real-world cases in clinical and public health informatics. The workshop will be useful for clinical and public health leaders and managers seeking to articulate and solve informatics problems in their organizations.
 View the video abstract

J. Finnell, Indiana University; L. Masson, Cedars Sinai Health System

The ability to write multiple-choice test items is a skill that is growing in importance for informatics practitioners and educators. The emergence of Clinical Informatics (CI) certification created the need for multiple-choice test items that adhere to national standards for use in high-stakes exams. AMIA established an item writing activity to generate items for a clinical informatics practice exam. CI fellowship programs use items to help fellows assess their mastery of the CI core content. Maintenance of Certification (MOC) requires that well written multiple-choice questions accompany learning content for credit to be offered (e.g., sessions at AMIA meetings). Applied Clinical Informatics requests that authors submit multiple-choice questions with manuscripts. AMIA’s work towards Advanced Health Informatics Certification (AHIC) will create additional demand for high quality informatics test items. Many individuals who write test questions for use in their educational programs are unfamiliar with the well-established set of rules for writing sound test items. Increased use of these test question guidelines would help students become familiar with the format of questions used on high-stakes exams as part of their educational program and could positively impact the quality of assessments used by educators. This workshop will present guidelines for writing high quality items, offer a recommended approach for writing clinical or health informatics items, and provide participants with an opportunity to write items that will be shared for feedback. Workshop faculty will share common pitfalls and strategies for effective item writing. After participating in this activity, the individual should be able to create items that comply with guidelines on creating one-best-answer multiple choice questions for high-stakes exams and self-assessments.

K. Unertl, Vanderbilt University; S. Haque, RTI International

Health information technology (HIT) implementation and adoption relies on successfully integrating technology and the context in which its implemented. The purpose of this workshop is to identify evidence-based practices from organizational studies that can inform HIT implementation and use. Participants will gain an understanding of principles, concepts and frameworks in organizational studies as well as practice in applying them with real-world case studies and ability to identify pathways to application in their own organizations.

4:00 p.m. – 6:00 p.m.

D. Aronsky, Semedy/Vanderbilt University Medical Center; A. Weitkamp, Vanderbilt University Medical Center; S. Maviglia, Semedy/Brigham and Women’s Hospital, Boston/Harvard Medical School; D. Wenke, Semedy; R. Rocha, Semedy/Brigham and Women’s Hospital, Boston/Harvard Medical School

Health care institutions build increasingly large amounts of clinical knowledge assets. They are responsible for the accuracy, transparency and updating of content. Inconsistent, incomplete, and outdated clinical knowledge assets represent unnecessary patient safety risks. Unfortunately, many organizations limit their knowledge management activities to a reactive and ad-hoc approach. This results in a lack of formal content review and maintenance process, as well as a system that supports an institution-wide knowledge management strategy. This workshop will introduce clinical knowledge management topics that include the cataloguing of knowledge assets, authoring and modeling of metadata, managing relationship and dependencies among data, importing and exporting knowledge assets from and to other clinical applications, guaranteeing structural and semantic integrity when knowledge assets change, and comprehensive asset lifecycle process. The workshop includes practical experiences, challenges, and lessons learned.

B. Levy, Geisinger Health, Geisinger Commonwealth School of Medicine; A. Carter, Childrens' Hospital of Atlanta

“Pathology Informatics involves collecting, examining, reporting and storing large complex sets of data derived from tests performed in clinical laboratories, anatomic pathology laboratories, or research laboratories in order to improve patient care and enhance our understanding of disease-related processes.” Informatics has been an integral part of the practice of pathology since the late 1940s, and laboratory data comprise a huge portion of the overall data in an EHR. Pathology informaticians are not only involved in the implementation and management of laboratory information systems within the pathology and laboratory environments but are also involved in managing laboratory data within EHRs and with enhancing the value of laboratory and clinical data to organizations and providers through advanced analytics. Yet, for many non-pathologist clinical informaticians, the world of pathology informatics (like the laboratory itself) can be a black box where lab test orders and specimens enter, and test results exit.

The purpose of this workshop is to provide clinical informaticians insight into the area of pathology informatics. Topics covered in the workshop include:

  • Value Provided by the Practice of Pathology Informatics
  • The Scope of Practice of Pathology Informatics
  • The Federal Regulatory World in Which Laboratories Operate and How Pathology Informatics Helps Fulfill Those Requirements
  • The Similarities and Differences Between an Electronic Health Record and a Laboratory Information System
  • The Emerging Areas of Genomics, Digital Pathology and Whole-slide Imaging
  • Data Analytics from Pathology’s Perspective
  • The Emergence of Computational Pathology

Participants will gain a better understanding of the value and importance of sound pathology informatics practice within their organizations as well as how to better leverage pathology informatics and informaticians to support the broader challenge of safe and efficient patient care throughout a health system.

C. Eckert, M. Ahmad, KenSci

Interpretable Machine Learning (ML) refers to machine learning models that can provide reasons or explanations for why certain patient-level predictions are made. In high-stakes domains outside of health care, merely providing traditional ML metrics like accuracy, precision, recall etc. are not sufficient. In health care, explanations for model results and patient-level risk predictions is imperative. Clinical providers and other decision-makers note interpretability as a priority for implementation and use since black box machine learning models seldom engender trust. Decisions based on machine learning predictions could inform diagnoses, clinical care pathways, patient risk stratification, and other considerations. It follows that for decisions of such importance, clinicians and other staff desire to know the “reasons” behind the prediction. This workshop will cover the definitions, nuances, challenges, and requirements for the design of interpretable and explainable ML models and systems in health care with an emphasis on the clinical application of this framework. We will discuss many uses in which interpretable machine learning models are needed in health care. Additionally, we will explore the landscape of recent advances to address the challenges of model interpretability in health care and also broadly describe how to select the most appropriate interpretable ML algorithm for a given problem (given certain clinically-oriented constraints). We will engage our audience by having interactive quizzes throughout the workshop. Based on the practice domains of our audience, we will also encourage participants to share their stories and learnings from applying ML in health care and how interpretability / explain-ability may have played a role. If the size of our audience is appropriate, we may also break the audience into groups to discuss the desiderata of explainable ML, its advantages, and potential pitfalls. We will also share additional resources and bibliography with the participants at the conclusion of the session for further learning.