More than 20 years ago, AMIA – in conjunction with the Medical Library Association, the Association of Academic Health Sciences Libraries, the American Health Information Management Association, the American Nurses Association, and the American College of Physicians – published a position paper, “Recommendations for Responsible Monitoring and Regulation of Clinical Software Systems.”1 In 1997, the consortium authors proposed four categories of clinical system risks and four classes of measured monitoring and regulatory actions that can be applied strategically based on the level of risk in a given setting. The consortium recommended that “FDA regulation should exempt most clinical software systems and focus on those systems posing highest clinical risk, with limited opportunities for competent human intervention.”2
In 2001, Sim, Gorman, Greene, et al. published “Clinical Decision Support Systems for the Practice of Evidence-based Medicine,” where they introduced the term “Evidence-adaptive clinical decision support systems.”3 This subclass of clinical decision support systems (CDSS) rely on a clinical knowledge base that “continually reflects the most up-to-date evidence from the research literature and practice-based sources.” Authors summarized proceedings from the 2000 AMIA Spring Symposium, identifying five broad areas for accelerating the development and adoption of clinical decision support systems for evidence-based medicine.
Despite the EHR adoption landscape of the era, these papers envisioned clinical environments that would leverage widespread use of CDSS; they envisioned that such systems would be adaptive to dynamic and growing bodies of data; and in need of innovative regulatory regimes. Since this time, EHRs have become ubiquitous and the decision support systems they have enabled are (1) increasingly driven by artificial intelligence and machine learning algorithms; (2) available as substitutable knowledge resources; and (3) used as tools to diagnose in Software-as-a-Medical Device products.
So too has the regulatory environment for such technology evolved. US and international regulators are pioneering an approach that shifts the emphasis of regulation from the functionality to the firm, relying on an assessment of how the firm’s software development practices “to provide reasonable assurance of safety and effectiveness of the organization’s [software] products.”4 While the object of this specific regulatory approach is SaMD, US regulators indicate that CDSS – especially adaptive CDSS – will be treated in a similar manner.5 Further, FDA officials released a discussion paper on a proposed regulatory framework for modifications to artificial intelligence/machine learning-based SaMD. Several questions posed in the discussion paper sought input from stakeholders on: (1) types of modifications a manufacturer of AI/ML-based SaMD might make and how FDA’s existing “total product lifecycle” approach could include so-called “good ML practices (GMLP); (2) ways to ensure the manufacturer has the ability to manage and control resultant risks of the modifications through “SaMD Pre-Specifications (SPS),” and “Algorithm Change Protocol (ACP),” or how the algorithm will learn and change while remaining safe and effective; (3) FDA’s approach for modifications after initial review with an established SPS and ACP; and (4) how manufacturers would be expected to commit to the principles of transparency and real-world performance monitoring for AI/ML-based SaMD.
AMIA’s work in 2019 towards improving how evidence is delivered back into the care continuum will focus on CDS in the era of big data and machine learning. AMIA members will again look beyond the horizon, as they have been doing for decades, to identify the opportunities and challenges of CDS given the growing velocity and veracity of data, the increasing diversity of data sources, and the advancing computational power of artificial intelligence.
1 Miller RA, Gardner RM, et al. Recommendations for Responsible Monitoring and Regulation of Clinical Software Systems. J Am Med Inform Assoc. 1997 Nov-Dec; 4(6): 442–457.
3 Sim I, Gorman B, Greene R, et al. Clinical Decision Support Systems for the Practice of Evidence-based Medicine. J Am Med Inform Assoc. 2001 Nov-Dec; 8(6): 527–534.
4 Food & Drug Administration. Software Precertification Program: Working Model – Version 1.0 – January 2019. 4.2 Excellence Appraisal Elements. Available at: https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/DigitalHealth...
5 CDS draft guidance: https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance...