The Case for Regulating EMRs
January/February 2011
The Case for Regulating EMRs
Papers reporting serious adverse events (Nebeker, 2005; Yong, 2005) relating to the use of commercial healthcare IT (HIT) applications received significant publicity in 2005. Many of the reports at that time focused on the configuration of decision support systems used in computerized physician order entry (CPOE) systems. After these initial stories, things seemed to quiet down. Yet these stories did not miss the attention of the Centers for Device and Radiological Health (CDRH) at the Food and Drug Administration (FDA).
Fast-forward 5 years, and the issue of patient safety and EMRs is once again in the news. In testimony at the Health and Human Services’ Health Information Technology Policy Committee, Adoption/Certification Workshop held on February 25, 2010, CDRH Director, Jeffrey Shuren, gently articulated FDA’s intent to regulate EMR applications (Shuren, 2010). He reiterated FDA’s legal and regulatory authority over medical device software, safety issues that have been reported to FDA regarding the safety of EMR software, and possible approaches the FDA could take to improve and ensure patient safety moving forward.
The fact that FDA can regulate software should be no surprise (Murray, 2010). The Center for Biologics Evaluation and Research (CBER) started regulating blood bank software via the premarket submission process in 1996. Since their inception, the FDA has considered Picture Archiving and Communications Systems (PACS) as accessories to diagnostic imaging modalities, and thus regulated. Many PACS components have since been reclassified in recognition of their independence from imaging modalities, but continue to be regulated as software medical devices.
Despite these exceptions, the HIT industry has managed to avoid FDA regulations by claiming their products simply automate administrative tasks and paperwork, such as workflow around and involving the electronic patient record. As the HIT industry has brought to market applications that stretch ever closer to the point of care, applications like the decision support systems used in CPOE have blurred the lines between “administrative” HIT systems and software medical devices. Shuren’s testimony noted that, “To date, FDA has largely refrained from enforcing our regulatory requirements with respect to HIT devices.”
Using their regulatory discretion, FDA has observed the market for these applications develop over the past several years, while keeping an eye on patient safety. In his testimony at the Adoption/Certification Workshop, Shuren reported a growing patient safety concern with some EMR applications (Murray 2010):
“In the past two years, we have received 260 reports of HIT-related malfunctions with the potential for patient harm—including 44 reported injuries and 6 reported deaths. Because these reports are purely voluntary, they may represent only the tip of the iceberg in terms of the HIT-related problems that exist.”
This testimony documents a real risk to patient safety (see Table 1). Yet, with all the focus on Meaningful Use, there has been little or no consideration to applying safety-critical systems engineering to the development, deployment, and ongoing management of EMRs. It is increasingly apparent that some HIT applications have moved beyond administrative workflow automation and now automate tasks at the point of care tied directly to the diagnosis and treatment of patients.
| Category |
Examples |
| Errors of Commission | Example 1: An error occurred in software used to view and document patient activities. When the user documented activities in the task list for one patient and used the “previous” or “next” arrows to select another patient chart, the first patient’s task list displayed for the second patient.
Example 2: A nuclear medicine study was saved in the wrong patient’s file. Investigation suggested that this was due to a software error. Example 3: A sleep lab’s workstation software had a confusing user interface, which led to the overwriting and replacement of one patient’s data with another patient’s study. |
| Errors of Omission or Transmission | Example 1: An EMR system was connected to a patient monitoring system to chart vital signs. The system required a hospital staff member to download the vital signs, verify them, and electronically post them in the patient’s chart. Hospital staff reported that, several times, vital signs have been downloaded, viewed, and approved, and have subsequently disappeared from the system.
Example 2: An operating room management software application frequently “locked up” during surgery, with no obvious indication that a “lock-up” was occurring. Operative data were lost and had to be re-entered manually, in some cases from the nurse’s recollection. Example 3: An improper database configuration caused manual patient allergy data entries to be overwritten during automatic updates of patient data from the hospital information system. |
| Errors in Data Analysis | Example 1: In one system, intravenous fluid rates of greater than 1,000mL/hr were printed as 1 mL/hr on the label that went to the nursing/drug administration area.
Example 2: A clinical decision support software application for checking a patient’s profile for drug allergies failed to display the allergy information properly. Investigation by the vendor determined that the error was caused by a missing codeset. Example 3: Mean pressure values displayed on a patient’s physiological monitors did not match the mean pressures computed by the EMR system after systolic and diastolic values were entered. |
| Incompatibility between Multi-Vendor Software Applications or Systems | Example 1: An Emergency Department management software package interfaces with the hospital’s core information system and the laboratory’s information system; all three systems are from different vendors. When lab results were ordered through the ED management software package for one patient, another patient’s results were returned.
Example 2: Images produced by a CT scanner from one vendor were presented as a mirror image by another vendor’s picture archiving and communication system (PACS) web software. The PACS software vendor stipulates that something in the interface between the two products causes some images to be randomly “flipped” when displayed. |
Table 1. Examples of Adverse Events Related to Health IT Reported to U.S. Federal Drug Administration (Shuren, 2010)
The Impact on Industry
The regulatory framework used by the FDA (2010) is embodied in the Quality System Regulation or QSR . Rather than test and “certify” a completed product, like the Federal Aviation Administration does with aircraft, the FDA regulates the processes by which medical devices are designed, manufactured, installed, serviced, and supported. This process orientation was originally selected as a way to engender innovation.
Being unregulated, the HIT industry takes a somewhat casual approach to product development. It is common practice for software to be developed without fully documented or complete requirements. And there is often little or no traceability between requirements and testing of the final product. Post-market surveillance of HIT software rarely extends beyond tracking the average time required to close out a service request. These practices have considerable impact on software quality. The QSR specifies that requirements must be complete and fully documented prior to beginning the design phase. And every requirement must be linked to a specific verification test to ensure the design implements that requirement accurately and reliably. Corrective Actions and Preventive Actions (CAPA) are also a part of the QSR, and define a process for investigating customer complaints (i.e, customer calls for support to the vendor) in an effort to identify latent design defects.
User interface design has a direct impact on usability, the successful automation of workflow, and hence, quality and patient safety. Workflow at the point of care is both complex and highly variable and represents a considerable user interface design challenge. The QSR does not directly address user interface design and usability, but this issue does come up during premarket submission reviews by the FDA. This issue is of sufficient importance that the FDA has published guidance on this topic (Sawyer, 1996).
One of the biggest patient safety challenges of current HIT applications is the extensive configurability of the product. The underlying software may be defect-free, but the configuration of that software can result in considerable patient safety risk. Further, due to workflow variability across providers, a configuration that is unsafe in one institution may be perfectly safe in another. The mis-configuration of decision support systems used in CPOE applications have been documented to have resulted in adverse events. When the FDA reviews pre-market submissions, they will likely focus on the safety-critical engineering processes defined by the manufacturer to ensure safe and effective software configuration.
The HIT industry often depends on third-party consultants to provide implementation services for the EMRs and other systems purchased by hospitals. These implementation services include the configuration of the HIT application. Because the configuration of a system can introduce defects that result in patient injury or death, it is possible that the FDA will designate the configuring of these systems as “manufacturing” and regulate these consulting firms as medical device manufacturers. Having two sets of “manufacturers” for an installed system, the original manufacturer and a consultant/systems integrator, is a model the FDA is already considering as a regulatory approach to medical device interoperability. It is most likely the FDA will begin by regulating HIT vendors, and only if this approach fails to provide sufficient patient safety and effectiveness, extend their oversight to HIT consultants.
A considerable challenge for industry will be the legacy code that makes up much of large HIT systems. These systems, including some EMRs, incorporate a significant amount of code that was written 20 or more years ago. Often legacy code is poorly documented and poorly structured—often referred to as “spaghetti code.” The original engineers who wrote the code are also long gone. After years of use, such software is often reliable. Making changes to legacy code however, is fraught with risk. The verification testing required to find and fix the resulting software bugs is often much more time consuming for legacy code.
The FDA will likely announce their intent to regulate HIT applications with the publishing of a draft rule. These types of draft rules often describe the products to be regulated in some detail, and provide a timetable for when the FDA plans to “exercise enforcement discretion.” These milestones will likely include deadlines for when manufacturers are expected to register with the FDA as medical device manufacturers, and when manufacturers are expected to submit a pre-market notification, and the deadline for obtaining final clearance from FDA. It is possible that manufacturers that fail to meet the clearance deadline will be required to refrain from selling their product until it is cleared.
Manufacturers should have the QSR fully implemented at or before registering as a medical device manufacturer with the FDA. Implementing the QSR is not much more difficult than implementing the policies and procedures required to receive ISO9001 certification. A software development shop that operates at level 3 or higher on the Capability Maturity Model (2010), is already doing much of what is required for the QSR.
The quicker a manufacturer must implement the QSR, the more disruptive and costly the implementation process. The ideal time frame for implementing the QSR is 12 to 18 months. A recent draft rule giving notice to regulated medical device connectivity applications (referred to by the FDA as Medical Device Data Systems, or MDDS) included 60 days for manufacturers to register with FDA and 180 days for manufacturers to receive clearance (Gee, 2008). The MDDS draft rule was published in 2008, and a final rule has yet to be published. The gap between the draft and final versions of the rule for EMRs could be of similar scope, or be no longer than a few months.
Besides implementing the QSR itself, becoming regulated by the FDA will also change some business practices. Providing software modifications, or “engineering specials” to induce a customer to buy will no longer be practical because the QSR does not make accommodations for “quick and dirty” software changes. Time to market for new products or versions of existing products will be lengthened due to the additional quality system process steps required. The additional overhead of the QSR may result in higher prices.
The HIT software market has generally low barriers to entry. In many cases, the regulation of a market by the FDA creates new barriers to entry. For HIT entrepreneurs, FDA regulation represents a new incremental cost to starting a business. Some HIT manufacturers rely on much smaller software developers to create small- to medium-size applications to compliment or fill feature gaps in their own systems. If these partner developers will be modifying or extending the functionality of a regulated software product, their product will likely be designated an accessory to the regulated device and thus also regulated. Playing pilot fish to the large HIT manufacturers may be considerably less attractive to these smaller development partners if they, too, must be regulated.
The Impact on Providers
An unanticipated consequence of regulating EMRs could be a transformation in the way these systems are managed and used by provider organizations. Legally, the FDA may only regulate medical device manufacturers. The FDA can classify a provider as a manufacturer, based on the provider’s behavior or actions, and regulate them as a manufacturer. Consequently, providers need to manage their activities to minimize the FDA’s interest in pursuing a provider as a manufacturer.
A common approach to IT adoption for a relatively small portion of early adopters is to develop their own applications. Legally, a provider who develops a medical device qualifies as a manufacturer and can be regulated should the FDA chose to exercise enforcement discretion. Provided the hospital does not sell their medical device software, or use it as a competitive advantage to promote their health care services, the FDA is likely to exercise regulatory discretion and make no attempt to regulate the provider. Any discretion on the part of the FDA could change at any time, and enforcement can be applied retroactively.
A much more common activity is for providers to modify the underlying software of the HIT systems they purchase. When that software is a regulated medical device, the provider engages in the modification of that medical device. The regulatory implications for providers with respect to modification are less clear than for creating medical device software from scratch (Gee, 2009). Yet the methods for avoiding the FDA’s attention and managing risk are similar to those for creating your own medical device software.
The next area of regulatory concern for providers is system configuration. After the manufacturer or a third party initially configures the system upon installation, responsibility for the continued proper configuration of the system lies with the customer. Due to the nature of how systems are configured, the potential for selling or promoting a configuration as a competitive advantage is unlikely. Should a market for system configuration files or scripts emerge, the FDA will likely have a strong interest in ensuring patient safety.
As Shuren mentioned in his remarks at the Adoption/Certification Workshop, adverse events related to the use of a regulated medical device must be reported to both the manufacturer and FDA. Manufacturers will use these reports to identify latent defects and improve product quality. The FDA can use reports to identify manufacturers for onsite inspections. Depending on the findings of the manufacturer’s investigation resulting from a complaint, focus may shift to the provider, especially in cases where the provider has modified the system.
Healthcare providers using regulated HIT systems will need to manage risks of potential FDA enforcement and liability that may result from adverse events. Providers have little experience with this because conventional electromechanical medical devices are rarely created by or modified by customers. Much as with manufacturers, the key to managing these risks is in the processes used by the provider. While current “best practices” are adequate to support mission-critical HIT systems, they are insufficient for life-critical regulated medical device software. The following provides an introduction to the changes providers must make to support their use of regulated EMRs.
Providers must have documented processes they use to manage risk and ensure safety and effectiveness. Documentation must also include training records. The documentation must be controlled whereby relevant documents are formally approved, secured, and made easily available to those needing the information to complete their jobs. The document control system also ensures that only the current and approved version of a document is available to users. Being able to store and retrieve documents on a shared network drive does not meet this requirement. Documents can be managed on paper, provided they are physically controlled. Electronic document management systems with electronic signatures are also available.
These processes will be documented as policies and procedures that will include broad topics such as the management of IT infrastructure and requirements for the physical installation, ongoing support and monitoring of systems. Areas that will require the biggest changes are likely to be risk management, change control and verification, and validation testing.
Providers will need to adopt a formal risk management process, such as ISO 14971 (Dolan). Risk management can drive both the identification and definition of policies and procedures adequate for the use and ongoing support and management of medical device software. The artifacts of a solid risk management process also serve as documentation of the provider’s thoughtful consideration and response to risks inherent with software medical devices. Any scrutiny by the FDA, accreditation bodies, or plaintiffs in legal proceedings will be greatly mitigated by a solid risk management process.
Change control and testing are two other key areas where current best practices are inadequate. While most providers follow a basic change control process, the rigor of that process, especially in the identification and mitigation of risk and documentation of the process, are likely insufficient. Test plans should be based upon a risk assessment, and include sufficient scope and detail to ensure adequate risk management. Current approaches to wireless LAN testing for firmware upgrades or introducing new products are inadequate.
Defining and implementing these new processes will require a team approach. The group most equipped to guide the support of medical devices is Clinical Engineering, as they are most familiar with the kind of life cycle management required for life-critical systems. The providers’ Risk Management department can contribute clinical and legal liability perspectives, and IT will play a central role as the group responsible for the medical device software. Providers that develop their own medical device software, modify medical device software, or engage in any manufacturer-like activities should seek FDA regulatory advice.
As manufacturers and providers both adopt these kinds of processes, time and costs will be impacted. Software developers, both manufacturers and providers, will see longer project time frames. Likewise, operating costs for both will also increase due to the overhead of these new safety critical processes. How this impacts the creation and adoption of systems, let alone meeting Meaningful Use milestones, remains to be seen. The increase in software quality and patient safety will offer at least some benefits to offset the increased effort.
Tim Gee is principal and founder of Medical Connectivity Consulting (MCC) (www.medicalconnectivity.com). Gee’s practice revolves around workflow automation through the integration of medical devices with information systems, and enabling technologies. MCC advises healthcare providers and manufacturers on needs assessment, requirements, workflow and patient flow analysis, regulatory issues, product development, sales, and marketing. Markets covered include both acute and ambulatory care.
Gee brings 25 years of experience with care delivery processes, therapeutic and diagnostic modalities in most clinical areas, including e-health transactions between physicians, hospitals, and payors. Gee speaks frequently at industry conferences and corporate events. He is program chair and principal organizer for the Medical Device Connectivity conference. Prior to founding MCC, Gee worked for medical device, healthcare IT, and wireless vendors, including: Philips Medical Systems, Welch Allyn, e-health connectivity startup Pointshare, AT&T Wireless, Corometrics, and Baxter Systems Division. Gee serves on the editorial advisory boards of a number of publications including Patient Safety & Quality Healthcare, participates in key industry interoperability development efforts, and is an advisor to a number of early stage companies. He may be contacted at tim@medicalconnectivity.com.
References
Capability Maturity Model. (2010, December 26). In Wikipedia, The Free Encyclopedia. Available at http://en.wikipedia.org/w/index.php?title=Capability_Maturity_Model&oldid=404318386
Dolan, A. M. ISO 14971-2007: Risk management for all medical devices—The new global era. Presented at 11th Conference of the Globabl Harmonization Task Force. An overview is available at http://www.ghtf.org/meetings/conferences/11thconference/D/02DOLAN.pdf
FDA. U.S. Federal Drug Administration. (2010, October 19). Quality system regulation. Available at http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/PostmarketRequirements/QualitySystemsRegulations/default.htm
Gee, T. (2008, March 1). FDA issues new MDDS rule. Medical Connectivity. Available at http://medicalconnectivity.com/2008/03/01/fda-issuesfrom-new-mdds-rule/
Gee, T. (2009, October 25). Impact of modifying FDA regulated devices. Medical Connectivity. Available at http://medicalconnectivity.com/2009/10/25/impact-of-modifying-fda-regulated-devices/
Murray, J. (2010) CDRH regulated software: An introduction. http://www.fda.gov/downloads/Training/CDRHLearn/UCM209129.pdf
Nebeker, J. (2005). High rates of adverse drug events in a highly computerized hospital. Archives of Internal Medicine, 165, 1111-1116
Sawyer, D. (1996, December). Do it by design – An introduction to human factors in medical devices. Available at http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm094957.htm
Shuren, J. (2010). Testimony of Jeffrey Shuren, Director of FDA’s Center for Devices and Radiological Health. http://healthit.hhs.gov/portal/server.pt/gateway/PTARGS_0_10741_910717_0_0_18/3Shuren_Testimony022510.pdf
Yong, Y. (2005). Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics, 116, 1506-1512.