PSQH Innovation Award Winner: Cleveland Clinic Develops Algorithm to Power Early Warning System
By Jay Kumar
Editor’s note: The second annual PSQH Innovation Awards recognize healthcare organizations who overcame quality improvement challenges. In this article, we highlight the winning submission selected from Cleveland Clinic. Thanks to Barbara J. Youngberg, JD, MSW, BSN, FASHRM, academic director and senior lecturer in residence, Loyola University Chicago, for her help in evaluating the submissions.
Subtle changes in a patient’s vital signs can signify clinical deterioration. After an incident in 2014, officials at Cleveland Clinic Medina (Ohio) Hospital began to look into mitigating factors, and what they found led to an improvement that has been selected as the winner of the second annual PSQH Innovation Awards.
According to Kathleen Mau, DNP, APRN, senior director for Cleveland Clinic’s Office of Nursing Education and Professional Development, officials conducted interviews and a literature review to determine the contributing factors to the incident.
Following this, the hospital and health system set up an early warning system (EWS) as a way to alert nurses to subtle changes in patient condition. But alerts alone aren’t enough, so they also developed an integrated workflow that supports patient assessment, contextual evaluation of clinical data, provider notification, interdisciplinary collaboration, and timely intervention, Mau said.
The team worked to create tools within the EMR to do the following:
> Improve recognition and intervention for early signs of clinical deterioration
> Increase unit-level situational awareness of patients experiencing signs of deterioration
> Increase interdisciplinary communication and collaboration regarding significance and management of subtle changes in patient condition
> Increase accountability for taking and recording vital signs in the EMR in a timely manner
> Increase frequency of vital sign monitoring and reassessment for patients with signs of early deterioration
“We needed every discipline’s input,” including everyone who has contact with the patient, Mau said. In addition, “we needed support from the executive leadership team.”
The interdisciplinary team that formed to develop the solution included nurses from advanced practice, quality, education, and informatics, as well as physicians and IT clinical and systems analysts. Mau said they brainstormed possible solutions, and during a six-month design phase, they met every two weeks to review and refine programs. Frontline caregivers tested and provided feedback on each iteration of the project, which was called VitalScout. Embedded in the EMR for use on non-ICU nursing units, VitalScout uses predictive analytics to automatically calculate risk scores and provide real-time, visual alerts to caregivers within the patient’s EMR.
To enhance situational awareness of risk scores for all patients on a unit, screensavers on every computer were reprogrammed to display de-identified, color-coded tiles indicating EWS scores without the need to log into the EMR, according to Mau. Clicking on a tile displays parameters driving the score, allowing for prioritization of care. Because providers and nurses viewed EMR data and reports differently, an interactive report was created to collate key data to one location accessible by all disciplines. This report provides data deemed essential for interpretation of the clinical significance of the patient’s risk score. Documentation templates are also embedded for quick documentation and to provide transparency of nursing findings and actions to the entire care team. Additionally, this report dynamically indicates when new vital signs have been recorded, if the EWS score has changed, and when vital signs were measured and recorded. Nurses can indicate that they have reviewed patient status using a “Marked as Reviewed” button; an automatic timer indicates when a new set of vital signs is required (every two hours for moderate EWS risk scores and every hour for high risk scores), and a color-coded alert for overdue vital sign reassessment promotes accountability.
Mau said VitalScout was first implemented at Medina Hospital in January 2015. Nurses attended a 90-minute hands-on training session highlighting VitalScout support of nursing workflow and intervention for early deterioration. While the technological aspects of the program were shown, the focus was on critical thinking, situational awareness, communication, and appropriate intervention. Unlicensed personnel attended a 30-minute class to highlight the importance of accurate and timely measurement and recording of vital signs. Providers received one-to-one training from physician specialists, and ancillary departments received focused in-services. Email communications and articles in hospital publications provided awareness of VitalScout to all employees. The VitalScout team provided on-site staff support during and in the two weeks after go-live. Performance improvement metrics were tracked and simulation education sessions were held four months post go-live.
The results
VitalScout was developed to help caregivers identify subtle changes in physiological parameters that could signal early signs of clinical deterioration and provide them with tools to support workflow, accountability for timely documentation and reassessment of vital signs, situational awareness, communication, critical thinking, and clinical decision-making. While anecdotal accounts of early intervention for subtle deterioration are prevalent, it is difficult to quantify what is prevented during these interventions.
Mau said during the initial pilot, several metrics were tracked, including the total time patients spent with VitalScout scores in the moderate/high risk range. By retrospectively applying the scoring tool to medical/surgical patients hospitalized in the 12 months pre-implementation (2014) and comparing the total number of hours at moderate/high risk scores to patients hospitalized in the 12 months post-implementation (2015), the team was able to demonstrate a 40% decrease (11,566 hours) (p < 0.0001) in hours patients spent at moderate/high risk levels for deterioration using VitalScout.
Decreases were also noted in the time it took to reassess vital signs following a moderate or high risk VitalScout alert, meeting the expectation that patients with moderate risk scores would be reassessed within two hours and patients with high risk scores would be reassessed within one hour to determine whether interventions worked and to monitor for further deterioration. At the moderate risk level, the length of time between VitalScout alert and reassessment of vital signs decreased by 25% (34 minutes) (p < 0.0001), while high risk level time decreased by 41% (31 minutes) (p < 0.0001) when post-implementation times were compared to pre-implementation times. All code blue and rapid response cases were also reviewed and compared for one year pre- and post-implementation, demonstrating that the number of codes decreased from 0.737 codes per 1,000 patient days to 0.543 codes per 1,000 patient days. Rapid response team (RRT) calls, on the other hand, increased from 2.672 RRTs per 1,000 patient days to 5.545 RRTs per 1,000 patient days without an associated increase in ICU transfers, suggesting that nurses may be recognizing signs of deterioration earlier and calling for RRT assistance sooner when alerted to early signs of deterioration post-implementation (Mau et al., 2019).
While reviewing code data post-implementation, some delays in nurse identification of respiratory compromise were noted in patients who had orders to titrate oxygen to maintain a specified oxygen saturation level. In these cases, nurses increased oxygen delivery to improve patient saturation levels, which in turn improved the VitalScout score and, in some instances, canceled the VitalScout alert (the score is re-calculated every time new parameter values are entered). Given this context, it was recognized that oxygen saturation monitoring alone is not a reliable indicator of clinical deterioration when data about oxygen delivery is excluded from the EWS algorithm (Mau et al., 2019).
This recognition led to the revision of VitalScout’s scoring tool, which went from a four-parameter to a five-parameter tool with the addition of oxygen supplementation. For each parameter (pulse, systolic blood pressure, respiratory rate, oxygen saturation, and use of supplemental oxygen) a value of 0–2 was assigned based on deviation from “in range” values, with the maximum VitalScout score increasing from 8 to 10 with the new parameter. The intervention algorithm that guides nurse response to alerts for scores in the moderate and high risk range was changed only to expand the high risk range to a score of 10. The range for moderate risk for deterioration remained at scores of 3–5.
In February 2016, the phase 2 pilot trialed the revised VitalScout scoring tool at Medina Hospital and at Hillcrest Hospital, a 500-bed tertiary care hospital in the Cleveland Clinic health system, to evaluate the revision and to determine if Medina Hospital’s findings could be replicated. In comparing 12-month pre-implementation moderate/high risk patient hours to moderate/high risk patient hours using the VitalScout original and revised scoring tools, Medina Hospital’s pre-implementation (2014) moderate/high risk patient hours accounted for 11.6% of cumulative hours, the initial VitalScout program (2015) accounted for 9.1%, and the revised VitalScout program (2016) demonstrated only 7.3% of patient hours (p < 0.0001) at moderate/high risk levels. Similarly, Hillcrest Hospital’s 12-month pre-implementation moderate/high risk hours accounted for 10.2% of cumulative hours, decreasing to 7.4% post-implementation (p < 0.0001). For reassessment of vital signs following a moderate risk alert, only 56% of Medina patients were reassessed within two hours pre-implementation, while 89% of post-implementation patients in the original VitalScout program and 98% of patients in the revised VitalScout program were reassessed within two hours (p < 0.0001). Hillcrest Hospital went from 47% to 82% of moderate-risk patients reassessed within two hours of an alert post-implementation (p < 0.0001). The statistically significant decrease in both metrics at both hospitals suggests staff are making use of the many VitalScout tools and demonstrates the positive impact VitalScout has had in supporting response to early signs of clinical deterioration and in decreasing the time non-ICU patients spend with at-risk vital signs.
A phenomenological study of Medina Hospital nurses sought to determine VitalScout impact on nursing practice. Thematic analysis of interviews led to a conceptual model showing that VitalScout directly impacted nursing practice by increasing awareness of early deterioration, triggering investigation, and supporting prioritization of care. This led to an indirect impact on organizational culture, resulting in improved communication/collaboration among all caregivers, increased accountability, and proactive response to early signs of deterioration (Burns et al., 2017).
To date, Cleveland Clinic has implemented VitalScout at 11 regional hospitals as well as its quaternary care main campus, with the most recent hospital go-live occurring in October 2019. For all hospitals using VitalScout for greater than one year, a statistically significant decrease (p < 0.0001) in patient hours spent at moderate/high risk was found when comparing one year pre- and post-implementation data. A statistically significant decrease (p < 0.0001) was also noted in time to reassess vital signs following a moderate/high risk alert using the same time frames.
Takeaways
The main takeaway, according to Mau, is that use of an EWS alone is not enough to mitigate failure to rescue related to recognition and timely intervention for early signs of deterioration. VitalScout provides caregivers with a comprehensive, visual, and intuitive experience for managing early clinical deterioration that is embedded within their usual workflow. Color-coded alerts to changing vital signs, screensavers to support situational awareness, a shared report that collates key data to one location, documentation templates, and timers are just a few of the tools created to support this workflow.
It was also recognized that while not every risk alert is an indicator of deterioration, every alert should be investigated to determine if the physiological changes are due to deterioration or if they might be an indicator of pain, anxiety, or some other cause. VitalScout tools and the intervention algorithm empower nurses and support assessment and critical thinking to determine the significance of an alert and appropriate intervention.
Challenges encountered by the team included ensuring the correct stakeholders were at the table, creating buy-in for an innovative practice change, and implementing this practice change at 12 hospitals, said Mau. It was vital to have team members representing all disciplines impacting this issue and to take the time to explore the workflow and perspectives of each discipline before developing a solution. The team recognized that if the workflow was too complicated or created additional work for staff, they would likely not use it. Just as vital was the need for clinicians and IT systems analysts to collaborate, with the analysts going so far as to speak to frontline caregivers and observe their daily workflow. This observation led to the suggestion of novel approaches that clinicians didn’t know were possible.
“The concern, especially at the clinical staff level, is it’s going to create more work,” Mau said. “We had to convince them that it could actually decrease workflow.”
Senior leadership from nursing, IT, and medicine championed the program during the development phase, ensuring the team had time, resources, and access to experts. Following the pilot, they coordinated presentations at key leadership meetings to gain support for program implementation at other system hospitals.
To support a systemwide rollout, a program manager was appointed and an implementation playbook was developed to provide step-by-step guidance and resources for implementing the program. At each new hospital, the chief nursing officer (CNO) appointed a hospital team to work alongside the VitalScout team. A standardized timeline and education plan was followed, with team members allotted time to meet role responsibilities. The CNO coordinated VitalScout presentations at all key leadership and physician meetings, and hospital leaders publicly endorsed the program.
While VitalScout was originally created for adult medical, surgical, and step-down units, it has been expanded to include postacute and drug/alcohol treatment units. RRT members now monitor the VitalScout screensaver, investigating changes and collaborating with staff nurses to determine the significance of a change and to address it proactively. A version of VitalScout appropriate for pediatric patients is currently in development.
Healthcare facilities looking to improve response and intervention for early signs of clinical deterioration can create a similar comprehensive program. By leveraging existing resources, they can develop tools within their EMR to support use of an EWS. A great starting place is to create a shared report within the EMR that collates key data to one location; this can support critical thinking and interdisciplinary collaboration to determine the significance of an EWS alert and the appropriate course of action.
“We’re able to better address the needs of the patient because we have better communication,” said Mau.
References
Burns, K., Reber, T., Theodore, K., Welch, B., Roy, D., & Siedlecki, S. L. (2017). Enhanced early warning system impact on nursing practice: A phenomenological study. Journal of Advanced Nursing, 74(5), 1150–1156. doi:10.1111/jan.13517
Mau, K., Fink, S., Hicks, B., Brookhouse, A., Flannery, A. M., & Siedlecki, S. L. (2019). Advanced technology leads to earlier intervention for clinical deterioration on medical/surgical units. Applied Nursing Research, 49, 1–4. doi:10.1016/j.apnr.2019.07.001
Honorable Mentions
The following submissions were selected as honorable mentions:
System-oriented projects
St. Luke’s Health System, Boise, Idaho
After a close call with infants being nearly exposed to recalled infant formula, the system set up a centralized program that automatically assigns daily alerts to more than 500 end users across an eight-hospital and 14,500-employee workforce. In the six months following June 2016, the system reduced unresolved assigned alerts by 90% and reduced resolution time by 90% (a month to three days).
Greater Hudson Valley Health System, Skokie, Illinois
GHVHS wanted to be able to measure harm and quality across all areas and relative to all payers through a single, comprehensive system that delivered the type of performance everyone involved would want for their own families. The system created a Harm Index to document and evaluate every patient harm and near-miss. With 2014 as the baseline year, the Harm Index produced a 35% decrease in total harms in its first year. From 2015 to YTD 2019, the total reduction was 48%. Since the goal is zero harms, however, GHVHS dug deeper to see how the trends looked per 1,000 discharges. The numbers here were similar, with a 34% decrease from 2014 to 2015 and a 51% decrease through YTD 2019.
Single-issue projects
Fox Chase Cancer Center, Philadelphia
The center wanted to implement a process for utilizing nasogastric tube feeding (NGT) for acute dysphagia during radiation treatment for head and neck cancers, recognizing that some patients would prefer percutaneous gastronomy (PEG) tube placement as an alternate method of nutrition. Fox Chase performed a thorough literature review comparing NGT use to PEG use in patients with acute head and neck cancer treatment-induced dysphagia. Three processes were developed, including identifying reactive tube needs, securing NGT to the nose, and NGT insertion. After collecting data on all patients undergoing radiation treatment for head and neck cancers, Fox Chase found that utilization of NGT in lieu of PEG has had a positive effect on multiple parameters for patients experiencing severe acute dysphagia during RT. Since the project launched, 16% of patients undergoing RT required a reactive feeding tube during treatment, about half received an NGT.
Henry Ford Health System, Detroit
The system wanted to improve its endoscope processing methods and education. The solution was to develop a hands-on, interactive course to focusing on issues identified during audits and issues identified by our endoscope repair vendor. The course was designed to be interactive, useful, relevant to the work that the staff are performing, and memorable. In 2018, major endoscope repairs decreased by 15%, and minor repairs increased by 203%. For 2019, the system is projecting that major repairs will decrease by 32% and minor repairs will increase by 21%. Since the beginning of the project, repair costs decreased $1.3 million.