Minimal Impact of Implemented Early Warning Score and Best Practice Alert for Patient Deterioration*

Minimal Impact of Implemented Early Warning Score and Best Practice Alert for Patient Deterioration*

by Bedoya, Armando D.; Clement, Meredith E.; Phelan, Matthew; Steorts, Rebecca C.; O’Brien, Cara; Goldstein, Benjamin A.  

Critical Care Medicine: January 2019 - Volume 47 - Issue 1 - p 49–55

Objectives: Previous studies have looked at National Early Warning Score performance in predicting in-hospital deterioration and death, but data are lacking with respect to patient outcomes following implementation of National Early Warning Score. We sought to determine the effectiveness of National Early Warning Score implementation on predicting and preventing patient deterioration in a clinical setting.

Design: Retrospective cohort study. Setting: Tertiary care academic facility and a community hospital. Patients: Patients 18 years old or older hospitalized from March 1, 2014, to February 28, 2015, during preimplementation of National Early Warning Score to August 1, 2015, to July 31, 2016, after National Early Warning Score was implemented. Interventions: Implementation of National Early Warning Score within the electronic health record and associated best practice alert.

Measurements and Main Results: In this study of 85,322 patients (42,402 patients pre-National Early Warning Score and 42,920 patients post-National Early Warning Score implementation), the primary outcome of rate of ICU transfer or death did not change after National Early Warning Score implementation, with adjusted hazard ratio of 0.94 (0.84–1.05) and 0.90 (0.77–1.05) at our academic and community hospital, respectively. In total, 175,357 best practice advisories fired during the study period, with the best practice advisory performing better at the community hospital than the academic at predicting an event within 12 hours 7.4% versus 2.2% of the time, respectively. Retraining National Early Warning Score with newly generated hospital-specific coefficients improved model performance.

Conclusions: At both our academic and community hospital, National Early Warning Score had poor performance characteristics and was generally ignored by frontline nursing staff. As a result, National Early Warning Score implementation had no appreciable impact on defined clinical outcomes. Refitting of the model using site-specific data improved performance and supports validating predictive models on local data.

Investigating the Impact of Different Suspicion of Infection Criteria on the Accuracy of Quick Sepsis-Related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores

Investigating the Impact of Different Suspicion of Infection Criteria on the Accuracy of Quick Sepsis-Related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores

Churpek, M M et al
Critical Care Medicine: November 2017 - Volume 45 - Issue 11 - p 1805–1812
Abstract

Objective: Objective:Studies in sepsis are limited by heterogeneity regarding what constitutes suspicion of infection. We sought to compare potential suspicion criteria using antibiotic and culture order combinations in terms of patient characteristics and outcomes. We further sought to determine the impact of differing criteria on the accuracy of sepsis screening tools and early warning scores. Design: Design:Observational cohort study. Setting: Setting:Academic center from November 2008 to January 2016. Patients: Patients:Hospitalized patients outside the ICU. Interventions: Interventions:None. Measurements and Main Results: Measurements and Main Results:Six criteria were investigated: 1) any culture, 2) blood culture, 3) any culture plus IV antibiotics, 4) blood culture plus IV antibiotics, 5) any culture plus IV antibiotics for at least 4 of 7 days, and 6) blood culture plus IV antibiotics for at least 4 of 7 days. Accuracy of the quick Sepsis-related Organ Failure Assessment score, Sepsis-related Organ Failure Assessment score, systemic inflammatory response syndrome criteria, the National and Modified Early Warning Score, and the electronic Cardiac Arrest Risk Triage score were calculated for predicting ICU transfer or death within 48 hours of meeting suspicion criteria. A total of 53,849 patients met at least one infection criteria. Mortality increased from 3% for group 1 to 9% for group 6 and percentage meeting Angus sepsis criteria increased from 20% to 40%. Across all criteria, score discrimination was lowest for systemic inflammatory response syndrome (median area under the receiver operating characteristic curve, 0.60) and Sepsis-related Organ Failure Assessment score (median area under the receiver operating characteristic curve, 0.62), intermediate for quick Sepsis-related Organ Failure Assessment (median area under the receiver operating characteristic curve, 0.65) and Modified Early Warning Score (median area under the receiver operating characteristic curve 0.67), and highest for National Early Warning Score (median area under the receiver operating characteristic curve 0.71) and electronic Cardiac Arrest Risk Triage (median area under the receiver operating characteristic curve 0.73). Conclusions: Conclusions:The choice of criteria to define a potentially infected population significantly impacts prevalence of mortality but has little impact on accuracy. Systemic inflammatory response syndrome was the least predictive and electronic Cardiac Arrest Risk Triage the most predictive regardless of how infection was defined.