Clinical Trial Outcomes

reduction in length of stay

reduction in length of stay

reduction in mortality

reduction in mortality

Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial

InSight’s effect on average patient length-of-stay (LOS) and in-hospital mortality rate was evaluated in an NIH- and NSF-supported randomized controlled clinical trial conducted in two medical-surgical intensive care units at the University of California, San Francisco (UCSF) Medical Center. Trial registration on ClinicalTrials.gov: NCT03015454.

InSight Sepsis Prediction

InSight analyzes correlations and trends among measurements commonly entered into the EHR to generate predictive risk scores for sepsis, severe sepsis, and septic shock, achieving an area under the receiver operator characteristic (ROC) curve of 0.92, 0.88, and 0.97, respectively. During the clinical trial, InSight demonstrated combined sensitivity of 0.90 and specificity of 0.90 and made predictions up to four hours in advance of sepsis onset.

InSight Sepsis Prediction
Figure 1. Receiver Operator Characteristic (ROC) curves for InSight and common scoring systems at time of (A) sepsis onset,
(B) severe sepsis onset, (C) four hours before septic shock onset. SIRS – systemic inflammatory response syndrome,
MEWS – modified early warning score, SOFA – sequential organ failure assessment.
Specificity
Sensitivity
4 hour  Look Ahead

Study Design

This RCT aimed to assess two main target outcomes, reduction of average length of stay (LOS) and reduction of in-hospital mortality rate, and allowed for a real-time comparison of InSight and the hospital’s rules-based severe sepsis detector. Adult patients admitted to participating units were randomly assigned to either the experimental group or the active comparator control group. Ultimately, 142 patients were enrolled, 75 of which were placed in the control group and 67 of which were placed in the experimental group. There were no statistically significant demographic differences between the two groups.

The experimental group was monitored by both InSight and the pre-existing EHR-based severe sepsis detector (a Best Practice Alert, or BPA, on the Epic EHR), which is a rules-based modified Systemic Inflammatory Response Syndrome (SIRS) screen. The control group received only the EHR-based modified SIRS screening. Patients in the control group who met the threshold cut-offs set by the severe sepsis predictor were indicated as such in the EHR. When a patient in the experimental group trended toward severe sepsis, InSight delivered a phone call warning to the charge nurse that prompted patient evaluation and possible treatment. Patients in both groups received the same standard of care. Following nurse suspicion of infection, a physician made an assessment of the patient and if appropriate, ordered administration of the sepsis bundle.

Outcomes

In the experimental group, average LOS decreased by 21%, a statistically significant reduction (p = 0.042) relative to the control group. In-hospital mortality rate declined by 58%, another statistically significant reduction (p = 0.018). Further, the number of septic shock cases fell from 5.3% in the control group to 1.5% in the experimental group and the experimental group was attended to and treated sooner than the control group. On average, patients in the experimental group were administered antibiotics 2.76 hours earlier and had blood cultures drawn 2.79 hours earlier than those in the control group.

As of March 2017, this is the only randomized controlled trial of a sepsis surveillance system that demonstrates statistically significant reductions in average LOS and in-hospital mortality. Prior attempts to investigate electronic sepsis surveillance tools primarily include retrospective studies and some non-interventional prospective observational studies. Both of these types of study designs can only estimate, but not clinically validate, impact on patient outcomes. Further, much of the literature in this field is dedicated to the study of rules-based sepsis detection rather than forecasting sepsis onset with machine-learning models.

Outcomes
Table 1. Differences in hospital length of stay (LOS), ICU LOS, and in-hospital mortality between the experimental
and control groups. The mean and the standard error (in parentheses) for each outcome are noted in the table.
All outcomes demonstrate statistically significant reductions when using InSight (P < .05).

Interested in learning more?

Contact Us