Trial Outcomes & Findings for Realtime Streaming Clinical Use Engine for Medical Escalation (NCT NCT04026555)
NCT ID: NCT04026555
Last Updated: 2025-01-14
Results Overview
Rate of escalation of care from floor to Stepdown, Telemetry, ICU, per 1,000 patient bed days.
Recruitment status
COMPLETED
Study phase
NA
Target enrollment
2780 participants
Primary outcome timeframe
10 months
Results posted on
2025-01-14
Participant Flow
All patients admitted to the study units were enrolled. There were no opt-outs.
Participant milestones
| Measure |
MEWS++ Monitoring
This consists of all the patients that will be receiving MEWS++ escalation monitoring and provider alerting.
MEWS++ Monitoring: Patient's electronic medical record data will undergo processing by a machine learning algorithm (MEWS++).
Predictor Score: A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Standard of Care Monitoring
Patients in the control arm will have a score calculated but no alert will be sent.
Predictor Score: A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
|---|---|---|
|
Overall Study
STARTED
|
1506
|
1274
|
|
Overall Study
COMPLETED
|
1506
|
1274
|
|
Overall Study
NOT COMPLETED
|
0
|
0
|
Reasons for withdrawal
Withdrawal data not reported
Baseline Characteristics
Realtime Streaming Clinical Use Engine for Medical Escalation
Baseline characteristics by cohort
| Measure |
MEWS++ Monitoring
n=1488 Participants
This consists of all the patients that will be receiving MEWS++ escalation monitoring and provider alerting.
MEWS++ Monitoring: Patient's electronic medical record data will undergo processing by a machine learning algorithm (MEWS++).
Predictor Score: A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Standard of Care Monitoring
n=1252 Participants
Patients in the control arm will have a score calculated but no alert will be sent.
Predictor Score: A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Total
n=2740 Participants
Total of all reporting groups
|
|---|---|---|---|
|
Age, Continuous
|
67.2 years
STANDARD_DEVIATION 17 • n=99 Participants
|
65.1 years
STANDARD_DEVIATION 17.8 • n=107 Participants
|
66.2 years
STANDARD_DEVIATION 17.4 • n=206 Participants
|
|
Sex: Female, Male
Female
|
762 Participants
n=99 Participants
|
666 Participants
n=107 Participants
|
1428 Participants
n=206 Participants
|
|
Sex: Female, Male
Male
|
726 Participants
n=99 Participants
|
586 Participants
n=107 Participants
|
1312 Participants
n=206 Participants
|
|
Ethnicity (NIH/OMB)
Hispanic or Latino
|
472 Participants
n=99 Participants
|
433 Participants
n=107 Participants
|
905 Participants
n=206 Participants
|
|
Ethnicity (NIH/OMB)
Not Hispanic or Latino
|
1016 Participants
n=99 Participants
|
819 Participants
n=107 Participants
|
1835 Participants
n=206 Participants
|
|
Ethnicity (NIH/OMB)
Unknown or Not Reported
|
0 Participants
n=99 Participants
|
0 Participants
n=107 Participants
|
0 Participants
n=206 Participants
|
|
Race/Ethnicity, Customized
White
|
541 Participants
n=99 Participants
|
401 Participants
n=107 Participants
|
942 Participants
n=206 Participants
|
|
Race/Ethnicity, Customized
Black
|
367 Participants
n=99 Participants
|
287 Participants
n=107 Participants
|
654 Participants
n=206 Participants
|
|
Race/Ethnicity, Customized
Asian
|
83 Participants
n=99 Participants
|
94 Participants
n=107 Participants
|
177 Participants
n=206 Participants
|
|
Race/Ethnicity, Customized
Other
|
480 Participants
n=99 Participants
|
448 Participants
n=107 Participants
|
928 Participants
n=206 Participants
|
|
Race/Ethnicity, Customized
Missing
|
17 Participants
n=99 Participants
|
22 Participants
n=107 Participants
|
39 Participants
n=206 Participants
|
|
BMI
|
27.1 kg per meters squared
STANDARD_DEVIATION 7.68 • n=99 Participants
|
26.5 kg per meters squared
STANDARD_DEVIATION 9.1 • n=107 Participants
|
26.9 kg per meters squared
STANDARD_DEVIATION 8.34 • n=206 Participants
|
|
Elixhauser Score
|
2.2 units on a scale
STANDARD_DEVIATION 1.4 • n=99 Participants
|
2.2 units on a scale
STANDARD_DEVIATION 1.4 • n=107 Participants
|
2.2 units on a scale
STANDARD_DEVIATION 1.4 • n=206 Participants
|
|
First Deterioration Score on Admission
|
0.56 proportion probability
STANDARD_DEVIATION 0.1 • n=99 Participants
|
0.57 proportion probability
STANDARD_DEVIATION 0.096 • n=107 Participants
|
0.57 proportion probability
STANDARD_DEVIATION 0.098 • n=206 Participants
|
PRIMARY outcome
Timeframe: 10 monthsRate of escalation of care from floor to Stepdown, Telemetry, ICU, per 1,000 patient bed days.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Overall Rate of Escalation
|
12.3 escalations per 1,000 patient bed days
|
11.3 escalations per 1,000 patient bed days
|
SECONDARY outcome
Timeframe: 10 monthsNumber of participants requiring blood pressure support agents such as initiation of vasopressor medication or administration of fluid bolus.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Number of Participants Requiring Blood Pressure Support
|
239 Participants
|
142 Participants
|
SECONDARY outcome
Timeframe: 10 monthsPopulation: No statistical comparison performed due to low rate of events.
Number of participants requiring respiratory support intervention such as initiation of nasal cannula to high flow or frequency of intubation.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Number of Participants Requiring Respiratory Support
|
15 Participants
|
5 Participants
|
SECONDARY outcome
Timeframe: 10 monthsPopulation: No statistical comparison performed due to no events.
The number of patients who had a cardiac arrest.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Number of Participants Who Experienced a Cardiac Arrest Episode
|
0 Participants
|
0 Participants
|
SECONDARY outcome
Timeframe: Duration of hospital stay, until discharge, regardless of stay length for patients who died in hospital, or 30 days after admission, starting from date of admission, up to 6 weeks.Number of Mortalities - Combined In-hospital and 30-day mortality. Mortalities only counted once. 30-day mortality includes those patients who died in-hospital within 30 days.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Mortality Rate
Combined in-hospital and 30-day
|
104 Participants
|
117 Participants
|
|
Mortality Rate
In-hospital mortality
|
81 Participants
|
83 Participants
|
|
Mortality Rate
30-d mortality
|
87 Participants
|
98 Participants
|
SECONDARY outcome
Timeframe: 10 monthsPopulation: No statistical analysis performed
The number is calculated as average number of alerts sent per day over all the days in the study period.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Notification Frequency - Number of Alerts Sent Per Day to Providers
Total Alerts
|
6.7 alerts per day
Standard Deviation 3.0
|
5.4 alerts per day
Standard Deviation 2.5
|
|
Notification Frequency - Number of Alerts Sent Per Day to Providers
Primary Alerts
|
4.5 alerts per day
Standard Deviation 2.3
|
3.5 alerts per day
Standard Deviation 1.9
|
|
Notification Frequency - Number of Alerts Sent Per Day to Providers
RRT Alerts
|
2.2 alerts per day
Standard Deviation 1.7
|
1.9 alerts per day
Standard Deviation 1.5
|
SECONDARY outcome
Timeframe: 10 monthsPopulation: Data were not collected for this outcome measure.
The average number of calls to RRT made per patient, regardless of alert. Not evaluated.
Outcome measures
Outcome data not reported
SECONDARY outcome
Timeframe: 10 monthsPopulation: Patients in the intervention arm.
The performance of the alert will be evaluated by calculating the sensitivity, specificity, positive predictive value, negative predictive value, precision, recall, and F1-score. This will be done both for the overall escalation rate and if possible for individual escalations (ICU, step-down, telemetry) and death.
Outcome measures
| Measure |
Intervention - Alert Sent
n=1506 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Sensitivity and Specificity of the RRT Alert
Sensitivity
|
0.88 proportion
|
—
|
|
Sensitivity and Specificity of the RRT Alert
Specificity
|
0.33 proportion
|
—
|
|
Sensitivity and Specificity of the RRT Alert
Positive Predictive Value
|
0.15 proportion
|
—
|
|
Sensitivity and Specificity of the RRT Alert
Negative Predictive Value
|
0.95 proportion
|
—
|
POST_HOC outcome
Timeframe: From time of alert until transfer to an ICU, assessed up to discharge from the hospital or deathTime in hours between alert and transfer to an ICU
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Time to ICU Escalation
|
49.7 hours
Interval 21.6 to 101.0
|
57.8 hours
Interval 25.2 to 113.0
|
POST_HOC outcome
Timeframe: Duration of hospital stay, starting from admission, up to the day of discharge, regardless of the length of hospital stay, up to 1 year.Population: Patients who survived to hospital discharge
Hazard ratio for faster earlier discharge for patients who got an alert
Outcome measures
| Measure |
Intervention - Alert Sent
n=1488 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated. If the prediction score exceeds a predetermined threshold, an alert will be sent to the provider. The alerting protocol is tiered, with both a low and high threshold. If the score is above the low threshold, nursing will be notified. If the score is above the high threshold, RRT will be notified.
|
Control - Standard of Care Monitoring
n=1252 Participants
A score predicting the likelihood that the patient will experience a deterioration in their clinical condition within six hours will be generated.Patients in the control arm will have a score calculated but no alert will be sent.
|
|---|---|---|
|
Likelihood of Earlier Hospital Discharge
|
1407 Participants
|
1,169 Participants
|
Adverse Events
MEWS++ Monitoring
Serious events: 0 serious events
Other events: 0 other events
Deaths: 104 deaths
Standard of Care Monitoring
Serious events: 0 serious events
Other events: 0 other events
Deaths: 117 deaths
Serious adverse events
Adverse event data not reported
Other adverse events
Adverse event data not reported
Additional Information
Dr. Matthew Levin
Icahn School of Medicine at Mount Sinai
Phone: 212-241-8382
Email: [email protected]
Results disclosure agreements
- Principal investigator is a sponsor employee
- Publication restrictions are in place