Trial Outcomes & Findings for Behavioral Economics to Improve Flu Vaccination Using EHR Nudges Replication (NCT NCT06626321)
NCT ID: NCT06626321
Last Updated: 2026-03-30
Results Overview
The primary outcome is flu vaccination completion during the first eligible primary care visit.
Recruitment status
COMPLETED
Study phase
NA
Target enrollment
26248 participants
Primary outcome timeframe
4 days from enrollment, at the eligible visit
Results posted on
2026-03-30
Participant Flow
Waiver of informed consent obtained. Patients were identified via Epic Clarity Query 4 days prior to their eligible primary care visit between October 21, 2024 - February 28, 2025. Patients were only counted once within the study window at their first primary care visit. Total protocol enrollment: 26,248 patients. No clinicians were individually enrolled as participants and no clinician outcomes were evaluated.
Clinics were randomized 2:1 to intervention and control. Patients seen at an intervention clinic and identified as high risk were further randomized 1:1 to standard messaging or an intensification nudge. The high risk individual randomization is nested within the intervention arm resulting in an overlap of patients where high risk individuals (15,163 of the 18,022) contribute both to estimation of the overall intervention effect and comparison of text messaging intensity among high risk patients
Unit of analysis: Clinics
Participant milestones
| Measure |
Control
Patients in clinics randomized to the control arm will receive standard of care.
|
Intervention
Patients in clinics randomized to the intervention arm will receive the toolkit of clinician and patient facing nudges. Patient nudges will be previsit text message reminders (standard messaging content). Clinician nudges will be monthly peer comparison feedback and default pended orders. No clinicians were individually enrolled as participants and no clinician outcomes were evaluated.
|
High Risk - Standard Messaging
Patients in the intervention clinics identified as high risk for noncompletion of the flu vaccine were randomized 1:1 to receive the high risk intensification messaging or remain in the standard messaging arm. Patients in the high risk standard messaging arm received the same standard messaging as average risk patients.
|
High Risk - Intensification Messaging
Patients in the intervention clinics identified as high risk for noncompletion of the flu vaccine were randomized 1:1 to receive the high risk intensification messaging or remain in the standard messaging arm. Patients in the high risk intensification group received an additional bidirectional texting component in addition to the standard messaging.
|
|---|---|---|---|---|
|
Clinic Level Randomization
STARTED
|
8226 10
|
18022 20
|
0 0
|
0 0
|
|
Clinic Level Randomization
COMPLETED
|
8226 10
|
18022 20
|
0 0
|
0 0
|
|
Clinic Level Randomization
NOT COMPLETED
|
0 0
|
0 0
|
0 0
|
0 0
|
|
Individual Randomization
STARTED
|
0 0
|
0 0
|
7593 0
|
7570 0
|
|
Individual Randomization
COMPLETED
|
0 0
|
0 0
|
7593 0
|
7570 0
|
|
Individual Randomization
NOT COMPLETED
|
0 0
|
0 0
|
0 0
|
0 0
|
Reasons for withdrawal
Withdrawal data not reported
Baseline Characteristics
Behavioral Economics to Improve Flu Vaccination Using EHR Nudges Replication
Baseline characteristics by cohort
| Measure |
Control
n=8226 Participants
Patients in clinics randomized to the control arm will receive standard of care.
|
Intervention
n=18022 Participants
Patients in clinics randomized to the intervention arm will receive the toolkit of clinician and patient facing nudges. Patient nudges will be pre-visit text message reminders (standard messaging content). Clinician nudges will be monthly peer comparison feedback and default pended orders.
|
Total
n=26248 Participants
Total of all reporting groups
|
|---|---|---|---|
|
Age, Continuous
|
66.99 years
STANDARD_DEVIATION 10.80 • n=4 Participants
|
66.12 years
STANDARD_DEVIATION 10.27 • n=28 Participants
|
66.40 years
STANDARD_DEVIATION 10.45 • n=10 Participants
|
|
Sex: Female, Male
Female
|
4268 Participants
n=4 Participants
|
9416 Participants
n=28 Participants
|
13684 Participants
n=10 Participants
|
|
Sex: Female, Male
Male
|
3958 Participants
n=4 Participants
|
8606 Participants
n=28 Participants
|
12564 Participants
n=10 Participants
|
|
Race (NIH/OMB)
American Indian or Alaska Native
|
12 Participants
n=4 Participants
|
25 Participants
n=28 Participants
|
37 Participants
n=10 Participants
|
|
Race (NIH/OMB)
Asian
|
108 Participants
n=4 Participants
|
250 Participants
n=28 Participants
|
358 Participants
n=10 Participants
|
|
Race (NIH/OMB)
Native Hawaiian or Other Pacific Islander
|
8 Participants
n=4 Participants
|
4 Participants
n=28 Participants
|
12 Participants
n=10 Participants
|
|
Race (NIH/OMB)
Black or African American
|
272 Participants
n=4 Participants
|
619 Participants
n=28 Participants
|
891 Participants
n=10 Participants
|
|
Race (NIH/OMB)
White
|
7400 Participants
n=4 Participants
|
16141 Participants
n=28 Participants
|
23541 Participants
n=10 Participants
|
|
Race (NIH/OMB)
More than one race
|
0 Participants
n=4 Participants
|
0 Participants
n=28 Participants
|
0 Participants
n=10 Participants
|
|
Race (NIH/OMB)
Unknown or Not Reported
|
426 Participants
n=4 Participants
|
983 Participants
n=28 Participants
|
1409 Participants
n=10 Participants
|
|
Race/Ethnicity, Customized
Not Hispanic or Latino/Unknown
|
7722 Participants
n=4 Participants
|
16829 Participants
n=28 Participants
|
24551 Participants
n=10 Participants
|
|
Race/Ethnicity, Customized
Hispanic or Latino
|
504 Participants
n=4 Participants
|
1193 Participants
n=28 Participants
|
1697 Participants
n=10 Participants
|
|
Lowest Quartile Income by Zip Code
Above Lowest Income Quartile by Zip Code
|
5665 Participants
n=4 Participants
|
13529 Participants
n=28 Participants
|
19194 Participants
n=10 Participants
|
|
Lowest Quartile Income by Zip Code
Lowest Income Quartile by Zip Code
|
2561 Participants
n=4 Participants
|
4493 Participants
n=28 Participants
|
7054 Participants
n=10 Participants
|
PRIMARY outcome
Timeframe: 4 days from enrollment, at the eligible visitThe primary outcome is flu vaccination completion during the first eligible primary care visit.
Outcome measures
| Measure |
Control
n=8226 Participants
Patients in clinics randomized to the control arm will receive standard of care.
|
Intervention
n=18022 Participants
Patients in clinics randomized to the intervention arm will receive the toolkit of clinician and patient facing nudges. Patient nudges will be previsit text message reminders (standard messaging content). Clinician nudges will be monthly peer comparison feedback and default pended orders.
|
High Risk - Standard Messaging
n=7593 Participants
Patients in the intervention clinics identified as high risk for noncompletion of the flu vaccine were randomized 1:1 to receive the high risk intensification messaging or remain in the standard messaging arm. Patients in the high risk standard messaging arm received the same standard messaging as average risk patients.
|
High Risk - Intensification Messaging
n=7570 Participants
Patients in the intervention clinics identified as high risk for noncompletion of the flu vaccine were randomized 1:1 to receive the high risk intensification messaging or remain in the standard messaging arm. Patients in the high risk intensification group received an additional bidirectional texting component in addition to the standard messaging.
|
|---|---|---|---|---|
|
Proportion of Patients Who Receive the Flu Vaccine at the Visit
|
689 Participants
|
2217 Participants
|
645 Participants
|
628 Participants
|
SECONDARY outcome
Timeframe: 3 monthsThe secondary outcome is flu vaccination completion within 3 months after the first eligible primary care visit.
Outcome measures
| Measure |
Control
n=8226 Participants
Patients in clinics randomized to the control arm will receive standard of care.
|
Intervention
n=18022 Participants
Patients in clinics randomized to the intervention arm will receive the toolkit of clinician and patient facing nudges. Patient nudges will be previsit text message reminders (standard messaging content). Clinician nudges will be monthly peer comparison feedback and default pended orders.
|
High Risk - Standard Messaging
n=7593 Participants
Patients in the intervention clinics identified as high risk for noncompletion of the flu vaccine were randomized 1:1 to receive the high risk intensification messaging or remain in the standard messaging arm. Patients in the high risk standard messaging arm received the same standard messaging as average risk patients.
|
High Risk - Intensification Messaging
n=7570 Participants
Patients in the intervention clinics identified as high risk for noncompletion of the flu vaccine were randomized 1:1 to receive the high risk intensification messaging or remain in the standard messaging arm. Patients in the high risk intensification group received an additional bidirectional texting component in addition to the standard messaging.
|
|---|---|---|---|---|
|
Proportion of Patients Who Receive the Flu Vaccine Within 3 Months After the Visit
|
947 Participants
|
2630 Participants
|
765 Participants
|
762 Participants
|
Adverse Events
Control
Serious events: 0 serious events
Other events: 2 other events
Deaths: 0 deaths
Intervention
Serious events: 0 serious events
Other events: 502 other events
Deaths: 0 deaths
Serious adverse events
| Measure |
Control
n=8226 participants at risk
Patients in clinics randomized to the control arm will receive standard of care.
|
Intervention
n=18022 participants at risk
Patients in clinics randomized to the intervention arm will receive the toolkit of clinician and patient facing nudges. Patient nudges will be pre-visit text message reminders (standard messaging content). Clinician nudges will be monthly peer comparison feedback and default pended orders.
|
|---|---|---|
|
Injury, poisoning and procedural complications
Flu Vaccine Reaction
|
0.00%
0/8226 • Each patient was monitored for adverse events for 3 months. Adverse events were collected at the end of the trial follow up period, 3 months after enrollment ended (May 2025), for all enrolled participants.
Per our DSMP, severe adverse events will not include death as no reasonable evidence exists to suggest death would result from outreach or communication to encourage flu vaccination. Therefore, All-Cause Mortality was not assessed. AEs are assessed for the Main Study Period arms only - Control (except Messaging, see footnote) and Intervention - as the patients in the High Risk arms are already accounted for in the Intervention arm total. AEs were not monitored or assessed for clinicians.
|
0.00%
0/18022 • Each patient was monitored for adverse events for 3 months. Adverse events were collected at the end of the trial follow up period, 3 months after enrollment ended (May 2025), for all enrolled participants.
Per our DSMP, severe adverse events will not include death as no reasonable evidence exists to suggest death would result from outreach or communication to encourage flu vaccination. Therefore, All-Cause Mortality was not assessed. AEs are assessed for the Main Study Period arms only - Control (except Messaging, see footnote) and Intervention - as the patients in the High Risk arms are already accounted for in the Intervention arm total. AEs were not monitored or assessed for clinicians.
|
Other adverse events
| Measure |
Control
n=8226 participants at risk
Patients in clinics randomized to the control arm will receive standard of care.
|
Intervention
n=18022 participants at risk
Patients in clinics randomized to the intervention arm will receive the toolkit of clinician and patient facing nudges. Patient nudges will be pre-visit text message reminders (standard messaging content). Clinician nudges will be monthly peer comparison feedback and default pended orders.
|
|---|---|---|
|
Social circumstances
Messaging Opt-out
|
—
0/0 • Each patient was monitored for adverse events for 3 months. Adverse events were collected at the end of the trial follow up period, 3 months after enrollment ended (May 2025), for all enrolled participants.
Per our DSMP, severe adverse events will not include death as no reasonable evidence exists to suggest death would result from outreach or communication to encourage flu vaccination. Therefore, All-Cause Mortality was not assessed. AEs are assessed for the Main Study Period arms only - Control (except Messaging, see footnote) and Intervention - as the patients in the High Risk arms are already accounted for in the Intervention arm total. AEs were not monitored or assessed for clinicians.
|
2.8%
498/18022 • Each patient was monitored for adverse events for 3 months. Adverse events were collected at the end of the trial follow up period, 3 months after enrollment ended (May 2025), for all enrolled participants.
Per our DSMP, severe adverse events will not include death as no reasonable evidence exists to suggest death would result from outreach or communication to encourage flu vaccination. Therefore, All-Cause Mortality was not assessed. AEs are assessed for the Main Study Period arms only - Control (except Messaging, see footnote) and Intervention - as the patients in the High Risk arms are already accounted for in the Intervention arm total. AEs were not monitored or assessed for clinicians.
|
|
Surgical and medical procedures
Duplicate Vaccination
|
0.02%
2/8226 • Each patient was monitored for adverse events for 3 months. Adverse events were collected at the end of the trial follow up period, 3 months after enrollment ended (May 2025), for all enrolled participants.
Per our DSMP, severe adverse events will not include death as no reasonable evidence exists to suggest death would result from outreach or communication to encourage flu vaccination. Therefore, All-Cause Mortality was not assessed. AEs are assessed for the Main Study Period arms only - Control (except Messaging, see footnote) and Intervention - as the patients in the High Risk arms are already accounted for in the Intervention arm total. AEs were not monitored or assessed for clinicians.
|
0.02%
4/18022 • Each patient was monitored for adverse events for 3 months. Adverse events were collected at the end of the trial follow up period, 3 months after enrollment ended (May 2025), for all enrolled participants.
Per our DSMP, severe adverse events will not include death as no reasonable evidence exists to suggest death would result from outreach or communication to encourage flu vaccination. Therefore, All-Cause Mortality was not assessed. AEs are assessed for the Main Study Period arms only - Control (except Messaging, see footnote) and Intervention - as the patients in the High Risk arms are already accounted for in the Intervention arm total. AEs were not monitored or assessed for clinicians.
|
Additional Information
Results disclosure agreements
- Principal investigator is a sponsor employee
- Publication restrictions are in place