A 680,000-person Megastudy of Nudges to Encourage Vaccination in Pharmacies

"Increasing compliance with public health recommendations is not just a persuasion problem. Decades of research show that many people fail to follow through on their intentions when it comes to decisions about their health..."

In the autumn of 2020, anticipating the need for insights about behaviourally informed COVID-19 vaccination messaging, a group of academic researchers in the United States (US) partnered with Walmart pharmacies to conduct a megastudy of text-based reminder messages to encourage influenza vaccination.

The researchers randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different reminders sent via short messaging service (SMS; commonly known as text messaging) using a variety of different behavioural science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages.

Independent scientific teams designed th 22 text-messaging interventions to test distinct hypotheses about how to encourage patients to get a flu shot at Walmart. For example, one used humour in an attempt to increase memorability ("Did you hear the joke about the flu? Never mind, we don't want to spread it around"). Another communicated that vaccination was a growing social norm ("More Americans are getting the flu shot than ever"). Yet another sought to increase "goal commitment" through an interactive prompt ("If you plan to get a flu shot at Walmart, commit by texting back: I will get a flu shot"). Some messages were more verbose; others were brief. Some texts were sent on one day only; others were sent on two separate days.

In total, 29.4% of patients in the business-as-usual control condition received a flu vaccine at a Walmart pharmacy between September 25 and December 31 2020. Each of the 22 interventions that were tested significantly increased vaccination rates relative to the control condition (all two-sided, unadjusted Ps < 0.02). Across the 22 experimental conditions, the mean treatment effect was a 2.0 percentage point increase in flu vaccinations relative to the control condition (a 6.8% lift, unadjusted P = 0.003).

The top-performing intervention included 2 text messages. The first conveyed: "It's flu season & you can get a flu shot at Walmart." The second message arrived 72 hours later and reminded patients that "A flu shot is waiting for you at Walmart." This intervention produced a 2.9 percentage point increase in flu vaccinations (a 9.9% lift; unadjusted and adjusted P < 0.001). After applying the James-Stein shrinkage procedure to account for "the winner's curse", the researchers still estimate that this intervention produced a 2.7 percentage point increase in vaccination over the control condition (a 9.3% lift).

Reflecting on the message content of this top-performing message, the researchers suggest that communicating that a vaccine is "waiting for you" accords with research on the endowment effect, showing we value things more if we feel they already belong to us. Further, this message may imply that the pharmacy is recommending vaccination due to have allocated a dose to you. Furthermore, saying a vaccine is "waiting for you" may suggest that getting a vaccine will be fast and easy.

To assess the ex ante predictability of this megastudy's results, the researchers collected forecasts of different interventions' efficacy from experts and lay people. Neither group anticipated that the "waiting for you" messaging would be the best-performing one, underscoring the value of simultaneously testing many different nudges.

In conclusion: "The results of this megastudy suggest that pharmacies can increase flu vaccination rates by sending behaviorally informed, text-based reminders to their patients....These insights from our megastudy of flu vaccinations can potentially inform efforts by pharmacies and hopefully also providers and governments around the world in the ongoing campaign to encourage full vaccination against COVID-19."

Full list of authors, with institutional affiliations: Katherine L. Milkman, University of Pennsylvania; Linnea Gandhi, University of Pennsylvania; Mitesh S. Patel, Ascension Health; Heather N. Graci, University of Pennsylvania; Dena M. Gromet, University of Pennsylvania; Hung Ho, The University of Chicago Booth School of Business; Joseph S. Kay, University of Pennsylvania; Timothy W. Lee, Northwestern University; Jake Rothschild, University of Pennsylvania; Jonathan E. Bogard, University of California; Ilana Brody, University of California; Christopher F. Chabris, Geisinger Health System; Edward Chang, Harvard University; Gretchen B. Chapman, Carnegie Mellon University; Jennifer E. Dannals, Dartmouth College; Noah J. Goldstein, University of California; Amir Goren, Geisinger Health System; Hal Hershfield, University of California; Alex Hirsch, University of Pennsylvania; Jillian Hmurovic, University of Pennsylvania; Samantha Horn, Carnegie Mellon University; Dean S. Karlan, Northwestern University; Ariella S. Kristal, Harvard Business School; Cait Lamberton, University of Pennsylvania; Michelle N. Meyer, Geisinger Health System; Allison H. Oakes, Anthem, Inc.; Maurice E. Schweitzer, University of Pennsylvania; Maheen Shermohammed, Geisinger Health System; Joachim Talloen, Carnegie Mellon University; Caleb Warren, University of Arizona; Ashley Whillans, Harvard University; Kuldeep N. Yadav, University of Pennsylvania; Julian J. Zlatev, Harvard University; Ron Berman, University of Pennsylvania; Chalanda N. Evans, Penn Medicine; Rahul Ladhania, University of Michigan School of Public Health; Jens Ludwig, University of Chicago; Nina Mazar, Boston University; Sendhil Mullainathan, University of Chicago; Christopher K. Snider, Penn Medicine; Jann Spiess, Stanford University; Eli Tsukayama, University of Hawai`i–West O`ahu; Lyle Ungar, University of Pennsylvania; Christophe Van den Bulte, University of Pennsylvania; Kevin G. Volpp, University of Pennsylvania; Angela L. Duckworth, University of Pennsylvania