A Theory and Evidence-Based Artificial Intelligence-Driven Motivational Digital Assistant to Decrease Vaccine Hesitancy: Intervention Development and Validation

"This digital assistant is effective in improving COVID-19 vaccine literacy and confidence through valid educational content and motivational conversations."

Vaccine hesitancy, defined as a delay in acceptance or refusal of vaccines despite their availability, is a complex and multifaceted issue that is influenced by diverse individual, group, and environmental factors. Didactic education has proven insufficient in effectively inducing behaviour change, but motivational interviewing (MI), an evidence-based psychological counseling technique that uses collaborative conversations, has shown effectiveness in reducing vaccine hesitancy by addressing individual concerns and empowering personal agency to vaccinate. It is possible that incorporating therapeutic dialogues with MI skills into artificial intelligence (AI)-driven chatbots could support self-efficacy during conversations. This study aimed to develop and validate an AI-driven motivational digital assistant in decreasing COVID-19 vaccine hesitancy among Hong Kong adults.

The Vaccine Hesitancy Determinants Matrix Model, developed by the World Health Organization (WHO) Strategic Advisory Group of Experts on Immunization, was used as a theoretical framework in this study to understand the influencing factors of vaccine hesitancy. This model has been widely used in different countries or regions to guide research exploring factors that influence vaccine hesitancy. The model lists three categories of factors: contextual influences, arising due to historic, sociocultural, environmental, health system/institutional, economic, or political factors; individual and group influences, arising from personal perception or social/peer environment influences; and vaccine/vaccination- specific issues, directly related to vaccines or vaccination.

From March to May 2022, one-to-one semi-structured interviews were conducted with Hong Kong adults who were hesitant towards COVID-19 vaccines (i.e., not taking COVID-19 vaccines or receiving involuntary COVID-19 vaccines). From October to December 2022, a multidisciplinary team of researchers in vaccines, psychology, and computer science developed "Auricle", an AI-driven motivational digital assistant, to address vaccine hesitancy. The Vaccine Hesitancy Matrix model and qualitative findings from the interviews guided the development of the intervention logic model and content with five web-based modules: Module 1: Basic Knowledge of COVID-19; Module 2: Basic Knowledge of COVID-19 Vaccine; Module 3: Common Questions about COVID-19 Vaccine; Module 4: Myths about COVID-19 Vaccines; and Module 5: Efforts of the Hong Kong Government. A motivational AI-driven chatbot, powered by natural language processing, tailored to each module was embedded to provide real-time, personalised, and interactive conversations on vaccine-related questions.

To develop the AI-driven motivational digital assistant, the researchers followed a multi-step process:

1. Module topics (e.g., myths about COVID-19 vaccines) were identified, and module contents (e.g., common myths/rumours regarding vaccine safety and efficacy) for each topic and their adaptations for conversational Q&A were developed. To enhance the trustworthiness of module content, two research team members conducted data searches from medical databases (e.g., MEDLINE), the WHO's COVID-19 special website, Hong Kong government websites, and other official sources to identify relevant information and receive regular updates. Multiple-choice questions were designed for each module to encourage reflection, interaction, and engagement.
2. Visual aids (e.g., videos and smart charts) were used to visualise the text information to improve readability and engagement.
3. Two research team members developed MI dialogues tailored to each module, using the four processes of "Engaging, Focusing, Evoking, and Planning" to guide the process. The four principles, including expressing empathy, developing discrepancy, rolling with resistance, and supporting self-efficacy, were incorporated into the dialogues to initiate motivation and commitment to vaccine uptake.
4. Bilingual translation (traditional Chinese and English) for educational content and MI dialogues was conducted.
5. Computer science professionals performed the coding of educational content into web pages, as well as the coding of educational content-adapted Q&A and MI dialogues into an AI-driven chatbot.

Expert evaluation of the intervention was conducted in March 2023, and a pilot test was performed in April 2023 with 12 participants to evaluate the feasibility, acceptability, and preliminary effectiveness of the intervention, followed by refinement based on user feedback. The content validity index from expert evaluation was 0.85. The pilot test showed significant improvements in vaccine-related health literacy (p = 0.021) and vaccine confidence (p = 0.027).

Specifically, users achieved high correct rates (89.33% to 98.33%) after learning the educational content of Modules 1 to 4. In general, increasing trends were observed in ratings of vaccine knowledge confidence (from 6.2 to 8.0), vaccine importance (from 6.6 to 7.2), and vaccine readiness (from 6.1 to 7.1). For the feedback collected by open questions, participants expressed high satisfaction with the programme. Users found the intervention helpful in addressing their concerns and providing valuable knowledge on COVID-19 vaccines. They appreciated the engaging communication style of the motivational AI-driven chatbot, clear navigation, measurable evaluations, and bilingual modes. Users also provided suggestions for improvement. Some users recommended incorporating additional interactive features or multimedia elements to enhance user engagement. Others suggested strengthening the intelligence of the chatbot to improve interaction and engagement. Based on the findings from the pilot test phase, iterative refinements were made to the intervention by professionals in computer science.

In reflecting on the process, the researchers suggest that, through therapeutic dialogues in the chatbot supporting empathy, a sense of personal agency, and evidence-based information, participants are provided with relevant and tailored information to clarify ambivalence and are motivated to make favourable decisions regarding vaccination. As an AI-driven digital assistant, this chatbot can be trained through numerous interactive conversations with users to achieve a more advanced and humanised performance.

Per the researchers, this approach could be applicable to a variety of existing vaccines and, particularly, to future newly developed vaccines. The next step is a fully powered randomised controlled trial (RCT) of the chatbot. Future studies could also compare the cost-effectiveness of an AI-driven motivational digital assistant versus human-delivered MI interventions in reducing vaccine hesitancy.