Outcomes and Costs of the Transition From a Paper-Based Immunization System to a Digital Immunization System in Vietnam: Mixed Methods Study

"...recommend that other provinces in Vietnam transition to paperless reporting by introducing the guidelines and standard operating procedures used in Hanoi and Son La and providing ongoing support through trainings, data review meetings, supportive supervision, and peer networks."



The National Immunization Information System (NIIS) was introduced nationwide in Vietnam in 2017. Health workers were expected to use the NIIS alongside the legacy paper-based system. Starting in 2018, Hanoi and Son La provinces transitioned to paperless reporting. Conducted in November 2021, this study assesses (i) changes in NIIS data quality and use, (ii) changes in immunisation programme outcomes, and (iii) the economic costs of using the NIIS versus the traditional paper system.



Key interventions implemented as part of the transition to paperless reporting included:

• Guidelines and training on the shift to paperless reporting, implemented through a training of trainers and cascaded training approach for health workers;
• Data quality and data use guidelines and training for health workers at the province, district, and commune health centre levels;
• Internet-based data review meetings at the district level, where all communes share progress on paperless transition and identify challenges to prioritise areas for support;
• Additional supportive supervision visits from the government and PATH at district and commune facilities, which shifted to internet-based supportive supervision visits during the COVID-19 pandemic; and
• Zalo (Vietnam's social media and chat app) groups for end users (at least 1 NIIS user per facility) in each district to exchange knowledge and experiences in NIIS use.

The mixed-methods study found that NIIS data timeliness significantly improved from pre- to postintervention in both Hanoi and Son La provinces. For example, the mean number of days from birth date to NIIS registration before and after intervention dropped from 18.6 (standard deviation (SD) 65.5) to 5.7 (SD 31.4) days in Hanoi (P<.001) and from 36.1 (SD 94.2) to 11.7 (40.1) days in Son La (P<.001). Data from Son La showed that the completeness and accuracy improved, while Hanoi exhibited mixed results, possibly influenced by the COVID-19 pandemic.



All respondents rated their facility's data quality in the NIIS as "good" or "very good" in terms of timeliness, completeness, and accuracy. All respondents mentioned human resources as the most important factor associated with data quality, including health workers' knowledge and skills (in data entry, analysis, quality assessment, and use), understanding the importance of data quality, and bandwidth to support the immunisation programme when working across health areas.



Data use improved; at postintervention, 100% (667/667) of facilities in both provinces used NIIS data for activities beyond monthly reporting compared with 34.8% (202/580) in Hanoi and 29.4% (55/187) in Son La at preintervention.



Across nearly all antigens, the percentage of children who received the vaccine on time was higher in the postintervention cohort compared with the preintervention cohort. With more timely data captured in the NIIS, health workers know which vaccines a child is due for and can send reminders or follow-up to deliver the vaccines on time. For the clients using the e-immunisation card, it may have also contributed to timely vaccination by reminding clients of their vaccination schedule and due dates.



In Son La, improvements in data quality and on-time vaccination were also associated with improvements in vaccination coverage. Full immunisation at 12 months was 88.3% (5,040/5,705) in the postintervention cohort compared with 63.3% (3,947/6,233) in the preintervention cohort. However, in Hanoi, despite improvements in on-time vaccination, there were declines in vaccination coverage, again likely because of the COVID-19 pandemic.



Up-front costs associated with developing and deploying the NIIS were estimated at US$0.48 per child in the study provinces. The commune health centre level showed cost savings from changing from the paper system to the NIIS, mainly driven by human resource time savings. At the administrative level, incremental costs resulted from changing from the paper system to the NIIS, as some costs increased, such as labour costs for supportive supervision and additional capital costs for equipment associated with the NIIS.



In conclusion, the Hanoi and Son La provinces successfully transitioned to paperless reporting while maintaining or improving NIIS data quality and data use. Future studies should monitor data quality and immunisation outcomes in other provinces as well as the sustainability of the observed changes in Hanoi and Son La.