The effect of using Open Data Kit for Rapid Convenience Monitoring to guide 2017 or 2018 measles vaccination campaign Nigeria: a case study of Lagos state
- 1. National Primary Health Care Development Agency, Abuja, Nigeria
- 2. World Health Organization, Country Office, Abuja, Nigeria
- 3. Technical Assistance Consultant, Gavi, The Vaccine Alliance
- 4. Global Public Health Solutions, Atlanta
- 5. Nigeria Governors Forum, Abuja, Nigeria
Abstract
Introduction: The lack of timely feedback of information during supplemental immunization activities to identify gaps and take decisions affect the quality of the exercise. Real-time monitoring using Open Data Kit (ODK); providing feedback on the performance of the vaccination team, with immediate corrective actions has been found to improve campaign outcome. To bridge this gap of immediate feedback to the vaccination teams, the ODK platform was used for conducting rapid convenience monitoring (RCM) in Lagos State for real-time data collection. The study described the process of RCM on the ODK platform for monitoring the 2017/2018 MVC in Lagos State to improve the campaign outcome.
Methods: RCM was conducted in Lagos State using ODK by supervisors and independent monitors who were trained on how to conduct the exercise. Inside household monitoring was done by sampling a total of 15 households with eligible children and checking for vaccination cards as evidence of vaccination. Outside household monitoring was done by identifying sites outside the houses and sampling four sites with a total of 20 children at each site checking for finger ink. Data submitted through ODK were analyzed using Microsoft Excel.
Results: A total of 2533 rapid convenience monitoring ODK submissions were recorded from the 20 Local Government Areas (LGAs) during the monitoring process. The RCM data for phase one indicated the need for action in days 1 to 3 of the campaign as the State was not reaching their 97% target. For phase two, the State had >97% in days 1 and 2 of the campaign but dropped to 83.1% and 95% in days 3 and 4 of the campaign prompting the need for action.
Conclusion: The use of an ODK platform to conduct RCM in the monitoring of 2017/18 MVC has contributed to improved quality and coverage of the campaign in Lagos State.
Keywords
Measles Vaccination Campaign; Rapid Convenience Monitoring; Open Data Kit; Coverage
Citation
Anugwalu CE, Oteri AJ, Bawa S, Dieng B, Jean Baptiste AE, et al. (2022) The effect of using Open Data Kit for Rapid Convenience Monitoring to guide 2017/2018 measles vaccination campaign Nigeria: a case study of Lagos state. Ann Vaccines Immunization 6(1): 1019.
INTRODUCTION
Measles is among the vaccine-preventable infectious diseases that have contributed to the mortality of children aged 1-59 months around the world [1]. In 2018, over 140 000 measles deaths were recorded globally mainly among children <5 years old, despite the availability of safe and cost-effective vaccine [2]. The World Health Organization (WHO), in 2009, recommended that measles supplemental immunization activities (SIAs) be conducted every 2-4 years in countries with <80% coverage of measles-containing vaccine (MCV) on routine immunization [3].
The Nigerian Measles Elimination Strategic Plan 2019-2028 aims to reduce the measles incidence to <5 cases per million population at the national level, and it is part of global efforts to achieve the elimination of the disease [4]. As part of these efforts, Nigeria conducted a nationwide follow-up measles vaccination campaign (MVC) between October 2017 and April 2018, the 6th after a catch up campaign in 2005/2006. Persistently low routine measles immunization coverage and relatively low measles follow up SIA coverage in 2013 (74%) and 2015 (84.5%) had resulted in a population immunity profile that has allowed increased transmission of measles virus among children. That was the basis for the 2017/2018 national measles vaccination campaign that had improved planning and coordination processes to achieve high campaign quality and the desired 95% coverage at the national level. Several innovations were built in the preparation of the campaign to improve the quality and reach all eligible children with the aim of reducing the disease burden [5].
Rapid Convenience Monitoring (RCM) was introduced to monitor program performance during (i.e., intra) or immediately after implementation to ensure a high-quality campaign [6]. The RCM is a quick and effective monitoring process that provides feedback on the performance of the vaccination team in the area covered and when conducted correctly, identifies unvaccinated children and the reasons for non-vaccination. Besides, corrective action is taken (e.g., reassigning teams to poorly vaccinated areas, modifying the timing of vaccination, or conducting mop-up vaccination activities) [7].
Before the 2017/18 SIAs, the RCM process was mainly conducted by monitors using paper based forms to collect information but to further enhance the process, a real-time electronic component through the use of the Open Data Kit (ODK), www.opendatakit.org) was introduced. ODK is a free and open-sourced software for collecting, managing and using data in resource-constrained environments. The collection of paperbased public health data in sub-Saharan Africa poses a significant challenge in the utilization of the data particularly to monitor the progress of ongoing immunization program [8]. Electronic data capture, on the other hand, is known for the timely availability of data from the field for analysis, better data consistency, and quality [9]. Real-time independent monitoring by the use of information technology has helped many countries (e.g., South Africa, Malawi, Nigeria, and Nepal) in having standardized tools for data collection [10].
Lessons learned from previous streams of the campaign showed a considerable gap in monitoring progress as information on missed children from the paper-based RCM data came after the campaign, which delayed the use of these data for action [11]. To bridge this gap, ODK, which was hitherto used mainly for polio vaccination campaign supervision was extended for conducting RCM in Lagos State for real-time data collection.
This paper aims to describe the use of RCM on the ODK platform for monitoring the 2017/2018 MVC in Lagos State to improve the campaign outcome
METHODS
Design
We described the conduct of Rapid Convenience Monitoring using the Open Data Kit application for real-time monitoring of the implementation of the 2017/18 MVC activities in Lagos State and demonstrated how these real-time data were used for prompt action to improve the vaccination coverage. The realtime ODK based RCM was deployed to all the LGAs in Lagos state during the 2 phases of the Measles SIAs.
Setting
The 2017/2018 MVC was conducted in four streams considering the different geopolitical zones and different phases within the states. Streams one and two, North West (NW) and North East (NE), were implemented in 2017 while streams three and four, North Central (NC) and southern states (i.e., South East (SE), South-South (SS) and South West (SW)) were implemented in 2018 [12]. However, Lagos state, due to its large population and the need for more resources, did not implement alongside other southern states.
The campaign was conducted in Lagos in two phases from 22nd to 27th March and 29th March to 3rd April 2018, each phase lasted for 6 days. During the campaign, the vaccinated children were given vaccination cards and also finger marked with indelible ink as evidence of vaccination to help verify vaccinated children. Lagos State is in the southwestern geopolitical zone of Nigeria with 20 local government areas (LGAs). It is the smallest state by land mass (3,496 km2 ) but densely populated and the second most populous in Nigeria with an estimated population in 2018 of 12,726,694 as projected from the 2006 national census [13]. The campaign targeted children aged 9-59 months, and the number of eligible children targeted in Lagos State for the 2017/2018 MVC was 3,660,480, per the State operational microplan estimates.
Planning and Coordination
Planning and coordination of the whole monitoring processes were done by the National Measles Technical Coordinating Committee which constitutes both government (National Primary Health Care Development Agency) and development partners (i.e., World Health Organization, United Nations Children Fund, United States Centres for Disease Control, African Field Epidemiology Network).
Development of real-time monitoring tool
(i) The standard RCM checklist paper form was uploaded to an online survey tool using the ODK platform. The tool captured variables including the following:Name and location of the area where the RCM is being conducted;
(ii) When (whether during or after the campaign);
(iii) Type (either inside or outside monitoring);
(iv)Number of Children seen, with card or finger marked distributed by age group;
(v) Number of households with eligible children sampled and households with eligible vaccinated; and (vi)Reasons for non-vaccination in case of a missed child
Trained independent monitors conducted the exercise and were provided with a stipend for the whole exercise to cover for mobile data expenses used for transmitting the RCM data. The data tool was downloaded on Android phones from the Google play store. The outcome of the RCM indicated whether the sampled area passed or failed the monitoring followed by the trigger for action, as shown in Table 1.
Table 1: Indicating RCM data triggers for action in the 2018 Measles Vaccination Campaign in Lagos State, Nigeria | ||
Indicators | Triggers for corrective action | |
Inside Household | i) Number of completely vaccinated households (out of 15) | <14 households completely vaccinated |
ii) Percentage of campaign-eligible children identified in the household during monitoring that were vaccinated during this campaign | <98% vaccinated children | |
Outside Household | i) Percentage of campaign-eligible children identified in the outside monitoring site during monitoring that were vaccinated during this campaign | <98% vaccinated children |
A failure indicated the need for mop-up to vaccinate the missed children.
Training of personnel
Management Support Teams (MST), including both government and partners, were trained using the WHO guidelines on RCM at the national level before deployment to the state. The MST supported the training of the national supervisors deployed to the state as well as the state supervisors and the independent monitors. The state supervisors and independent monitors were trained on how to conduct RCM and transmit the data using ODK on Android phones. Selection criteria for the independent monitors included, individuals who have had experience with Supplemental Immunization Activities (SIAs) with at least a diploma in health or related field, can speak and read English and familiar with the area of assignment and speak the local language of the area
Intra Campaign RCM process
The campaign was in two phases, each phase was for 6 days, and RCM was conducted from the second day of each phase. Supervisors and independent monitors sampled catchment areas covered by the team the previous day to assess the performance of vaccination teams, regardless of their administrative coverage. As the aim is to rapidly identify areas with unvaccinated children, monitors were deployed to areas where children are most likely to have been missed (i.e. hard-to-reach areas, history of noncompliance, slums and gated estates)
Inside household monitoring
Each supervisor or independent monitor sampled a total of 15 households with eligible children for vaccination status. Started by randomly selecting a household and systematically skipping 2 houses until the number of 15 households are reached. The total number of children seen was recorded indicating those that were vaccinated based on the availability of vaccination cards and those not vaccinated based on the absence of vaccination cards.
Outside household monitoring
In the same settlement, each supervisor or independent monitor identified sites where eligible children are found (e.g., schools, street, markets, church, playground) and sampled four sites with a total of 20 children at each site. The 20 children to be sampled were distributed across all ages among the eligible children (i.e., 9-11 months, 12-59 months). Finger marking was used as means to identify vaccinated children outside, since the children will not be with their vaccination cards outside their homes
The trigger for action in (Table 1) was used to determine whether a particular area passed or failed the process. The ward target was 100% vaccinated children, LGA target was 98%, while that of the state was 97%; to enable achievement of 95% at the national level. For wards with <100% coverage, the state team was alerted immediately to send back vaccination teams to mop up these areas. RCM data from Lagos State were submitted real-time through the ODK platform directly to the national (NPHCDA) server. These data were analyzed immediately at the national level, and feedback was sent to states for action (Figure 1).
Figure 1: Flowchart showing how Rapid Convenience Monitoring data is used for action
Independent monitors filled two or more RCM forms on the ODK platform for both inside and outside household monitoring.
Data collection and Analysis
We collected the RCM data submitted through ODK to the NPHCDA server and analyzed them using Microsoft Excel. The results were presented in tables and charts.
RESULTS
A total of 2533 RCM-ODK entries distributed in various proportions among the 20 LGAs of the state was submitted real-time. Eti-Osa had the highest RCM-ODK submissions with 228(9%) while the lowest was Epe with 61(2.4%). Apapa, Kosofe, Lagos Island, Ikorodu and Ajeromi/Ifelodun LGAs had RCM-ODK submissions between 4.2% to 4.8% (Table 2).
Table 2: Distribution of RCM submissions by monitors, 2018 Measles Vaccination Campaign by LGAs, Lagos State, Nigeria | |
Local Government Area | Number of ODK Submission (%) |
Agege | 81(3.2) |
Ajeromi/Ifelodun | 122(4.8) |
Alimosho | 156(6.2) |
Amuwo Odofin | 92(3.6) |
Apapa | 106(4.2) |
Badagry | 167(6.6) |
Epe | 61(2.4) |
Eti Osa | 228(9.0) |
Ibeju Lekki | 130(5.1) |
Ifako/Ijaye | 146(5.8) |
Ikeja | 171(6.8) |
Ikorodu | 117(4.6) |
Kosofe | 108(4.3) |
Lagos Island | 111(4.4) |
Lagos Mainland | 73(2.9) |
Mushin | 92(3.6) |
Ojo | 74(2.9) |
Oshodi/Isolo | 149(5.9) |
Shomolu | 157(6.2) |
Surulere | 192(7.6) |
Total | 2533(100.0) |
The RCM data for phase one indicated the need for action in days 1 to 3 of the campaign as the State was not reaching their 97% target while days 4 to 6 of the campaign had >97%. For phase two, the State had >97% in days 1 and 2 of the campaign but dropped to 83.1% and 95% in days 3 and 4 of the campaign indicating the need for action. However, days 5 and 6 of the campaign showed improvement in the State with >97% cumulative coverage (Table 3).
Table 3: RCM Data by days in both phase one and phase two in Lagos state, 2017/18 Measles Vaccination Campaign | ||||||
Days | Number of RCM Conducted | Number of Children Seen | Number Vaccinated | % Vaccinated | Decision | Mean time to action (hrs) |
Phase One | 1,390 | 16,138 | 15,689 | 97.2 | ||
Day 1 | 24 | 393 | 380 | 96.7 | Action | 48 |
Day 2 | 147 | 1,694 | 1,556 | 91.9 | Action | 24 |
Day 3 | 353 | 4,070 | 3,933 | 96.6 | action | 24 |
Day 4 | 223 | 2,548 | 2,491 | 97.8 | No action | |
Day 5 | 193 | 2,343 | 2,341 | 99.9 | No action | |
Day 6 | 212 | 2,474 | 2,429 | 98.2 | No action | |
Mop-Up | 238 | 2,616 | 2,559 | 97.8 | No action | |
Phase Two | 1,143 | 13,905 | 13,529 | 97.3 | ||
Day 1 | 179 | 2,026 | 1,994 | 98.4 | No action | |
Day 2 | 34 | 404 | 393 | 97.3 | No action | |
Day 3 | 6 | 65 | 54 | 83.1 | action | 24 |
Day 4 | 291 | 3,595 | 3,416 | 95 | action | 48 |
Day 5 | 435 | 5,315 | 5,192 | 97.7 | No action | |
Day 6 | 162 | 1,758 | 1,750 | 99.5 | No action | |
Mop-Up | 36 | 742 | 730 | 98.4 | No action | |
Total | 2,533 | 30,043 | 29,218 | 97.3 |
The range of time to act was between 24-48 hours.
Despite the ≥100% administrative coverage reported from almost all the 20 LGAs, missed children were reported through the RCM. Alimosho, Eti-Osa and Ikeja LGAs which had >100% coverage from the administrative data had 93%, 90.8% and 92.9 % respectively according to the RCM data (Table 4).
Table 4: Administrative Data and Rapid Convenience Monitoring (RCM) in Lagos State, 2017/18 Measles Vaccination Campaign | ||||
LGA | Administrative Data | RCM Data | ||
Target Population | No of Children Vaccinated (%) | Total number of children monitored | Total number of children vaccinated (%) | |
Agege | 1,56,862 | 161, 865 (103.2) | 899 | 854 (95.0) |
Ajeromi/Ifelodun | 3,44,868 | 356, 466 (103.4) | 1406 | 1375 (97.8) |
Alimosho | 4,35,421 | 491, 517 (112.9) | 1706 | 1586 (93.0) |
Amuwo Odofin | 98,151 | 103, 273 (105.2) | 1108 | 1108 (100.0) |
Apapa | 51,443 | 37, 523 (72.9) | 2111 | 2111 (100.0) |
Badagry | 2,01,649 | 201, 468 (99.9) | 1965 | 1933 (98.4) |
Epe | 33,524 | 36, 817 (109.8) | 831 | 831 (100.0) |
Eti Osa | 74,557 | 79, 834 (107.1) | 2532 | 2353 (92.9) |
Ibeju Lekki | 29,565 | 38, 025 (128.6) | 1597 | 1597 (100.0) |
Ifako/Ijaye | 1,48,781 | 155, 072 (104.2) | 1439 | 1413 (98.2) |
Ikeja | 1,15,962 | 120, 899 (104.2) | 1893 | 1719 (90.8) |
Ikorodu | 3,64,631 | 417, 658 (114.2) | 1146 | 1127 (98.3) |
Kosofe | 1,20,824 | 125, 743 (104.1) | 1419 | 1419 (100.0) |
Lagos Island | 55,868 | 74, 098 (132.6) | 1199 | 1163 (97.0) |
Lagos Mainland | 1,10,452 | 112, 941 (102.3) | 1073 | 1070 (99.7) |
Mushin | 2,84,263 | 286, 321 (100.7) | 931 | 901 (96.8) |
Ojo | 3,98,129 | 468, 433 (117.7) | 624 | 617 (98.9) |
Oshodi/Isolo | 3,21,084 | 349, 605 (108.9) | 1864 | 1803 (96.7) |
Shomolu | 1,32,456 | 138, 739 (104.7) | 2086 | 2023 (97.0) |
Surulere | 1,65,492 | 175, 193 (105.9) | 2214 | 2115 (95.5) |
Total | 36,43,982 | 3, 931, 490 (107.9) | 30043 | 29218 (97.3) |
Table 5
Table 5: Reasons for Non-vaccination by household, RCM, Measles Vaccination Campaign 2018, Lagos state. | |
Reasons | Number (%) |
Vaccine Safety | 38 (34.2) |
Religious Belief | 7 (6.3) |
No felt need | 12 (10.8) |
Political differences | 0 (0.0) |
No caregiver consent | 28 (25.2) |
Unhappy with immunization team | 0 (0.0) |
Reason not given | 13 (11.7) |
Other reasons | 13 (11.7) |
Total | 111 (100.0) |
highlights the reasons for unvaccinated children as captured by RCM. 34.2% of the households stated vaccine safety, 25.2% no caregiver consent, while 11.7% did not state any reason.
Though the ODK submission of the RCM by independent monitors and supervisors was made daily, (Figure 2)
Figure 2: RCM-ODK submissions in different LGAs of Lagos state in 2018 MVC
shows the cumulative geocoordinates for the numbers of days the RCM was conducted in the LGAs. The densely populated areas of the map (middle) contain 12 out of the 20 LGAs in Lagos state.
DISCUSSION
We found that the daily follow up on the RCM data from the ODK enhanced the monitoring of the campaign real-time in Lagos State at the National level as well as the state level by indicating areas where children were missed, reasons for nonvaccination and informing actions to bridge the gaps identified. The real-time RCM data facilitated decision making on areas that the teams needed to go back and mop up. The ODK submissions by independent monitors and supervisors were georeferenced as such considered credible and equally used as an accountability tool to ensure they were actually at the locations they were supposed to monitor.
Similarly, a study on use of ODK for RCM by Oh et al., in Nepal, 2016, demonstrated increased reporting of timeliness and completeness of monitoring data needed for swift corrective actions by the higher level in a measles vaccination campaign [14]. Timely reporting improves supervision of the measles vaccination by the national and state level due to the timely availability of the RCM data for action as similarly observed in the Lagos state during the 2018 MVC. Similar actions of reassigning teams to poorly vaccinated settlements and mop up due to the RCM were equally used in the Lagos state situation. ODK-based RCM made it possible for timely follow up on the areas with missed children. Also, there was minimal chances of errors of entry and delayed transmission as experienced with the paper-based RCM data, which sometimes get delayed until the end of the campaigns days. The RCM tool also provided timely information on the reasons for the unvaccinated children identified, which provided the evidence base for rapid response teams to address problems of vaccine hesitancy and resolved cases of noncompliance.
Using administrative data to monitor coverage during the campaign is very unreliable with most times coverages surpassing 100% [15, 16&17]. This tendency also occurred in as almost all the LGAs in the state had >100% administrative coverage during the campaign. Although RCM information is not for estimating coverages, its role should be to identify areas with unvaccinated children and appropriate action instituted to address the gap. The appropriate actions taken following RCM findings, ensure that the quality of the campaign is improved with likely acceptable outcomes. The high administrative coverage may be indicative of denominator issues, numerator issues or data entry errors, which make the administrative coverage data unreliable. There can be bias in the administrative coverage data due to incomplete tallying or reporting of the number of doses administered, poorly documented shifts in population, dependence on outdated census data and vaccination of individual outside the geographical area or target population [18].
This study, though descriptive of what happened during the 2018 measles vaccination campaign has its own limitations. First, the selection of the wards and settlements was non-probability sample; a convenience sampling method with the propensity for identifying areas likely to have unvaccinated children. Secondly, there was no specific call in template designed to record the number of children vaccinated during the mop-up activities to know the percentage of children reached compared to the initially reported findings. However, some supervisors were sent back to confirm that mop ups had been conducted. Also, feedback of actions taken was not provided real-time but through reports to the respective levels, partly because the reasons for unvaccinated children varied across the board, some of which required longer time for intervention by the rapid response teams. There were reported mobile network outages and other network problems, which affected the timely sending of the data. Nevertheless, this did not affect the data entry as entering the data in the ODK platform does not require a cellular network. The supervisors sent these data when they reached areas with a network on the same day. On the other hand, the RCM provided the opportunity for consciously targeting areas with inequity for selection, to ensure they are not left out.
The 2018 MVC had an administrative coverage of 107.9% and post-campaign coverage survey of 93.2% [19]. The ODK RCM process deployed, where areas with unvaccinated children were identified and covered contributed to the improved coverage. Though RCM cannot be used to estimate vaccination coverage because it does not yield statistically valid estimates as it does not use a probability sample of the population [18]. But, it is a convenient and straightforward means of identifying poorly covered areas for corrective measures. The Pan American Health Organization (PAHO), recommends that RCM be used as an efficient method for validating the coverage [20]. This implies that for an area with about 100% administrative coverage, the RCM data should conform to this data with little or no missed children from the same area.
Electronic data collection automatically adds up information, eliminating errors in data entry, loss of the paper or typo in summaries; and this corroborates with a study in Brazil by Teixeira et al., 2011 which found that automated calculations and analysis removed manual calculation errors which were an issue with the paper-based RCM data [6].
CONCLUSION AND RECOMMENDATION
The use of an ODK platform to conduct RCM in the monitoring of 2018 MVC in Lagos State was shown to have contributed to the positive outcomes of the campaign by improving the coverage. The improved coverage was achieved as a result of the real-time availability of the RCM data and prompt action by the National and state team to cover the area where the RCM indicated unvaccinated children. Subsequent SIAs in Nigeria should employ real-time RCM to obtain data on the progress of vaccination campaigns in the states.
ACKNOWLEDGEMENT
We wish to acknowledge the immense support of the State primary health care board, the vaccination team members and the community for the immense contribution towards the successful implementation of the Measles vaccination campaign 2018. We also acknowledge Cheryl Williams and the entire US CDC and team that trained the Independent Monitors on the use of ODK for RCM as well as the World Health Organization who ensured the independent monitors had some extra funds for their data.
FUNDING
This research was part of the documentation of the best practices from the 2017/2018 Measles vaccination campaign and did not receive any specific grant from funding agencies in the public, commercial, or not for profit sectors.
REFERENCES
2. Mahase E. Measles: 142000 people died in 2018, mostly aged under 5 BMJ 2019; 367:16830 doi:10.1136/bmj.16830 (Accesed: 7 Deember 2020)
11.National Primary Health Care Development Agency. 2017-18 Measles Vaccination Campaign Report. 2018.
12.FMOH, NPHCDA 2017/2018 Measles Vaccination Campaign Field Guide.
13.National Population Commission. NPopC_Population_ Estimates2007_2018.
17.National Primary Health Care Development Agency and Partners. Measles Vaccination Campaign Survey 2016 - Final Technical Report. 2016.
18.Luman ET, Cairns KL, Perry R, Gittelman D. Use and abuse of rapid monitoring to assess coverage during mass vaccination campaigns. Bull World Health Organ.. 2007. 85(9): 651.