Introduction: This study compared four established methods of evaluating facility-level emergency obstetric capacity and evaluated the extent to which these measures were associated with volume of deliveries.

Methods: Using health facility assessment data from the 2013-2016 Service Provision and Assessment surveys conducted in Malawi, Haiti, Tanzania, Nepal, Bangladesh and Senegal, facilities were classified into three levels of obstetric capacity (comprehensive, basic and less than basic) using four different emergency obstetric care (EmOC) assessment methods. These methods included

1) Performance of signal function in the past 3 months based on facility self-report;

2) Interviewer-observed availability of the facility’s structural capacity to perform signal functions;

3) Recent or previous performance of signal functions, and

4) A composite index of obstetric care. Poisson regression models were used to examine the performance of each EmOC method in predicting facility volume of deliveries.

Results: There is a huge unmet need for EmOC regardless of the method used. The composite index emerged as the optimal EmOC method for characterizing facility obstetric capacity based on its consistency in classifying obstetric facilities across countries, association with delivery volume in four out of six countries, coverage of multiple domains of obstetric care, and the association of those domains with facility volume of deliveries in all six countries examined. Findings showed heterogeneity across EmOC methods and countries, suggesting that these methods work differently in predicting delivery volume across settings.

Conclusion: The variations in facility readiness to provide comprehensive care depending on the EmOC method used highlights the importance of the lens through which obstetric capacity is evaluated. This heterogeneity in performance of the EmOC methods across countries suggests that there may not be a one-size fits all approach to evaluate obstetric capacity but rather multiple methods are encouraged to guide stakeholders on what closely reflects the true obstetric capacity in any setting

Signal functions, Obstetric assessment, Obstetric care, health facilities, health systems, Service delivery, Low-and-middle-income countries

Amadi-Mgbenka C, Borrell LN, Jones HE, Maroko A, Bolumar F (2021) A comparison of four obstetric assessment methods for determining facility obstetric capacity across six low-and-middle-income countries: The need for consistency in practice. J Prev Med Healthc 3(1): 1020.

More than 90% of maternal deaths occur in low-andmiddle-income countries (LMICs), and are largely attributed to preventable pregnancy-related causes.(1) Despite the global decline in maternal mortality rates (MMR) from 385 to 216 deaths/100,000 live births between 1990 and 2015, maternal mortality disproportionately affects LMICs.(2) Among the LMICs included in this study including Malawi, Nepal, Bangladesh, Haiti, Tanzania and Senegal, maternal mortality ranges between 214 to 556 deaths per 100,000 live births. Hence, mitigating maternal mortality constitutes one of the key Sustainable Development Goals, (2) and comprehensive emergency obstetric care (EmOC) plays a critical role in this process. Comprehensive EmOC has been identified as the most critical intervention for preventing maternal deaths resulting from pregnancy complications.(3,4) Comprehensive EmOC comprises nine (9) signal functions including: administration of parenteral antibiotics, parenteral anticonvulsants, parenteral uterotonics, removal of retained products, assisted vaginal delivery, manual removal of the placenta, resuscitation of the newborn, cesarean section and blood transfusion.(4) The provision of these signal functions across health facilities are routinely assessed by researchers and program personnel, and they serve as a benchmark for evaluating facility readiness to prevent maternal deaths due to pregnancy complications.(4)

The common methods of measuring EmOC include: 1) performance of signal functions in the past 3 months based on facility-report (4); 2) interviewer-observed availability of the facility’s structural capacity to perform signal functions (5); and 3) recent or previous performance of signal functions based on facility report (6,8) and 4) composite index comprising 53 obstetric care indicators.(9) However, most of these methods may not adequately capture important characteristics that reflect the capacities of health facilities in varied settings to provide comprehensive emergency obstetric care. For instance, the first and third methods listed restrict the assessment of facility obstetric capacity to the performance of the nine signal functions. However, these methods fail to account for the array of structural measures that are indicative of EmOC capacity (e.g. availability of clinical guidelines, staff training and supervision, and 24/7 availability of providers, among others) (Appendix 1 and 2). (9,10) Although the second method considers specific structural measures in addition to performance of signal functions (Appendix 1), it lacks comprehensiveness and contextual relevance. In particular, the availability and performance of EmOC in diverse settings is mostly driven by disparities in contextual and facility characteristics, which are not accounted for by most of the existing methods.(11,12,13,14) Some of these disparities include health provider absenteeism, availability of authorized health providers, volume of deliveries, and resource limitations.(4,5,10,15) The fourth method—a composite index of obstetric care—is considered a comprehensive approach since it aggregates a variety of structural and process measures. (9) Yet, the variation in number and type of indicators included across composite indices, alongside the different methods for aggregating these indicators introduce inconsistencies in its definition. The resulting public health and policy implications are that some of these methods misrepresent the real situation of women’s access to emergency obstetric care within their facility service environment. For example, the use of performance of signal functions alone as an indicator of EmOC capacity (method 1 above) has been shown to significantly overestimate health facility readiness to manage obstetric complications,(5) with one study across 44 facilities in Kenya showing this approach overestimated facility readiness to manage pregnancy complications by 54.5%.(5)

Providing an accurate picture of facility obstetric capacity is key to informing policies, especially in LMICs where maternal mortality rates are high. We therefore compare the four different methods of measuring EmOC facility readiness across six countries and identify how each of these methods performs in terms of predicting facility volume of deliveries. Given that delivery volume is associated with indicators of obstetric care,(15) the average facility delivery volume provides a suitable outcome measure for discriminating among the various EmOC assessment methods.

Data sources

This cross-sectional study utilized the most recent Service Provision Assessment (SPA) surveys conducted by the Demographic Health Survey program (DHS) across six countries: Malawi, Haiti, Nepal, Bangladesh, Tanzania and Senegal.(16) The SPA surveys provide a comprehensive overview of a country’s health service delivery.? The SPA utilizes four data collection tools including facility inventory questionnaires, health worker interviews, client interviews and protocol observations for antenatal, family planning and sick child visits. ?This study only included data from the facility inventory and health worker interviews. The facility inventory provided data on the structural and process measures of obstetric care available at the facility level, while the health worker interviews provided data on the provision of obstetric services by health providers. The obstetric services comprise signal functions and their associated structural measures. The availability of obstetric infrastructure and routine performance of signal functions were most commonly assessed at the facility level, while provider-specific performance of signal functions and training requirements were assessed at the provider-level. The providers were linked to their facilities using a unique facility identifier. ?The SPA surveys chosen for the present study included: Malawi (?2013-2014), Haiti (2017- 2018), Bangladesh (2014), Nepal (2015), Tanzania (2014-2015), and Senegal (2016). We selected these countries because their SPA surveys were the most recently conducted across low-andmiddle income countries. In addition, these countries have very high maternal mortality rates among countries with available SPA data.(17) The study populations consisted of facilities that provide delivery services and report having providers who offer delivery/newborn services—and had received training in these services in the past 24 months. The sample sizes included Malawi (n=459 out of 977), Haiti (n=300 out of 1033), Bangladesh (n=449 out of 1548), Nepal (n=507 out of 992), Tanzania (n=790 out of 1188) and Senegal (n=306 out of 484). (Table 1). We excluded facilities that were not offering delivery or newborn services since they are not required to provide obstetric services.

Measures

Independent variables: We utilized four previously established EmOC assessment methods in classifying facilities according to their levels of obstetric capacity. The methods includMethod 1- Facility-reported performance of signal functions: This method classifies facilities as comprehensive or basic EmOC based on the reported performance of all nine signal functions in the past three months.(5) This is the standard method of EmOC evaluation specified by stakeholders including the United Nations and Columbia University’s Averting Maternal Death and Disability group.(4)Method 2- Validation of facility-reported signal functions using the facility’s structural capacity: This method classifies facilities based on the reported performance of only the signal functions corresponding with the observed availability of the structural measures associated with them. This method validates the reported performance of signal functions using the structural capacities to perform them, thereby preventing the lack of correspondence between the facilityreported obstetric performance and structural capacity. (5,18) For instance, if a facility reports the administration of antibiotics, a signal function, then they should have the required medical supplies including ampicillin, amoxycillin, and gentamycin (Appendix 1)Method 3- Recent and prior performance of signal functions: This method classifies facilities based on either the reported performance of signal functions in the past three months or at a prior time. This method is premised on the finding that the lack of recent performance of some signal functions (in the prior three months to facility assessment) in a facility may be largely attributed to an absence of recent demand for those functions. (13, 6,19Method 4- Composite indices of structural and process measures: This method involved aggregating a total of 53 structural and process measures of obstetric care to reflect obstetric capacity. The resulting index included measures of both emergency obstetric care (signal functions) as well as routine obstetric care such as the use of partographs for the “Active Management of the Third Stage of Labor,” presence of infection control equipment, and more. (20) Consistent with a recent report on effective coverage of facility deliveries, the composite index was developed using a weighted additive approach across six domains of the SPA in which each indicator was weighted proportional to the total number of indicators in each domain. Facilities were assigned an aggregated score (out of 100). (9) Details on the development of the composite index have been published in a prior DHS study.(9) The domains and their description are available in Appendix 2.

Based on EmOC methods 1, 2 and 3, facilities in each of the countries were classified into three categories: comprehensive, basic, or less than basic EmOC. These levels of obstetric capacity have been established in the existing literature.4 Notably, comprehensive EmOC facilities are those that report providing all nine signal functions in the past 3 months, basic EmOC facilities are those capable of providing seven signal functions (or less) out of the nine signal functions excluding blood transfusion and cesarean section, while ‘less than basic EmOC facilities’ provide less than seven signal functions (excluding blood transfusion and cesarean section).(4,9,20) To facilitate comparisons between the composite index (a continuous score based on 53 obstetric indicators) and the first three EmOC methods, cut points were selected to mirror the proportion of facilities providing comprehensive, basic, and less than basic EmOC capacity. They included 1) the 90th percentile (high capacity); 2) 75th to less than 90th percentile (medium capacity); and 3) less than 75th percentile (low capacity).

Outcome variable

The outcome variable, facility volume of delivery, was a continuous measure that specified at the provider-level. Specifically, eight providers on average were interviewed in each facility. The providers were asked for the total number of deliveries they conducted in the past six months. We aggregated the average number of deliveries across the providers to the facility level. The rationale for selecting facility volume of deliveries as an outcome to investigate the predictive capacities of the four EmOC assessment methods was consistent with prior studies showing that delivery volume is significantly associated with structural and process indicators of obstetric care.(15,21) The regression models were set to predict volume of deliveries among facilities with high and medium obstetric capacities (reference=low obstetric facilities), since the existing literature has established that higher obstetric capacity tends to drive facility volume of deliveries and individual choice of delivery care.(15,22)

Covariates

Consistent with previous studies,(15,23) several facility characteristics were considered to describe contextual differences across the countries, and were controlled for in the analysis. They included: geographic location (urban, rural), facility type (hospital, health/welfare center, dispensary, clinic, health post, maternity, stand alone, health complex), and managing authority (public, private).

Statistical analysis

Descriptive statistics were computed to characterize the distribution of facilities and providers as well as the nine signal functions for each EmOC assessment method across the six countries. Kappa statistics were estimated to examine the agreement between the sub-levels of obstetric capacity. Specifically, we assessed the agreement of methods 1, 2, and 3 with method 4, as the latter was qualitatively different from the first three methods. Poisson regression models were fitted to examine the associations between each EmOC assessment method and the average volume of deliveries across the facilities. Models were also fitted to examine the association between the domains of the composite index (EmOC method 4) with facility volume of deliveries. An interaction term between country and each method was included in the model to determine whether the association between EmOC method and average volume of deliveries varied significantly by country. SAS survey procedures were employed in computing weighted percentages and standard errors. SAS Proc GENMOD was used for the regression analysis. Sample sizes presented in the tables are unweighted while proportions, standard errors, Kappa, prevalence ratios and 95% confidence intervals are weighted. All the analyses were performed using SAS v. 9.4.

Table 1: Distribution of selected characteristics among facilities offering delivery services in Malawi, Haiti, Bangladesh, Nepal, Tanzania and Senegal, 2014-2018 SPA.

Study population characteristics

Across the six countries, Senegal and Tanzania had the highest proportions of facilities providing delivery services (75.8% and 76.1% respectively), and Bangladesh had the lowest (18.1%) (Table 1). More than half of the facilities in all six countries were situated in rural areas. The majority of these facilities were health/welfare centers, clinics, dispensaries, and health posts. Haiti had the largest proportion of hospitals (30.1%). Delivery services were predominantly managed by public authorities across all countries. In particular, at least twothirds of health facilities providing delivery services in Malawi, Bangladesh, Nepal, Tanzania, and Senegal were managed by public authorities; while in Haiti, it was less than half. About half of the facilities in Malawi (53.8%) and Haiti (49.3%) had 24-hour coverage of health providers compared to around onethird of facilities in Bangladesh and Tanzania, and only 8.5% of facilities in Senegal (Table 1). The median deliveries conducted by the health providers in the last six months varied across the countries, ranging from 10 deliveries per provider in Haiti and Nepal to 30 deliveries per provider in Malawi. Senegal had the second highest number of deliveries (Median: 25, IQR: 10-50).

Emergency obstetric care provision based on performance of signal functions alone (Method 1)

Regardless of countries, the most common signal functions available included administration of parenteral oxytocic drugs, assisted vaginal delivery, parental antibiotics, and neonatal resuscitation (Table 2). However, there were variations across countries. For instance, administration of oxytocic drugs was reported for almost all facilities in Malawi (98.2%) but only by 53.4% of facilities in Bangladesh. Assisted vaginal delivery was the second most common signal function performed by at least 50% of the facilities across the countries. However, in Nepal only 16.4% of facilities assisted vaginal deliveries. Similarly, neonatal resuscitation was performed in 87.9% of facilities in Malawi but only in 40.0% of Nepal facilities. The signal functions least available across countries were administration of parental anticonvulsant for hypertension, blood transfusion and c-sections. The lowest availability for these functions were reported in Nepal (10.4%) and Tanzania (15.7%) for anticonvulsants; and Tanzania (6.0% and 6.9%) and Senegal (2.9% and 4.5%) for blood transfusion and cesarean sections, respectively.

Emergency obstetric care provision based on signal functions validated by structural measures (Method 2)

There were considerable reductions in the proportion of health facilities providing signal functions when comparing their self-reported provision in the past three months (method 1), (Table 2) with the availability of the required structural measures to perform them (method 2), (Table 3) across the six countries. The greatest percentage decline was observed in the proportion of facilities providing assisted vaginal delivery, ranging from 9% in Nepal to 91% in Senegal. In almost all the countries, the smallest decline was observed in the proportion of facilities providing parenteral oxytocic drugs (2% or less) except for Bangladesh, which showed a 20% decline in capacity (Method 1 vs 2: 53.4% vs 31.7%).

Emergency obstetric care provision based on either recent or prior performance of signal functions (Method 3)

There were considerable increases in the proportion of facilities providing signal functions when comparing their recent self-reported provision in the past three months (method 1), (Table 2) with either a recent or prior provision (method 3), (Table 4). The greatest increase was observed in the proportion of facilities providing manual removal of placenta, ranging from 10.0% in Bangladesh to 29.0% in Malawi. The smallest increase was observed in the proportions of facilities providing blood transfusion (0.9% in Tanzania to 1.6% in Malawi) and cesarean section (0.4% in Tanzania to 1.1% in Malawi).

Emergency obstetric care provision based on a composite index of obstetric capacity (Method 4)

The overall domain scores for the composite index were very similar across the six countries. Malawi had the highest average score (Mean=65.9), followed by Haiti (Mean=63.9), Tanzania (Mean=63.4), Senegal (Mean=60.6), Nepal (Mean=58.5) and Bangladesh (Mean=56.9) (Table 5). The domains with the highest scores across countries included 1) newborn signal functions and immediate care, 2) equipment, and 3) medicines and commodities, although the domain scores varied across countries. For instance, Malawi had the highest domain score (Mean=16.1) while Haiti had the lowest (Mean=13.5) for newborn signal functions and immediate care. The domains with the lowest scores across countries included 1) general requirements, 2) comprehensive emergency obstetric care, and 3) guidelines, staff training and supervision. The countries with the lowest scores included Bangladesh (Mean=8.2) and Senegal (Mean=6.2) for general requirements; Malawi (Mean=8.3) and Nepal (Mean=7.1) for comprehensive emergency obstetric care; and Tanzania (Mean=7.7) and Nepal (Mean=5.2) for guidelines, staff training and supervision.

Levels of emergency obstetric capacity across the four EmOC assessment methods

Table 6 shows that regardless of the method used, facilities across the six countries were mostly classified as having less than basic EmOC capacity (Methods 1-3) or as low capacity (Method 4). There is a huge unmet need for even basic EmOC across the countries. The four methods differed in their classification of comprehensive EmOC or high capacity facilities. Methods 1 and 4, however, were similar in their classification of comprehensive EmOC facilities, as indicated by the proportions in each comprehensive EmOC category. Method 3 yielded a slightly higher proportion of comprehensive EmOC facilities compared with the other methods. Method 2 differed considerably from methods 1, 3 and 4 in its classification of comprehensive EmOC facilities. In particular, none of the facilities across the countries qualified as comprehensive EmOC facilities using method 2, except for 0.2% of facilities in Bangladesh. Method 3 seemed to discriminate better across countries in relation to the three categories of classification. In particular, there seemed to be a larger proportion of comprehensive and basic EmOC facilities compared with the other three methods.

Based on EmOC methods 1, 3, and 4, Haiti, Malawi and Bangladesh emerged as the top performing countries in terms of having higher proportions of facilities that provide providing comprehensive EmOC or a high obstetric capacity. Based on EmOC method 2, only Bangladesh had at least one facility providing comprehensive EmOC, while none of the other countries had any comprehensive EmOC capacity. Results of the kappa agreement statistics showed an overall moderate agreement between the levels of EmOC method 4 and the other three methods, across all the countries (Kappa range: 0.41-0.60; Appendix 3).

Association between the EmOC methods with facility volume of deliveries

The four EmOC methods were significantly associated with facility volume of deliveries in at least one of the six countries, with variations across countries (Table 7 and 8). In adjusted models, EmOC methods 1 and 3 were significantly associated with higher volume of deliveries in three countries (Haiti, Tanzania, and Senegal). EmOC method 2 was associated with higher volume of deliveries in Nepal only. EmOC method 4 was associated with higher volume of deliveries in four countries (Haiti, Tanzania, Senegal, and Bangladesh). Using EmOC method 1, comprehensive EmOC was associated with higher volume of deliveries in Haiti (Adjusted Prevalence Ratio APR: 2.16, 95% CI 1.36,2.44), Tanzania (APR: 2.23, 95% CI 1.56, 3.49) and Senegal (APR: 2.34, 95% CI 1.63, 3.36) (Table 8). However, basic EmOC was only significantly associated with volume of deliveries in Haiti (APR: 2.06, 95% CI 1.18, 3.59) and Senegal (APR: 1.53, 95% CI 1.16, 2.01). As for method 4 (the composite index), facilities providing high obstetric capacity were significantly associated with volume of deliveries in Haiti (APR: 2.10, 95% CI 1.27, 3.47) and Tanzania (APR: 1.73, 95% CI 1.19, 2.52). However, facilities providing medium obstetric capacity were significantly associated with volume of deliveries in Bangladesh (APR:1.96, 95% CI 1.15, 3.35), and Senegal (APR:1.34, 95% CI 1.07, 1.68). A test of interaction between each EmOC assessment method and the countries confirmed the observed heterogeneity of the association between the EmOC methods and facility volume of delivery: EmOC method 1*country (P=0.0002); EmOC method 2*country (P=0.06); EmOC method 3*country (P=<0.0001); EmOC method 4*country (P=<0.0001; Appendix 4).All of the six domains of obstetric care were significantly associated with facility volume of deliveries in at least one country (Appendix 5 and 6). Four of the six domains (comprehensive obstetric care; newborn signal functions and immediate care; general requirements; and medicines and commodities) were significantly associated with higher volume of deliveries in at least one of the countries. Two domains, medicines and commodities; and guidelines, staff training and supervision, were significantly associated with lower volume of deliveries in just one of the countries.

Table 2: Distribution of facility-reported performance of signal functions in past 3 months (Method 1), in Malawi, Haiti, Bangladesh, Nepal, Tanzania and Senegal: 2014-2018 SPA:

Table 3: Distribution of facility-reported performance of signal functions in past 3 months validated by interviewer observation of structural measures (Method 2), in Malawi, Haiti, Bangladesh, Nepal, Tanzania and Senegal: 2014-2018 SPA:

Table 4: Distribution of facility-reported performance of signal functions at least once or in past 3 (Method 3), in Malawi, Haiti, Bangladesh, Nepal, Tanzania and Senegal: 2014-2018 SPA:

Table 5: Distribution of domain characteristics used in the composite index (Method 4), in Malawi, Haiti, Bangladesh, Nepal, Tanzania and Senegal: 2014-2018 SPA:

Table 6: Distribution of facilities according to their levels of emergency obstetric capacity using methods,1,2, 3, and 4 in Malawi, Haiti, Bangladesh, Nepal, Tanzania, and Senegal: 2014-2018 SPA:

Table 7: Prevalence ratio estimates of facility volume of deliveries by EmOC method: in Malawi, Haiti, Bangladesh, Nepal, Tanzania, and Senegal: 2014-2018 SPA

Table 8: Adjusted prevalence ratio estimates of facility volume of deliveries by EmOC method in Malawi, Haiti, Bangladesh, Nepal, Tanzania, and Senegal: 2014-2018 SPA:

Using nationally representative data from health facility surveys in six LMICs, we found considerable variations in signal function performance within and between countries depending on the EmOC assessment method employed. Our findings showed that there is a huge unmet need for even basic EmOC across the countries, regardless of how it is measured. However, there were variations depending on the approach to measurement that was utilized. The four EmOC methods differed in their classification of comprehensive EmOC or high capacity obstetric facilities. Methods 1 and 4, however, were similar in their classification of comprehensive EmOC facilities. Method 3 yielded a slightly higher proportion of comprehensive EmOC facilities. Method 2 differed considerably from methods 1, 3 and 4 in its classification of comprehensive EmOC facilities. In particular, only Bangladesh had at least one facility providing comprehensive EmOC, while none of the other countries qualified as having comprehensive EmOC facilities. This study also showed that all four EmOC methods were significantly associated with facility volume of deliveries in at least one of the six countries, with variations across countries. The composite index emerged as the optimal EmOC method for characterizing facility obstetric capacity based on its consistency in characterizing obstetric facilities across countries, association with delivery volume in 4 out of 6 countries, coverage of multiple domains of obstetric care, and the association of those domains with facility volume of deliveries in all six countries examined.

The similarity in comprehensive EmOC facility categories using methods 1 and 4 could be as a result of both methods incorporating the self-reported performance of signal functions. Method 3 yielded a slightly higher proportion of comprehensive EmOC facilities and its distribution seemed to discriminate better across countries in terms of the three categories of classification. This could be attributed to the greater flexibility of this method, since this method captures facilities that performed the signal functions in the past three months in addition to those that had ever performed them. Prior studies have not compared the performance of methods 1 and 3, however, advocates of method 3 have suggested that this method accounts for the reality that a lack of recent performance of signal functions in a facility does not reflect the lack of capacity to do so but rather, performance of signal functions largely depends on their demand in terms of referrals or visits from women who need such care. (6) Method 2 varied considerably from the rest of the methods. Specifically, none of the facilities across the countries qualified as comprehensive EmOC facilities using method 2, except for 0.2% of facilities in Bangladesh. The striking differences observed when comparing method 2 to other methods could be reflective of the extent of self-report bias. For instance, method 2 validates the self-reported signal function performance across other methods by evaluating correspondence between self-report and actual capacity to perform signal functions. This will ultimately yield a much lower proportion of facilities. In particular, all the countries, except for Bangladesh, had no facilities that qualified as being comprehensive. Prior studies have compared the performance of signal functions using methods 1 and 2.(5) Our study showed a 2 to 20 percentage-point reduction in the capacities of health facilities to perform signal functions, when the reported performance of signal functions alone (method 1) was compared to the observed structural capacity to perform them (method 2). This finding was consistent with prior studies showing that signal functions alone (method 1) overestimate the provision of emergency obstetric care.(5) In particular, a prior study showed that based on signal functions alone (EmOC method 1), 86% of facilities in Kakamega county, Kenya reported that they had administered parenteral anticonvulsants in the past 3 months. However, when the availability of structural measures required to administer anticonvulsants were assessed, only 6% of those facilities could have performed that signal function.(5) In the present study, we had similar findings. For instance, in Bangladesh, around 30% of facilities reported a recent administration of anticonvulsants, however, less than 10% of facilities actually had the structural capacity to administer anticonvulsants.

When classifying facilities using a composite index (EmOC method 4), the average obstetric capacity scores obtained for some of the countries were somewhat consistent with prior literature [9]. A similar study using DHS data employed a weighted additive index and obtained average obstetric scores that were similar to the present study for Nepal (57.7 vs 57.9) and Senegal (60.0 vs 62.5) [9].However, the prior study differed from the present one for other countries including Malawi (67.4 vs. 63.9), Haiti (52.7 vs 65.1), Bangladesh (46.5 vs 58.9), and Tanzania (52.7 vs. 63.4). These differences could be attributed to variations in the development of the composite index. For instance, in calculating index scores across facilities, provider data were collapsed to the facility level. The present study achieved this summarization of provider data by restricting the data to only providers who had received training in delivery or newborn services in the past 24 months. Hence, depending on the variable chosen for collapsing to the facility level, the resulting facility sample will likely differ, which will influence the results obtained. In terms of comparability between the EmOC categories of method 4 with the other three methods, our findings showed a moderate agreement in general. To the best of our knowledge, no study has compared concordance between the composite indices of obstetric care and existing assessment methods.

Our findings regarding the association of the EmOC methods with higher volume of deliveries were consistent with prior studies showing that delivery volume is associated with facility obstetric capacity, using signal functions alone.(15,21) However, our study showed that the composite index of obstetric care was associated with volume of deliveries in four out of the six countries studied, compared with the other methods. Methods 1 and 3 were significantly associated with facility volume of deliveries in three countries, while method 2 had a significant association in just one country. This is consistent with prior studies showing that composite indices are better predictors of various facility and individual outcomes. One study that utilized principal component analysis in developing an index of obstetric care found that the index scores were better predictors (in comparison with signal functions alone) of the number of deliveries (Variance explained, R2 =0·61 vs R2 =0·31), number of low birthweight babies (0·68 vs 0·26), maternal deaths (0·81 vs 0·34) and neonatal deaths (0·79 vs 0·36).(21) Another related study showed that a composite index of reproductive and childhood care was strongly correlated with mortality and childhood malnutrition compared with a single count of interventions.(24)

The present study also has several limitations. The composite measure of EmOC capacity used in the present study included variables from only one facility assessment tool (SPA). This tool does not capture the array of measures covered across various EmOC tools that could potentially be included in the index. Also, the composite index was restricted to providers who offered delivery or newborn services and had received training in these services in the last 24 months. This led to a smaller sample size of facilities compared to the sample of facilities providing delivery care. Also, the EmOC measures included in the various methods only account for the process of patient service delivery, or the extent to which providers adhere to stated standards. However observing delivery process is pertinent since the availability of obstetric capacity does not guarantee performance—such metrics were not captured by the SPA.(9). Also, the outcome measure, volume of delivery, was assessed at the providerlevel rather than the facility level. Since providers may often conduct deliveries/work at different facilities, the volume of delivery measure is not unique to a given facility leading to double counting of deliveries. Prior literature has shown that providers frequently work/conduct deliveries across multiple facilities, and this phenomenon is common across LMICs. (25,26) Another limitation of the present study is missing data (although negligible) across some of the SPA indicators. A key challenge of the weighted additive index (which was utilized in creating the composite index) is that it assumes that all the included indicators are of equal importance in the provision of adequate obstetric care. Other approaches to creating composite indices such as principal component analysis, and expert panel ratings suggest that some of the indicators may be of greater importance than others.(9,27) Other limitations of this study include the self-reported nature of the SPA data and the lack of standardization across settings and providers. The latter may have underestimated our results.

Despite these limitations, the present study has several strengths. To the best of our knowledge, this is the first study to compare obstetric care performance across countries using four established assessment methods. These methods have been used in conducting EmOC assessments across various countries.(5,6,15,28) In addition, this is the first study to examine concordance between method 4 (composite index) with other assessment methods. This study also utilized the SPA surveys which provides comprehensive data on various obstetric care indicators, and this allowed for proper specification of each of the EmOC methods. Comparisons between the different EmOC assessment methods were done across six LMICs which further strengthens the study by accounting for contextual differences in obstetric capacities across various settings. The use of multiple countries in different contexts including Africa and Asia contributes to the generalizability of the study findings.

The discrepancies in comprehensive EmOC capacities across facilities based on the EmOC methods raise considerable concern and highlights the importance of the lenses through which obstetric capacity is evaluated across countries. Our findings showing heterogeneity across EmOC measurement methods suggest the need to standardize and validate how best to measure facility readiness to provide EmOC to ensure policy makers and other stakeholders are making resource allocation decisions based on the best available data. The heterogeneity in performance of the EmOC methods across countries suggests that the need for consensus in measuring EmOC across settings although there may need to be some context-specific tailoring. Hence, using one type of EmOC assessment method may not fully capture the true performance of facilities. Although EmOC method 4 emerged as the optimal assessment method, it is suggested that subsequent EmOC assessments should employ additional methods as a sensitivity approach to provide a more accurate picture of obstetric capacity. Prior research has largely supported the use of composite measures of obstetric care which allow for greater flexibility in assessing care, thus accounting for facilities characteristics and performance measures in their entirety. The present study further supports this notion showing that composite measures are more predictive of facility service outcomes in multiple settings. Further studies in a similar direction are encouraged to employ EmOC assessment tools (other than the SPA) which capture a wider array of indicators of emergency obstetric care. Taken together, such efforts could largely improve facility obstetric assessments while informing policy making and resource allocation, especially in LMICs with very low EmOC capacity. Findings from such research could potentially inform stakeholder decisions as they rapidly work towards developing new measures of capacity for EmOC with the ultimate goal of reducing maternal and infant mortality.

1. Banke-Thomas A, Wright K, Sonoiki O, et al. Assessing emergency obstetric care provision in low- and middle-income countries: a systematic review of the application of global guidelines. Glob Health Action. 2016; 9: 31880.

2. Maternal Health Task Force. The Sustainable Development Goals and Maternal Mortality.

3. Ameh C, Van den Broek N. Making It Happen: Training health-care providers in emergency obstetric and newborn care. Best Pract Res Clin Obstet Gynaecol. 2015; 29(8): 1077-1091.

4. Bailey P, Lobis S, Maine D, Fortney J. Monitoring emergency obstetric care: a handbook. 2009.

5. Cranmer JN, Dettinger J, Calkins K, Kibore M, Gachuno O, et al. Beyond signal functions in global obstetric care: Using a clinical cascade to measure emergency obstetric readiness. PLoS One. 2018; 13(2): e0184252.

6. Iyer V SK, Mehta R, Mavalankar D. Availability and provision of emergency obstetric care under a public–private partnership in three districts of Gujarat, India: lessons for Universal Health Coverage. BMJ global health. 2016; 1(1): e000019.

7. Bintabara D, Ernest A, Mpondo B. Health facility service availability and readiness to provide basic emergency obstetric and newborn care in a low-resource setting: evidence from a Tanzania National Survey. BMJ Open. 2019; 9(2): e020608.

8. Allen SM, Opondo C, Campbell OMR. Measuring facility capability to provide routine and emergency childbirth care to mothers and newborns: An appeal to adjust for delivery caseload of facilities. PLoS One. 2017; 12(10): e0186515.

9. Wang W, Mallick L, Allen C, Pullum T. Effective coverage of facility delivery in Bangladesh, Haiti, Malawi, Nepal, Senegal, and Tanzania. PLoS One. 2019; 14(6): e0217853.

10. Tembo T CG, Vwalika B, Sitali L. Signal functions for emergency obstetric care as an intervention for reducing maternal mortality: a survey of public and private health facilities in Lusaka District, Zambia. BMC pregnancy and childbirth. 2017; 17(1): 288.

11. Roy L, Biswas TK, Chowdhury ME. Emergency obstetric and newborn care signal functions in public and private facilities in Bangladesh. PLoS One. 2017; 12(11): e0187238.

12. Kyei-Onanjiri M, Carolan-Olah M, Awoonor-Williams JK, McCann TV. Review of emergency obstetric care interventions in health facilities in the Upper East Region of Ghana: a questionnaire survey. BMC Health Serv Res. 2018; 18(1): 184.

13. Nesbitt RC, Lohela TJ, Manu A, et al. Quality along the continuum: a health facility assessment of intrapartum and postnatal care in Ghana. PLoS One. 2013; 8(11): e81089.

14.Fakih B NA, Ali AO, Mkopi A, Hassan A, Ali AM, et al. The status of maternal and newborn health care services in Zanzibar. BMC pregnancy and childbirth. 2016; 16(1): 134.

15. Kruk ME, Leslie HH, Verguet S, Mbaruku GM, Adanu RMK, et al. Quality of basic maternal care functions in health facilities of five African countries: an analysis of national health system surveys. Lancet Glob Health. 2016; 4(11): e845-e855.

16.The Demographic and Health Survey Program. SPA Overview. 17.United Nations Childrens’ Fund (UNICEF). Maternal Mortality. 2019.

18. Bailey P, Awoonor-Williams J, Lebrun V, et al. Referral patterns through the lens of health facility readiness to manage obstetric complications: national facility-based results from Ghana. Reproductive health. 2019; 16(1): 19.

19. Nesbitt RC, Lohela TJ, Manu A, et al. Correction: Quality along the Continuum: A Health Facility Assessment of Intrapartum and Postnatal Care in Ghana. PLoS One. 2015; 10(10): e0141517.

20. Wang W. Influence of service readiness on use of facility delivery care: a study linking health facility data and population data in Haiti. 2014.

21. Boyer C, Jackson E, Bawah A, Schmitt M, Awoonor-Williams J, et al. Estimating indices of health system readiness: an example from rural northern Ghana. The Lancet Global Health. 2015; 3(S14).

22. Peet ED, Edward N. Okeke. Utilization and quality: How the quality of care influences demand for obstetric care in Nigeria. PloS one. 2019; 14(2): e0211500.

23. Larson E, Hermosilla S, Kimweri A, Mbaruku GM, Kruk ME. Determinants of perceived quality of obstetric care in rural Tanzania: a cross-sectional study. BMC Health Serv Res. 2014; 14: 483.

24. Wehrmeister FC, Restrepo-Mendez MC, Franca GV, Victora CG, Barros AJ. Summary indices for monitoring universal coverage in maternal and child health care. Bull World Health Organ. 2016; 94(12): 903- 912.

25. Hipgrave DB, Krishna H. Dual practice by doctors working in South and East Asia: a review of its origins, scope and impact, and the options for regulation. Health policy and planning 2014; 29(6): 703-716.

26. Mumtaz Z, Afshan B, Sarah S. Challenges to achieving appropriate and equitable access to Caesarean section: ethnographic insights from rural Pakistan. Journal of Biosocial Science. 2019; 52(4): 1–13.

27. Gabrysch S, Civitelli G, Edmond KM, et al. New signal functions to measure the ability of health facilities to provide routine and emergency newborn care. PLoS Med. 2012; 9(11): e1001340.

28. Sabde Y DV, Randive B, Chaturvedi S, Sidney K, Salazar M, et al. The availability of emergency obstetric care in the context of the JSY cash transfer programme in Madhya Pradesh, India. . BMC pregnancy and childbirth 2016; 16(1): 116.