1  

Susan Chen 03/25/2011

Bidan Desa: Did It Work?

I. Research Design

Do government efforts to provide health care have an impact on the populations that the

programs target? This study explores the question in the context of Indonesia, analyzing the effects of

the Village Midwife (or Bidan Desa) program that began in the early 1990s. Analysis is conducted to

determine if provision of midwife services to reproductive age women would affect health outcomes.

The Village Midwife program places trained midwives in villages and townships in an effort

to increase women’s access to reproductive health care. Village midwives have a number of duties,

including provision of health and family planning services, promoting community participation in

health, working with traditional birth attendants, and referring complicated cases to health clinics and

hospitals (Government of India, 1989).

The quasi-experiment focuses on rural regions and uses data from the government-sponsored

longitudinal Indonesia Family Life Survey (IFLS). The IFLS is a panel survey of individuals, and

households. The first round of data (IFLS1) was collected in 1993 and included interviews with 7,224

households and with 22,347 individuals within those households (Frankenberg, 2001). The analysis

looks at a sample of 8824 individuals in 13 provinces. This large sample size should allow for

significant and robust analysis. Thirty-nine percent of participants (2926) were women between ages

19-45 (reproductive age), 22 percent (2017) were women over age 45, 25 percent (2132) were men

between ages 19-45, and 20 percent (1749) were men over age 45. In 1997, a survey (IFLS2) was

conducted to re-interview all IFLS1 individuals (International Household Survey Network, 2009).

The dependent variable in these analyses is health status, or the difference between the Body

Mass Index in 1997 and BMI in 1993. Positive values of the dependent variable represent a positive

2  

change in health status between 1993 and 1997.

II. Effect of Having Midwife on Average BMI

From 1993 to 1997, all villages—with and without midwives—showed statistically significant

overall gains in BMI. Thus, there is an overall improvement in health from 1993 to 1997.

Surprisingly, villages without midwives showed a significantly higher BMI than villages with

midwives. This appears to indicate that villages with midwives reduce BMI, and therefore are

detrimental to improving health outcomes. However, this conclusion neglects to consider the fact that

midwives were assigned to rural areas far away from health centers so the health of the population

tended to be worse than that of the urban population. As a result, these areas may already have a low

BMI to begin with (selection bias). The communities that gained a Village Midwife may be more

likely to have residents with low income and education or low levels of socioeconomic development.

The baseline characteristics (individual and community) may vary.

The ability to make a causal claim from this data alone is weak. First, while BMI is one

important health statistic, it is not adequate in itself as an overall health indicator. Statistics should

also take into account disease prevalence, for example. Thus, there is a construct validity problem

with using BMI to indicate health status. The study analysis also fails to consider omitted variables

that can affect BMI; for example, the changing economic environment and shift in food prices.

Because the Village Midwife program is not the only aspect of the health environment that

may have changed during the 1990s, it is important to control for other dimensions of the health

service environment, and for levels of infrastructure more generally. Therefore, the study has

constructed measures for health access. Access to the Village Midwife program is measured as

whether or not a Village Midwife was available in each year of the survey. Access to public clinics is

measured as the distance from the community to the nearest health center. Access to outreach efforts

3  

was measured by a variable indicating whether or not the community receives monthly visits from

health center staff. Physical infrastructure is measured by whether the community’s main road is

paved and whether a public phone is located in the community.

III. Changes in BMI 1993-1997

In considering how access to services affects behaviors or outcomes, one must consider that

services may be systematically placed to increase the likelihood that they reach people with particular

characteristics. For example, health centers may be targeted toward areas where the population is

poor. To address the issue of non-random placement of midwives, this study examines change in

health status as a function of whether the community gained a Village Midwife.

A difference in difference analysis shows no significant difference between villages that

gained a midwife and those that did not. Both villages increased BMI over the years. A difference in

difference analysis, by measuring changes in BMI, helps control for the fact that the villages selected

to receive midwives may have different initial health characteristics than those that did not.

This pre-post non-equivalent group design reduces, but does not eliminate, threats to internal

validity. For example, suppose national events affected urban areas differently than rural areas, BMI

differences can still reflect more than just the treatment effect. Furthermore, the construct validity

problem with using BMI to indicate health status lingers.

Table 1: Changes in BMI and Changes in Midwife Status Midwife in Village No Midwife in Village Differences in BMI Mean BMI (SE in parenthesis)

1993 1997

20.83 (0.09) 21.10 (0.05)

21.37 (0.04) 22.41 (0.06)

-.55** (0.1) -1.31** (0.08)

Gained Midwife Did Not Gain Midwife Difference in Differences

Change in BMI from 1993-1997

+0.33** (0.03)

+0.37** (0.02)

-0.04 (0.03)

**Significant at 5% level

4  

IV. Effects of Gaining a Midwife on BMI Changes Among Different Demographic Groups

Indonesia’s midwivery program targets women of reproductive age. Therefore, health

outcomes (positive BMI changes) are expected to have largest effect on this subgroup with the

introduction of midwives. After conducting an OLS regression of 4 subgroups (women age 45 and

younger, older than 45, men age 45 and younger, older than 45), the results suggest that the Village

Midwife program has positively affected the health status of the group toward whom it is targeted:

women of reproductive age (age 45 and younger).

The fact that the positive effects of the program are limited to reproductive-age women adds

strength to the argument that the association is causal, rather than arising from some other factor that

has changed concurrently with changes in access to Village Midwives (if such a factor were driving

the improvements in health status, it would likely affect both women and men, rather than only

reproductive-age women).

Significant tests confirm that among subgroups gaining a midwife, the only significant

differences appear between young reproductive-age women and all the other subgroups. For women

older than 45, there is no impact on BMI of gaining a midwife. Nor is there any impact on BMI of

gaining a midwife for men. The interaction between gaining a Village Midwife and age is negative,

which implies that the health-enhancing effects of Village Midwives decline as women get older.

This finding in intuitive, since the services that Village Midwives typically offer (prenatal, delivery,

family planning services), are particularly relevant for younger women.

This pre-post non-equivalent group design reduces threats to internal validity and makes a

more confident statement of causality. However, some variables like disease may affect different

groups differently so causation claims are not unassailable.

5  

Table 2: Changes In BMI Among Different Subgroups

(SE in parenthesis) BMI Change BMI Change Adjusted for Individual

Characteristics

BMI Change Adjusted for Individual and

Community Characteristics

Did Not Gain Midwife Women Age Over 45 N=2017

-0.59** (0.06)

-0.55** (0.06)

-0.59** (0.06)

Men Age 45 and Below N=2132

-0.31** (0.06)

-0.32** (0.06)

-0.31** (0.06)

Men Age Over 45 N=1749

-0.63** (0.07)

-0.63** (0.07)

-0.65** (0.07)

Gained a Midwife Women Age 45 and Below N=1308

+0.11*** (0.06)

+0.15** (0.06)

+0.13** (0.06)

Women Age Over 45 N=929

-0.20** (0.09)

-0.20** (0.09)

-0.19** (0.09)

Men Age 45 and Below N=964

-0.20** (0.09)

-0.21** (0.09)

-0.19** (0.09)

Men Age Over 45 N=820

-0.26** (0.10)

-0.26** (0.10)

-0.26** (0.10)

Reference Group Women Age 45 and Below Who Did Not Gain A Midwife, N=2926

0.70 (0.04)

0.56 (0.05)

0.56 (0.10)

**BMI change is significant at 0.05 level, ***BMI change is significant at 0.10 level The dependent variable is specified as BMI(97)- BMI(93). V. Midwife Gains and BMI Changes, Controlled for Individual Characteristics

Correlations at a point in time between characteristics of the health service environment and

health outcomes will be biased by failure to address the individual characteristics of the midwives.

By examining changes in health at the individual level as a function of changes in health programs,

the researchers hold constant aspects of the community in which an individual lives that may affect

both access to services and health status.

I further explored whether the effect of gaining a Village Midwife varies across subgroups,

adjusting for individual effects. In this analysis, after controlling for individual effects such as

6  

income and education, young women gaining midwives are still the subgroup that gains significantly

more than any other subgroup. These results further strengthen our causal claim because individual

differences are accounted for.

VI. Midwife Gains and BMI Changes, Controlled for Individual & Community Characteristics

A skeptic might argue that close proximity to a health care center or living in a community

that has a monthly visit from health center staff might affect health outcomes positively. Levels of

socioeconomic development vary by whether the community has a public phone and whether the

main road in the community is paved. The next analysis controls for both individual and the broader

community effects.

The results from this analysis further strengthen our causal claim in that young women

gaining midwives are still the subgroup that gains significantly more than any other subgroup, even

when adjusted for village-by-village differences in infrastructure.

In combination, the results from Parts IV, V, and VI indicate a strong causal claim between

gaining a midwife and increases in BMI among young women. The Village Midwife program was

implemented to address concerns about maternal health. To this extent, the study achieved its

intended affect. Accounting for individual and community characteristics, the study mitigates biased

estimates of the impact of services. The results suggest that efforts of the Ministry of Health to

rapidly expand access to midwifery services has had a pay off in terms of the health status of women

of reproductive age.

Nevertheless, there are still three threats to validity that might be problematic when

generalizing the results above.

1) Construct Validity: BMI is not the only indicator of overall health, although it is an important

one. Other outcomes such as infant mortality or disease rates might be relevant.

7  

2) External Validity: This program targets poor, rural areas and cannot be generalized to urban

contexts without recognizing cultural differences in maternal attitudes and behaviors.

3) Internal Validity: The BMI variation among various subgroups might be due to differences in

collection times. People’s BMIs change at different times of the year. Young men might have

low BMIs if they have been farming in the Summer; young women might have low BMIs

when they are taking exams in the Spring.

VII. Hypothetical Time Series Analysis

To further strengthen the causal claim, we can control for time effects. A time series analysis

requires different observations for every individual. Consequently, each individual will have multiple

observations in the study, corresponding to the number of times they were surveyed. To conduct a

time-series analysis, I would collect data from a non-equivalent group design that observed many

comparison groups while considering the various arrival times of midwives. The data would be

collected 3-5 years before the arrival of midwives, to isolate health trends existing pre-arrival, and 3-

5 years after their arrival, to adjust for seasonal changes within one year (internal validity problem).

In addition, I can collect data on other health indicators such as infant mortality and disease

rates to remedy the construct validity concern with using solely the BMI measure. This extra control

will provide a fuller picture of health outcomes.

With these controls, I expect the results of the regression to be more confidently reflective of

the effects of the intervention. The effects from pre-existing health trends and seasonal changes will

be excluded. Thus, the study would increase the ability to make a causal claim about the intervention.

Hypothetical Regression Discontinuity Analysis

In regression discontinuity designs, participants are assigned to comparison groups solely on

the basis of a cutoff score. In this study, the Indonesian Ministry of Health would have to set a cutoff

8  

BMI average to determine whether or not a village would gain a midwife. For example, the Ministry

of Health can allocate a midwife to villages with an average BMI of below 19, and those villages

with an average BMI above 19 would not receive a midwife. Assuming this analysis is rigorously and

fully implemented by those assigning midwives to villages, this study can then compare villages that

received midwives and were within a short range of the baseline BMI cutoff. As an example, villages

with an average BMI of 18-19 can be compared with those with an average BMI of 19-20.

This type of analysis is beneficial for comparing groups that are just above the cutoff (control

group) with those that are just below the cutoff (treatment group). If we discover that villages on

opposing sides of the cutoff yielded different effects, this finding will strengthen the internal validity

of our study design and make a strong case that the intervention is the main causal driver. However,

there are a number of problems with this type of analysis. While the results of the study would

suggest a strong causal claim, the claim can only be extended to programs with an average cutoff

BMI of 19. The causal claim would be very strong for the given cutoff. However, the results cannot

be generalized to other cutoff levels (e.g. 18 or 20). Other studies may show significantly varied

effects for different subgroups. For example, one might predict that a similar village midwife

program will have more drastic results for those with a lower average BMI at baseline.

References Frankenberg, Elizabeth and Duncan Thomas. "Women's Health and Pregnancy Outcomes: Do

Services Make a Difference?." Demography 38.2 (2001): 253-265. Government of Indonesia. 1989. Panduan Bidan di Tingkat Desa. Jakarta: Direktorat Jenderal

Pembinaan Kesehatan Masyarakat. International Household Survey Network (2009). IHSN - Indonesian Family Life Survey. Retrieved

March 20, 2011 from http://surveynetwork.org