SWB Week: David Fitch on Efficiency in the Evaluation of Experimental Village Health Programs

Greetings, I am David J. Fitch, IDIES, Universidad Rafael Landívar, Guatemala and a member of SWB.

Learning Lessons: UNICEF estimates Cuban and US infant mortality (IM) rates are 6 and 8 per 1,000 live births, showing a better record for a very poor country than a very rich country. Other poor countries and countries giving them foreign aid should try to learn from Cuba.

Tip: Use efficient design and analysis. Consider the design and experimental evaluation of a Guatemalan village health program where currently the child death rate is high.  The concern here is efficiency – increasing the probability of a significant experimental-control group difference .per dollar spent.

Example design:

• Construct a population of Guatemalan villages with high child death rate.
• Randomly select two samples of 10 villages.
• Place a Cuban MD and send a village person to Cuba to receive medical training in each village of one sample.

The high cost of the experimental program per village motivates the design efficiency.

• Collect baseline data for 16 covariates in each of the 20 villages.
• Assemble village child death rates in recent years and compute covariate correlations with death rate.

Tip: Use relevant statistical experience.

• With a sample of 20, two estimators derived from the 16 covariates plus the dummy – 1 for experimental, 0 for control, it is reasonable to use multiple regressions.
• Enter 16 village estimated covariate measures into a 20×16 matrix where the elements are , r is the correlation between the covariate and mortality at the time – baseline and at the end of say five years.  Compute a covariance matrix for each time period.  Without the r, C is a correlation matrix, the one usually used in factor analysis.  With the rthose covariates more predictive of death have greater variance and hence greater weight in each factor.  At each time period a principal axes factor analysis gives us, with  and our matrix of factor scores is .  We use the first two factor scores.  Compute two regression equations:
  • baseline data
  • end-of-five-year data
• A t test would test for the significance of the difference of the differences.

Tip: Make the statistics clear. For this example:

• Consider the 10 control villages.
• Use the baseline regression equation on mortality for each of the 10 control villages.
• Then use the regression equation on mortality for the end-of-five-year data.
• Calculate the difference for each of the 10 control villages.
• Do the same for the 10 experimental villages.
• ·If there was a significant difference between the two difference distributions in favor of the experimental program, there is evidence to support the program.

The American Evaluation Association is celebrating Statistics Without Borders Week. The contributions all week come from SWB members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.