On Bathwater, Babies, and Designing Programs for Impact: Evaluations of the Integrated Community Case Management Strategy in Burkina Faso, Ethiopia, and Malawi

The adage “ don ’ t throw the baby out with the bathwater ” advises us not to discard the essential along with the unes-sential, whether due to impatience or frustration. 1 This issue of the journal includes reports on prospective evaluations of the integrated community case management (iCCM) of childhood illness strategy in Burkina Faso, 2 Ethiopia, 3 and Malawi. 4 iCCM seeks to reduce child mortality by making effective treatment of pneumonia, diarrhea, and malaria available from trained health workers at community level. 5 In this commen-tary, we show results from the three countries, highlighting hard-won lessons about iCCM ’ s potential to prevent unneces-sary deaths among children. a 6

The evaluation in each setting was conducted in collaboration with local research institutions, based on a locally adapted version of the common evaluation framework for maternal, newborn, and child health (MNCH). 7 The evaluation designs were built on impact models reflecting the pathway to child survival. Beginning with policy and program inputs, the pathway moves to project implementation and intermediate outputs such as improved provision of quality services, strengthened health systems, and increased utilization. Continuing to coverage and behavioral outcomes, the pathway culminates in impact of under-five and postneonatal mortality. We focus on mortality among children aged 2-59 months here, because the iCCM strategy did not include guidelines for children under 2 months of age. Detailed methods are described in the individual papers; Table 1 provides a summary. All sites documented program inputs and contextual factors that may have affected program implementation or effectiveness. The evaluation teams shared intermediate and final results with governments and partners as a basis for strengthening iCCM implementation.
Selected results across the three countries are shown in Table 2. We reanalyzed some variables to maximize compa-rability across countries; in some instances, results presented here are not included in the series papers.
iCCM was not associated with accelerated mortality declines in children aged 2-59 months during the 2-to 4-year evaluation periods in these three settings. The reasons for this lack of association likely varied across settings, as does the likelihood that these programs may achieve measureable impact after additional years of sustained implementation.
In Burkina Faso, iCCM implementation did not reflect current best practice for designing and implementing effective MNCH programs. The program relied on community health worker (CHW) lay volunteers. The initial training was organized as a "cascade" in which representatives from districts were trained and then requested to organize subdistrict-level training sessions to train over 3,000 CHWs in the space of a few months, with minimal support or supervision. The training included written materials even though many of the CHWs were illiterate. Systems for ensuring the availability of drugs and reinforcing health worker performance were not consistently in place at the time of training, or indeed throughout the evaluated implementation period. There were no systematic efforts to generate community demand for and utilization of iCCM services. Not surprisingly, then, the quality of iCCM service provision was poor, the services were grossly underutilized, and there was no improvement in intervention areas in the proportion of sick children who received correct treatment of their disease ( Table 2). The Burkina Faso iCCM program will not have an impact unless it is redesigned to fully capacitate health workers and educate the community to increase demand. Greater attention to accountability and best practices in women's and children's health should help prevent similar examples of incomplete and therefore ineffective programs in the future. 12 However, in both Malawi and Ethiopia, iCCM was implemented more strongly and achieved important intermediate outputs. In both settings, government policies were in place to support full implementation of the strategy, and high numbers of paid community-level workers were well trained and initially well equipped. The percent of children receiving correct treatment from CHW for iCCM illnesses was moderately high in Ethiopia (78%) and lower in Malawi (63%) ( Table 2) but was at least as good as the care received in first-level health facilities. 11,13 In both settings, unfortunately, children with more severe illnesses were less likely than those without life-threatening conditions to be managed correctly, pointing to continuing needs for refinement of the iCCM approach to improve and sustain health worker performance. 11,13 Health systems supports for iCCM were quite strong in Ethiopia, particularly high levels of clinical supervision, which requires further investigation to generate best practice models. Careful monitoring was used to generate remedial actions when problems were identified. In Malawi, *Address correspondence to Elizabeth Hazel, Institute for International Programs, Johns Hopkins University (IIP-JHU), 615 North Wolfe Street, Baltimore, MD 21205. E-mail: ehazel1@jhu.edu only 58% of iCCM workers received clinical supervision and only 38% reported no stock-outs of essential iCCM drugs in the previous 3 months. 4 Although far from perfect, both Ethiopia and Malawi implemented iCCM strongly and at scale, providing an important foundation for continued improvement.
However, in both Ethiopia and Malawi, there was no change in overall care seeking for iCCM illnesses and no impact on mortality ( Table 2). The evaluation results also make clear that the potential effectiveness of iCCM was capped by low utilization. At baseline in Ethiopia, only 23% of mothers took their sick child to any formal provider for care, and care seeking rates did not increase over time. In contrast, care seeking in Malawi was quite high at baseline and remained the same at endline. Care seeking from iCCM providers increased slightly in both settings, but at such low absolute proportions that no population impact could be expected. No matter how strong iCCM service provision may be, the strategy cannot save lives unless mothers and other caregivers take their children for care. Future iCCM programs must be designed with this in mind, and include strong, locally defined components designed to increase care seeking and utilization.
In Malawi, particularly, the results suggest that geographic targeting of iCCM based on physical access to fixed health facilities may no longer be effective, or at least not sufficient. iCCM in Malawi was implemented in "hard-to-reach" areas of each district, as defined by district management teams, Qualitative study of health system supports for iCCM 10 iCCM = integrated community case management. based on distance to a health post. Before 2008, this appeared to make good public health sense, when overall coverage for treatment interventions was low. By 2010, however, there were no differentials in children's receipt of appropriate treatment based on mothers' reports of whether distance to a health facility was a barrier to family health care. 4 The assumption that reaching the unreached is limited by geography is no longer true, at least in Malawi.
Returning to the bathwater and baby adage, the iCCM in 2015 can be considered as the baby and the challenges of fully implementing iCCM as the bathwater. Like a human infant, iCCM programs will require steadfast attention, investment, and developmental guidance to mature and achieve their mortality-reduction potential. iCCM is not the answer in all settings-where first-level facilities are accessible and are or can be improved to provide quality services, such as in urban settings, there may be no need for iCCM, unless it can be demonstrated that the strategy overcomes barriers to care seeking that go beyond physical access. In settings where iCCM is needed, further research, development, and investment will be required to ensure that implementation plans reflect an accurate understanding of "which" women and children are not using services, "how best" to reach and motivate them to seek care, and "what role" iCCM can play in this pathway to survival. Ongoing programs of evaluation and implementation research can continue to generate information about the bathwater and the parts of iCCM and its context that must be discarded or refreshed. Local capacity to collect and analyze relevant data is a prerequisite for generating essential knowledge and putting this knowledge to use in the service of women and children.