Lessons from the recent trial of a pay-for-performance programme in Afghanistan

Elina Dale, Anubhav Agarwal, Cyrus Engineer

Significant resources in global health are spent on pay-for-performance (P4P), also known under a more general term as results-based financing (RBF). Originating in the UK and USA, P4P has now become – to borrow a phrase from Cheryl Cashin – the new “it-girl” in health financing. However, as recent experience from Afghanistan shows, implementation is not always easy and P4P interventions must be better designed if they are to achieve real population health gains.

From 2010 to 2012 a P4P programme in Afghanistan provided quarterly bonus payments to health-care providers for increases in the use of maternal and child health (MCH) services, adjusted by a quality of care score. Our study, a large-scale cluster randomized trial, demonstrates that the programme did not produce the intended results. There were no observable improvements in any of the five key MCH coverage indicators measuring contraceptive prevalence, skilled birth attendances, vaccinations, and antenatal and postnatal check-ups. No changes were observed in the equity of care. While the programme appeared to increase time spent with patients, resulting in more complete histories and physical examinations, and improved patient counselling, other measures of quality, such as availability of medicines and supplies, did not substantially change.

Do we really understand the problem we are trying to address?
Sophie Witter in her recent blog on “growing pains (and gains)” of P4P highlighted the need to make the theory of change in P4P programmes more explicit. The intervention in Afghanistan confirms this general finding. The P4P programme that is the focus of our study had no component addressing the important barriers faced by communities and households to accessing the services (demand-side barriers). Community health workers who could promote outreach were not part of the P4P programme, and rewards for them were left at the discretion of each facility. Other interventions that could address demand-side barriers to utilization include conditional cash transfers that have demonstrated strong results in many different settings, from Mexico to India. Before a P4P intervention is considered, there must be a clear understanding of whether the key bottlenecks lie on the demand or the supply side, as well as the hypothesized theory of change.

Putting health workers in a double bind
Related to the lack of a clear conceptual framework discussed above, solutions should have plausible links to problems they are trying to address. In some cases, the main obstacle to good performance may not be lack of motivation, but a lack of means.

In Afghanistan many male doctors continued treating female patients under the Taliban, even though it was strictly forbidden. Giving these doctors and nurses the means to perform, such as facilities with reliable electricity and water supplies, has to be the first step in improving their performance. The P4P programme in Afghanistan included no accompanying efforts to improve working conditions at facility level. Facilities were given autonomy in deciding how to distribute earned payments, and there was no set minimum threshold that had to be spent on facility level improvements. This may have put health workers in a situation where they felt pressure to perform without having the capability to do so. In a P4P intervention in Rwanda, approximately 1/5 of funds were allocated for overall facility-level investments such as infrastructure and medical supplies. This is a common feature in many P4P programmes that unfortunately was not implemented in Afghanistan.

How can we motivate if we don’t communicate?
Almost half of the health workers from the programme’s treatment facilities stated they did not receive P4P. Thus, they either actually did not receive any payments, or were not aware of having received them. Health workers did not receive timely communication about the programme, especially regarding how payments would be made to individuals. Theory of motivation clearly states that the relationship between external tangible rewards and motivation depends vitally on the context in which it is provided. If employees understand the goals and processes of the programme, they are more likely to accept and have a stronger commitment to them.

Concluding remarks
Afghanistan’s P4P programme received over $175 million from the International Development Association, and $12 million from the Health Results Innovation Trust Fund. It is just one of over 30 LMICs, where a large resource pool has been committed to support P4P interventions. This evaluation experience provides an important lesson for future initiatives. It is vital to identify the important bottlenecks to service use and address them holistically in the intervention design. The expectation that more money will simply result in better performance is too simplistic to be translated into real population health gains.

Read more:

Engineer CY, Dale E, Agarwal A, Agarwal A, Alonge O, Edward A, Gupta S, Schuh HB, Burnham G, Peters DH. Effectiveness of a pay-for-performance intervention to improve maternal and child health services in Afghanistan: a cluster-randomized trial. Int J Epidemiol. 2016, doi: 10.1093/ije/dyv362. [Free to access until 31 May 2016]

Elina M. Dale, PhD (dalee@who.int) is a health financing specialist with ten years of experience in design and implementation of health system strengthening reforms, policy analysis, and research in global health. She joined WHO, Geneva in late 2014 as a Technical Officer in Health Financing Policy Unit. Prior to joining WHO in Geneva, she was a post-doctoral fellow at Johns Hopkins School of Public Health, Department of International Health, where she also completed her PhD. Her research at Hopkins focused on performance-based financing, job satisfaction and motivation, and quality of care.

Anubhav Agarwal, MPH (aagarwal@mail.harvard.edu) is a Visiting Scientist in the Department of Global Health and Population at the Harvard T.H. Chan School of Public Health  and a Consultant at The World Bank. He recently graduated from the Harvard School of Public Health with a Master’s Degree in Global Health. Prior to starting his graduate studies, he spent five years in Afghanistan with the Indian Institute of Health Management Research where he worked closely with the Ministry of Public Health of Afghanistan on monitoring and evaluation. 

Cyrus Engineer, DrPH, MHS, MHA (cengineer@towson.edu) is Assistant Professor & Director of the healthcare management program at Towson University, Baltimore. He is adjunct Assistant Professor at Johns Hopkins School of Public Health. His research focuses on health system performance especially linked to quality and patient safety. Most recently he was the Country Director for Johns Hopkins University projects in Afghanistan.