Credibility, Validity, and Assumptions in Program Evaluation Methodology

This book focuses on methods of choice in program evaluation. Credible methods choice lies in the assumptions we make about the appropriateness and validity of selected methods and the validity of those assumptions. As evaluators make methodological decisions in various stages of the evaluation process, a number of validity questions arise. Yet unexamined assumptions are a risk to useful evaluation. The first edition of this book discussed the formulation of credible methodological arguments and methods of examining validity assumptions. However, previous publications suggest advantages and disadvantages of using various methods and when to use them. Instead, this book analyzes assumptions underlying actual methodological choices in evaluation studies and how these influence evaluation quality. This analysis is the basis of suggested tools.

The second edition extends the review of methodological assumptions to the evaluation of humanitarian assistance. While evaluators of humanitarian action apply conventional research methods and standards, they have to adapt these methods to the challenges and constraints of crisis contexts. For example, the urgency and chaos of humanitarian emergencies makes it hard to obtain program documentation; objectives may be unclear, and early plans may quickly become outdated as the context changes or is clarified. The lack of up-to-date baseline data is not uncommon. Neither is staff turnover. Differences in perspective may intensify and undermine trust. The deviation from ideal circumstances challenges evaluation and calls for methodological innovation. And how do evaluators work with assumptions in non-ideal settings? What tools are most relevant and effective? This revised edition reviews major evaluations of humanitarian action and discusses strategies for working with evaluation assumptions in crises and stable program settings.

Apollo M. Nkwake is a Global Lead-Senior Technical Advisor for Monitoring, Evaluation and Learning (MEL) at Action Against Hunger USA. He previously worked as Associate Research Professor for MEL at The George Washington and Tulane Universities. He was a Senior Research Associate at Education Development Center. He also served as Senior MEL Manager at World Agroforestry Center. Apollo holds a PhD in Social Development from University of Cape Town and holds Canadian Evaluation Society’s Credentialed Evaluator designation. Dr. Nkwake is a recipient of American Evaluation Association’s 2017 Marcia Guttentag Promising New Evaluator Award. He has authored six books, several journal papers/book chapters and guest edited several special journal volumes. He is also the author of: Working with Assumptions in International Development Program Evaluation. He is Associate Editor for American Journal of Evaluation, a member of the International Evaluation Academy Council and Evaluation Reviewer for UNDP’s Independent Evaluation Group.