A chapter in the forthcoming edited volume, Randomized Control Trials in Development: the Gold Standard Revisited, by Florent Bédécarrats, Isabelle Guerin, and François Roubaud.
The use of RCTs in development economics has attracted a consistent drumbeat of criticism, but relatively little response from so-called randomistas (other than a steadily increasing number of practitioners and papers). Here I systematize the critiques and discuss the difficulty in responding directly to them. Then I apply prominent RCT critic Lant Pritchett’s PDIA framework to illustrate how the RCT movement has been responsive to the critiques if not to the critics through a steady evolution of practice. Finally, I assess the current state of the RCT movement in terms of impact and productivity.
Publications
Viewing all posts with tag: Randomized Control Trials
The Disruptive Power of RCTs
A chapter in the forthcoming edited volume, Randomized Control Trials in Development: the Gold Standard Revisited, by Florent Bédécarrats, Isabelle Guerin, and François Roubaud.
The use of RCTs in development economics has attracted a consistent drumbeat of criticism, but relatively little response from so-called randomistas (other than a steadily increasing number of practitioners and papers). Here I systematize the critiques and discuss the difficulty in responding directly to them. Then I apply prominent RCT critic Lant Pritchett’s PDIA framework to illustrate how the RCT movement has been responsive to the critiques if not to the critics through a steady evolution of practice. Finally, I assess the current state of the RCT movement in terms of impact and productivity.
What We Know So Far: 9 RCTs on Microcredit
What we know so far: 9 randomized control trials (RCTs) about the impact of microcredit.
The Wisdom of the Group: How Lessons from Savings Groups Can Guide Financial Product Innovation
In this note we focus on the savings group as a model for delivering products to address this market failure. Reviewing recent research, we extract the mechanisms that make savings groups effective. We then explore the potential to apply these factors to formal products that make sense for both providers and consumers.
Substitution Bias & External Validity: Why an innovative anti-poverty program showed no net impact
The net impact of development interventions can depend on the availability of close substitutes to the intervention. We analyze a randomized trial of an innovative anti-poverty program in South India which provides “ultra-poor” households with inputs to create a new, sustainable livelihood. We find no statistically significant evidence of lasting net impact on consumption, income or asset accumulation. Instead, income from the new livelihood substituted for earnings from wage labor. A very similar intervention made a large difference elsewhere in South Asia, however, where wage labor alternatives were less compelling. The analysis highlights the roles of substitution bias and dropout bias in shaping evaluation results and delimiting external validity.
Latest Findings from Randomized Evaluations of Microfinance
In 2009, the results from two microcredit impact studies in Hyderabad, India, and Manila, the Philippines were released to mixed responses (Banerjee, Duflo, Glennerster, and Kinnan 2010; Karlan and Zinman 2011). Some media declared microfinance a failure (Bennett 2009). Many in the microfinance community dismissed these randomized studies as too limited to be a true reflection of the entire sector .
Evaluation Fundamentals
Impact evaluations try to measure the change in a participant’s life that occurred because of an intervention. The “intervention” could be a policy, a project, an insurance product, or a specific feature of a product. For instance, the intervention could relate to a particular product feature, such as the extent of coverage, a change of pricing structure, or variations in the distribution channel.
Targeting the Ultra Poor
Can the poorest be reached with finance? If yes, there are two main routes. The first option is for institutions to extend existing products and services to even poorer customers. The other is to design independent approaches that target the particular challenges faced by the ultra poor.
An Introduction to Impact Evaluations with Randomized Designs
Randomized experiments are increasingly popular ways to evaluate the impacts of development interventions. They provide hope that we can overcome important biases common to nearly all statistical evaluations. When done well, randomized control trials (RCTs) can provide clear, transparent, and credible evidence in complicated contexts, and it’s not surprising that they dominate clinical research in medicine.
Access to Finance: Chapter 2, Handbook of Development Economics, Volume 5
Expanding access to financial services holds the promise to help reduce poverty and spur economic development. But, as a practical matter, commercial banks have faced challenges expanding access to poor and low-income households in developing economies, and nonprofits have had limited reach. We review recent innovations that are improving the quantity and quality of financial access. They are taking possibilities well beyond early models centered on providing “microcredit” for small business investment. We focus on new credit mechanisms and devices that help households manage cash flows, save, and cope with risk. Our eye is on contract designs, product innovations, regulatory policy, and ultimately economic and social impacts. We relate the innovations and empirical evidence to theoretical ideas, drawing links in particular to new work in behavioral economics and to randomized evaluation methods.