Around the world, low immunizations rates for children are a persistent problem. Now, an experiment conducted in India shows that an inexpensive combination of methods, including text reminders and small financial incentives, has a major impact on immunization.
Led by MIT economists, the research finds that a trifecta of incentives, text messages, and information provided by local residents creates a 44 percent increase in child immunizations, at low cost. Alternately, without financial incentives, but still using text messages and local information, there is a 9 percent increase in immunizations at virtually no expense — the most cost-effective increase the researchers found.
“The most effective package overall has incentives, reminders, and enlisting of community ambassadors to remind people,” says MIT economist Esther Duflo, who helped lead the research. “The cost is very low. And an even more cost-effective package is to not have incentives — you can increase immunization just from reminders through social networks. That’s basically a free lunch because you are making a more effective use of the immunization infrastructure in place. So the small cost of the program is more compensated by the fact that the full cost of administering an immunization goes down.”
The experiment is also notable for the sophisticated new method the research team developed to combine a variety of these approaches in the experiment — and then see precisely what effects were produced by different combinations as well as their component parts.
“What is good about this is that it triangulates and links all these pieces of evidence together,” says MIT economist Abhijit Banerjee, who also helped lead the project. “In terms of our confidence in saying this is a reasonable policy recipe, that’s very important.”
A new paper detailing the results and the method, “Selecting the Most Effective Nudge: Evidence from a Large-Scale Experiment on Immunization,” is being published in the journal Econometrica. Duflo and Banerjee are among 11 co-authors of the paper, along with several staff members of MIT’s Abdul Latif Jameel Anti-Poverty Lab (J-PAL).
Duflo and Banerjee are also two of the co-founders of MIT’s Abdul Latif Jameel Anti-Poverty Lab (J-PAL), a global leader in field experiments about antipoverty programs. In 2019 they were awarded the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, along with Michael Kremer of Harvard University.
Analyzing 75 approaches at once
About 2 million children die per year globally from diseases that are vaccine-preventable. As of 2016, when the current study began, only 62 percent of children in India were fully immunized against tuberculosis, measles, diptheria, tetanus, and polio.
Prior research by Duflo and Banerjee has helped validate the value of finding new ways to boost immunizations rates. In one prior study the economists found that immunization rates for rural children in the state of Rajasthan, India, jumped from 5 percent to 39 percent when their families were offered a modest quantity of lentils as an incentive. (That finding was mentioned in their Nobel citation.) Subsequently, many other researchers have studied new methods of increasing immunization.
To conduct the current study, the research team partnered with the state government of Haryana, India, to conduct an experiment spanning more than 900 villages, from 2016 through 2019.
The researchers based the experiment around their three basic ways of encouraging parents to get their children vaccinated: financial incentives, text messages, and information from local “ambassadors,” that is, well-connected residents. The research team then developed a set of varying combinations of these elements. In some cases they would offer more incentives, or fewer, along with different amounts of text messages, and different kinds of exposure to local information.
In all, the researchers wound up with 75 combinations of these elements and developed a new method to evaluate them all, which they call treatment variant aggregation (TVA). Essentially, the scholars developed an algorithm that used a systematic data-driven approach to pool together variations that were ultimately identical, and noted which ones were ineffective. To select the best package, they also adjusted their results for the so-called “winner’s curse” of social-science studies, in which the policy option that works best in a particular experiment will tend to be the one that did better due to random chance.
All told, the scholars believe they have developed a way of evaluating many “treatments” — the individual elements, such as financial incentives — within the same experiment, rather than just trying out one concept, like distributing lentils, per every large study.
“It’s not one experiment where you compare A with B,” says Banerjee, who is also the Ford Foundation International Professor of Economics. “What we do here is evaluate a combination of things. Even in scenarios where you see no effect, there is information to be harvested. It may be that in a combination of treatments, maybe one element works well, and the others have a negative effect and the net is zero, but there is information there. So, you want to keep track of all the possibilities as you go along, although it is a mathematically difficult exercise.”
The researchers were also able to discern that differences among local populations have an impact on the effectiveness of the different elements being tested. Generally, groups with lower immunization rates will respond more to incentives to immunize.
“In a way, we are landing back where we were in [the lentil study in] Rajasthan, where low immunization rates lead to super-high effects for these incentives,” says Duflo, who is also the Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics. “We replicated the result in this context.” However, she reinforces, the new method allows scholars to acquire more information about that process more quickly.
An actionable path
The research team is hopeful that the new TVA method will gain wider adoption among scholars and lead to more experiments with multifaceted approaches, in which numerous potential solutions are evaluated simultaneously. The method could apply to antipoverty research, medical trials, and more.
Beyond that, they note, these kinds of results give governments and other organizations the ability to see how different policy options will play out, in both medical and fiscal terms.
“The reason why we did this was to be able to give the government of Haryana an actionable path, moving forward,” Duflo says.
She adds: “People before thought in order to say something with confidence, you should try just one treatment at a time,” meaning, one type of intervention at a time, such as incentives, or text messages. However, Duflo notes, “I’m very happy to say you can have more than one, and you can analyze all of them. It takes many steps, but such is life: many steps.”
In addition to Duflo and Banerjee, the co-authors of the study are Arun G. Chandrasekhar of J-PAL; Suresh Dalpath of the Government of Haryana; John Floretta of J-PAL; Matthew O. Jackson, an economist at Stanford University; Harini Kannan of J-PAL; Francine Loza of J-PAL; Anirudh Sankar of Stanford; Anna Schrimpf of J-PAL; and Maheshwor Shrestha of the World Bank.
The research was made possible through cooperation with the Haryana Department of Health and Family Welfare.