Robert Morello and Dr. Daniel Nielson, Department of Political Science
A structural problem plagues the delivery of foreign aid: the beneficiaries have little ability or means to inform the donors of the projects’ initial needs, progress, or effects. As development projects are typically executed, donors carry out assessments that often involve surveys or other before-and-after appraisals. But the approach is nearly always observational, and thus the results are likely biased toward positive findings. Thus, the feedback mechanisms that could regulate public spending in domestic contexts are broken or missing when it comes to foreign aid. Some scholars have identified the absence of feedback as the key reason why aid sometimes fails to create intended benefits. Easterly1 argues that aid fails because aid agency bureaucrats in faraway countries determine what impoverished people need, instead of the people themselves. The poor rarely, if ever, have the opportunity to provide input on their needs or feedback about whether aid addressed the needs. Therefore, aid agencies frequently have misplaced incentives and efforts.
One potential resolution to this failing involves grassroots monitoring through crowdsourcing, which gives large groups of citizens a means of reporting directly on relevant conditions in their neighborhoods and villages. Several attempts have been made to crowdsource grassroots monitoring; however, UNICEF’s innovative Ureport system promises to complete this task a much larger scale than any previous attempt. The program aggregates RapidSMS text messages from citizen monitors in Uganda, sending a poll question each week to registered participants. It also offers the opportunity to answer a burning question that remains about crowdsourcing: What motivates grassroots monitors?
This report used randomized evaluation to scientifically determine best practices for motivating Ureporters to respond to poll questions, sending texts message to UNICEF or report. Our results will suggest methods for revolutionizing recipient feedback mechanisms in international development, revealing the best way to harness the crowd for future grassroots monitoring.
- Treatment #1: This treatment Ureporters were provided with human feedback from BYU research assistants and the Uganda Scout Association to their Ureports.
- Treatment #2: This treatment encouraged answers to poll questions with a material incentive. Ureporters were asked to answer all poll questions to be entered to win a Google solar charger. After two week, the Ureporters were reminded of the incentive.
- Treatment #3: In this treatment, we created a social network through text messages. In societies that actively use Twitter, hash tags connect people to exchange information on similar interests. This treatment created a similar effect.
We found that that the treatments had varied effects. Compared to the control group, the Human Response intervention elicited a statistically significant more number of messages per Ureporter and an almost doubled the response rate to poll questions. The Response Lottery increased the number of message per Ureporter, but did not increase the response rate. The Twitter treatment depressed the number of message per Ureporter as well as depressing response rates. (see figure 1)
| Average Messages | Response Rate | |
|---|---|---|
| Control | 0.189 | 12.3% |
| Human Response | 0.300*** | 23.7%*** |
| Response Lottery | 0.253** | 14.8% |
| 0.054*** | 4.1%*** | |
| ** Significance at the 0.05 level *** Significance at the .001 level |
These initial results start to explain the Ureport crowd’s motivations. First, the Twitter treatment may indicate an effect similar to survey fatigue. In these districts, the simple suggestion of joining an SMS group seems to have depressed both the number of messages and the response ates. The intrinsic incentive to a join a group and actively participate in it seems to have overwhelmed these participants. Second, the response lottery treatment had an interesting effect. Adding an extrinsic incentive increases the number of messages received but does not increase the response rate. It created a subsection of the population that became excited about Ureport,
but the excitement did not spread generally. This subset increased the number of Ureports sent, but the added extrinsic incentive did not increase response rates or the number of Ureporters that were active in the system. The human response treatment had the intended effect. It increased the number of messages received as expected. It also increased the response rate. When Ureporter has someone actively listening to them, they were more likely to talk. This treatment is the only one in this study which increased the response rates. Therefore, we conclude that increasing the intrinsic incentive is the only way to increase response rates, but as we learned from the Twitter treatment, it is possible to overwhelm.
Intrinsic incentives are the crowd’s primary motivations. However, the crowd can easily be overwhelmed. In order to increase the crowd’s participation, intrinsic incentives should be increased to a currently unknown point. Donors need to listen to monitors at the grassroots level, and let them know that they are listening. This intrinsic motivation seems to have the only positive effect in the study. Therefore, we recommend using intrinsic incentives to encourage grassroots monitoring through crowdsourcing. These findings were presented to UNICEF Uganda to guide its future work with the Ureport system as well as at the World Bank. A publication of these results is forthcoming.
Grassroots monitoring may be one of the keys to improving development projects. This research starts to shed light on the crowd’s motivations to perform such work. It creates a theoretical basis for future designs of community monitoring through crowdsourcing. As the designs improve, we anticipate the effectiveness of the monitoring will also improve. And, as many scholars and practitioners have argued improved monitoring will improve development outcomes, alleviating poverty and improving the standard of living for billions around the globe.