Statistics over stories
NREGA must be improved upon through randomised evaluation, not anecdotal evidence….Rohit Kumar Singh
NREGA must be improved upon through randomised evaluation, not anecdotal evidence….Rohit Kumar Singh
The National Rural Employment Guarantee Scheme, more popularly known as NREGA, has come under sharp focus since the results of the recently concluded Lok Sabha elections were announced. While many commentators have attributed the UPAs return to power to the success of NREGA, some are still skeptical about its efficacy and impact.
Irrespective of the degree of its success, most experts agree that for NREGA to be more effective in the current economic context, it needs to be re-engineered; both (a) by the policy makers at the level of programme design and (b) by the implementation machinery at the level of actual delivery. This assumes importance when one considers the mammoth size of the programme it is probably the largest-ever public employment initiative anywhere in the world. For the purpose of the current discussion, we will restrict ourselves to the challenges in re-engineering the programme design.
At the design level, the two main criticisms of NREGA have been that (a) the quality of assets created under the programme leaves much to be desired; and (b) by having only unskilled physical labour as its mainstay, there is no incentive for rural youth to acquire higher skills; rather it acts as a disincentive. Both these criticisms need to be tested on an objective basis.
Here is the place we need to take a leaf out of the private sector book: we should make changes in programme design (interventions) not based on anecdotal evidence not even based on post-facto reviews by the so-called experts but based on randomised evaluation. This would help us arrive at a measurement of the efficacy of such interventions, in an objective and rational manner.
For the uninitiated, randomised evaluation measures the impact by analysing what would have happened in the absence of the interventions. For this purpose, a counterfactual is constructed by selecting a group that is not affected by the interventions. The randomisation is done at an appropriate level to create a control group that mimics the counterfactual.
For example, in the case of NREGA, many have attributed the low quality of assets created under the programme to the mandatory 60:40 ratio that has been fixed for labour to material costs. It is interesting how many of us in the bureaucracy love these guidelines and ratios. And, it is not just the Indian bureaucracy. Tony Emerson, editor of Newsweek International, while discussing NREGA during an informal conversation, told me about a news story that he covered in Newfoundland, Canada in the 1990s. In a programme similar to NREGA, people who worked 10 weeks were eligible for government-paid unemployment benefits for the rest of the 42 weeks of the year. As the lack of fish resulted in further deterioration of the fishing industry, people complained that getting work for even the mandatory 10 weeks was becoming difficult. The government came up with a programme to construct fishing related structures on the coast so that people got at least 10 weeks of employment, making them eligible for the benefits. The result redundant fishing structures got created that were of little or no use since the fish were drying up anyway!
Coming back to our 60:40 ratio mandated under NREGA. This stipulation implies that for a project costing Rs 50 lakhs, at least Rs 30 lakhs must be spent as wages. It appears reasonable when the primary objective is to create employment, but has an adverse impact on the quality of output as mostly kuchcha works fit this criterion. As a result, many village works, especially the link roads are vulnerable to medium to heavy rains. Intuitively, one could say that by tweaking the ratio in favour of the material component, this problem could be resolved. But the million dollar question is will there be any impact, and if at all, what will be its extent, when this ratio is changed to say, 50:50 or even 40:60 (labour:material).
This is where randomised evaluation comes handy. To measure the possible impact, we could initially select a set comprising, say, 100 panchayats located in one or a couple of similarly situated districts. These panchayats would then be randomly assigned to two groups the treatment group and the control group. In the control group, the existing labour:material ratio may be continued whereas the panchayats in the treatment group would be allowed to spend more on the material component if they see fit. It is important to collect baseline data initially and monitor the process so that the integrity of the experiment is not compromised. Then, after a suitable time period, follow up data (on the quality of roads or other assets created under NREGA) would need to be collected for both the groups of panchayats in identical ways. This data can then be used to estimate the impact of the intervention and to assess whether such impact is statistically and practically significant.
A similar experiment could easily be designed for including other basic skills like carpentry, masonry, welding and even data entry in the NREGA domain. For example, data entry could conveniently be dovetailed with the National Rural Health Mission (NRHM) or Sarva Shiksha Abhiyan (SSA) to create local panchayat level databases on immunisation of infants, expecting and lactating mothers (under NRHM) and attendance-cum-performance tracking of school children (under SSA). It is interesting to note that the wages paid by the private sector for basic data entry are similar to the wages paid for unskilled manual labour under NREGA. However, in case of data entry, there is a distinct incentive to upgrade skills to move to the next wage stratum.
Across the world, many social programmes are now being evaluated on a randomised basis to the benefit of governments, policy planners and donor agencies like the World Bank etc. Probably the best example is the evaluation of the de-worming programme of school children in the district of Busia, Kenya (by Edward Miguel and Michael Kremer of Harvard University). Here, randomised evaluation led to the conclusion that the most cost-effective intervention to increase school attendance among rural children was providing de-worming tablets to the kids. Or closer home, where randomised evaluation by Sewa Mandir in rural Udaipur (by Esther Duflo, Rema Hanna and Stephan Ryan of Poverty Action Lab, MIT) confirmed that the digital camera intervention in rural government schools resulted in controlling teacher absenteeism and improved academic performance of the school children.
As the country expectantly looks forward to many more meaningful social initiatives, it will be prudent to simultaneously plan for concurrent randomised evaluation of the new programmes.
The writer is an IAS officer working in Rajasthan. The views expressed are personal