Interpreting the Data Revolution: Proceed with Caution (Part 1)

Written by David Roodman, freelance policy consultant based in Washington, DC. He blogs at davidroodman.com.

This is the latest post in our blog series on ‘What kind of ‘data revolution’ do we need for post-2015?, and is the first part of a two-part blog. 

In its report last summer, the High-Level Panel of Eminent Persons on the Post-2015 Development Agenda (HLP) produced a fantastic meme, a call for a “data revolution.” The sexy meme could be a boon for those wanting to reverse the chronic underfunding of the public goods that are national statistics.

But, partly because of the vagueness of the term, partly because most of its proponents are in donor countries, I worry that the “data revolution” will become an aid fad, a good impulse unmoored from critical thinking by collective excitement, leading to waste.

Where there are both pitfalls and promise, there is a puzzle. Having thought about this some, I will present my current understanding in three lists: interpretations of the data revolution, cautions, and positive recommendations.

First, the interpretations. From the text of the HLP report, I infer four meanings of “data revolution”:

1. A technology revolution: Exploiting new technological possibilities for monitoring impacts, outcomes, and states of poverty, through mobile phones, smart cards, and other technologies.

2. Open data: Making data public to support entrepreneurs and policy makers and help citizens hold government accountable.

3. Capacity building in national statistical agencies: Building up the domestic institutions that collect most statistics on human development.

4. A big survey push: Building a comprehensive baseline for the post-2015 goals, which would, in some 150 countries, collect nationally representative data for the indicators subject to targets, with enough resolution to permit breakouts by “gender, geography, income, disability, and other categories.”

Other interpretations include improved tracking of aid flows and improved quality of administrative data, which is generated not by surveys but by the routine operation of government.

Reacting to those ideas, I formulate several cautions. My purpose is not to counsel despair to data revolutionaries. Rather, if the “data revolution” is an idea worthy of serious consideration, then I want to suggest it must be able to survive and benefit from tough questioning. The cautions:

– “Data,” understood broadly, is like “documents”: inarguably important, but not a good organizing concept for a broad social project. Do we also need a “document revolution”? Specificity is essential in discussing what a “data revolution” should be and whether and how it is worth pursuing.

– Local decision makers are more central to the fight against poverty than the international community clustered around formulation of post-2015 development goals. Their needs for data are diverse and dynamic, rendering unrealistic any rhetoric about supplying the needs of all actors in development. In addition, a drive for a comprehensive baseline for the post-2015 goals could unwittingly override the needs of these local problem solvers for different data or the same data collected different ways.

– While new technologies hold great promise for data collection, when it comes to tracking progress on international development goals, there is as yet no easy substitute for traditional survey design and execution. (But see this good review of possible roles for technology.)

– Although inspired by results-based management — guidance of an organization through repeated measurement of outcomes — the Millennium Development Goals are not an instance of RBM, because no one manages the world. The appearance of RBM — the targets and timetables — has helped the MDGs cut through the media clutter and educate the public about the extent of poverty in the world and the realistic hope for reducing it. The MDGs probably help explain the substantial rise in European aid during the 2000’s, and perhaps have lifted or shifted spending on health and education by developing-country governments. It is plausible that the MDGs have accelerated progress on a few outcomes, such as child mortality, but it is hard to be sure. The upshot: public education being the primary channel for change, a comprehensive 2015 baseline for the MDG successors is ideal but not urgent.

– Low-tech is still often optimal where labor is cheap. Donors should not overly push the capital- and technology-intensive methods they know best. (Interestingly, Kenya’s mobile phone–based money transfer system, an emblem of the power of high tech in poor places, complements low-wage labor: it deploys the mobile phone as a skin for the labor-intensive businesses of manning kiosks and moving cash.)

– Although national statistical production is chronically underfunded, it is not a machine whose output is automatically improved by more funding. Perverse incentives can undercut the effectiveness of statistical capacity, no matter how well funded. Morten Jerven has documented how politically charged the population census is in Nigeria is, because it can affect elections. Justin Sandefur has shown that teachers in Kenya over-report school enrollments, presumably to increase the funding they get. The impact of more aid for statistics thus depends on context and on how funding is provided.

In the next part of this blog, I will outline some potential ways forward.

Click here to read the second part.

3 Comments on "Interpreting the Data Revolution: Proceed with Caution (Part 1)"

  1. There are two other aspects of the data revolution that you should consider, David.

    One is the push to have large corporations report more systematically on the sustainability of their operations, both in terms of environmental footprint as well as social footprint, like jobs created and wages paid, standards of quality raised/training of suppliers and distributors. By bringing a greater understanding of private sector activity into Boardrooms, private sector operations can be made more aligned with broader development objectives.

    The second is the idea of integrating natural resource use more centrally into national accounts so that a range of environmental externalities are actually taken into account when development projects (public and private) are considered.

  2. “Capacity building” is a going to be a tough one.

    Here’s a potential pitfall, for example. Maybe we should all agree to a standardized form of data collection, management, and analytics for open data’s sake, so our respective datasets may have more relevance outside our organization’s own publications. However, we’ve got to agree to things like what type of software should we promote? Expensive license-based softwares like Stata that may be too expensive for organizations, or free open source software like R that are quite more complicated?

    Extrapolate the problems we have training our less tech-friendly co-workers and partners here in the “industrialized world”, to the developing world where existing supporting infrastructure may be lacking. That’s gonna a fun hurdle to jump.

    Excellent article, David.

  3. Homi: noted.
    Timothy: I’m not that concerned about this kind of standardization. There are already standards for formatting and sharing data (CSV, XML, OData) making it possible to import into lots of different kinds of software. I don’t think any central body needs to advise people on whether to use R or Stata or whatever, any more than we need a central body to develop standards for the format of books. Data generation and analysis happen organically. Maybe they don’t happen enough, but there’s only so much we can do about that, just as there’s only so much that we can do about the fact that policymakers people don’t read enough relevant analysis. What we can do is focus on certain of data that are particularly valuable and under-supplies and consider whether it makes sense for outsiders to subsidize them.
    –David

Leave a comment

Your email address will not be published.


*