Written by Rolf Luyendijk, Sr. Statistics and Monitoring Specialist, UNICEF Data and Analytics Section, New York.
This is the 15th post in our blog series on ‘What kind of ‘data revolution’ do we need for post-2015?’
Whereas several blogs in this series have highlighted the need for more frequent collection of data and the exploration of different sources of data and information, it is often the absence of a results- and evidence-based management culture, rather than a lack of data, that undermines transparency and accountability and holds back development. The analysis and effective use of data and information collected is critical to results-based management and to inform policy formulation and evaluation. Too much information sits idle in ledgers, in reports, in databases and information systems and too little, way too little, is effectively analysed and used. A focus just on collecting more data and information is insufficient.
As one example, both the UNICEF supported Multiple Indicator Cluster Surveys (MICS) and the USAID supported Demographic and Health Surveys (DHS) have provided a wealth of comparable information over a 20 to 25 year period – information that to date is largely untapped at a national level. These kinds of data are critical to effective planning and evaluation and provide a crucial complement to real-time data sources for day-to-day management. The latest MICS and DHS findings do enter into national situation analyses and progress assessment and have been very effectively used at a global level for tracking progress towards the international development goals, as well as for the analysis of a huge number of outcome indicators related to other areas of human development. But the wealth of information which can be disaggregated by different stratifiers of inequality, such as wealth, gender, ethnicity, religion, urban/rural areas, and broad geographical areas has gone largely unexplored at both a national and an international level.
As an illustrative example, the DHS surveys have included an asset-based classification of household wealth since the early 1990s. Nevertheless, it took until the mid-2000s before there were a critical number of analyses that made a larger non-academic audience aware that the MICS and DHS carried such information.
The mere increased availability of data does not necessarily lead to better policy making or better informed governments or decision makers. The widespread gut reaction that more data collection is needed when “we don’t know what’s going on” at all levels is not the right one. The first reaction, as with good academic research, should not be to collect more information, but to assess what information and data have already been collected – an inventory of what we already know.
New technologies and near infinite data storage capacity should allow us under the Data Revolution to build the infrastructure to compile, summarize, and digitize existing information at national and local levels and to create an open platform for data sharing. This would be foremost an exercise of data mining and compilation, rather than one of data collection. Importantly, it would allow the linking of data from different sources—household surveys and censuses but also data from information systems and ‘big data’ sources. New analytical techniques need to be developed that allow using household surveys to validate and complement more timely but less reliable data. The modalities still need to be worked out.
Result- and evidence-based management, which has been so hugely beneficial to the private sector, and increasingly so to the public sector, combines the collection of information with analysis, use and communication for learning what works and for increasing effectiveness and efficiency in achieving desired outcomes or results. Several blog posts in this series have pointed to strengthening the capacity of national statistical offices (NSOs) to do this, but I believe that strategy to be too limited and ineffective.
Indeed as this example from Nigeria related by Mićo Tatalović illustrated, the NSO should make their data freely and openly available and accessible in a format that allows for easy interpretation by users. But most NSOs don’t have the sector specific expertise or technical capacity to prepare further analysis of their data – let alone sector-specific analyses. Moreover, one could argue that the responsibility of the NSO is to independently collect national data and ensure their quality, and present and disseminate them for further analysis. This independence of the NSO and the data they produce should not be jeopardized by asking NSOs to engage in policy evaluation based on their data. This is best left to line-ministries and civil society groups who are more invested in what the data actually show.
Instead of strengthening the analytical capacity at NSOs for sector data, I want to argue that under the Data Revolution, governments should be encouraged to involve research institutes, think tanks, universities and civil society organizations – including youth groups, consumer actions groups, etc. – in policy relevant analysis of the wealth of data and information that are already out there and that are coming in at an unprecedented rate through new technologies and social media. This will gradually build up the citizen accountability mechanisms that have been so effective in influencing national policy debates in many of the more developed countries.
The widespread call for more timely collection of data, I fear, will only result in more unused information if that call does not go hand in hand with a) open access to data and information and b) the development and strengthening of national capacity to analyse and present these data in policy relevant way. The Data Revolution and the Global Partnership on Development Data needs to acknowledge that and include a strategy to develop this capacity.