The economic data for most countries in sub-Saharan Africa, and in particular GDP numbers, are so bad that they are almost useless without an understanding of the economic and statistical histories of individual countries. The misuse of such data has distorted policy decisions by governments, multilateral bodies, and development agencies, as well as the broader perception of Africa. And a huge body of academic work by economists that relies on such data is a case of "garbage in, garbage out".

"The tendency in development economics since the middle of the 1990s has been toward large studies with cross-country regressions, without any appreciation of problems with the data. This is particularly likely to yield nonsense or misleading findings, resulting in inaccurate economic histories of postcolonial Africa."

This is one depressing message of Poor Numbers, but Jerven is less interested in how these numbers have been (mis)used than in how they have been created, exploring their changing political and organisational contexts through case studies of individual countries and statistical offices.

Jerven begins with a brief introduction to the concept of Gross Domestic Product and some of its complexities. (While it has no technical details, Poor Numbers is nevertheless aimed at people with at least some familiarity with development issues, African history and geography, and economics.) Some of the problems in Africa include a large informal sector, measures mostly based on production rather income or expenditure data, irregular and low quality data collection, a mix of surveys and administrative data, and under-resourced statistical offices. One fundamental problem is that of baselines: full censuses and large surveys are rare, so annual production figures are based on cumulative changes since a baseline year (which are sometimes simply inferred from population growth or rainfall figures). Especially as the sectoral composition of the economy changes over time, this can produce serious underestimates of GDP.

To illustrate how much the results can vary, Jerven compares the three major sources of national income data: the World Development Indicators, the Penn World Tables, and the datasets of Angus Maddison. These differ not just in absolute GDP numbers but in their relative rankings: of the forty-five countries in sub-Saharan Africa, a fifth differ by more than ten places between the lists. These differences result from absent data, gap-filling procedures, and infrequent updating of base years: a rebasing of Ghana's series in 2010, accepted by the World Bank in 2011, resulted in its GDP rising by 60%. The inaccuracy of this data has been known for decades, but there has been little investigation of estimation procedures and assumptions, error margins, or the extent to which errors are systematic rather than random.

Jerven goes on to place the history of African economic statistics within the context of broader political economy. This is country-specific, but examples give a feel for how the complex history of national income data can make it impossible to determine the effects of structural adjustment or other policies.

"in the late 1990s both Zambia and Tanzania underwent a massive upward reappraisal of national income after structural adjustment. Both countries had followed a path of state-led development from the late 1960s until the crisis in the 1980s. During this period ... data on trade, services, and (by implication) production were collected by the parastatal companies, which were assumed to represent the whole economy. When those state agencies were unable to offer services or unable to offer services for an acceptable price, economic actors turned to informal and parallel operators. That is why the national income estimates recorded a massive decline in the late 1970s and early 1980s. It is impossible to correctly gauge the movement and/or the size of this unrecorded component. ... The resulting national income series is potentially misleading as scholars who wish to compare income across countries and across time approach per capita estimates."

To illustrate the kind of assumptions involved in generating numbers, and the controversies that can involve, Jerven presents three case studies: population census data from Nigeria, crop production figures from Nigeria, and national income revision for Tanzania.

"Measurement is not simply a technocratic exercise. The political economy in which the "facts" are embedded does matter. There is a clear trend of discontinuity in census-taking in Nigeria, from the colonial problem of evasion to the postcolonial race to be included. It is also a reminder of the importance and difficulty of getting "levels" right, and further, that the measure of change might be severely distorted when the levels are biased."

"In the Handbook of Economic Growth, Durlauf et al. argued that negative output shocks are a typical phenomenon among low-income countries. Not realizing that the "output shock" was a purely statistical artifact, they included the period from 1987 to 1990 in Tanzania on their "top ten list" of output collapses. It seems that when economic development experts are not country experts, the road from fact to fiction is short."

Quality data remains critical for decision-making: randomized trials and microlevel tests of data are useful, but governments still need reliable macro data to guide policy. Statistical offices sit in a "policy circle" involving governments and multilaterals such as the World Bank and the IMF, and are driven by changing priorities such as the Millennium Development Goals. Jerven has attempted his own survey of the capacity and constraints on statistical offices, looking at some individual countries in detail and offering suggestions as to what can be done to support them.

"Statistical officers who want to revise and rebase the national accounts series invariably rely on external consultants and funding. Since 2007, a technical assistance program supported by Norway ... has ensured that the methods used in Malawi (though not necessarily the raw data) meet European standards, and the base year and statistical methods have been updated ... the bigger problem in Malawi has been serious concerns about the independence of the statistical services."

Despite all of the problems, and the risks of "garbage in, garbage out", Jerven's conclusion stresses the importance of numbers, calling for "qualitative rigor" and interdisciplinary approaches, and defends the concept of GDP as "too important to be ignored", even if "the numbers are too poor to be trusted blindly".

"GDP should not be treated as an objective number but rather as a number that is a product of a process in which a range of arbitrary and controversial assumptions are made. ... the metric should be used with the utmost care."

Jerven also touches here on problems with other datasets: metrics of growth, corruption, battle deaths, democracy and so forth have similar problems to GDP, and are also misused despite warnings. The problems are not just conceptual: "scholars pay great attention to defining the concepts and devote great effort to theorizing the existence of the phenomenon and spend comparatively little time critically probing the numbers that are supposed to represent them".

November 2015

External links:

- buy from Bookshop.org
- buy from Amazon.com or Amazon.co.uk
- share this review on Facebook or Twitter

Related reviews:

- books about Africa + African history
- books about economics + finance
- books published by Cornell University Press