Friday, 25 January 2013

The limits of ‘evidence’: Evidence-Based Policy-making for African agriculture

By James Sumberg, Martha Awo, John Thompson, George T-M Kwadzo and Dela-Dem Doe Fiankor, Researchers, STEPS Centre Livestock project
Photo: Cavorting Chickens from rachelstrohm's photostream on Flickr
Agricultural policy makers in Africa are now being dragged into the era of 'evidence-based' policy (EBP) making. But the quality and availability of evidence in some countries - and debates about what even counts as evidence - create some interesting challenges.

Globally the proponents of EBP have been criticised for adopting a simplistic, linear understanding of the relationship between evidence and action, and for their normative approach to the desirable relationships between research-based knowledge and policy formulation. However, the literature on EBP, and particularly that associated with the 'realist synthesis', increasingly recognises that there are in fact different 'evidence bases'; that the notion of evidence can be quite slippery and contested; and that different kinds of evidence can be interpreted and valued differently by different groups and individuals. (That's a theme that will be explored in detail at the STEPS Centre's symposium on scientific advice in a couple of weeks' time.)

Despite the varied view of evidence in the literature, the idea that policy makers should take more account of 'evidence' (e.g. of what worked where, for whom and under what conditions) is now generally accepted. Many governments and donor agencies emphasise the central importance of EBP in improving development interventions and outcomes and in holding the policy actors to account. Of course, before the point of evaluating the impacts of various policy options, 'evidence' is also critical for establishing basic trends, constraints and dynamics within a sector or around a particular problem of interest.

But do African policy makers have access to good evidence? The challenges around the availability and quality of baseline data relating to food and agriculture in Africa (e.g. crop areas, yields, livestock populations and offtake levels) are long-standing and well recognised. More often than not, policy analysts, advocates and programme and project developers rely on national data series available through government statistics offices and FAOSTAT from the U.N. Food and Agriculture Organization because it is 'the best data available' (or more often because it is 'the only data available').

The coverage and quality of these data vary, depending on the importance and nature of the crop or species in question, the production systems used, or the market arrangements that are in place.  As one might expect, the data that relate to minor crops and livestock species, those dominated by small-scale producers and mixed cropping systems, or where there is little government intervention, are often more limited and/or of lower quality. It goes without saying that these 'evidence issues' have critical implications for efforts to establish evidence-based agricultural policy-making in Africa.

As our project on the 'livestock revolution' in West Africa is finding, chicken meat seems to be an increasingly important part of diets in Ghana (though in truth it's difficult to support this statement with 'hard' evidence, even if it's well supported anecdotally) so it is not surprising that the origin, availability, price and quality of chicken meat – and the state and future of the domestic poultry sector – are issues of intense public concern and policy debate in the country.

The main lines along which policy is argued in Ghana have remained relatively constant for at least a decade. These include the restriction (or not) of imports of frozen chicken meat through higher tariffs or an outright ban; the provision (or not) of direct support to domestic producers; the desirability (or not) of national self-sufficiency in poultry products; and the pros and cons of supporting commercial versus backyard producers.

The arguments that are used to support policy advocacy in relation to poultry highlight national food security (made more poignant by the food price spikes since 2007); the potential for employment generation (and more recently specifically for young people); product quality and public health concerns; the increasing importance of 'biosecurity'; the link often made between small-livestock like poultry and poverty alleviation, especially for women; and the Government of Ghana's willingness (or not) to stand up to international financial institutions.

While contestation around evidence – in terms of quality, relevance and meaning – is central to most policy processes, what is striking in the case of poultry in Ghana is the absence of credible information about even the most basic trends in chicken meat consumption or about the structure, health, growth (or demise?) of the domestic poultry sector. There are good reasons to believe that data available through FAOStat and those from the Statistics Research and Information Directorate (SRID) of the Ministry of Food and Agriculture (MOFA) (on which the FAOStat data are based) are so fundamentally flawed as to be of little if any value for policy making.

In principle, this absence of credible empirical evidence about the poultry sector creates a situation where policy debates and processes are dominated by claims and counter-claims that are repeated and embellished, and few of which can be tested. While there are many signs that the consumption of imported frozen chicken parts has increased, there is little data with which to address some key questions (is the domestic poultry sector really in a 'severe slump' and 'on its knees'? Would higher tariffs on the importation of frozen chicken really result in an additional 500,000 jobs in the domestic poultry sector? Is locally produced chicken meat really safer, more nutritious and tastier than frozen imported chicken?)

While claims and counter-claims, and narratives and counter-narratives, often play important parts in policy debates and processes, and while the 'meaning' of various bits of evidence can be hotly contested, those promoting evidence-based policy always assume the existence and availability of something akin to meaningful and systematic evidence. It would appear that for policy questions relating to poultry production and consumption in Ghana this assumption does not hold, and certainly not if we look primarily to readily available official statistics for that evidence.

In the relative absence of 'good' evidence, then, what should governments do?

One possible response is for government (perhaps with the help of development partners and the poultry industry) to invest in better systems for the collection and analysis of official statistics. Without doubt this is both logical and necessary, but will also be expensive and methodologically challenging, and will not yield results for many years. In any case, much strategic thinking is needed to determine which data are most essential and how these can be collected, stored and analysed in as systematic and cost-effective manner as possible.

Another is to look to non-traditional sources for 'evidence'. But what are these sources? How can they be tapped in a cost-effective manner? And what might be the implications of their collection and use for commercial confidentiality in what is a very competitive sector?

A third response is to reflect critically on the relevance of the EBP agenda in such situations: if EBP is the new gold standard, how can policy outcomes be improved when the traditional evidence vault is empty? This is not to suggest that the policy process should be simply cut free from the anchor of evidence. Rather, it is to be realistic about the contexts and circumstances in which agricultural policy is argued in Africa today, and more open about the need for new and creative approaches to the problem.

But this is a false choice: ultimately all three can help. Statistical systems need to be improved, new sources of evidence need to be developed and a more critical and politically informed understanding of the interplay between evidence and policy needs to be promoted.

In the meantime, policy processes relating to poultry consumption and production in Ghana will be prey to claim and counter-claim, narrative and counter-narrative. In such a situation, heuristics such as 'cheap food', 'national self-sufficiency' or 'food sovereignty' may provide the best basis for consistent policy.

For more information about the STEPS Centre's project on poultry in Ghana, visit the Livestock project page on our website.