Measuring global hunger isn't easy. This new approach wants to change that

The first two Sustainable Development Goals (SDGs) are to end poverty and hunger. Both goals relate to the proportion of the population living below acceptable thresholds – poverty lines in one case, and dietary energy standards in the other. These goals involve the lower tail of distributions so statistics on living standards have to reliably measure not just means and totals but also variances. Debates about hunger measurement highlight the difficulty of achieving this and the inadequacy of current survey practice.

The global agency that measures hunger is the Food and Agriculture Organisation (FAO). The FAO use country averages of annual dietary energy per person, from aggregate Food Balance Sheets, and they distribute these across the population by using adjusted estimates of the inter-household variance in calories from surveys. Specifically, FAO dampen variances because the short reference periods for surveys give just a snapshot of the household’s diet for a week, fortnight or month, which causes excess variability (Cafiero 2014).

Proponents of other approaches argue for directly measuring hunger from household surveys which are deemed by the World Bank to be reliable enough for counting the poor (Carletto et al. 2012). However, the advocates of the direct survey approach do not emphasise that these diet snapshots will overstate variance and exaggerate the rate living below dietary thresholds.

The ideal (but impractical) survey to match what FAO attempt to measure – chronic hunger on an annual basis – would observe the same households over the course of a year. Such a design would show that many short-term shocks tend to cancel out over time. In contrast, a snapshot from a short reference period, even if for a month, will have a higher variance than the true annual variance because some of the shocks, but not their reversal, occur in that short reference period. This snapshot will overstate the chronic hunger rate, as shown in the figure below using a dietary threshold of 2,000 calories per person per day.

Figure 1. Daily calorie distribution

Thus it should not surprise if surveys report more hunger than FAO estimates. For example, 59% of the population in 12 African countries are undernourished according to surveys, while FAO estimate just a 39% hunger rate for the same countries (de Haen et al. 2011). Yet what is poorly understood is that these gaps are inherent features of the two approaches. Moreover, any consideration of the intra-year fluctuations causing these gaps is based on misleading narratives about ‘seasonality’.

A new approach to measuring chronic hunger

In a recent paper, I propose a new way to measure chronic hunger from surveys, which accounts for excess variability from just observing a snapshot of diets (Gibson 2016). This method also can identify the transient component of hunger, which is a type of welfare fluctuation that is neglected in the literature compared to the emphasis placed on transient poverty (and a type of hunger neglected by the FAO).

The proposed method needs surveys to see the same households in at least two, non-adjacent periods in the year. This survey design is rare. Dupriez et al. (2014) survey statistics offices in 100 low- and middle-income countries to obtain metadata on their food consumption surveys and find just two that use this type of intra-year panel. Many more surveys in their sample use revisits for short, adjacent, periods (e.g. every second day) to check on diary-keeping by respondents; the median diary-keeping survey has interviewers make five visits in two weeks.

However, seeing the same household repeatedly for, say, two weeks to implement a diary is less informative than seeing it for a week, and then again for another week, six months later. Seeing the same household at two or more times of the year reveals more about outcomes with low auto-correlation since a snapshot of these mis-measures their long-run average. The benefit from repeated observations has previously been noted for incomes, expenditures, and microenterprise profits, which have low auto-correlations (McKenzie 2012), and the new results show that calories are another outcome of interest with low auto-correlations.

To get a correct estimate of annual variances from snapshot surveys, the correlations between values of a living standards indicator in separate periods for the same households are needed. These correlations are implicitly assumed to be 1.0 if short reference period survey data are treated as equivalent to annual data. If correlations are only 0.7, monthly reference period surveys overstate annual variances by 40%, and by 80% if the correlations are as low as 0.5. The correlation-based method has been found to almost exactly replicate what benchmark annual data from year-long diaries show, for variance-based statistics such as inequality and poverty indices (Gibson et al. 2003) but has not previously been used to measure hunger.

Evidence on intra-year calorie correlations and chronic hunger from Myanmar

To illustrate the method of measuring chronic hunger from diet snapshots, I use survey data from Myanmar, where households were visited twice – six months apart. This survey of over 18,000 households thoroughly covers 228 food groups, and focuses on quantity consumed rather than on acquisitions (thus, dealing with storage and other leakages). Food consumed outside of the home is covered by 24 categories, and for all foods the survey uses the most natural local units (with metric-equivalent conversion done at the data processing stage). Thus, there should be more confidence in the food quantity and calorie data from this survey than from many other household surveys that are less suited to nutritional analyses.

The correlation between per capita calories in these two times of the year is just 0.45. This low correlation suggests that shocks to calories are partially reversed over the rest of the year. However, if data from a snapshot survey are used to represent the long-run situation, shocks are wrongly locked in as if they occur in every month of the year. Notably, the correlations are lower in per capita terms than in household terms because household size also fluctuates over time, especially in urban areas.

Two implications of these low correlations for hunger measurement can be seen in the figure below, which shows calorie densities and the prevalence of hunger. First, chronic hunger is exaggerated by unadjusted short-reference period data; these suggest 26% of the population get fewer than 2,000 calories per day. This hunger rate falls to just 14% once the excess variance in short reference period surveys is corrected.

Figure 2. Adjusted and unadjusted daily calorie distributions

Second, there is a lot of transitory hunger; the 12-point gap between the 14% and the 26% estimates in Figure 2 are people below 2,000 daily calories in a month but with annual average daily calories likely exceeding 2,000. Thus, short reference period surveys will identify some unknown mix of chronic and transient hunger. Since the appropriate interventions differ – smoothing in one case and raising the mean in the other – distinguishing these two types of hunger should be important to policymakers.

These low correlations have little to do with seasonality. They are lower in urban areas than in rural areas, and they do not differ between the heavily irrigated region of the Irrawaddy River delta and the seasonal dry zone. Instead, low correlations show that households cannot easily smooth their calories over time, due to shocks that may have demographic, health, income, or food price and food availability origins. Thus, surveys that stagger fieldwork over the months of the year to deal with seasonality, but see each household in just one period of the year, cannot capture these shocks and their (partial) reversal. Consequently, such surveys will overstate annual variances and overstate the chronic hunger rate.

Conclusions

Many household surveys are designed to get base weights for the CPI. To get these means and totals, a short reference period for each household – with sub-samples seen in different months of the year – is fine. However, this design overstates variances and mixes together chronic and transient welfare components.

Different survey designs, such as intra-year panels, are needed to accurately measure variances and the lower tail of distributions. This need is especially important because the SDGs aim to completely eliminate chronic hunger (and poverty). Thus, there is no allowance for measured hunger rates that may be above zero because they include transitory hunger picked up by short-reference period surveys.

References

Cafiero, C (2014) “Advances in hunger measurement: Traditional FAO methods and recent innovations”, FAO Statistics Division Working Paper Series, No 14-04.

Carletto C, A Zezza and R Banarjee (2012) “Towards better measurements of household food security: Harmonizing indicators and the role of household surveys”, Global Food Security, 2(1):30-40.

De Haen, H, S Klasen and M Qaim (2011) “What do we really know? Metrics for food insecurity and undernutrition”, Food Policy, 36(6): 760-769.

Dupriez, O, L Smith and N Troubat (2014) Assessment of the Reliability and Relevance of the Food Data Collected in National Household Consumption and Expenditure Surveys, FAO, IHSN and World Bank.

Gibson, J (2016) “Measuring chronic hunger from diet snapshots: Why 'bottom up' survey counts and 'top down' FAO estimates will never meet”, Working Paper 7/16, University of Waikato.

Gibson, J, J Huang and S Rozelle (2003) “Improving estimates of inequality and poverty from urban China's household income and expenditure survey”, Review of Income and Wealth, 49(1): 53-68.

McKenzie, D (2012) “Beyond baseline and follow-up: The case for more T in experiments”, Journal of Development Economics, 99(2): 210-221.