Is this the fundamental flaw in nearly all global poverty projects?

Since 1960, the world has spent roughly $5.7 trillion on foreign aid. Of that enormous sum, how much has been effective in fighting poverty?

The shocking truth is that we don’t really know. The International Monetary Fund (IMF) admits, “the effectiveness of foreign aid remains an unresolved issue.” A 2018 study concurs: “The poverty-reducing effects of aid are not well-documented.”

Why is this? If we’ve spent so much money fighting poverty over the years, why don’t we know better what works and what doesn’t?

While assessing impact is inherently complex, the real problem is this: international development projects are not designed to measure long-term results.

Even though the entire premise of international development is for people and countries to become self-sustaining in the long term, this is precisely what is not measured in the vast majority of economic development projects.

I have worked in international development for nearly 40 years. Yet, I can recall only a handful of projects, out of hundreds, which required any kind of impact evaluation after a project’s close.

More often, projects simply require a baseline assessment (measuring conditions at a project’s start) and an end-line assessment (measuring conditions at a project’s end). This last assessment occurs when project staff are still on the ground and in touch with programme participants. But, after the programme staff depart, we hardly ever measure whether participants sustain the progress themselves.

Last year, my non-profit organization, TechnoServe, operated more than 120 poverty reduction projects in 35 countries. Of those, only one project had funding to measure impact after it closed. Even that was funded separately by a private donor.

International development and evaluation expert, Jindra Cekan, PhD, has done a rough calculation of post-project evaluations overall, extrapolating from OECD public foreign aid funding data and Mali project data from USAID. Her conservative estimate is that – of all global development projects since 1960 – only 0.002% have been evaluated after they ended.

So, for every 50,000 global development projects, only one is ever evaluated for long-term impact.

Some may argue that end-line assessments are a good enough measure of success. But, in the rare cases where end-line and post-project evaluations are conducted, they often show different results.

To cite just one case: a USAID project in Malawi reported in its 2014 end-line survey that certain new practices had been “successfully introduced” among programme participants. However, an evaluation five years afterwards found “only limited evidence of sustained use of these practices and no differences [with control groups] in longer-term adoption.”

In other words, what seemed to work at the time the project ended, no longer worked a few years later.

Is this how we want to run development projects? If we simply assume short-term success is good enough, we will continue applying approaches that may not work in the long term. And, we will spend significant time, effort and money perpetuating a system of foreign aid, rather than self-sufficiency.

Ultimately, the cost of finding out what works will be lower than the cost of not knowing what works.

I realize the risk behind this. It’s hard to publicize unfavourable results. Development implementers feel pressure to prove their effectiveness to win new projects. Development funders feel pressure to show that aid dollars – especially from taxpayers – are working.

That’s why systemic change is needed. Post-project evaluations should become the norm, not the exception. This will lessen the pressure on individual long-term assessments to show positive results. And, of course, many post-project assessments will likely show a continued positive impact.

Most importantly, regardless of what the results show, we can learn from them. Recent ex-post evaluations on six USAID water and sanitation projects, for instance, yielded a treasure trove of actionable insights. Another post-project evaluation of a market systems development (MSD) programme in Ghana, noted that such ex-post evaluations were rare for MSD projects. “This evidence gap is particularly problematic,” the report said, “given the increasing adoption of MSD approaches by USAID and implementers.”

In other words, there is currently little evidence that these approaches work in the long term. But taxpayer money is nevertheless funding their replication all over the world.

Long-term impact evaluations won’t magically provide all the answers. But they will force us to ask the right question: how can we solve poverty for good?

If adopting regular post-project evaluations seems unrealistic, I would note that one development organization is already doing it. Japan International Cooperation Agency (JICA) conducts ex-post evaluations on all projects over a certain size and publishes the results.

While some types of development projects (e.g. infrastructure) more easily lend themselves to impact evaluations, projects of all sectors and geographies deserve assessment after they end.

The first step must be to include long-term impact measurement into project design from the outset. Charities and development organizations should not be left to find independent funding for their own long-term evaluations (as our organization has had to do repeatedly).

Secondly, economic development projects should be properly resourced to track beneficiaries for years after an intervention ends – often a challenge in remote and rural communities. To achieve this, the sector needs more specialized staff with strong evaluation expertise and access to the latest monitoring and evaluation tools.

Finally, we need to adopt standard, sector-wide operating procedures to measure economic impact years after projects end. This requires initial investment into more research to develop the proper protocols and metrics to provide comparable, meaningful data.

Finding genuine, long-term solutions to poverty is one of the greatest responsibilities that humanity has. To take this responsibility seriously, the entire development sector must invest more in finding and funding what really works.