Big data could have saved ancient civilizations, and it could save us

The burning of the Ancient Library of Alexandria has come to symbolize the tragedy of irretrievably losing valuable cultural information and knowledge. The Egyptian centre of scholarship was one of the largest and most important libraries of the ancient world, standing from its construction in the 3rd century BC until the Romans conquered Egypt in 30 BC.

In those days, much of the known world’s recorded knowledge was kept in the library. Today, our information is decentralized across huge regions, and thanks to digital technology, there is a lot more of it. So much more, in fact, that it’s been theorized that every last papyrus scroll in the vast Library of Alexandria could now fit onto an ordinary flash drive.

90% of the world’s data has been created in the last two years. According to the documentary The Human Face of Big Data (shown worldwide as part of SAP’s Our Digital Future film series), the typical person in the Western world is now exposed to as much data in one day as someone in the 15th century would have seen in their entire life.

For convenience, we call this phenomenon ‘big data’. We won’t always call it that. The term denotes something still new and exciting. Since the recent explosion of data generation, there hasn’t yet been time to scrape the surface of its potential to inform decisions.

There is such an abundance of data that we don’t yet know if it will actually hinder us more than help us. We could be suffering from debilitating information overload. Big data and digital technology is moving us into totally unchartered territory. We’re just beginning to understand how we can use all that data to improve the world and human lives.

Rather than just feeling smug about its ability to help us live better, we have to also be wary of following a path of self-destruction. We can’t simply dismiss doomsday believers as “negative” or “gloomy” thinkers.

The Library of Alexandria was the ancient world’s attempt at ‘big data’, making full use of the technology of the day. You can be sure the great minds of the day were scrutinizing the library’s ‘data’ to make intellectual connections, further human knowledge, and preserve and advance their civilization.

Little did they know at the time, their efforts were in vain; that the conquering Roman Empire would undo centuries of work invested into the library’s body of knowledge. Little did the Romans know, emperors’ squabbles and decadence would eventually lead to their downfall. Every ancient civilization collapsed for some reason or another (though never just one reason).

The world of today, however, is very different. We are in an age of independent states, rather than empires, and because of modern technology we are fast moving towards a global civilization. The fall of our global civilization would be a terminal disaster for the entire world.

The Human Face of Big Data spends some time discussing how we can get out of the problems we’ve made. Climate change, overpopulation, conflict over finite resources, invasive species, nuclear instability – it would be foolish to think any of these problems or threats will solve themselves or never occur.

As we tackle these problems with the help of big data, can we dare to dream that our global civilization will be the first that doesn’t unwittingly destroy itself? Can we ever be sure our actions won’t lead to our collapse? To begin answering these questions, let’s look back at a few ancient civilizations (chiefly by way of Jared Diamond’s excellent bookCollapse), consider how they collapsed, and think about how big data might have saved them…

Greenland Norse Vikings

In AD 984, Vikings settled in Greenland, and by 1450 they had died out. They inadvertently caused soil erosion and deforestation, which meant they weren’t able to make the charcoal they needed to support themselves as an Iron Age society. Dwindling trade with neighboring mother country Norway didn’t help the Vikings either, nor did their hostile relationship with the Eskimos with whom they shared Greenland. Those Eskimos may have blocked Norse access to the outer fjords, which they depended on for seals.

What if the Vikings had the ‘big data’ we have today? They might have built a vast sensor network feeding into a database system that measured how much deforestation they could safely carry out. They might have used insight from data to figure out how to share fjord access with the Eskimos harmoniously, while mapping an efficient trade route around the sea ice separating Norway and Greenland.

Easter Islanders

Out of hundreds of islands in the Pacific Ocean, none has suffered a case of deforestation as severe as that which destroyed the civilization on Easter Island in the 1600s. Jared Diamond has called it "the clearest example of a society that destroyed itself by over-exploiting its own resources." A combination of environmental factors led to the deforestation, but on such a small island, how could the Easter Islanders not have seen what they were doing? Diamond asked, “what did they say when they were cutting down the last palm tree?” In the future, people might be asking the same about us.

With big data, the civilization might have been able to identify and address problems caused by volcanic activity, latitude, rainfall patterns and the lack of “continental dust from Asia” that protects the Pacific islands by restoring soil fertility.

Indus Valley Civilisation

The largest of the early urban civilizations, the Indus once covered more than a million square kilometers and may have accounted for 10% of the world’s population. After a period of stability and great technological advancement, when the civilization’s rivers flooded adequately to support farming, climate change caused the floods to dry up, and the cities had to be abandoned.

We can’t imagine that happening in today’s developed cities, in an age where the likes of Buenos Aries have sensor networks monitoring and controlling the flow and pressure of their entire water supply. Even if natural climate change did make a region uninhabitable over time, we’d surely have the foresight to know it was coming.

If big data is to ‘save’ our global civilization, achieving something our ancestors didn’t, it will depend on more than just data. If big data really does have the potential to solve civilization-ending problems, our fate will depend on humans acting on insights.

The most urgent problems facing today’s global civilization are of our own making. There are many things we still don’t understand, and many things we haven’t started or stopped doing to solve these problems. Big data will be crucial in helping us change that. Unlike ancient civilizations, digital technology can ensure our future.